Immiscible Diffusion Policy

Preserving Multimodal Robot Actions
through Label-Free Noise Assignment

Xiao Zhang^1,*Yuxin Chen^1,*Zhixuan Liang^2,*Guojian Zhan¹Chenran Li¹Chenfeng Xu³Masayoshi Tomizuka¹Yiheng Li¹

¹ University of California, Berkeley² Princeton University³ The University of Texas at Austin

* Equal contribution

Paper PDF Code DP + DP3 arXiv

Preserving more than one way to act

A robot can often complete the same task in several ways: push an object clockwise or counterclockwise, pick it up with either hand, or place it in different valid orientations. A diffusion policy should be able to reproduce these alternatives from the demonstrations.

In our experiments, however, the policy can concentrate on a single behavior even when the training data and individual batches are balanced. A different random noise sample does not necessarily produce a different action modality.

Retaining those alternatives matters when a change in the environment makes the preferred behavior infeasible. For example, an obstacle may block one pushing direction while leaving the other available. We study how to preserve this diversity without requiring modality labels or changing policy inference.

Experimental Push T starting scene with an illustrative clockwise pushing path

Experimental Push T starting scene with an illustrative counterclockwise pushing path

Modality collapse in Diffusion Policy

Diffusion policies can exhibit modality collapse even when the demonstration data is well balanced. Although the demonstrations cover multiple valid behaviors in balanced proportions, the learned policy can concentrate on one behavior or a subset of behaviors. We observe this even when mirrored demonstrations enforce exact symmetry within every training batch.

In Push T, vanilla policies assign an average of 97.6% of rollouts to a single modality across 10 training seeds, with the preferred direction varying across seeds. Eight of the 10 policies generate only one modality across all 50 evaluation rollouts, despite equal representation of both directions in every training batch.

The same failure extends across all seven tasks: vanilla policies strongly favor one modality on the simulation tasks with two modalities, lose a modality on both tasks with four modalities, and use only one hand on both humanoid tasks.

Immiscible Diffusion Policy

Our analysis connects modality collapse to the way actions are paired with noise during training. Independent pairing can mix diffusion paths from different action modalities, weakening the denoiser’s ability to distinguish them.

Immiscible Diffusion Policy assigns each action chunk to exactly one noise sample within a training batch. Hungarian matching organizes the mapping from noise to actions, helping distinct modalities remain reachable. The assignment requires no modality labels and leaves the policy architecture and inference procedure unchanged.

Method overview: Hungarian assignment of noise to actions reduces mixing between diffusion paths and preserves clockwise and counterclockwise Push-T behaviors

Sample noise

Draw one Gaussian noise sample per action chunk. Standardize the action coordinates across the batch for computing the matching cost.

Match one to one

Use Hungarian assignment to minimize total Euclidean distance between action chunks and noise. Every noise sample is used exactly once.

Train as usual

Use the reassigned noise in the diffusion objective. Keep the policy architecture, observation conditioning, and inference procedure unchanged.

The assignment minimizes the sum of Euclidean distances between flattened action chunks standardized across the batch and the sampled noise. Standardization is used only to compute this cost; the diffusion objective uses the original normalized actions.

π* = arg min_{π ∈ S_B} ∑_{i = 1}^B ‖ã_i − ε_π(i)‖₂

π permutes the B noise samples in a training batch.

Algorithm 1 Immiscible Diffusion Policy training step

Input: A batch of observations and normalized action chunks {(o_i, a_i)}_i=1^B, policy parameters θ, and diffusion schedule ᾱ.

Notation: D is the number of action coordinates per chunk; μ_B and σ_B are their coordinate means and standard deviations across the batch. m_i is the existing loss mask, or all ones when no mask is used.

Draw one Gaussian noise sample per action chunk: ε_i ∼ 𝒩(0, I_D).
Flatten each action chunk into a vector: x_i ← vec(a_i).
Standardize coordinates for matching: ${x̃}_{i} \leftarrow \frac{x_{i} - μ_{B}}{\max (σ_{B}, 10^{- 6})}$
Compute Euclidean distances between actions and noise: $C_{i j} \leftarrow {‖ {x̃}_{i} - vec (ε_{j}) ‖}_{2}$
Minimize total matching cost with the Hungarian algorithm: $π \leftarrow \underset{π \in S_{B}}{arg min} \sum_{i = 1}^{B} C_{i, π (i)}$
Reorder the original noise samples using the assignment: ε′_i ← ε_π(i).
Choose a corruption level for each chunk: t_i ∼ Uniform({1, …, T}).
Apply forward diffusion to the original normalized actions: ${a_{i}}^{t_{i}} \leftarrow \sqrt{{ᾱ}_{t_{i}}} a_{i} + \sqrt{1 - {ᾱ}_{t_{i}}} {ε′}_{i}$
Select the prediction target: y_i ← ε′_i for noise prediction, or y_i ← a_i for action prediction.
Compute the masked mean squared prediction error: $ℒ \leftarrow \frac{1}{B D} \sum_{i = 1}^{B} {‖ m_{i} ⊙ (f_{θ} ({a_{i}}^{t_{i}}, t_{i}, o_{i}) - y_{i}) ‖}_{2}^{2}$
Update the policy parameters with the existing optimizer: θ ← OptimizerStep(θ, ∇_θℒ).

View the full codebase on GitHub

Experimental results

We evaluate five simulation tasks and two tasks on a Unitree G1 humanoid, using Diffusion Policy (DP) and 3D Diffusion Policy (DP3). The experiments span state, RGB, and point cloud observations, with two or four valid modalities defined by pushing direction, arm choice, placement orientation, or rotation direction.

We evaluate how often each demonstrated modality appears in policy rollouts. Across the three simulation tasks with two modes, Immiscible increases the frequency of the minority mode by 6.0 to 14.6 times. On both tasks with four modes, it recovers a modality that is absent from vanilla rollouts. On each G1 task, both hand modalities appear in a 50:50 split.

Simulation

Push-T

Behaviors in this task

Clockwise pushing
Counterclockwise pushing

Push T task setup showing both valid pushing directions

Example Immiscible Diffusion Policy rollout for Push-T

Rollout modality distribution % of rollouts

Modality preservation during training

Figure 5

Training curves show ratios of the dominant mode for three tasks with two modes and modality composition for two tasks with four modes; dashed lines mark assignment activation — Dashed lines mark when noise assignment is activated. Results average the training seeds reported in Table I; shaded regions in the upper plots show standard error.

Upper plots: how strongly the policy favors one behavior. For the three tasks with two modalities, the curves show the fraction of rollouts assigned to the dominant modality. A value near 1 means nearly every rollout follows the same behavior; 0.5 represents equal use of both behaviors. As training progresses, vanilla Diffusion Policy increasingly favors one modality. After noise assignment is activated, Immiscible keeps more rollouts in the alternative modality.

Lower plots: which behaviors remain available. For Place Shoe and Rotate QR Code, each colored band shows the fraction of rollouts using one of the four modalities. A band that shrinks to zero indicates that the corresponding behavior is no longer generated. Vanilla loses a demonstrated modality in both tasks, while Immiscible preserves all four with a more balanced distribution. These plots measure behavioral diversity over training; task coverage and success are reported separately in the table below.

All task results

Table I

Table I. Expert data lists the demonstration proportions reported in the paper. Policy results are percentages, reported as mean ± standard error across training seeds. Rollout frequencies are ordered under vanilla, with the same order retained for Immiscible; the physical behavior at each rank can vary across seeds. The G1 tasks use one seed, so no standard error is reported.
Task	Expert data (%)	Policy	Frequencies in vanilla order (%)	Task metric (%)	Seeds
Push-T	43.1 / 56.9	Vanilla	97.60 ± 1.62 / 2.40 ± 1.62	Coverage: 95.16 ± 0.30	10
Push-T	43.1 / 56.9	Immiscible	65.00 ± 7.64 / 35.00 ± 7.64	Coverage: 95.39 ± 0.10	10
Stack Blocks	50.0 / 50.0	Vanilla	96.67 ± 2.72 / 3.33 ± 2.72	Success: 96.67 ± 2.72	3
Stack Blocks	50.0 / 50.0	Immiscible	80.00 ± 0.00 / 20.00 ± 0.00	Success: 96.67 ± 2.72	3
Place Bread	51.0 / 49.0	Vanilla	95.56 ± 3.63 / 4.44 ± 3.63	Success: 100.00 ± 0.00	3
Place Bread	51.0 / 49.0	Immiscible	66.67 ± 8.31 / 33.33 ± 8.31	Success: 97.78 ± 1.81	3
Place Shoe	25.7 / 23.8 / 24.3 / 26.2	Vanilla	37.80 ± 14.50 / 37.80 ± 15.50 / 24.40 ± 1.80 / 0.00 ± 0.00	Success: 100.00 ± 0.00	3
Place Shoe	25.7 / 23.8 / 24.3 / 26.2	Immiscible	24.92 ± 1.75 / 29.68 ± 4.89 / 29.37 ± 5.53 / 16.03 ± 8.16	Success: 93.33 ± 3.14	3
Rotate QR Code	23.3 / 24.8 / 26.7 / 25.2	Vanilla	57.80 ± 11.90 / 35.60 ± 12.70 / 6.70 ± 3.10 / 0.00 ± 0.00	Success: 86.67 ± 5.44	3
Rotate QR Code	23.3 / 24.8 / 26.7 / 25.2	Immiscible	44.40 ± 9.60 / 17.80 ± 6.50 / 26.70 ± 6.30 / 11.10 ± 6.50	Success: 73.33 ± 3.14	3
Fruit-to-Plate	50.0 / 50.0	Vanilla	100.00 / 0.00	Success: 75.00	1
Fruit-to-Plate	50.0 / 50.0	Immiscible	50.00 / 50.00	Success: 60.00	1
Can-to-Bin	50.0 / 50.0	Vanilla	100.00 / 0.00	Success: 100.00	1
Can-to-Bin	50.0 / 50.0	Immiscible	50.00 / 50.00	Success: 90.00	1

Conclusion

We identify action modality collapse as an important limitation of diffusion policies for robotic action generation. Policies can concentrate on a single modality or a subset of modalities even when demonstrations are balanced and symmetry is enforced within every training batch. Our analysis connects this behavior to independent pairing of actions and noise, which increases the mixing and crossing of diffusion paths and weakens the denoiser’s sensitivity to differences between modalities in clustered action spaces of low dimensionality.

Immiscible Diffusion Policy addresses this issue by coupling action chunks with noise samples during training, without modality labels or changes to the policy architecture or inference procedure. Across simulation and real world tasks, it improves modality preservation while maintaining strong task performance, preserving suppressed behaviors and recovering valid modalities absent from vanilla rollouts. These results show that explicit coupling between actions and noise offers a simple and broadly applicable approach to preserving action multimodality in robot planning with diffusion policies.

BibTeX

Immiscible Diffusion Policy
Under review for ICRA 2027.

arXiv:2610.09369

BIBTEX

@article{zhang2026immiscible,
  title={Immiscible Diffusion Policy: Preserving Multimodal Robot Actions
         through Label-Free Noise Assignment},
  author={Zhang, Xiao and Chen, Yuxin and Liang, Zhixuan and Zhan, Guojian
          and Li, Chenran and Xu, Chenfeng and Tomizuka, Masayoshi and Li, Yiheng},
  journal={arXiv preprint arXiv:2610.09369},
  year={2026}
}

Download .bib