Diffusion policy's RGB encoder shared by all cameras. Is this intended?

[HiroIshida]

In the diffusion policy implementation, it seems that RGB encoder is shared by all camera images.

lerobot/lerobot/common/policies/diffusion/modeling_diffusion.py

Line 185 in c351e1f

self.rgb_encoder = DiffusionRgbEncoder(config)

However, in the original paper, different image encoders are used for different cameras.

@alexander-soare Is this design choice? (e.g. single encoder is better than multiple encoders?)

Chi, Cheng, et al. "Diffusion policy: Visuomotor policy learning via action diffusion." The International Journal of Robotics Research (2023): 02783649241273668.

[alexander-soare]

@HiroIshida thanks for raising this. It was a choice for simple over complicated, mostly for speed of development. We're not against using multiple image encoders going forward.