Does OpenVLA baseline work in the simulated environments?

[StoneT2000]

Was taking a look at the OpenVLA baseline in colab but it seems the final video is a failure. For some context I'm in the middle of porting the simpler environments to the latest ManiSkill (runs faster + easy support of batched evaluation via GPU parallel rendering). Was planning on using this repository probably for testing/evaluation (it's very clean code! and only pytorch, jax dependencies of octo eval is causing some problems)

[sebbyjp]

Hi sounds very cool and thank you! It wasn't able to do any of the tasks when we tried but maybe with more fine-tuning or co-training in maniskill. If your project involves running robotics transformers seamlessly in simulation environments please let me know if you're looking for contributions :)