Ayzaan Wahid

Hey everyone, sorry for the dependency issues. Looks like there are issues both with python upgrade from 3.10 -> 3.11, and with some pypi versions being unavailable. I'll update the...

Hi @MaximilianToelle, I've just pushed a portion of the code for normalization which you can find in this commit: https://github.com/google-research/language-table/commit/2cabececffa3c1420345fd7df522a7664532b2c0 This contains the normalization code which you can take a...

The training code is available now! The python dependency chain was a bit tricky, mainly because of a TF dependency on specific protouf versions. I've added a `requirements_static.txt` with pinned...

@MaximilianToelle, to answer your questions: * Ah yes this is wrong and/or bad variable naming, since we're taking the mean it's not a per example loss. In this case it...

Yes the model we used in real used `lava_image_encoder="resnet"` with this config to freeze parameters: ``` config.agent.pretrained_checkpoints = [ ( "", [("params/text", "params/encoder/TextEncoder_0")], ), ("", [("params", "params/encoder/ResNetVisualEncoder_0/MultiscaleResNet_0"), ("batch_stats", "batch_stats/encoder/ResNetVisualEncoder_0/MultiscaleResNet_0")]) ]...

Hi, thanks for the question. Overall we tried to minimize the number of modalities, and ended up being able to learn good policies using only RGB. Perhaps this is because...

It's possible to experiment with this in sim by producing a dataset that also outputs a depth image, using the provided oracle policies.