Questions on Pi0FASTConfig Settings

[destroy314]

I noticed that most of the Pi0FASTConfig in the default config have parameters like action_dim set:

openpi/src/openpi/training/config.py

Lines 429 to 430 in 29068dd

	name="pi0_fast_droid",
	model=pi0_fast.Pi0FASTConfig(action_dim=8, action_horizon=10),

openpi/src/openpi/training/config.py

Lines 476 to 477 in 29068dd

	name="pi0_fast_libero",
	model=pi0_fast.Pi0FASTConfig(action_dim=7, action_horizon=10, max_token_len=180),

I am a bit confused about this:

1. Why modify it instead of aligning the action space with padding like pi0?
2. Why wouldn't this cause a shape conflict when loading model weights?
3. Why are these parameters not set in the pi0 fast lora configuration?
   

   openpi/src/openpi/training/config.py

   Lines 489 to 490 in 29068dd

   	name="pi0_fast_libero_low_mem_finetune",
   	model=pi0_fast.Pi0FASTConfig(paligemma_variant="gemma_2b_lora"),

4. Why are these parameters set here, even though it seems like get_freeze_filter() does not use them?
   

   openpi/src/openpi/training/config.py

   Lines 500 to 502 in 29068dd

   	freeze_filter=pi0_fast.Pi0FASTConfig(
   	action_dim=7, action_horizon=10, max_token_len=180, paligemma_variant="gemma_2b_lora"
   	).get_freeze_filter(),

I don't fully understand how the Pi0FAST model works, thank you for your answer.

[mishmish66]

That appears to be a mistake as far as I can tell, good catch!

[mishmish66]

As for as why reshape rather than pad, that has to do with the specifics of how the FAST tokenizer works.

It's actually kind of agnostic to the number of output channels, the same output could be reshaped to arbitrary output dimension with FAST (though it likely wouldn't work well without finetuning)

The tokens generated by the VLA are expanded by BPE into a flat vector, which is reshaped into a DCT matrix of the required dimension and then that is transformed through the inverse DCT to an action chunk. This operation is kind of agnostic to channel counts by design (that's part of what makes it cool!) so to change output dimension it's easier to just change the reshape operation rather than pad, and since the base model is trained on a variety of action dimensions it should relatively easily adjust to this.