Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[viz] Fixes & updates to html visualizer #617

Merged
merged 8 commits into from
Jan 9, 2025
Merged

[viz] Fixes & updates to html visualizer #617

merged 8 commits into from
Jan 9, 2025
+56 −46

Conversation

mishig25
Copy link
Collaborator

@mishig25 mishig25 commented Jan 7, 2025 •
edited
Loading

As Remi noted, viz was failing for yaak-ai/lerobot-driving-school.

The reason was: previously viz tool assumed float32 features to have exact names observation.state & action. However, yaak-ai/lerobot-driving-school had a field action.continuous.

This PR fixes this issue by removing hardcoded names observation.state & action in favour for dynamically selecting all float32 features.

image

@mishig25 mishig25 marked this pull request as ready for review January 7, 2025 14:57
@mishig25
Copy link
Collaborator Author

mishig25 commented Jan 7, 2025

I dont think the failing test is related to this PR

@mishig25
Copy link
Collaborator Author

mishig25 commented Jan 7, 2025

cc: @Cadene @aliberts

@mishig25
Copy link
Collaborator Author

mishig25 commented Jan 7, 2025 •
edited
Loading

pushed 094e94c that fixes an issue when motor names are not explicitly provided in the data

for cadene/agibotworld_task_327

image

@mishig25 mishig25 changed the title [viz] Generalize float32 columns [viz] Fixes & updates to html visualizer Jan 7, 2025
@mishig25
Copy link
Collaborator Author

mishig25 commented Jan 8, 2025

pushed dcd821b that makes it possible to filter videos

Screen.Recording.2025-01-08.at.11.43.21.mov

@Cadene
Copy link
Collaborator

Cadene commented Jan 8, 2025

@mishig25 We just merged a fix for the poetry check not passing, could you rebase?

@mishig25
Copy link
Collaborator Author

mishig25 commented Jan 8, 2025

done so, and the checks are now passing

@aliberts
Copy link
Collaborator

aliberts commented Jan 9, 2025 •
edited
Loading

Thank you @mishig25!
I'm trying to run this on pusht using

python lerobot/scripts/visualize_dataset_html.py \
    --repo-id lerobot/pusht

But I get this:
image

(it works on cadene/agibotworld_task_327)

@mishig25
fix shape issue
Loading
Loading status checks…
29994ba
@mishig25
Copy link
Collaborator Author

mishig25 commented Jan 9, 2025

@aliberts fix 29994ba is pushed

python lerobot/scripts/visualize_dataset_html.py --repo-id lerobot/pusht should work now

image

aliberts
aliberts approved these changes Jan 9, 2025
View reviewed changes
Copy link
Collaborator

@aliberts aliberts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for that feature, that's quite useful :)

@aliberts aliberts merged commit 25a8597 into huggingface:main Jan 9, 2025
5 checks passed
@mishig25 mishig25 deleted the viz_f32_features branch January 9, 2025 11:05
villekuosmanen added a commit to villekuosmanen/lerobot that referenced this pull request Jan 10, 2025
@villekuosmanen @HiroIshida
@alexander-soare @ohharsen @ivelin @brainwavecoder9 @resolver101757 @J4nn1K @KasparSLT @aliberts @Cadene @michel-aractingi @ma3oun @Raziel90 @s1lent4gnt @mishig25 @helper2424 @CharlesCNorton @philfung
feat: sync upstream changes (#4)
9e38286 
* feat: enable to use multiple rgb encoders per camera in diffusion policy (huggingface#484)

Co-authored-by: Alexander Soare <[email protected]>

* Fix config file (huggingface#495)

* fix: broken images and a few minor typos in README (huggingface#499)

Signed-off-by: ivelin <[email protected]>

* Add support for Windows (huggingface#494)

* bug causes error uploading to huggingface, unicode issue on windows. (huggingface#450)

* Add distinction between two unallowed cases in name check "eval_" (huggingface#489)

* Rename deprecated argument (temporal_ensemble_momentum) (huggingface#490)

* Dataset v2.0 (huggingface#461)

Co-authored-by: Remi <[email protected]>

* Refactor OpenX (huggingface#505)

* Fix missing local_files_only in record/replay (huggingface#540)

Co-authored-by: Simon Alibert <[email protected]>

* Control simulated robot with real leader (huggingface#514)

Co-authored-by: Remi <[email protected]>

* Update 7_get_started_with_real_robot.md (huggingface#559)

* LerobotDataset pushable to HF from any folder (huggingface#563)

* Fix example 6 (huggingface#572)

* fixing typo from 'teloperation' to 'teleoperation' (huggingface#566)

* [vizualizer] for LeRobodDataset V2 (huggingface#576)

* Fix broken `create_lerobot_dataset_card`  (huggingface#590)

* Update README.md (huggingface#612)

* Fix Quality workflow (huggingface#622)

* fix(docs): typos in benchmark readme.md (huggingface#614)

Co-authored-by: Simon Alibert <[email protected]>

* fix(visualise): use correct language description for each episode id (huggingface#604)

Co-authored-by: Simon Alibert <[email protected]>

* typo fix: batch_convert_dataset_v1_to_v2.py (huggingface#615)

Co-authored-by: Simon Alibert <[email protected]>

* [viz] Fixes & updates to html visualizer (huggingface#617)

* fixes to SO-100 readme (huggingface#600)

Co-authored-by: Philip Fung <no@one>
Co-authored-by: Simon Alibert <[email protected]>

---------

Signed-off-by: ivelin <[email protected]>
Co-authored-by: Hirokazu Ishida <[email protected]>
Co-authored-by: Alexander Soare <[email protected]>
Co-authored-by: Arsen Ohanyan <[email protected]>
Co-authored-by: Ivelin Ivanov <[email protected]>
Co-authored-by: Daniel Ritchie <[email protected]>
Co-authored-by: resolver101757 <[email protected]>
Co-authored-by: Jannik Grothusen <[email protected]>
Co-authored-by: KasparSLT <[email protected]>
Co-authored-by: Simon Alibert <[email protected]>
Co-authored-by: Remi <[email protected]>
Co-authored-by: Michel Aractingi <[email protected]>
Co-authored-by: Simon Alibert <[email protected]>
Co-authored-by: berjaoui <[email protected]>
Co-authored-by: Claudio Coppola <[email protected]>
Co-authored-by: s1lent4gnt <[email protected]>
Co-authored-by: Mishig <[email protected]>
Co-authored-by: Eugene Mironov <[email protected]>
Co-authored-by: CharlesCNorton <[email protected]>
Co-authored-by: Philip Fung <[email protected]>
Co-authored-by: Philip Fung <no@one>
@mishig25 mishig25 mentioned this pull request Jan 11, 2025
Open
chrisheninger pushed a commit to chrisheninger/lerobot that referenced this pull request Jan 26, 2025
@mishig25 @chrisheninger
[viz] Fixes & updates to html visualizer (huggingface#617)
dd4d83f
michel-aractingi pushed a commit that referenced this pull request Feb 3, 2025
@mishig25 @michel-aractingi
[viz] Fixes & updates to html visualizer (#617)
100f54e
menhguin pushed a commit to menhguin/lerobot that referenced this pull request Feb 9, 2025
@mishig25
[viz] Fixes & updates to html visualizer (huggingface#617)
be72c19
aliberts pushed a commit that referenced this pull request Feb 9, 2025
@Cadene @aliberts
Fix autocalib moss (#486)
5ceb342 
[Fix] Move back to manual calibration (#488)

feat: enable to use multiple rgb encoders per camera in diffusion policy (#484)

Co-authored-by: Alexander Soare <[email protected]>

Fix config file (#495)

fix: broken images and a few minor typos in README (#499)

Signed-off-by: ivelin <[email protected]>

Add support for Windows (#494)

bug causes error uploading to huggingface, unicode issue on windows. (#450)

Add distinction between two unallowed cases in name check "eval_" (#489)

WIP

Fix autocalib moss (#486)

[Fix] Move back to manual calibration (#488)

feat: enable to use multiple rgb encoders per camera in diffusion policy (#484)

Co-authored-by: Alexander Soare <[email protected]>

Fix config file (#495)

fix: broken images and a few minor typos in README (#499)

Signed-off-by: ivelin <[email protected]>

Add support for Windows (#494)

bug causes error uploading to huggingface, unicode issue on windows. (#450)

Add distinction between two unallowed cases in name check "eval_" (#489)

Rename deprecated argument (temporal_ensemble_momentum) (#490)

Dataset v2.0 (#461)

Co-authored-by: Remi <[email protected]>

Refactor OpenX (#505)

Fix missing local_files_only in record/replay (#540)

Co-authored-by: Simon Alibert <[email protected]>

Control simulated robot with real leader (#514)

Co-authored-by: Remi <[email protected]>

Update 7_get_started_with_real_robot.md (#559)

LerobotDataset pushable to HF from any folder (#563)

Fix example 6 (#572)

fixing typo from 'teloperation' to 'teleoperation' (#566)

[vizualizer] for LeRobodDataset V2 (#576)

Fix broken `create_lerobot_dataset_card`  (#590)

Update README.md (#612)

Add draccus, create MainConfig

WIP refactor train.py and ACT

Add policies training presets

Update diffusion policy

Add pusht and xarm env configs

Update tdmpc

Update vqbet

Fix poetry relax

Add feature types to envs

Add EvalPipelineConfig, parse features from envs

Add custom parser

Update pretrained loading mechanisms

Add dependency fixes & lock update

Fix pretrained_path

Refactor envs, remove RealEnv

Fix typo

Enable end-to-end tests

Fix Makefile

Log eval config

Fix end-to-end tests

Fix Quality workflow (#622)

Remove amp & add resume test

Speed-up tests

Fix poetry relax

Remove config yaml for robot devices (#594)

Co-authored-by: Simon Alibert <[email protected]>

fix(docs): typos in benchmark readme.md (#614)

Co-authored-by: Simon Alibert <[email protected]>

fix(visualise): use correct language description for each episode id (#604)

Co-authored-by: Simon Alibert <[email protected]>

typo fix: batch_convert_dataset_v1_to_v2.py (#615)

Co-authored-by: Simon Alibert <[email protected]>

[viz] Fixes & updates to html visualizer (#617)

Fix logger

Remove hydra-core

Add aggregate_stats

Add estimate_num_samples for images, Add test image

Remove NoneSchedulerConfig

Add push_pretrained

Remove eval.episode_length

Fix wandb_video

Fix typo

Add features back into policy configs (#643)

fixes to SO-100 readme (#600)

Co-authored-by: Philip Fung <no@one>
Co-authored-by: Simon Alibert <[email protected]>

Fix for the issue #638 (#639)

Fix env_to_policy_features call

Fix wandb init

remove omegaconf

Add branch arg

Move deprecated

Move training config

Remove pathable_args

Implement custom HubMixin

Fixes

Implement PreTrainedPolicy base class

Add HubMixin to TrainPipelineConfig

Udpate example 2 & 3

Update push_pretrained

Bump`rerun-sdk` dependency to `0.21.0` (#618)

Co-authored-by: Simon Alibert <[email protected]>

Fix config_class

Fix from_pretrained kwargs

Remove policy_protocol

Camelize PretrainedConfig

Additional fix while retraining policies (#629)

Co-authored-by: Simon Alibert <[email protected]>

Actually reactivate tdmpc online test

Update example 4

Remove advanced example 1

Remove example 5

Move example 6 to advanced

Use HubMixin.save_pretrained

Enable config_path to be a repo_id

Dry has_method

Update example 4

Update README

Cleanup pyproject.toml

Update eval docstring

Update README

Clean example 4

Update README

Make 'last' checkpoint symlink relative

Fix cluster image (#653)

Simplify example 4

fix stats per episodes and aggregate stats and casting to tensor
JIy3AHKO pushed a commit to vertix/lerobot that referenced this pull request Feb 27, 2025
@mishig25 @JIy3AHKO
[viz] Fixes & updates to html visualizer (huggingface#617)
a73e13a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aliberts aliberts

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@mishig25 @Cadene @aliberts