Testing Isaac-Lift-Cube-Franka-v0 task

[Kt981]

Hi,

I've trained Franka robot with the Isaac-Lift-Cube-Franka-v0 task for 8 hours and didn't get a good result- the robot turns around each time, regardless of the cube position. I used the last checkpoint for the test, but the output was: Loading checkpoint from: C:\IsaacLab\IsaacLab_git\logs\sb3\Isaac-Lift-Cube-Franka-v0\2025-01-14_14-31-33\model_9984000_steps.zip. The last model was created after 16384000 steps, what could be the problem?
This is the full output during playing:

[INFO][AppLauncher]: Loading experience file: C:\IsaacLab\IsaacLab_git\source\apps\isaaclab.python.rendering.kit
[Warning] [omni.isaac.kit.simulation_app] Modules: ['omni.kit_app'] were loaded before SimulationApp was started and might not be loaded correctly.
[Warning] [omni.isaac.kit.simulation_app] Please check to make sure no extra omniverse or pxr modules are imported before the call to SimulationApp(...)
Loading user config located at: 'c:/isaaclab/isaaclab_git/isaaclab/lib/site-packages/omni/data/Kit/Isaac-Sim/4.2/user.config.json'
[Info] [carb] Logging to file: c:/isaaclab/isaaclab_git/isaaclab/lib/site-packages/omni/logs/Kit/Isaac-Sim/4.2/kit_20250115_145546.log
2025-01-15 12:55:46 [0ms] [Warning] [omni.kit.app.plugin] No crash reporter present, dumps uploading isn't available.
2025-01-15 12:55:49 [3,487ms] [Warning] [gpu.foundation.plugin] Skipping unsupported non-NVIDIA GPU: Intel(R) UHD Graphics 770
2025-01-15 12:55:49 [3,514ms] [Warning] [gpu.foundation.plugin] Skipping unsupported non-NVIDIA GPU: Intel(R) UHD Graphics 770

|---------------------------------------------------------------------------------------------|
| Driver Version: 566.36 | Graphics API: D3D12
|=============================================================================================|
| GPU | Name | Active | LDA | GPU Memory | Vendor-ID | LUID |
| | | | | | Device-ID | UUID |
| | | | | | Bus-ID | |
|---------------------------------------------------------------------------------------------|
| 0 | NVIDIA GeForce RTX 4090 | Yes: 0 | | 24142 MB | 10de | 3f5b0100.. |
| | | | | | 2684 | 0 |
| | | | | | 1 | |
|---------------------------------------------------------------------------------------------|
| 1 | Intel(R) UHD Graphics 770 | | | 128 MB | 8086 | ef97eb40.. |
| | | | | | a780 | 0 |
| | | | | | N/A | |
|=============================================================================================|
| OS: Windows 11 Enterprise, Version: 10.0 (23H2), Build: 22631, Kernel: 10.0.22621.3733
| Processor: Intel(R) Core(TM) i7-14700K | Cores: 20 | Logical: 28
|---------------------------------------------------------------------------------------------|
| Total Memory (MB): 130813 | Free Memory: 79903
| Total Page/Swap (MB): 150269 | Free Page/Swap: 91941
|---------------------------------------------------------------------------------------------|
2025-01-15 12:55:52 [6,565ms] [Warning] [omni.replicator.core.scripts.annotators] Annotator PostProcessDispatch is already registered, overwriting annotator template
C:\IsaacLab\IsaacLab_git\isaaclab\lib\site-packages\h5py_init_.py:36: UserWarning: h5py is running against HDF5 1.14.4 when it was built against 1.14.2, this may cause problems
_warn(("h5py is running against HDF5 {0} when it was built against {1}, "
[INFO]: Parsing configuration from: omni.isaac.lab_tasks.manager_based.manipulation.lift.config.franka.joint_pos_env_cfg:FrankaCubeLiftEnvCfg
[INFO]: Parsing configuration from: C:\IsaacLab\IsaacLab_git\source\extensions\omni.isaac.lab_tasks\omni\isaac\lab_tasks\manager_based\manipulation\lift\config\franka\agents\sb3_ppo_cfg.yaml
2025-01-15 12:56:01 [15,503ms] [Warning] [omni.isaac.lab.envs.manager_based_env] Seed not set for the environment. The environment creation may not be deterministic.
[INFO]: Base environment:
Environment device : cuda:0
Environment seed : None
Physics step-size : 0.01
Rendering step-size : 0.02
Environment step-size : 0.02
[INFO]: Time taken for scene creation : 2.913668 seconds
[INFO]: Scene manager:
Number of environments: 128
Environment spacing : 2.5
Source prim name : /World/envs/env_0
Global prim paths : []
Replicate physics : True
[INFO]: Starting the simulation. This may take a few seconds. Please wait...
2025-01-15 12:56:07 [20,888ms] [Warning] [omni.hydra.scene_delegate.plugin] Calling getBypassRenderSkelMeshProcessing for prim /World/envs/env_0/Object/visuals.proto_visuals_id0 that has not been populated
[INFO]: Time taken for simulation start : 13.621719 seconds
[INFO] Command Manager: contains 1 active terms.
+------------------------------------------+
| Active Command Terms |
+-------+-------------+--------------------+
| Index | Name | Type |
+-------+-------------+--------------------+
| 0 | object_pose | UniformPoseCommand |
+-------+-------------+--------------------+

[INFO] Recorder Manager: contains 0 active terms.
+---------------------+
| Active Recorder Terms |
+-----------+---------+
| Index | Name |
+-----------+---------+
+-----------+---------+

[INFO] Action Manager: contains 2 active terms.
+------------------------------------+
| Active Action Terms (shape: 8) |
+-------+----------------+-----------+
| Index | Name | Dimension |
+-------+----------------+-----------+
| 0 | arm_action | 7 |
| 1 | gripper_action | 1 |
+-------+----------------+-----------+

[INFO] Observation Manager: contains 1 groups.
+-----------------------------------------------------------+
| Active Observation Terms in Group: 'policy' (shape: (36,)) |
+----------+-------------------------------------+----------+
| Index | Name | Shape |
+----------+-------------------------------------+----------+
| 0 | joint_pos | (9,) |
| 1 | joint_vel | (9,) |
| 2 | object_position | (3,) |
| 3 | target_object_position | (7,) |
| 4 | actions | (8,) |
+----------+-------------------------------------+----------+

[INFO] Event Manager: contains 1 active terms.
+--------------------------------------+
| Active Event Terms in Mode: 'reset' |
+---------+----------------------------+
| Index | Name |
+---------+----------------------------+
| 0 | reset_all |
| 1 | reset_object_position |
+---------+----------------------------+

[INFO] Termination Manager: contains 2 active terms.
+------------------------------------+
| Active Termination Terms |
+-------+-----------------+----------+
| Index | Name | Time Out |
+-------+-----------------+----------+
| 0 | time_out | True |
| 1 | object_dropping | False |
+-------+-----------------+----------+

[INFO] Reward Manager: contains 6 active terms.
+-----------------------------------------------------+
| Active Reward Terms |
+-------+-----------------------------------+---------+
| Index | Name | Weight |
+-------+-----------------------------------+---------+
| 0 | reaching_object | 1.0 |
| 1 | lifting_object | 15.0 |
| 2 | object_goal_tracking | 16.0 |
| 3 | object_goal_tracking_fine_grained | 5.0 |
| 4 | action_rate | -0.0001 |
| 5 | joint_vel | -0.0001 |
+-------+-----------------------------------+---------+

[INFO] Curriculum Manager: contains 2 active terms.
+--------------------------+
| Active Curriculum Terms |
+---------+----------------+
| Index | Name |
+---------+----------------+
| 0 | action_rate |
| 1 | joint_vel |
+---------+----------------+

Creating window for environment.
[INFO]: Completed setting up the environment...
Loading checkpoint from: C:\IsaacLab\IsaacLab_git\logs\sb3\Isaac-Lift-Cube-Franka-v0\2025-01-14_14-31-33\model_9984000_steps.zip
== CURRENT SYSTEM INFO ==

• OS: Windows-10-10.0.22631-SP0 10.0.22631
• Python: 3.10.11
• Stable-Baselines3: 2.4.0
• PyTorch: 2.4.0+cu121
• GPU Enabled: True
• Numpy: 1.26.0
• Cloudpickle: 3.1.0
• Gymnasium: 1.0.0
• OpenAI Gym: 0.23.1

== SAVED MODEL SYSTEM INFO ==

• OS: Windows-10-10.0.22631-SP0 10.0.22631
• Python: 3.10.11
• Stable-Baselines3: 2.4.0
• PyTorch: 2.4.0+cu121
• GPU Enabled: True
• Numpy: 1.26.0
• Cloudpickle: 3.1.0
• Gymnasium: 1.0.0
• OpenAI Gym: 0.23.1

C:\IsaacLab\IsaacLab_git\isaaclab\lib\site-packages\torch\storage.py:414: FutureWarning: You are using torch.load with weights_only=False (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for weights_only will be flipped to True. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=True for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
return torch.load(io.BytesIO(b))
Setting seed: 42
C:\IsaacLab\IsaacLab_git\isaaclab\lib\site-packages\stable_baselines3\common\on_policy_algorithm.py:150: UserWarning: You are trying to run PPO on the GPU, but it is primarily intended to run on the CPU when not using a CNN policy (you are using ActorCriticPolicy which should be a MlpPolicy). See DLR-RM/stable-baselines3#1245 for more info. You can pass device='cpu' or export CUDA_VISIBLE_DEVICES= to force using the CPU.Note: The model will train, but the GPU utilization will be poor and the training might take longer than on CPU.
warnings.warn(
2025-01-15 12:57:14 [87,878ms] [Warning] [omni.isaac.lab.sim.simulation_context] Simulation is stopped. The app will keep running with physics disabled. Press Ctrl+C or close the window to exit the app.
2025-01-15 12:57:15 [89,471ms] [Warning] [omni.usd] Unexpected reference count of 3 for UsdStage 'anon:0000016EEB1E2A70:World0.usd' while being closed in UsdContext (this may indicate it is still resident in memory).