NuPlan dataset for closed-loop testing using the world model

[happy-yan]

The documentation doesn't give any indication of how NuPlan is tested, so we'll assume it's by running the simulate_model_for_world_model_reactive_agents.sh file.

SAVE_DIR='/home/public/data/nuPlan-v1.1/data/test/0831simulation'
MODEL_PATH="/home/public/data/ckpt/epoch_llama_sm.ckpt"

EXPERIMENT="tmp"

export NUPLAN_MAPS_ROOT="/home/public/data/maps"
export NUPLAN_DATA_ROOT="/home/public/data/nuPlan-v1.1/splits/test/"
export NUPLAN_HYDRA_CONFIG_PATH=/root/workspace/GUMP/nuplan_extent/planning/script/config
export CUDA_VISIBLE_DEVICES=0

export PYTHONPATH=$PWD:$PYTHONPATH

python nuplan_extent/planning/script/run_simulation.py
experiment_name=$EXPERIMENT
group=$SAVE_DIR
+simulation='closed_loop_reactive_agents'
planner=ml_planner
planner.ml_planner.model_config='${model}'
planner.ml_planner.checkpoint_path=$MODEL_PATH
common_cfg=gump
common_cfg.input_cfg.input_channel_indexes=[0,1,2,3,4,5]
common_cfg.output_cfg.trajectory_steps=16
common_cfg.output_cfg.time_horizon=8.
scenario_builder=nuplan
scenario_filter=nuplan_challenge_scenarios
scenario_builder.data_root=$NUPLAN_DATA_ROOT
scenario_builder.scenario_mapping.subsample_ratio_override=0.5
model=policy_structured_plan_model_raster_naive
model.feature_builders.0.subsample_ratio_override=0.5
scenario_filter.timestamp_threshold_s=15
number_of_gpus_allocated_per_simulation=1.0
scenario_builder.map_root=$NUPLAN_MAPS_ROOT
scenario_filter.num_scenarios_per_type=1
~callback.simulation_nuboard_video_callback
simulation_history_buffer_duration=6.0
worker.threads_per_node=8
max_callback_workers=8
observation=world_model_agents_observation
callback.simulation_feature_video_callback.visualize_all_scenarios=True
scenario_filter.limit_total_scenarios=1 \

The result was File "/usr/local/lib/python3.9/dist-packages/hydra/_internal/defaults_list.py", line 773, in config_not_found_error
raise MissingConfigException(
hydra.errors.MissingConfigException: In 'horizon_simulation': Could not load 'callback/simulation_nuboard_video_callback'.

Config search path:
provider=hydra, path=pkg://hydra.conf
provider=main, path=file:///root/workspace/GUMP/nuplan_extent/planning/script/config
provider=hydra.searchpath in main, path=pkg://nuplan.planning.script.config.common
provider=hydra.searchpath in main, path=pkg://nuplan.planning.script.config.simulation
provider=hydra.searchpath in main, path=pkg://nuplan_extent.planning.script.config.common
provider=hydra.searchpath in main, path=pkg://nuplan_extent.planning.script.experiments
provider=hydra.searchpath in main, path=pkg://nuplan_extent.planning.script.config.simulation
provider=hydra.searchpath in main, path=file://root/workspace/GUMP/nuplan_extent/planning/simulation/callback
provider=schema, path=structured://

However，we make sure the file is under the corresponding path. Besides, we sincerely hope to see your Downstream Task: Reinforcement Learning

[Yihanhu]

Hi @happy-yan , thank you for your interest. The content related to this part hasn't been released yet. We are currently optimizing the entire pipeline in terms of both efficiency and performance. At the moment, I can't guarantee a specific time for the code release, as we are somewhat short on manpower.