VLR Vision Language Model | Large Scale Training

FINETUNING

• Go to finetune.sh and change the model type
• Using finevision (local path already in)
• Just run ./finetune.sh

The code is optimized for the Marenostrum 5 HPC system, with H100s.

Features

• distributed checkpoints
• compile
• deterministic
• better args + config
• data parallel
• FSDP
• compile + checkpoints
• static shape compile
• FSDP multinode
• data packing

Models Supported

• Qwen2.5-VL series
• Qwen3-VL series

DISCLAIMER

This code was originally the Qwen3-VL codebase developed by Qwen team, Alibaba Cloud. We didnt change the license.