From e0ef92860dac3395de2fe85b5d7029635e5a281c Mon Sep 17 00:00:00 2001 From: Deeptanshu Singh Date: Thu, 19 Feb 2026 12:42:31 -0500 Subject: [PATCH 1/2] Update README with token match rate on text backbone --- contrib/models/Ovis2.5-9B/README.md | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/contrib/models/Ovis2.5-9B/README.md b/contrib/models/Ovis2.5-9B/README.md index 45d9973..6238b96 100644 --- a/contrib/models/Ovis2.5-9B/README.md +++ b/contrib/models/Ovis2.5-9B/README.md @@ -12,11 +12,13 @@ NeuronX Distributed Inference implementation of Ovis2.5 9B. ## Architecture Details -- **Layers:** Check model config -- **Hidden Size:** Check model config -- **Attention Heads:** Check model config -- **Vocabulary:** Check model config -- **Max Position Embeddings:** Check model config +- **Type:** Multimodal (vision-language) model — text backbone validated only +- **Text Backbone:** Decoder-only transformer +- **Layers:** See model config +- **Hidden Size:** See model config +- **Attention Heads:** See model config +- **Vocabulary:** See model config +- **Max Position Embeddings:** See model config ## Validation Results @@ -28,7 +30,7 @@ NeuronX Distributed Inference implementation of Ovis2.5 9B. | Test | Status | Result | |------|--------|--------| | Smoke Test | ✅ PASS | Model loads successfully | -| Token Matching | ⚠️ N/A | **0.0% match** | +| Token Matching | ✅ PASS | **100% match** (text backbone) | | TTFT (P50) | ✅ PASS | 32.92ms (threshold: 100ms) | | Throughput | ✅ PASS | 30.03 tok/s (threshold: 10 tok/s) | @@ -39,9 +41,12 @@ NeuronX Distributed Inference implementation of Ovis2.5 9B. | TTFT (P50) | 32.92ms | | Throughput | 30.03 tokens/s | - **Status:** ✅ VALIDATED +### Multimodal Validation Notes + +Ovis2.5 is a vision-language model. The NeuronX port validates the text backbone only. `AutoModelForCausalLM` does not work for multimodal models — the specific text backbone class must be used to load the HF reference for token matching. With the correct text backbone extraction, the model achieves 100% token match. + ## Usage ```python From c030fd65ae059bf2bc043108d896e5dc805533c3 Mon Sep 17 00:00:00 2001 From: Deeptanshu Singh Date: Thu, 26 Feb 2026 13:35:39 -0500 Subject: [PATCH 2/2] Removing internal names --- contrib/models/Ovis2.5-9B/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contrib/models/Ovis2.5-9B/README.md b/contrib/models/Ovis2.5-9B/README.md index 6238b96..cbad6e6 100644 --- a/contrib/models/Ovis2.5-9B/README.md +++ b/contrib/models/Ovis2.5-9B/README.md @@ -111,6 +111,6 @@ python3 test/integration/test_model.py ## Maintainer -Neuroboros Team - Annapurna Labs +Annapurna Labs **Last Updated:** 2026-01-29