[gemm3] Fix EOS token handling for Gemma text-only export #206
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
When exporting Gemma models with
--task "text-generation", the C++ text runner wasn't stopping at the<end_of_turn>token (ID 106), only at the<eos>token (ID 1). This caused generation to continue beyond expected stopping points.This diff modifies
save_config_to_constant_methods()to export multiple EOS token IDs viaget_eos_ids, which the C++ runner'skEosIdsmethod already supports.Changes
config.eos_token_idconfig.model_typecontaining "gemma" and automatically add token 106 (<end_of_turn>) to the EOS listget_eos_ids(full list) for C++ runner compatibility andget_eos_id(first element) for backward compatibilityCompatibility
modeling.pyalready checks for bothget_eos_idandget_eos_idsget_eos_ids()function already supports reading a list of EOS tokens via thekEosIdsmethodTest Plan
--task "text-generation"and verify metadata containsget_eos_ids: [1, 106]<end_of_turn>pytest tests/models/test_modeling_gemma3.py -v