As a Nabledge developer, I want to investigate AI model selection capabilities so that users can choose between premium and standard models

[kiyotis]

Situation

User request nablarch/nabledge#3 asks for the ability to select AI models when using Nabledge. Currently, users cannot control which model (premium/standard) is used when invoking Nabledge skills, leading to cost management challenges with premium request limits.

Pain

Users struggle with premium request limits and cannot optimize their usage by selecting appropriate models for different tasks. Development team lacks clarity on whether GitHub Copilot supports model selection for skills and how to implement this feature if possible.

Benefit

• Users can optimize costs by choosing standard models for simple tasks
• Users can reserve premium models for complex implementation and design consultations
• Development team can provide clear guidance on model selection capabilities

Success Criteria

• Investigate whether GitHub Copilot supports model selection for skills
• If supported, document implementation approach in .lw/research/
• If not supported, document limitations and potential workarounds
• Update Nabledgeで使用するAIモデルを選択できるようにして欲しい nabledge#3 with findings and feasibility assessment

[kiyotis]

Investigation Results: AI Model Selection

1. Specifying LLM Model in Chat

Claude Code ✅ Fully Supported

Multiple methods available (priority order):

• Interactive command: /model <alias|name> (e.g., /model opus, /model sonnet)
• Startup flag: claude --model <alias|name>
• Environment variable: ANTHROPIC_MODEL=<alias|name>
• Settings file: Add "model": "opus" to .claude/settings.json

Available models: opus, sonnet, haiku, opusplan, sonnet[1m], and more

Documentation: https://code.claude.com/docs/en/model-config.md

GitHub Copilot ✅ Supported

• Chat panel dropdown: Select model from "CURRENT-MODEL" dropdown at bottom of chat interface (available in VS Code, JetBrains, Visual Studio, browser, etc.)
• Available models: Multiple models including Claude Sonnet 4.5, Claude Opus 4.5, Claude Opus 4.6, GPT models, and "Auto" option
• Organization policy: Copilot Business users need organization permission to switch models

Documentation: https://docs.github.com/en/copilot/how-tos/use-ai-models/change-the-chat-model

---

2. Specifying LLM Model for Skills/Agents

Claude Code ⚠️ Partial Support

Default behavior: Skills do NOT have independent model settings by default. They inherit from execution context.

Model inheritance depends on execution mode:

1. Inline execution (default):

   • Skill runs in main conversation
   • Uses whatever model is selected in main chat

2. Fork execution (context: fork):

   • Skill runs in subagent
   • Uses subagent's model configuration (default: inherit from main conversation)
   • Subagents can specify model: sonnet/opus/haiku/inherit in frontmatter

Skill can request a model:

---
name: my-skill
model: sonnet  # Requests Sonnet, but not guaranteed
---

⚠️ This is a request, not a guarantee. Actual model depends on conversation/subagent settings.

Workaround for guaranteed model control:

• Use fork execution with custom subagent
• Set subagent model via CLAUDE_CODE_SUBAGENT_MODEL environment variable
• Configure subagent's frontmatter with explicit model

Documentation:

• Skills: https://code.claude.com/docs/en/skills.md
• Subagents: https://code.claude.com/docs/en/sub-agents.md

GitHub Copilot ❓ Unknown

Coding Agent: Model selection available when assigning issues or starting tasks (default: "Auto")

Custom Extensions/Agents: No official documentation found regarding model selection for custom extensions. It's unclear whether:

• Extensions inherit user's selected chat model
• Extensions can specify their own model preference
• Extensions use a fixed default model

Documentation: https://docs.github.com/en/copilot/how-tos/use-copilot-agents/coding-agent/changing-the-ai-model

---

3. Other Approaches

Claude Code

1. Effort level control (Opus 4.6 only):

   • /model interface with slider
   • CLAUDE_CODE_EFFORT_LEVEL=low|medium|high
   • Settings: "effortLevel": "low|medium|high"
   • Controls adaptive reasoning depth

2. Model alias configuration:

   • ANTHROPIC_DEFAULT_OPUS_MODEL
   • ANTHROPIC_DEFAULT_SONNET_MODEL
   • ANTHROPIC_DEFAULT_HAIKU_MODEL

3. Enterprise restrictions:

   • Admins can limit available models via availableModels in managed settings

GitHub Copilot

1. Auto mode: Automatically selects best model based on availability and rate limiting
2. Organization policies: Enterprise admins control model access permissions
3. Premium request management: Different models consume monthly allowances differently

---

4. Recommended Approach for Nabledge

Analysis

Based on investigation results:

Claude Code: Skills can request a model via frontmatter, but cannot guarantee which model runs them in default (inline) mode. Users control the actual model through main conversation settings.

GitHub Copilot: Custom extension model control is not documented. Behavior unclear.

Proposed Solutions

Option A: Document Model Request Pattern (Minimal Change)

Add model frontmatter to nabledge-6 skill with recommended model:

---
name: nabledge-6
model: sonnet  # Request standard model for cost optimization
---

Pros: Simple, no breaking changes
Cons: Not guaranteed, user's main conversation setting takes precedence

Option B: Provide Fork Mode Variant (Advanced Control)

Create alternative workflow that uses fork execution with explicit model:

---
name: nabledge-6-standard
context: fork
agent: custom-nabledge-agent  # Define agent with model: sonnet
---

Pros: Guaranteed model control
Cons: More complex setup, requires custom subagent definition

Option C: User Documentation Only

Document that users should:

• Set main conversation model before invoking nabledge skills
• Use /model sonnet for standard tasks, /model opus for complex tasks
• Provide clear guidance in README on when to use which model

Pros: No code changes, works for both Claude Code and GitHub Copilot
Cons: Relies on user discipline

Recommendation

Implement Option C + Option A:

1. Add clear user documentation on model selection best practices
2. Add model: sonnet to skill frontmatter as a hint (Option A)
3. Document in Nabledgeで使用するAIモデルを選択できるようにして欲しい nabledge#3 that skill-level model control is limited by platform capabilities
4. Consider Option B (fork mode) as future enhancement if users need guaranteed model control

This approach provides immediate value through documentation while respecting platform limitations.