Skip to content

# Bug: Local Project Resources Not Prioritized - Agents and Workflows Fail #80

New issue
New issue
Open
Open
# Bug: Local Project Resources Not Prioritized - Agents and Workflows Fail#80
@Gorby200

Description

@Gorby200
Gorby200
opened on Feb 15, 2026

Summary

Local project configurations (agents and workflows) are not accessible or prioritized, making it impossible to create custom CodeMachine-based projects with project-specific agents and workflows. This violates the fundamental "local over global" principle that developer tools should follow.

Environment: Latest CodeMachine CLI v0.8.0 from repository (Commit f8bdb47), all platforms, reproducible 100%

Bug #1: Initial Attempt to Run Local Workflow (First Blocker)

What happened: Created a new CodeMachine-based project and added .codemachine/workflows/my-workflow.workflow.js with custom agents defined in .codemachine/config/main.agents.js. When running cm, the system loaded the wrong workflow entirely - it loaded the global ali.workflow.js from the CodeMachine installation instead of the local project workflow.

Expected: Load .codemachine/workflows/my-workflow.workflow.js

Actual: Loaded ~/.codemachine/resources/0.8.0/templates/workflows/ali.workflow.js

Why this matters: Without the ability to use local workflows, every CodeMachine project is forced to use the same global workflow. This makes it impossible to:

  • Create project-specific workflows
  • Test different workflow configurations
  • Customize workflows for specific use cases
  • Maintain workflow configurations in version control with the project

Root cause: getTemplatePathFromTracking() in src/shared/workflows/template.ts checked the global templates directory before checking the local .codemachine/workflows/ directory. This violated the principle that local project resources should always take precedence over global defaults.

Fix: Reversed the search order to check .codemachine/workflows/ first, then global templates, then imported packages.

Bug #2: Workflow Files Embedded in Binary (Discovered During Debugging Bug #1)

What happened: While investigating why the wrong workflow was loading, discovered that global workflow files were being automatically imported when the CodeMachine runtime module loaded. This happened because scripts/build.ts collected ALL files from the templates/ directory, including .workflow.js files, and embedded them into the compiled binary.

Evidence from generated file (src/shared/runtime/resource-manifest.ts):

// Auto-generated during build
import "../../../templates/workflows/ali.workflow.js" with { type: "file" };
import "../../../templates/workflows/_example.workflow.js" with { type: "file" };

Why this matters:

  1. Timing issues: These imports execute immediately when the runtime module loads, before any local project configuration is registered
  2. Wrong workflow priority: Global workflows embedded in binary are always found before checking local .codemachine/workflows/
  3. Bloat: Embeds workflow files in binary even though they should be loaded dynamically at runtime
  4. Maintenance burden: Changing global workflows requires rebuilding the entire binary instead of just editing a file

Root cause: scripts/build.ts collected all files without filtering:

const resourceFiles = [
  ...collectFiles('templates'), // ❌ Includes .workflow.js files!
];

This violated the separation between code (which should be embedded) and configuration (which should be loaded dynamically).

Fix: Added filter to exclude workflow files from embedding:

const resourceFiles = [
  ...collectFiles('templates').filter(f => !f.endsWith('.workflow.js')), // ✅
];

Bug #3: Agents Found During Loading, Missing During Execution (Third Blocker)

After fixing Bugs #1 and #2, the correct local workflow finally loaded successfully, but immediately failed when trying to execute the first agent step.

What happened: Workflow template loaded with all agents visible:

[resolveStep] Available agents: 11 total (9 local + 2 global) ✅
[resolveStep] Agent found: my-custom-agent ✅

But execution failed immediately:

[loadAgentConfig] Found 2 agents from collectAgentDefinitions
[loadAgentConfig] Available agents: cm-workflow-builder, cm-workflow-builder-quick
Error: Unknown agent id: my-custom-agent

Evidence from logs showing the inconsistency:

2026-02-14T22:23:21.690Z [resolveStep] Agent found: spec-reader
2026-02-14T22:23:21.902Z [loadAgentConfig] Looking for agent: spec-reader
2026-02-14T22:23:21.905Z [AgentCatalog] getImportedMainAgents function exists: false
2026-02-14T22:23:21.915Z [loadAgentConfig] Found 2 agents from collectAgentDefinitions
2026-02-14T22:23:21.915Z [loadAgentConfig] Agent NOT found: spec-reader

Why this matters:

  • Complete breakdown: Workflow can be defined but not executed
  • Data inconsistency: Two different parts of the system see completely different agent lists
  • Impossible to debug: The error message shows only 2 agents exist, but logs show 11 were registered
  • Fundamental architecture flaw: CodeMachine had two separate agent lookup systems that were never synchronized

Root cause: CodeMachine has two independent agent lookup mechanisms that operate on completely different data sources:

System 1: Workflow Template Resolution

  • Location: src/workflows/utils/config.ts
  • Storage: In-memory _importedMainAgents array
  • Used by: resolveStep() during workflow template loading
  • What it sees: All registered agents (local + global)
  • Code:
// src/workflows/utils/config.ts
let _importedMainAgents = []; // Local storage only

export function getAllMainAgents() {
  return [..._importedMainAgents, ...getMainAgents()];
}

System 2: Agent Execution

  • Location: src/shared/agents/discovery/catalog.ts
  • Storage: None! Directly scans filesystem
  • Used by: loadAgentConfig() during agent execution
  • What it sees: Only global agents from filesystem
  • Code:
// src/shared/agents/discovery/catalog.ts
export async function collectAgentDefinitions(projectRoot) {
  const byId = new Map();

  // ❌ NO imported agents here!
  // Only scans filesystem:
  // - projectRoot/config/main.agents.js
  // - global config/main.agents.js
  // - imported packages
  // But _importedMainAgents from System 1 is never checked!
}

The critical flaw: These two systems are never synchronized. When local agents are registered in System 1, System 2 has no way to access them. They're completely isolated.

Fix: Created unified agent registry (src/shared/agents/imported-registry.ts) that both systems now use, ensuring they always see the same agent list.

Root Cause Analysis

Bug #1: Local Resources Not Prioritized (Wrong Search Order)

Location:

  • src/shared/workflows/template.ts - Workflow path resolution
  • src/workflows/run.ts - Agent registration
  • src/workflows/preflight.ts - Agent registration

Problem 1: Workflow resolution checked global before local

The function getTemplatePathFromTracking(cmRoot) was supposed to find workflows for a project, but it searched in the wrong order:

// BEFORE (BUGGY):
export async function getTemplatePathFromTracking(cmRoot: string) {
  const activeTemplate = await getActiveTemplate(cmRoot);

  // ❌ WRONG: Checked GLOBAL templates FIRST
  const localPath = path.join(templatesDir, activeTemplate); // Global directory!
  if (existsSync(localPath)) {
    return localPath; // Returns global workflow, ignoring local!
  }

  // Local .codemachine/workflows/ checked LAST or never checked at all
  // This means global workflows always win
}

Consequences:

  • Local workflow in .codemachine/workflows/my-workflow.workflow.js completely ignored if global templates/workflows/my-workflow.workflow.js exists
  • Custom project workflows always lost to global CodeMachine workflows
  • Impossible to override global workflows with project-specific versions
  • Projects cannot maintain their own workflow configurations

Problem 2: Local agent configs never registered

The workflow runner only registered imported package agents, completely ignoring local project agent configurations:

// BEFORE (BUGGY):
export async function runWorkflow(options: RunWorkflowOptions = {}) {
  clearImportedAgents();

  // ❌ WRONG: Only registered imported packages, NOT local config!
  const importedPackages = getAllInstalledImports();
  for (const imp of importedPackages) {
    registerImportedAgents(imp.resolvedPaths.config);
  }

  // Local .codemachine/config/main.agents.js never registered!
  // Local agents completely ignored!
}

Consequences:

  • Local agents in .codemachine/config/main.agents.js never loaded
  • Only global CodeMachine agents and imported package agents available
  • Project-specific agent configurations completely ignored
  • Every project forced to use the same global agent definitions

Fix applied:

// AFTER (FIXED):

// Workflow resolution:
export async function getTemplatePathFromTracking(cmRoot: string) {
  const activeTemplate = await getActiveTemplate(cmRoot);

  // ✅ CORRECT: Check LOCAL project workflows FIRST (highest priority)
  const localWorkflowsPath = path.join(cmRoot, 'workflows', activeTemplate);
  if (existsSync(localWorkflowsPath)) {
    return localWorkflowsPath; // Local wins!
  }

  // Then check global templates directory
  const globalPath = path.join(templatesDir, activeTemplate);
  if (existsSync(globalPath)) {
    return globalPath;
  }

  // Finally check imported packages
  // ...
}

// Agent registration:
export async function runWorkflow(options: RunWorkflowOptions = {}) {
  clearImportedAgents();

  // ✅ CORRECT: Register LOCAL project agents FIRST (highest priority)
  const localConfigDir = path.join(cwd, '.codemachine', 'config');
  if (existsSync(localConfigDir)) {
    registerImportedAgents(localConfigDir);
    debug('[Workflow] Registered local agents from %s', localConfigDir);
  }

  // Then register imported package agents
  const importedPackages = getAllInstalledImports();
  for (const imp of importedPackages) {
    registerImportedAgents(imp.resolvedPaths.config);
  }
}

Priority order now enforced:

  1. Local project - .codemachine/config/main.agents.js (highest priority)
  2. Imported packages - from installed CodeMachine extensions
  3. Global CLI - CodeMachine's built-in agents (lowest priority)

This ensures projects can override any global configuration, which is fundamental for developer tools.

Bug #2: Workflow Files in Binary (Build-time Issue)

Location: scripts/build.ts

The problem:
The build script collected ALL files from the templates/ directory and embedded them into the compiled binary. This included .workflow.js files which are configuration that should be loaded dynamically at runtime.

// BEFORE (BUGGY):
const resourceFiles = [
  ...collectFiles('config'),
  ...collectFiles('prompts'),
  ...collectFiles('templates'), // ❌ Includes .workflow.js files!
  join(repoRoot, 'package.json'),
];

This generated src/shared/runtime/resource-manifest.ts with automatic imports:

// Auto-generated by build.ts
import "../../../templates/workflows/ali.workflow.js" with { type: "file" };
import "../../../templates/workflows/_example.workflow.js" with { type: "file" };

Why this caused problems:

  1. Timing issues: When runtime module is imported, these workflow files are immediately loaded as part of module initialization. This happens BEFORE local project agents are registered, causing resolveStep() to execute prematurely.

  2. Wrong workflow priority: Since global workflows are embedded in binary and imported automatically, they're always found before checking local .codemachine/workflows/. This reinforced Bug Check if the AI engine exists before running the command. #1.

  3. Separation of concerns violated: Workflow files are configuration, not code. They should be loaded dynamically at runtime, not embedded in compiled binary. Code should be embedded, configuration should not.

  4. Maintenance burden: Changing or debugging global workflows requires:

    • Rebuilding entire binary
    • Reinstalling CodeMachine globally
    • Instead of just editing a text file

Evidence from logs:
When the runtime module loaded, global workflows were imported automatically, visible in logs:

2026-02-14T22:09:43.093Z [loadWorkflowModule] Loading: C:\Users\user\.codemachine\resources\0.8.0\templates\workflows\ali.workflow.js
2026-02-14T22:09:43.163Z [TemplateLoader] Template loaded successfully: .../ali.workflow.js

This happened before the local project workflow was even looked for.

Fix applied:

// AFTER (FIXED):
const resourceFiles = [
  ...collectFiles('config'),
  ...collectFiles('prompts'),
  ...collectFiles('templates').filter(f => !f.endsWith('.workflow.js')), // ✅ Exclude
  join(repoRoot, 'package.json'),
];

Why this works:

  • Workflow files now loaded dynamically at runtime from filesystem
  • Local project workflows (.codemachine/workflows/) checked before global templates
  • Prevents premature resolveStep() execution during module import
  • Respects separation between code (embedded) and configuration (dynamic)
  • Global workflows can now be edited by users without rebuilding binary

Bug #3: Dual Agent Registration Systems (Data Inconsistency)

Location:

  • src/workflows/utils/config.ts - System 1
  • src/shared/agents/discovery/catalog.ts - System 2

The architectural flaw:

CodeMachine evolved to have two separate agent lookup mechanisms that were never designed to work together:

System 1: Workflow Template Resolution

  • Purpose: Find agents when loading workflow templates
  • When: During resolveStep() calls in workflow files
  • Storage: In-memory _importedMainAgents array
  • Data sources:
    • Local project .codemachine/config/main.agents.js (if registered)
    • Imported packages
    • Global CLI config/main.agents.js
  • What it sees: All registered agents (9 local + 2 global = 11 total)
  • Used by: Workflow template loading, resolveStep()

Code location (src/workflows/utils/config.ts):

// Storage for imported agents (merged from external packages)
let _importedMainAgents: AgentConfig[] = [];

export function getAllMainAgents(): AgentConfig[] {
  const imported = _importedMainAgents;
  const main = getMainAgents();
  return [...imported, ...main]; // Merges imported + global
}

// Registration function
export function registerImportedAgents(configPath: string): void {
  const mainAgentsPath = path.join(configPath, 'main.agents.js');
  const importedMain = require(mainAgentsPath) as AgentConfig[];
  _importedMainAgents = [...importedMain, ..._importedMainAgents];
  // ...
}

System 2: Agent Execution

  • Purpose: Find agents when executing them during workflow runs
  • When: During loadAgentConfig() calls in agent runner
  • Storage: None! Direct filesystem scanning
  • Data sources:
    • Scans filesystem for config/main.agents.js in various locations
    • NO ACCESS to _importedMainAgents from System 1
  • What it sees: Only agents from filesystem (2 global agents)
  • Used by: Agent execution, loadAgentConfig()

Code location (src/shared/agents/discovery/catalog.ts):

export async function collectAgentDefinitions(projectRoot: string): Promise<AgentDefinition[]> {
  const byId = new Map<string, AgentDefinition>();

  // ❌ NO imported agents here!
  // System 1's _importedMainAgents is never checked
  // Only direct filesystem scanning:

  const roots = [
    projectRoot,                          // Checks projectRoot/config/main.agents.js
    ...CLI_ROOT_CANDIDATES,                // Checks global config/main.agents.js
    ...importedRoots,                      // Checks imported packages
  ];

  for (const root of roots) {
    const moduleCandidate = path.join(root, 'config', 'main.agents.js');
    if (existsSync(moduleCandidate)) {
      const agents = loadAgentsFromModule(moduleCandidate);
      // Add to byId
    }
  }

  return Array.from(byId.values());
}

The critical disconnect:

When a local project has agents in .codemachine/config/main.agents.js:

  1. Workflow starts → run.ts calls registerImportedAgents('.codemachine/config')
  2. System 1 (_importedMainAgents) now has local agents ✅
  3. Workflow file loads → resolveStep('my-agent') called
  4. resolveStep() calls getAllMainAgents() → finds 11 agents ✅
  5. Workflow attempts to execute → loadAgentConfig('my-agent') called
  6. loadAgentConfig() calls collectAgentDefinitions(projectRoot)
  7. System 2 scans filesystem → finds only 2 global agents ❌
  8. Error: "Unknown agent id: my-agent" ❌

Evidence from debug logs:

# During template loading (System 1):
2026-02-14T22:23:21.690Z [resolveStep] Agent found: spec-reader
2026-02-14T22:23:21.689Z [Config] getAllMainAgents: 9 imported + 2 main = 11 total

# During execution (System 2):
2026-02-14T22:23:21.902Z [loadAgentConfig] Looking for agent: spec-reader
2026-02-14T22:23:21.905Z [AgentCatalog] getImportedMainAgents function exists: false
2026-02-14T22:23:21.915Z [loadAgentConfig] Found 2 agents from collectAgentDefinitions
2026-02-14T22:23:21.915Z [loadAgentConfig] Agent NOT found: spec-reader

Why this happened historically:

  • System 1 was added first for workflow templates
  • System 2 was added later for agent execution
  • They were never integrated or synchronized
  • Each system maintained its own agent source
  • No one realized they needed to share data

Fix applied: Created shared agent registry

Created new file src/shared/agents/imported-registry.ts:

// Storage for imported agents from external packages and local projects
let _importedMainAgents: AgentDefinition[] = [];

export function registerImportedAgents(agents: AgentDefinition[]): void {
  _importedMainAgents = [...agents, ..._importedMainAgents];
}

export function getImportedMainAgents(): AgentDefinition[] {
  return _importedMainAgents;
}

export function clearImportedAgents(): void {
  _importedMainAgents = [];
}

Updated System 1 (src/workflows/utils/config.ts):

import {
  registerImportedAgents as registerImportedAgentsShared,
  getImportedMainAgents as getImportedMainAgentsShared,
} from '../../shared/agents/imported-registry.js';

export function getAllMainAgents(): AgentConfig[] {
  const imported = getImportedMainAgentsShared(); // Uses shared registry
  const main = getMainAgents();
  return [...imported, ...main];
}

export function registerImportedAgents(configPath: string): void {
  // ... load agents ...
  registerImportedAgentsShared(importedMain); // Register in shared registry
}

Updated System 2 (src/shared/agents/discovery/catalog.ts):

import { getImportedMainAgents } from '../imported-registry.js';

export async function collectAgentDefinitions(projectRoot: string): Promise<AgentDefinition[]> {
  const byId = new Map<string, AgentDefinition>();

  // ✅ NOW includes imported agents from shared registry!
  const importedAgents = getImportedMainAgents();
  for (const agent of importedAgents) {
    byId.set(agent.id, { ...agent, id });
  }

  // ... rest of filesystem scanning ...

  return Array.from(byId.values());
}

Result: Both systems now use the same registry, ensuring they always see identical agent lists. The architecture is now consistent and maintainable.

Solution Summary

Fix #1: Local Resources Priority

Enforced "local over global" principle throughout codebase.

Files changed:

  • src/shared/workflows/template.ts - Check .codemachine/workflows/ first before global templates
  • src/workflows/run.ts - Register .codemachine/config/main.agents.js first before imported packages
  • src/workflows/preflight.ts - Added local agent registration with highest priority

Key changes:

// Workflow resolution: local → global → imported
const localWorkflowsPath = path.join(cmRoot, 'workflows', activeTemplate);
if (existsSync(localWorkflowsPath)) return localWorkflowsPath;

// Agent registration: local → imported → global
const localConfigDir = path.join(cwd, '.codemachine', 'config');
if (existsSync(localConfigDir)) {
  registerImportedAgents(localConfigDir);
}

Fix #2: Exclude Workflow Files from Build

Ensured workflow files are loaded dynamically at runtime, not embedded in binary.

Files changed:

  • scripts/build.ts - Filter out .workflow.js files from resource embedding

Key change:

const resourceFiles = [
  ...collectFiles('templates').filter(f => !f.endsWith('.workflow.js')), // ✅
];

Result:

  • Workflows loaded from filesystem at runtime
  • Local workflows checked before global
  • No premature resolveStep() execution
  • Proper separation of code vs configuration

Fix #3: Unified Agent Registry

Created single source of truth for all imported agents.

Files changed:

  • src/shared/agents/imported-registry.ts - NEW unified registry
  • src/workflows/utils/config.ts - Uses shared registry
  • src/shared/agents/discovery/catalog.ts - Integrated with shared registry

Key changes:

// New shared registry (imported-registry.ts):
let _importedMainAgents = [];
export function registerImportedAgents(agents) { /* ... */ }
export function getImportedMainAgents() { return _importedMainAgents; }

// Both systems now use it:
const imported = getImportedMainAgentsShared(); // Same data everywhere

Result: Both template loading and agent execution see identical agent lists. No more "agent found in loading but missing in execution" inconsistencies.

Files Changed

Modified

For Fix #1 (Local Resources Priority):

  • src/shared/workflows/template.ts - Reversed search order: local → global → imported
  • src/workflows/run.ts - Added local agent registration before imported packages
  • src/workflows/preflight.ts - Added ensureImportedAgentsRegistered(cwd) with local priority

For Fix #2 (Build Workflow Files):

  • scripts/build.ts - Filter to exclude .workflow.js files: .filter(f => !f.endsWith('.workflow.js'))

For Fix #3 (Unified Agent Registry):

  • src/workflows/utils/config.ts - Refactored to use shared registry instead of local _importedMainAgents
  • src/shared/agents/discovery/catalog.ts - Integrated with shared registry, now includes imported agents
  • src/agents/runner/config.ts - Added debug logging for agent lookup (helps diagnose issues)

Added

  • src/shared/agents/imported-registry.ts - NEW unified agent registry shared between all agent lookup systems

Impact

Before Fix

Workflow issues:

  • ❌ Local workflows in .codemachine/workflows/ ignored (global always loaded)
  • ❌ Global workflows embedded in binary, can't be easily modified
  • ❌ Wrong workflow loaded, no way to override with project-specific version

Agent issues:

  • ❌ Local agents in .codemachine/config/main.agents.js never loaded
  • ❌ Inconsistent agent lookup: template loading sees 11 agents, execution sees only 2
  • ❌ "Unknown agent id" errors despite agents being registered
  • ❌ Impossible to create project-specific agent configurations

Architectural issues:

  • ❌ Two separate agent lookup systems with different data
  • ❌ No synchronization between systems
  • ❌ Global resources always took precedence over local
  • ❌ Violated fundamental "local over global" principle

Developer experience:

  • ❌ Cannot customize workflows per project
  • ❌ Cannot define project-specific agents
  • ❌ Cannot test different configurations locally
  • ❌ Cannot maintain configurations in project version control

After Fix

Workflow improvements:

  • ✅ Local workflows found first and prioritized over global
  • ✅ Workflows loaded dynamically from filesystem (not embedded in binary)
  • ✅ Projects can override any global workflow
  • ✅ Easy to maintain project-specific workflow configurations

Agent improvements:

  • ✅ Local agents registered automatically with highest priority
  • ✅ Consistent agent lookup: both systems see same agent list
  • ✅ Custom agents work correctly in all contexts
  • ✅ Project-specific agent configurations fully supported

Architectural improvements:

  • ✅ Unified agent registry shared across all systems
  • ✅ Consistent data throughout application lifecycle
  • ✅ Local-over-global principle enforced everywhere
  • ✅ Proper separation: code embedded, config dynamic

Developer experience:

  • ✅ Projects can define custom workflows and agents
  • ✅ Local configs override global defaults
  • ✅ Easy to test different configurations
  • ✅ Project configurations live in version control
  • ✅ Can create project-specific CodeMachine-based tools

Backward Compatibility

✅ Fully backward compatible

  • No API changes
  • No configuration changes required
  • Existing projects and workflows continue to work
  • Only internal mechanism changes (not user-facing)
  • Global workflows and agents still work as before
  • Projects without local configs fall back to global defaults

Additional Notes

Contributor Note

These bugs were discovered while setting up a new CodeMachine-based project with custom local agents and workflows.

The bugs would manifest with any attempt to:

  • Create project-specific workflows in .codemachine/workflows/
  • Define local agents in .codemachine/config/main.agents.js
  • Override global configurations with project-specific versions

Pull Request Offer

Would you like this as a PR?

If the CodeMachine maintainers are interested, I can:

  1. ✅ Clean up and refactor the fix into production-ready code
  2. ✅ Update documentation and add code comments
  3. ✅ Submit as a well-structured pull request to the main repository

What you'll get:

  • Clean, production-ready code
  • Comprehensive comments explaining changes
  • Ready for code review and integration

Let me know if you'd like me to prepare a PR! I'm happy to contribute this fix back to the community.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions