Feature Request: Asset Location Tracking in Analysis Results

[magne4000]

Summary

Add location tracking to the analyze function to capture where assets, dependencies, and imports are referenced in source code. This enables downstream tools (like bundlers and transformers) to rewrite asset references while preserving source maps and code structure.

Motivation

Currently, @vercel/nft identifies which assets/dependencies a file uses but doesn't track where in the source code those references occur. This limits the ability to:

1. Transform asset references: Build tools that need to rewrite asset paths (e.g., Vite plugins, custom bundlers) must re-parse and duplicate nft's sophisticated detection heuristics
2. Generate accurate source maps: Without position information, transformations can't maintain proper source map mappings

Use Case

Vite/Rolldown/Esbuild Plugin for Asset Transformation

// Input code
const fontPath = path.join(__dirname, 'fonts', 'somefile.ttf');
const data = fs.readFileSync(fontPath);

// With location tracking, a Vite plugin can:
// 1. Detect the asset reference at the exact location
// 2. Emit the asset to the build output
// 3. Replace the expression with the new asset URL
// 4. Maintain accurate source maps

// Output code
const fontPath = new URL('./assets/somefile-abc123.ttf', import.meta.url).href;
const data = fs.readFileSync(fontPath);

Proposed API

Enhanced AnalyzeResult Interface

export interface AssetLocation {
  start: number;        // Character offset where reference starts
  end: number;          // Character offset where reference ends
  source: string;       // Original source code text (maybe not necessary)
  type:                 // Type of asset reference
    | 'fs.readFileSync'
    | 'fs.readFile' 
    | 'fs.createReadStream'
    | 'path.join'
    | 'path.resolve'
    | '__dirname'
    | '__filename'
    | 'import.meta.url'
    | 'bindings'
    | 'node-gyp-build'
//  | ...
    | 'other';
}

export interface AnalyzeResult {
  assets: Set<string>;
  deps: Set<string>;
  imports: Set<string>;
  isESM: boolean;
  
  // New: Location tracking
  assetsLocation: Map<string, AssetLocation[]>;
}

Example Usage

const result = await analyze('app.js', code, job);

// Access location information
for (const [assetPath, locations] of Object.entries(result.assetsLocation)) {
  console.log(`Asset: ${assetPath}`);
  
  for (const loc of locations) {
    console.log(`  Found at ${loc.start}-${loc.end}`);
    console.log(`  Type: ${loc.type}`);
    console.log(`  Code: ${loc.source}`);
  }
}

// Use in transformations
const s = new MagicString(code);
for (const [assetPath, locations] of Object.entries(result.assetsLocation)) {
  const newPath = transformAsset(assetPath);
  
  // Replace in reverse order to preserve positions
  for (const loc of locations.sort((a, b) => b.start - a.start)) {
    s.overwrite(loc.start, loc.end, newPath);
  }
}

Coverage

I think that the following detection heuristics can be enhanced:

Fully Trackable (13 cases):

• ✅ require(), module.require(), require.resolve()
• ✅ ESM import/export declarations
• ✅ Dynamic import() expressions
• ✅ fs.* function calls (readFileSync, readFile, etc.)
• ✅ path.* functions (join, resolve)
• ✅ __dirname, __filename references
• ✅ import.meta.url
• ✅ Special bindings (BINDINGS, NODE_GYP_BUILD, NBIND_INIT, etc.)

Partially Trackable (3 cases):

• ⚠️ Glob patterns (emitAssetDirectory, emitWildcardRequire)
  • Can track the expression that triggered the glob
  • Cannot track individual files discovered from filesystem

Open Questions

1. Opt-in flag: Should location tracking be behind a feature flag to avoid performance overhead for users who don't need it?

   const result = await analyze(id, code, { 
     ...job,
     trackLocations: true 
   });

2. Glob patterns: How should we represent assets discovered through glob patterns? Current proposal is to track the pattern expression but not individual files.

[styfle]

We don't export analyze(). Its not part of the public interface. So how would this look on nodeFileTrace()?

https://github.com/vercel/nft/blob/main/src/index.ts

[magne4000]

Here is an example with nodeFileTrace used in a Vite plugin:

const someVitePlugin = {
  name: 'some-plugin',
  transform(code, id) {
    const result = await nodeFileTrace(id, {
      depth: 1,
      readFile() {
        return code;
      }
    });

    const s = new MagicString(code);
    for (const [assetPath, locations] of Object.entries(result.assetsLocation)) {
      const newPath = transformAsset(assetPath);
      
      // Replace in reverse order to preserve positions
      for (const loc of locations.sort((a, b) => b.start - a.start)) {
        s.overwrite(loc.start, loc.end, newPath);
      }
    }

    return {
      code: s.toString(),
      map: s.generateMap({ hires: true }),
    }
  }
}

[styfle]

Opt-in flag: Should location tracking be behind a feature flag to avoid performance overhead for users who don't need it?

Yes

Glob patterns: How should we represent assets discovered through glob patterns?

Seems like it should track the individual files. We should have those since they end up in the fileList output.

[magne4000]

I'll be experimenting with a POC in the coming weeks, that way we'll be able to decide upon a definitive API for this feature.