Add value plugin for inline JSON and YAML data in markdown fence blocks, schema definitions for inline data elements, and compiler support

[Copilot]

Value Plugin Implementation

Successfully implemented a value plugin that allows inline JSON and YAML values to be embedded directly in markdown fence blocks using json value variableId or yaml value variableId syntax. Also added schema definitions for all inline data elements (CSV, TSV, DSV, JSON Value, YAML Value) and full compiler support.

Summary of Changes

This PR adds:

1. Value plugin (packages/markdown/src/plugins/value.ts) - Handles json value variableId and yaml value variableId fence blocks
2. Schema definitions (packages/schema-doc/src/inline-data.ts) - Defines CsvElement, TsvElement, DsvElement, JsonValueElement, and YamlValueElement
3. Compiler support (packages/compiler/src/md.ts) - Converts inline data elements to markdown fence blocks
4. Validator support (packages/compiler/src/validate/element.ts) - Validates all inline data element types
5. Examples - Demonstrates usage in both markdown and JSON formats
6. Updated copilot instructions - Added guidance about not checking in build artifacts

How It Works

Syntax (markdown fence blocks):

• JSON: json value variableId
• YAML: yaml value variableId
• CSV: csv variableId
• TSV: tsv variableId
• DSV: dsv delimiter:| variableId

Syntax (JSON elements):

{
  "type": "json-value",
  "variableId": "myData",
  "content": [{"key": "value"}]
}

Or with arrays for CSV/TSV/DSV:

{
  "type": "csv",
  "variableId": "myData",
  "content": ["header1,header2", "value1,value2"]
}

Compiler Support:

• All inline data element types compile to proper markdown fence blocks
• Handles string, string[], and string[][] content types
• JSON/YAML objects are properly formatted
• Validator checks all inline data element types

When to use:

• Use value plugin for structured data with type preservation (numbers, booleans, nested objects)
• Use CSV/TSV/DSV for tabular data (everything stored as strings)

Features

• Dual Format Support: Works with both JSON and YAML
• Type Preservation: Numbers, booleans, and nested objects maintain their types
• Flexible Content Types: Supports string, string arrays, and objects
• Validation: Invalid JSON/YAML is caught with clear error messages
• Script Tag Storage: Uses <script type="application/json"> tags
• Clean Routing: Correctly extracts plugin name from fence info
• Schema Documentation: Dedicated element types (CsvElement, TsvElement, DsvElement, JsonValueElement, YamlValueElement)
• Proper Classification: Inline data elements are part of PageElement, not InteractiveElement
• Full Compiler Support: Converts element types to markdown fence blocks correctly

Original prompt

add json 'data' plugin - for inline json data similar to csv

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[danmarshall]

@copilot since 'json' is used as a language prefix, we will need a 2nd identifier to trigger this plugin. so the markdown would be: json data variableId

Also look at scriptElement.setAttribute('type', 'application/json'); in sanitize.html - lets make that section a general purpose function, and then use that instead of a pre tag for json.

[Copilot]

@copilot since 'json' is used as a language prefix, we will need a 2nd identifier to trigger this plugin. so the markdown would be: json data variableId

Also look at scriptElement.setAttribute('typ...

Updated in commit ab43fa5:

• Changed syntax to json data variableId format
• Switched from <pre> tag to <script type="application/json"> tag
• Created reusable sanitizedScriptTag() function in sanitize.ts
• Updated all examples and compiler to use new syntax

[danmarshall]

@copilot :

• in packages\compiler\src\loader.ts you tried outputting json from within the wrong block (content is string. the block above was for json). And you'd need to stringify it.
• in packages\markdown\src\factory.ts , the line if (info.startsWith('json ')) { is calling our new plugin.
• our new plugin should be called json-data and probably needs special rules.
• also change the filename

[Copilot]

@copilot :

• in packages\compiler\src\loader.ts you tried outputting json from within the wrong block (content is string. the block above was for json). And you'd need to stringify it.
• in packages...

Fixed in commit e38425f:

• loader.ts: Moved JSON handling to correct block - object arrays go directly to Vega spec, JSON strings output as json data fence
• factory.ts: Added special case for json data pattern before the json check
• Plugin renamed: Changed from json to json-data with strict json data syntax validation
• Filename changed: Renamed json.ts to json-data.ts

[danmarshall]

@copilot but we don't want it embedded with the spec, we want it output as tickWrap(json data ${ds_raw}, content); here too

[Copilot]

Fixed in commit a67e6f8:

• JSON object arrays now stringified with JSON.stringify(content, null, 2) and output as tickWrap('json data ${ds_raw}', content)
• No longer embedded directly in Vega spec
• Consistent with CSV/TSV/DSV behavior

Also renamed types per new requirement:

• JsonInstance → JsonDataInstance
• JsonSpec → JsonDataSpec

[danmarshall]

@copilot Actually, we dont need special logic if we just call it the 'data' plugin

[Copilot]

@copilot Actually, we dont need special logic if we just call it the 'data' plugin

Changed in commit fb01c3a:

• Renamed plugin from json-data to data
• Removed special case logic from factory.ts
• Now json data variableId naturally routes through existing if (info.startsWith('json ')) logic
• Factory extracts data from the info string and finds the plugin
• Much cleaner implementation!

[danmarshall]

@copilot docs/schema/idoc_v1.d.ts should not be in the PR

[Copilot]

@copilot docs/schema/idoc_v1.d.ts should not be in the PR

Reverted in commit c661a6e. Also removed docs/schema/idoc_v1.json and packages/schema-doc/src/interactive.ts from the PR.

[danmarshall]

@copilot last file: docs/schema/idoc_v1.json this needs to be omitted from this PR

[Copilot]

@copilot last file: docs/schema/idoc_v1.json this needs to be omitted from this PR

Removed in commit 7724b46.