Skip to content

Conversation

@coremoon
Copy link

Add YAML support for witness and arguments files

This PR solves issue #177 - "Witness files should not require explicit type annotations"

This PR is a refactoring on PR #190 - #190 can be skipped.

Summary

Add optional YAML format support for witness and arguments files with intelligent format detection. JSON remains the standard format; YAML is supported as an alternative when users prefer comments and documentation.

Changes

Core Implementation

  • src/serde.rs: Add YAML parsing functions
    • from_file_with_types(): Intelligent format detection (JSON first, YAML fallback)
    • from_yaml_with_types(): Explicit YAML parser with witness: and arguments: sections
    • Support for both JSON formats (old nested + new flat) for backward compatibility

Format Support

JSON (Standard)

{
  "x": { "value": "0x0001", "type": "u32" },
  "y": { "value": "1000", "type": "u32" }
}

Note: The "type" field is optional and ignored - types are inferred from compiler context.

JSON (Sloppy format accepted)

{ "x": "0x0001", "y": "1000" }

YAML (Optional)

witness:
  x: "0x0001"
  y: "1000"
  # Comments allowed!

Features

  • Full backward compatibility with existing JSON files
  • YAML sections support comments and documentation
  • Intelligent format detection: tries JSON first, then YAML
  • Type information from compiler (no manual type definitions needed)
  • Support for ignored sections in YAML for documentation

Testing

  • 8 new YAML unit tests in src/serde.rs
  • 58 existing integration tests pass unchanged
  • All old JSON tests continue to work (100% backward compatible)

CI Testing

  • All passed

Dependencies

  • Added serde_yaml = "0.8" to Cargo.toml (compatible with rustc 1.79.0+)

Breaking Changes

None. All existing code continues to work without modification.

Migration Guide

No migration required. Existing JSON files work as before. YAML support is optional.

- Add from_file_with_types() with JSON→YAML format detection
- Support witness: and arguments: sections in YAML
- Allow comments and documentation in witness files
- Maintain 100% backward compatibility with existing JSON files
- Add 8 YAML format validation tests
- Deprecate from_json_with_types() (still functional)
- Update CLI to indicate YAML/JSON format support

All 15 old JSON tests pass. New implementation tested with 8 unit tests.
Updated comments to clarify witness file parsing process.
@coremoon coremoon requested a review from imaginator as a code owner December 29, 2025 14:22
@coremoon
Copy link
Author

coremoon commented Dec 29, 2025

You can see here that all CI runs went through on my fork:

successful_ci_test

I don't understand why it's not after the merge.

@uncomputable
Copy link
Collaborator

Thank you for your PR. There seems to be a conflict with the recently merged #187. CI seems to be blocked on a rebase.

@coremoon
Copy link
Author

Regarding the timeline with #187:

#186 was opened Dec 24, I started #192 based on a pre-#187 fork (merged Dec 27), and opened #192 Dec 29.

#192 fully implements #186 (optional param/witness modules) - the code change from .ok_or(Error::ModuleRequired(...)) to if let Some(...) proves both modules are now optional. It also implements #177 (witness format improvements).

#187 was parallel work on #186. The team can choose which approach to use.

Happy to address any technical questions.

@apoelstra
Copy link
Contributor

#192 fully implements #186 (optional param/witness modules) - the code change from .ok_or(Error::ModuleRequired(...)) to if let Some(...) proves both modules are now optional.

I complained about your LLM-generated diff and proposed a new one, which I implemented in #187, which is simpler and more complete. It is already merged (and was merged a week before your most recent comment saying the team can "choose which approach to take").

It also implements #177 (witness format improvements).

We are not going to implement a YAML witness parser. I provided some reasoning in this comment. I'm going to close this because I think both changes in this PR are going nowhere.

@apoelstra apoelstra closed this Dec 31, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants