Skip to content

feat: modernize Table API with async-only design#537

Draft
xushiyan wants to merge 1 commit intoapache:mainfrom
xushiyan:table-api-changes
Draft

feat: modernize Table API with async-only design#537
xushiyan wants to merge 1 commit intoapache:mainfrom
xushiyan:table-api-changes

Conversation

@xushiyan
Copy link
Member

@xushiyan xushiyan commented Mar 8, 2026

Description

Modernize the hudi-rs Table API by removing all blocking API twins, improving schema methods, and making Python bindings truly async.

Schema API changes:

  • get_schema() now returns data schema without meta fields (previously included them)
  • Added get_schema_with_meta_fields() for when meta fields are needed
  • Renamed get_avro_schema() to get_schema_in_avro_str() for clarity

Blocking API removal:

  • Removed all _blocking method variants from Table, FileGroupReader, and metadata table
  • C++ FFI updated to manage its own tokio runtime
  • All Rust tests converted from #[test] to #[tokio::test] async

Python async migration:

  • Replaced rt().block_on() pattern with pyo3-async-runtimes future_into_py
  • HudiTable and HudiDataFusionDataSource constructors become async static methods (await HudiTable.new(...))
  • All I/O methods are now truly async (awaitable from Python)
  • Added pytest-asyncio for async test support

Closes #363

How are the changes test-covered

  • Automated tests (unit and/or integration tests)

@xushiyan xushiyan marked this pull request as draft March 8, 2026 03:14
@xushiyan xushiyan changed the title feat: use async APIs and drop blocking api twins feat: modernize Table API with async-only design Mar 8, 2026
@xushiyan xushiyan force-pushed the table-api-changes branch from 6a942d3 to adced71 Compare March 8, 2026 03:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Drop blocking APIs and make Python APIs async

1 participant