Skip to content

Conversation

@JohnCari
Copy link
Contributor

@JohnCari JohnCari commented Jan 3, 2026

Summary

This PR adds support for pushing aggregate operations (COUNT, SUM, AVG, MIN, MAX) to foreign data sources via the PostgreSQL GetForeignUpperPaths callback. This enables FDWs to execute aggregate queries remotely, significantly reducing data transfer for analytics queries.

Key Changes

  • New types in interface.rs:

    • AggregateKind enum: Represents supported aggregate functions (Count, CountColumn, Sum, Avg, Min, Max)
    • Aggregate struct: Contains aggregate operation details including kind, column, distinct flag, and alias
    • Helper methods: sql_name(), deparse(), deparse_with_alias() for SQL generation
  • New ForeignDataWrapper trait methods:

    • supported_aggregates(): Declare which aggregates the FDW can push down (default: empty = no pushdown)
    • supports_group_by(): Declare GROUP BY pushdown support (default: false)
    • get_aggregate_rel_size(): Cost estimation for aggregate queries
    • begin_aggregate_scan(): Initialize aggregate query execution
  • New module upper.rs:

    • Implements GetForeignUpperPaths callback for aggregate pushdown
    • Extracts aggregate information from GroupPathExtraData
    • Extracts GROUP BY columns from query
    • Creates and registers foreign upper paths for aggregate queries

Backward Compatibility

All new trait methods have default implementations, ensuring existing FDWs continue to work without modification. FDWs opt-in to aggregate pushdown by overriding supported_aggregates() to return a non-empty vector.

Example Usage

impl ForeignDataWrapper<MyError> for MyFdw {
    fn supported_aggregates(&self) -> Vec<AggregateKind> {
        vec![
            AggregateKind::Count,
            AggregateKind::Sum,
            AggregateKind::Avg,
            AggregateKind::Min,
            AggregateKind::Max,
        ]
    }

    fn supports_group_by(&self) -> bool {
        true
    }

    fn begin_aggregate_scan(
        &mut self,
        aggregates: &[Aggregate],
        group_by: &[Column],
        quals: &[Qual],
        options: &HashMap<String, String>,
    ) -> Result<(), MyError> {
        // Build and execute remote aggregate query
        let select_items: Vec<String> = group_by.iter()
            .map(|c| c.name.clone())
            .chain(aggregates.iter().map(|a| a.deparse_with_alias()))
            .collect();
        // Execute query...
        Ok(())
    }
}

Related Issue

Closes #22

Test Plan

  • Verify existing FDWs without aggregate support continue to work unchanged
  • Test COUNT(*) pushdown with a simple FDW implementation
  • Test GROUP BY pushdown with multiple columns
  • Test multiple aggregates in single query (e.g., SELECT MIN(x), MAX(x), COUNT(*) FROM t)
  • Verify HAVING clause correctly prevents pushdown (not supported)
  • Verify unsupported aggregates correctly fall back to local execution
  • Test EXPLAIN output shows aggregate pushdown when applicable

Add support for pushing aggregate operations (COUNT, SUM, AVG, MIN, MAX)
to foreign data sources via the GetForeignUpperPaths callback.

New types:
- AggregateKind enum: Represents supported aggregate functions
- Aggregate struct: Contains aggregate operation details

New ForeignDataWrapper trait methods:
- supported_aggregates(): Declare which aggregates the FDW can push down
- supports_group_by(): Declare GROUP BY pushdown support
- get_aggregate_rel_size(): Cost estimation for aggregate queries
- begin_aggregate_scan(): Initialize aggregate query execution

New module:
- upper.rs: Implements GetForeignUpperPaths callback for aggregate pushdown

All new methods have default implementations for backward compatibility.
- Add debug2! logging throughout upper.rs for better observability:
  - HAVING clause rejection
  - Unsupported aggregate function names
  - DISTINCT modifier processing
  - Extracted aggregates list
  - GROUP BY columns
  - Cost estimation values

- Update docs/contributing/native.md with aggregate trait methods
- Add comprehensive Aggregate Pushdown section to docs/guides/query-pushdown.md
@burmecia
Copy link
Member

burmecia commented Jan 5, 2026

Thanks for the PR! Could you format the code so CI can pass?

@JohnCari
Copy link
Contributor Author

JohnCari commented Jan 5, 2026

Yes ofcourse, I will do it at night when I get home and thank you! @burmecia

@JohnCari
Copy link
Contributor Author

JohnCari commented Jan 8, 2026

@burmecia working on this now

Apply rustfmt formatting to satisfy CI checks:
- Break long lines in debug2! macro calls
- Adjust line formatting in interface.rs
…er_path

- Replace direct List field access with pgrx::PgList::from_pg() for proper iteration
- Fix create_foreign_upper_path signature for different PostgreSQL versions:
  - PG13-16: 9 parameters (no disabled_nodes, no fdw_restrictinfo)
  - PG17: 10 parameters (added fdw_restrictinfo)
  - PG18: 11 parameters (added disabled_nodes)
- Use conditional compilation (#[cfg(feature = "...")]) for version-specific parameters
Replace pgrx::PgList usage with direct pg_sys::List element access.
This avoids type availability issues during cargo test since PgList
may not be exported at the pgrx crate level in all configurations.

The new list_iter helper function iterates over List elements using
raw pointer access to the elements array, which is always available
as part of the pg_sys FFI bindings.
- Change type_oid: 0 to type_oid: pgrx::pg_sys::Oid::INVALID in Aggregate::deparse doctest
- Change trait method doc examples to rust,ignore since they show implementation patterns
  with &self parameters that don't compile as standalone functions
@JohnCari
Copy link
Contributor Author

JohnCari commented Jan 8, 2026

@burmecia I finished formatting the code, it was harder and took me longer than i expected but finally finished, now all seven checks pass, let me know if you need anything else for this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GetForeignUpperPaths & aggregate pushdown

2 participants