Skip to content

Conversation

@KevBrowne
Copy link
Contributor

@KevBrowne KevBrowne commented Jan 9, 2026

Overview

Ran into issue when testing rescale

Issue

2026-01-08 19:22:50,394 INFO  com.pinterest.flink.streaming.connectors.psc.table.PscDynamicTableFactory [] - scan.parallelism configured: 100 (will override partition-based parallelism)
2026-01-08 19:22:50,462 WARN  com.pinterest.flink.streaming.connectors.psc.table.PscTableCommonUtils [] - scan.enable-rescale is true, but partition count could not be determined. Rescale will not be applied.

We are unable to create a Metadata client, which fails at
Invalid metadataClient configuration

Solution

we need to set PSC_METADATA_CLIENT_ID similar to how PscSourceEnumerator.java calls the PscMetadataClient

Test Plan


$psc  mvn -pl psc-flink test -Dtest=PscTableCommonUtilsTest,PscDynamicTableFactoryTest -Dgpg.skip=true -Djacoco.skip=true 2>&1 | tail -10
[INFO] Results:
[INFO] 
[INFO] Tests run: 86, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  22.529 s
[INFO] Finished at: 2026-01-12T18:23:37-05:00
[INFO] -------------------------------------------------

@KevBrowne KevBrowne requested a review from a team as a code owner January 9, 2026 16:53
@jeffxiang jeffxiang merged commit f750364 into pinterest:main Jan 13, 2026
2 checks passed
KevBrowne added a commit to KevBrowne/psc that referenced this pull request Jan 13, 2026
* fix rescale configs

* update to ID

* add test

---------

Co-authored-by: Kevin Browne <kbrowne@kbrowne-JG645XV.dyn.pinadmin.com>
jeffxiang pushed a commit that referenced this pull request Jan 13, 2026
* fix rescale configs

* update to ID

* add test

---------

Co-authored-by: Kevin Browne <kbrowne@kbrowne-JG645XV.dyn.pinadmin.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants