Skip to content

Conversation

@jeqo
Copy link
Contributor

@jeqo jeqo commented Dec 26, 2025

Properly separating storage back-end clients between read/write and background jobs.
SharedState now owns the StorageBackends lifecycle, properly closing all of them.
So, Reader and Writer component do not have to close storages anymore.

Alongside, making background components internal constructors visible for easier testing.
This allows to make internal SharedState constructor private.

On the first commit, simplify how getTopicConfig returns configurations, doing the "current configs" calculation on the MetadataView itself.

jeqo added 2 commits December 26, 2025 16:39
Properly separating storage back-end clients between read/write and
background jobs.
SharedState now owns the StorageBackends lifecycle, properly closing all
of them.
So, Reader and Writer component do not have to close storages anymore.

Alongside, making background components internal constructors visible
for easier testing.
This allows to make internal SharedState constructor private.
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors the SharedState class to properly own and manage the lifecycle of storage backends, separating them into three dedicated instances for fetch, produce, and background operations. The refactoring also simplifies the MetadataView.getTopicConfig method to directly return LogConfig instead of Properties, eliminating the need for manual config merging in consuming code.

Key changes:

  • SharedState now creates and owns three separate StorageBackend instances (fetchStorage, produceStorage, backgroundStorage) and is responsible for closing all of them
  • Reader and Writer components no longer close storage backends; SharedState handles all storage lifecycle management
  • Background components (FileMerger, FileCleaner) now have package-visible constructors for easier testing
  • MetadataView.getTopicConfig() now returns LogConfig directly, with default config merging happening internally

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
storage/inkless/src/main/java/io/aiven/inkless/common/SharedState.java Refactored to own three separate StorageBackend instances, added comprehensive resource cleanup in close() and exception handlers
storage/inkless/src/main/java/io/aiven/inkless/produce/Writer.java Removed storage parameter and close logic from internal constructor
storage/inkless/src/main/java/io/aiven/inkless/produce/AppendHandler.java Simplified to use Function<String, LogConfig> instead of manually merging configs
storage/inkless/src/main/java/io/aiven/inkless/merge/FileMerger.java Added package-visible constructor for testing, uses backgroundStorage from SharedState
storage/inkless/src/main/java/io/aiven/inkless/delete/FileCleaner.java Updated to use backgroundStorage from SharedState
storage/inkless/src/main/java/io/aiven/inkless/delete/RetentionEnforcer.java Simplified to directly use getTopicConfig without manual config merging
storage/inkless/src/main/java/io/aiven/inkless/delete/DeleteRecordsInterceptor.java Refactored to accept ControlPlane and MetadataView directly instead of SharedState
storage/inkless/src/main/java/io/aiven/inkless/consume/Reader.java Removed storage close logic
storage/inkless/src/main/java/io/aiven/inkless/consume/FetchHandler.java Updated to use fetchStorage from SharedState
storage/inkless/src/main/java/io/aiven/inkless/control_plane/MetadataView.java Changed getTopicConfig return type from Properties to LogConfig
core/src/main/scala/kafka/server/metadata/InklessMetadataView.scala Implemented new getTopicConfig signature, merging default and topic configs internally
Test files Updated to reflect new constructor signatures and simplified mocking requirements

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Utils.closeQuietly(produceStorage, "produceStorage");
Utils.closeQuietly(fetchStorage, "fetchStorage");
Utils.closeQuietly(batchCoordinateCache, "batchCoordinateCache");
Utils.closeQuietly(objectCache, "objectCache");
Copy link

Copilot AI Dec 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The storageMetrics object is not closed in the exception handler after being created. If an exception occurs after storageMetrics is created but before the method returns successfully, it will not be cleaned up. Add Utils.closeQuietly(storageMetrics, "storageMetrics"); to the exception handler to prevent resource leaks.

Suggested change
Utils.closeQuietly(objectCache, "objectCache");
Utils.closeQuietly(objectCache, "objectCache");
Utils.closeQuietly(storageMetrics, "storageMetrics");

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants