-
Notifications
You must be signed in to change notification settings - Fork 6
refactor(inkless): SharedState to own StorageBackends #469
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Properly separating storage back-end clients between read/write and background jobs. SharedState now owns the StorageBackends lifecycle, properly closing all of them. So, Reader and Writer component do not have to close storages anymore. Alongside, making background components internal constructors visible for easier testing. This allows to make internal SharedState constructor private.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR refactors the SharedState class to properly own and manage the lifecycle of storage backends, separating them into three dedicated instances for fetch, produce, and background operations. The refactoring also simplifies the MetadataView.getTopicConfig method to directly return LogConfig instead of Properties, eliminating the need for manual config merging in consuming code.
Key changes:
- SharedState now creates and owns three separate StorageBackend instances (fetchStorage, produceStorage, backgroundStorage) and is responsible for closing all of them
- Reader and Writer components no longer close storage backends; SharedState handles all storage lifecycle management
- Background components (FileMerger, FileCleaner) now have package-visible constructors for easier testing
- MetadataView.getTopicConfig() now returns LogConfig directly, with default config merging happening internally
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| storage/inkless/src/main/java/io/aiven/inkless/common/SharedState.java | Refactored to own three separate StorageBackend instances, added comprehensive resource cleanup in close() and exception handlers |
| storage/inkless/src/main/java/io/aiven/inkless/produce/Writer.java | Removed storage parameter and close logic from internal constructor |
| storage/inkless/src/main/java/io/aiven/inkless/produce/AppendHandler.java | Simplified to use Function<String, LogConfig> instead of manually merging configs |
| storage/inkless/src/main/java/io/aiven/inkless/merge/FileMerger.java | Added package-visible constructor for testing, uses backgroundStorage from SharedState |
| storage/inkless/src/main/java/io/aiven/inkless/delete/FileCleaner.java | Updated to use backgroundStorage from SharedState |
| storage/inkless/src/main/java/io/aiven/inkless/delete/RetentionEnforcer.java | Simplified to directly use getTopicConfig without manual config merging |
| storage/inkless/src/main/java/io/aiven/inkless/delete/DeleteRecordsInterceptor.java | Refactored to accept ControlPlane and MetadataView directly instead of SharedState |
| storage/inkless/src/main/java/io/aiven/inkless/consume/Reader.java | Removed storage close logic |
| storage/inkless/src/main/java/io/aiven/inkless/consume/FetchHandler.java | Updated to use fetchStorage from SharedState |
| storage/inkless/src/main/java/io/aiven/inkless/control_plane/MetadataView.java | Changed getTopicConfig return type from Properties to LogConfig |
| core/src/main/scala/kafka/server/metadata/InklessMetadataView.scala | Implemented new getTopicConfig signature, merging default and topic configs internally |
| Test files | Updated to reflect new constructor signatures and simplified mocking requirements |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| Utils.closeQuietly(produceStorage, "produceStorage"); | ||
| Utils.closeQuietly(fetchStorage, "fetchStorage"); | ||
| Utils.closeQuietly(batchCoordinateCache, "batchCoordinateCache"); | ||
| Utils.closeQuietly(objectCache, "objectCache"); |
Copilot
AI
Dec 26, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The storageMetrics object is not closed in the exception handler after being created. If an exception occurs after storageMetrics is created but before the method returns successfully, it will not be cleaned up. Add Utils.closeQuietly(storageMetrics, "storageMetrics"); to the exception handler to prevent resource leaks.
| Utils.closeQuietly(objectCache, "objectCache"); | |
| Utils.closeQuietly(objectCache, "objectCache"); | |
| Utils.closeQuietly(storageMetrics, "storageMetrics"); |
Properly separating storage back-end clients between read/write and background jobs.
SharedState now owns the StorageBackends lifecycle, properly closing all of them.
So, Reader and Writer component do not have to close storages anymore.
Alongside, making background components internal constructors visible for easier testing.
This allows to make internal SharedState constructor private.
On the first commit, simplify how
getTopicConfigreturns configurations, doing the "current configs" calculation on the MetadataView itself.