Skip to content

feat: update inch options and url to support influxdb 3 core and improve batch production#47

Open
philjb wants to merge 1 commit intomasterfrom
pjb/46/v3-core-write-support
Open

feat: update inch options and url to support influxdb 3 core and improve batch production#47
philjb wants to merge 1 commit intomasterfrom
pjb/46/v3-core-write-support

Conversation

@philjb
Copy link
Contributor

@philjb philjb commented Aug 19, 2025

Adds an option to use the v3 native write endpoint so advanced control of write behavior can be used such as the "no-sync" option.

Additionally, modifies the channel for buffered write batches to reflect concurrency setting, fixes an issue with retrying a write that was needlessly recompressing or copying the write batch, reduced allocations and copies by reusing buffers and letting bytes Buffer's take ownership of existing buffers.

Updates golang to version 1.24

@philjb philjb force-pushed the pjb/46/v3-core-write-support branch from 96f61ef to d599cb2 Compare February 4, 2026 22:24

// generateBatches returns a channel for streaming batches.
func (s *Simulator) generateBatches() <-chan []byte {
ch := make(chan []byte, 10)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a channel of size 10 means 10 batches get buffered (naturally) but if the concurrency is higher, they might drain unsmoothly creating delays for the concurrent writers because a batch isn't available yet: the writing might stutter on inch's data generation size.

If the channel for buffering batches is slightly bigger than concurrency, the individual writeres should see a smoother pattern where the next batch is always available when it is finished with the last and ready to write the next batch (without waiting on generation).

This pr increases cpu usage but increasing the concurrent work batch generation and writing to the db under test. Also increases memory usage if concurrency is high, but really high concurrency might be better simulated with multiple inch instances running.

Adds an option to use the v3 native write endpoint so advanced control
of write behavior can be used such as the "no-sync" option.

Additionally, modifies the channel for buffered write batches to reflect
concurrency setting, fixes an issue with retrying a write that was
needlessly recompressing or copying the write batch, reduced allocations
and copies by reusing buffers and letting bytes Buffer's take ownership
of existing buffers.

Updates golang to version 1.24

* closes #46
@philjb philjb force-pushed the pjb/46/v3-core-write-support branch from d599cb2 to 59d5235 Compare February 4, 2026 23:22
@philjb philjb changed the title feat: update inch options and url to support influxdb 3 core feat: update inch options and url to support influxdb 3 core and improve batch production Feb 4, 2026
@philjb philjb marked this pull request as ready for review February 4, 2026 23:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

update inch to support influxdb v3 core and enterprise "native" write endpoint

1 participant