net: pool bufio reader and writer to reduce memory allocation (#776)#1085
net: pool bufio reader and writer to reduce memory allocation (#776)#1085Samriddha9619 wants to merge 1 commit intopingcap:mainfrom
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Hi @Samriddha9619. Thanks for your PR. I'm waiting for a pingcap member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
Welcome @Samriddha9619! It looks like this is your first PR to pingcap/tiproxy 🎉 |
|
|
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9aba8c4de5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
pkg/proxy/net/packetio.go
Outdated
| errs = append(errs, err) | ||
| } | ||
| */ | ||
| freeBasicReadWriter(p.readWriter) |
There was a problem hiding this comment.
Close now returns the underlying basicReadWriter buffers to sync.Pool before the connection is actually closed, which mutates bufio.Reader/Writer state (Reset(nil)) while other goroutines may still be in ReadPacket/WritePacket; this happens in normal shutdown paths where SQLServer.Close can call ClientConnection.Close while processMsg is still reading, and onConn's deferred Close can race as well. That can lead to races/panics and even duplicate Put of the same reader/writer, allowing later connections to share a single buffer instance and corrupt traffic.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Maybe overwrite basicReadWriter.Close()?
func (brw *basicReadWriter) Close() error {
err := brw.Conn.Close()
brw.Free()
return err
}
Samriddha9619
force-pushed
the
fix/issue-776-bufio-pool
branch
from
9aba8c4 to
e69d439
Compare
|
To use Codex here, create a Codex account and connect to github. |
Samriddha9619
force-pushed
the
fix/issue-776-bufio-pool
branch
from
e69d439 to
12bd672
Compare
Samriddha9619
force-pushed
the
fix/issue-776-bufio-pool
branch
from
12bd672 to
b5903d0
Compare
| brw := &basicReadWriter{ | ||
| Conn: conn, | ||
| } | ||
| if bufferSize == DefaultConnBufferSize { |
There was a problem hiding this comment.
I'm wondering how to take full advantage of the buffer pool.
If the global buffer size config changes, all the memory in the pool is useless and becomes a leak. Meanwhile, subsequent allocations can't use the pool.
There was a problem hiding this comment.
What if we check the capacity on Get() using the built-in .Size() method? If the buffer from the pool doesn't match the current bufferSize config, we can just discard it and allocate a fresh one.
Does this approach look good? I will verify and update the PR as soon as I get some time
2 participants
What problem does this PR solve?
Issue Number: close #776
Problem Summary:
In short-lived connection workloads, buffered IO is created frequently, leading to heavy memory allocation and GC pressure.
What is changed and how it works:
sync.Poolforbufio.Readerandbufio.WriterinbasicReadWriter.pooledflag to safely manage the buffer lifecycle.freeBasicReadWriterto recursively unwrap connection layers (TLS, compression) and return standard-sized buffers to the pool on connection close to prevent memory leaks.Check List
Tests - [x] Unit test
Notable changes
Release note
Please refer to Release Notes Language Style Guide to write a quality release note.