perf: reuse HTTP connexions between sessions#325
Open
bolinocroustibat wants to merge 8 commits intomainfrom
Open
perf: reuse HTTP connexions between sessions#325bolinocroustibat wants to merge 8 commits intomainfrom
bolinocroustibat wants to merge 8 commits intomainfrom
Conversation
bolinocroustibat
requested review from
Pierlou and
maudetes
and removed request for
Pierlou
bolinocroustibat
changed the title
bolinocroustibat
force-pushed
the
keep-http-client
branch
from
da645c4 to
27f5cca
Compare
bolinocroustibat
changed the title
# Conflicts: # CHANGELOG.md # udata_hydra/crawl/check_resources.py # udata_hydra/routes/checks.py # udata_hydra/utils/file.py
# Conflicts: # CHANGELOG.md
# Conflicts: # CHANGELOG.md
️✅ There are no secrets present in this pull request anymore.If these secrets were true positive and are still valid, we highly recommend you to revoke them. 🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request. |
bolinocroustibat
added
the
code quality
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reuse HTTP connexions between sessions
Refactors the HTTP client usage so that the application relies on a shared aiohttp.ClientSession instead of creating a new session for every request. This should reduce connection-setup overhead and enable connection pooling. Even though we currently don’t make many consecutive requests to the same host — so the impact will be limited for now — using a shared client keeps things consistent, more readable, and prepares us for scenarios where reuse matters.
Changes:
• Introduce centralized HTTP client management in
utils/http.py.• Replace direct ClientSession instantiations with calls to
get_http_client().Benchmarks:
See benchmarks in this PR on another repo. Additional benchmarking with many successive requests to the same server would better quantify the performance gains.