Refactor ClientRequest #11012

Dreamsorcerer · 2025-05-25T13:05:57Z

Goals

Separate logic between the proxy version and the main version, which have different requirements. This reduces need for asserts etc. where we know an attribute will not be None in one case, but not the other. The proxy code now uses the internal ClientRequestBase, while ClientRequest is for the main request and will be exposed to middlewares.
Rename some attributes/methods to better reflect what middleware users should be accessing (and matching what we've actually documented).
Ensure body is always Payload (it was only not when None was passed previously), and strip out the redundant code that tries to send an empty body.
Make it easier to make additions to ClientRequest in future releases without breaking subclasses.

for more information, see https://pre-commit.ci

aiohttp/client_reqrep.py

…into clientrequest-refactor

aiohttp/client_reqrep.py

codecov · 2025-05-25T13:34:08Z

Codecov Report

❌ Patch coverage is 99.80769% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 98.73%. Comparing base (149a810) to head (fec5fef).
⚠️ Report is 59 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
aiohttp/connector.py	90.90%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #11012      +/-   ##
==========================================
- Coverage   98.74%   98.73%   -0.02%     
==========================================
  Files         127      127              
  Lines       43535    43497      -38     
  Branches     2327     2321       -6     
==========================================
- Hits        42989    42947      -42     
  Misses        390      390              
- Partials      156      160       +4

Flag	Coverage Δ
CI-GHA	`98.60% <98.84%> (-0.03%)`	⬇️
OS-Linux	`98.34% <98.46%> (-0.03%)`	⬇️
OS-Windows	`96.67% <98.46%> (-0.02%)`	⬇️
OS-macOS	`97.56% <98.46%> (-0.03%)`	⬇️
Py-3.10.11	`97.10% <95.96%> (-0.05%)`	⬇️
Py-3.10.18	`?`
Py-3.10.19	`97.60% <95.57%> (?)`
Py-3.11.13	`?`
Py-3.11.14	`97.81% <96.53%> (?)`
Py-3.11.9	`97.31% <96.92%> (-0.04%)`	⬇️
Py-3.12.10	`97.41% <97.11%> (-0.05%)`	⬇️
Py-3.12.11	`?`
Py-3.12.12	`97.91% <96.73%> (?)`
Py-3.13.7	`?`
Py-3.13.8	`98.17% <97.11%> (?)`
Py-3.14.0	`98.11% <96.99%> (-0.05%)`	⬇️
Py-3.14.0t	`97.18% <96.59%> (-0.05%)`	⬇️
Py-pypy3.10.16-7.3.19	`90.85% <95.38%> (-2.97%)`	⬇️
VM-macos	`97.56% <98.46%> (-0.03%)`	⬇️
VM-ubuntu	`98.34% <98.46%> (-0.03%)`	⬇️
VM-windows	`96.67% <98.46%> (-0.02%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

aiohttp/client_reqrep.py

codspeed-hq · 2025-05-25T14:08:59Z

CodSpeed Performance Report

Merging #11012 will not alter performance

_{Comparing clientrequest-refactor (fec5fef) with master (42fc48a)}

Summary

✅ 59 untouched

for more information, see https://pre-commit.ci

tests/test_benchmarks_client_request.py

…into clientrequest-refactor

aiohttp/client_reqrep.py

…into clientrequest-refactor

bdraco · 2025-05-26T06:56:30Z

Sorry about the conflicts

Bumps [pip](https://github.com/pypa/pip) from 24.3.1 to 25.0. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pypa/pip/blob/main/NEWS.rst">pip's changelog</a>.</em></p> <blockquote> <h1>25.0 (2025-01-26)</h1> <h2>Deprecations and Removals</h2> <ul> <li>Deprecate the <code>no-python-version-warning</code> flag as it has long done nothing since Python 2 support was removed in pip 21.0. (<code>[#13154](pypa/pip#13154) <https://github.com/pypa/pip/issues/13154></code>_)</li> </ul> <h2>Features</h2> <ul> <li>Prefer to display :pep:<code>639</code> <code>License-Expression</code> in <code>pip show</code> if metadata version is at least 2.4. (<code>[#13112](pypa/pip#13112) <https://github.com/pypa/pip/issues/13112></code>_)</li> <li>Support :pep:<code>639</code> <code>License-Expression</code> and <code>License-File</code> metadata fields in JSON output. <code>pip inspect</code> and <code>pip install --report</code> now emit <code>license_expression</code> and <code>license_file</code> fields in the <code>metadata</code> object, if the corresponding fields are present in the installed <code>METADATA</code> file. (<code>[#13134](pypa/pip#13134) <https://github.com/pypa/pip/issues/13134></code>_)</li> <li>Files in the network cache will inherit the read/write permissions of pip's cache directory (in addition to the current user retaining read/write access). This enables a single cache to be shared among multiple users. (<code>[aio-libs#11012](pypa/pip#11012) <https://github.com/pypa/pip/issues/11012></code>_)</li> <li>Return the size, along with the number, of files cleared on <code>pip cache purge</code> and <code>pip cache remove</code> (<code>[#12176](pypa/pip#12176) <https://github.com/pypa/pip/issues/12176></code>_)</li> <li>Cache <code>python-requires</code> checks while filtering potential installation candidates. (<code>[#13128](pypa/pip#13128) <https://github.com/pypa/pip/issues/13128></code>_)</li> <li>Optimize package collection by avoiding unnecessary URL parsing and other processing. (<code>[#13132](pypa/pip#13132) <https://github.com/pypa/pip/issues/13132></code>_)</li> </ul> <h2>Bug Fixes</h2> <ul> <li>Reorder the encoding detection when decoding a requirements file, relying on UTF-8 over the locale encoding by default, matching the documented behaviour. (<code>[#12771](pypa/pip#12771) <https://github.com/pypa/pip/issues/12771></code>_)</li> <li>The pip version self check is disabled on <code>EXTERNALLY-MANAGED</code> environments. (<code>[#11820](pypa/pip#11820) <https://github.com/pypa/pip/issues/11820></code>_)</li> <li>Fix a security bug allowing a specially crafted wheel to execute code during installation. (<code>[#13079](pypa/pip#13079) <https://github.com/pypa/pip/issues/13079></code>_)</li> <li>The inclusion of <code>packaging</code> 24.2 changes how pre-release specifiers with <code><</code> and <code>></code> behave. Including a pre-release version with these specifiers now implies accepting pre-releases (e.g., <code><2.0dev</code> can include <code>1.0rc1</code>). To avoid implying pre-releases, avoid specifying them (e.g., use <code><2.0</code>). The exception is <code>!=</code>, which never implies pre-releases. (<code>[#13163](pypa/pip#13163) <https://github.com/pypa/pip/issues/13163></code>_)</li> <li>The <code>--cert</code> and <code>--client-cert</code> command-line options are now respected while installing build dependencies. Consequently, the private <code>_PIP_STANDALONE_CERT</code> environment variable is no longer used. (<code>[aio-libs#5502](pypa/pip#5502) <https://github.com/pypa/pip/issues/5502></code>_)</li> <li>The <code>--proxy</code> command-line option is now respected while installing build dependencies. (<code>[aio-libs#6018](pypa/pip#6018) <https://github.com/pypa/pip/issues/6018></code>_)</li> </ul> <h2>Vendored Libraries</h2> <ul> <li>Upgrade CacheControl to 0.14.1</li> <li>Upgrade idna to 3.10</li> <li>Upgrade msgpack to 1.1.0</li> <li>Upgrade packaging to 24.2</li> </ul>  </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pypa/pip/commit/f47b5874299848c688336ae7c8d69534013fe2c6"><code>f47b587</code></a> Bump for release</li> <li><a href="https://github.com/pypa/pip/commit/74a7f3335338712af44be95241daf62e756f27ec"><code>74a7f33</code></a> Update AUTHORS.txt</li> <li><a href="https://github.com/pypa/pip/commit/a008888a5b123e8d5e4667bdd21e4b42f3fc034c"><code>a008888</code></a> Merge pull request <a href="https://redirect.github.com/pypa/pip/issues/13171">#13171</a> from pypa/dependabot/github_actions/github-actions-...</li> <li><a href="https://github.com/pypa/pip/commit/d265fb7427c3ba4dbd10e4874a0bebea2e59350e"><code>d265fb7</code></a> Merge pull request <a href="https://redirect.github.com/pypa/pip/issues/13174">#13174</a> from ichard26/changelog</li> <li><a href="https://github.com/pypa/pip/commit/d35384ef91cb372a5223a01f980e5deb84c8fde5"><code>d35384e</code></a> Copyedit news entries before 25.0</li> <li><a href="https://github.com/pypa/pip/commit/adc4f9951b51b6a06e405b8960dd0c5f030f0fb5"><code>adc4f99</code></a> Reorder requirements file decoding (<a href="https://redirect.github.com/pypa/pip/issues/12795">#12795</a>)</li> <li><a href="https://github.com/pypa/pip/commit/40c42149a51a63e8416c047d5ddc0da1694387ea"><code>40c4214</code></a> Bump pypa/gh-action-pypi-publish in the github-actions group</li> <li><a href="https://github.com/pypa/pip/commit/6b0fb904803fbb3ce7da63966b2759407b6cd9dc"><code>6b0fb90</code></a> Merge pull request <a href="https://redirect.github.com/pypa/pip/issues/13048">#13048</a> from sbidoul/trusted-publisher-sbi</li> <li><a href="https://github.com/pypa/pip/commit/c7fb1e13ec79b1b48481ac245144c2b368e64f7d"><code>c7fb1e1</code></a> Merge pull request <a href="https://redirect.github.com/pypa/pip/issues/13145">#13145</a> from befeleme/pip-show-pep639</li> <li><a href="https://github.com/pypa/pip/commit/41c807c5938d269703c6ff2644fb3b7dc88eda4e"><code>41c807c</code></a> Show License-Expression if present in package metadata</li> <li>Additional commits viewable in <a href="https://github.com/pypa/pip/compare/24.3.1...25.0">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pip&package-manager=pip&previous-version=24.3.1&new-version=25.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

for more information, see https://pre-commit.ci

…into clientrequest-refactor

for more information, see https://pre-commit.ci

aiohttp/client_reqrep.py

bdraco · 2025-10-18T18:53:51Z

Down to one perf regression

bdraco · 2025-10-18T19:15:35Z

I changed RequestInfo to lazy construction. That should take care of the other one. Since they already have to adjust subclassing for ClientRequest we might as well do it now and they can adjust ClientResponse at the same time if they have swallowed some of the args.

Dreamsorcerer · 2025-10-19T13:18:28Z

aiohttp/client_reqrep.py

        timer: BaseTimerContext | None,
-        request_info: RequestInfo,
-        traces: list["Trace"],
+        request_info: RequestInfo | None,  # Backwards compat; not normally used anymore


@bdraco Should we just remove this parameter?

I think the class should only ever be constructed internally. The change already breaks subclasses, so we probably want to add kwargs again, like we have for ClientRequest.

Dreamsorcerer · 2025-10-19T13:20:08Z

aiohttp/client_reqrep.py

+        original_url: URL | None = None,
    ) -> None:
        # URL forbids subclasses, so a simple type check is enough.


Suggested change

original_url: URL | None = None,

) -> None:

# URL forbids subclasses, so a simple type check is enough.

original_url: URL | None = None,

**kwargs: object,

) -> None:

# kwargs exists so authors of subclasses should expect to pass through unknown

# arguments. This allows us to safely add new arguments in future releases.

# But, we should never receive unknown arguments here in the parent class, this

# would indicate an argument has been named wrong or similar in the subclass.

assert not kwargs, "Unexpected arguments to ClientResponse"

# URL forbids subclasses, so a simple type check is enough.

CHANGES/11012.breaking.rst

Refactor ClientRequest

18564e9

Dreamsorcerer requested a review from asvetlov as a code owner May 25, 2025 13:05

Dreamsorcerer added the backport:skip Skip backport bot label May 25, 2025

[pre-commit.ci] auto fixes from pre-commit.com hooks

f4d733f

for more information, see https://pre-commit.ci

github-advanced-security bot found potential problems May 25, 2025

View reviewed changes

aiohttp/client_reqrep.py Fixed Show fixed Hide fixed

aiohttp/client_reqrep.py Fixed Show fixed Hide fixed

Dreamsorcerer added 2 commits May 25, 2025 14:11

Add kwargs

22d724a

Merge branch 'clientrequest-refactor' of github.com:aio-libs/aiohttp …

6fa8996

…into clientrequest-refactor

Dreamsorcerer commented May 25, 2025

View reviewed changes

aiohttp/client_reqrep.py Outdated Show resolved Hide resolved

Tweaks

8257048

Dreamsorcerer commented May 25, 2025

View reviewed changes

aiohttp/client_reqrep.py Show resolved Hide resolved

Dreamsorcerer added 2 commits May 25, 2025 14:26

Fix

7171e0a

Fix

9023653

bdraco reviewed May 25, 2025

View reviewed changes

aiohttp/client_reqrep.py Outdated Show resolved Hide resolved

Dreamsorcerer added 2 commits May 25, 2025 14:50

Fix benchmarks

6a1e81a

Fix benchmarks

ef47363

Dreamsorcerer and others added 2 commits May 25, 2025 15:56

Remove unneeded code

477d52d

[pre-commit.ci] auto fixes from pre-commit.com hooks

8fbac96

for more information, see https://pre-commit.ci

github-advanced-security bot found potential problems May 25, 2025

View reviewed changes

tests/test_benchmarks_client_request.py Fixed Show fixed Hide fixed

Dreamsorcerer added 3 commits May 25, 2025 16:06

Fix

cb5f104

Merge branch 'clientrequest-refactor' of github.com:aio-libs/aiohttp …

3820c39

…into clientrequest-refactor

Merge branch 'master' into clientrequest-refactor

2559a0b

Dreamsorcerer commented May 25, 2025

View reviewed changes

aiohttp/client_reqrep.py Outdated Show resolved Hide resolved

Dreamsorcerer added 3 commits May 25, 2025 17:32

Readd shortcut

19dfe3a

Merge branch 'clientrequest-refactor' of github.com:aio-libs/aiohttp …

580387b

…into clientrequest-refactor

Tweaks

0cdf184

Dreamsorcerer mentioned this pull request May 26, 2025

Move ClientResponse to top of file #11025

Merged

Dreamsorcerer and others added 16 commits October 15, 2025 15:06

Update test_proxy.py

54e9ee7

Merge branch 'master' into clientrequest-refactor

89f6de2

[pre-commit.ci] auto fixes from pre-commit.com hooks

4c8df69

for more information, see https://pre-commit.ci

Fixes

57e84cc

Fixes

861fa1a

Fixes

4895d27

[pre-commit.ci] auto fixes from pre-commit.com hooks

3bb91ca

for more information, see https://pre-commit.ci

Fixes

e929ff5

Merge branch 'clientrequest-refactor' of github.com:aio-libs/aiohttp …

6145b46

…into clientrequest-refactor

Fixes

157be4d

Fixes

1d542e6

Fixes

6855d0a

[pre-commit.ci] auto fixes from pre-commit.com hooks

61f9aa7

for more information, see https://pre-commit.ci

Fixes

cb9ec6e

Fixes

f1549ea

Create 11012.breaking.rst

c392b49

Dreamsorcerer requested a review from webknjaz as a code owner October 15, 2025 17:25

psf-chronographer bot added the bot:chronographer:provided There is a change note present in this PR label Oct 15, 2025

bdraco reviewed Oct 18, 2025

View reviewed changes

aiohttp/client_reqrep.py Outdated Show resolved Hide resolved

bdraco reviewed Oct 18, 2025

View reviewed changes

aiohttp/client_reqrep.py Outdated Show resolved Hide resolved

solve one perf regression

f19ab2a

bdraco added 2 commits October 18, 2025 09:00

build request info once

9eceeef

lazy construct RequestInfo

238096f

Dreamsorcerer commented Oct 19, 2025

View reviewed changes

CHANGES/11012.breaking.rst Outdated Show resolved Hide resolved

Dreamsorcerer added 2 commits October 19, 2025 14:23

Update CHANGES/11012.breaking.rst

5780c0a

str annotations

fec5fef

Uh oh!

Refactor ClientRequest #11012

Are you sure you want to change the base?

Refactor ClientRequest #11012

Conversation

Dreamsorcerer commented May 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Goals

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov bot commented May 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

codspeed-hq bot commented May 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CodSpeed Performance Report

Merging #11012 will not alter performance

Summary

Uh oh!

Uh oh!

Uh oh!

bdraco commented May 26, 2025

Uh oh!

Uh oh!

Uh oh!

bdraco commented Oct 18, 2025

Uh oh!

bdraco commented Oct 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Dreamsorcerer Oct 19, 2025

Choose a reason for hiding this comment

Uh oh!

Dreamsorcerer Oct 19, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Dreamsorcerer commented May 25, 2025 •

edited

Loading

codecov bot commented May 25, 2025 •

edited

Loading

codspeed-hq bot commented May 25, 2025 •

edited

Loading

bdraco commented Oct 18, 2025 •

edited

Loading