-
-
Notifications
You must be signed in to change notification settings - Fork 619
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(helper/proxy): introduce proxy helper #3589
base: main
Are you sure you want to change the base?
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #3589 +/- ##
==========================================
+ Coverage 94.71% 94.73% +0.01%
==========================================
Files 158 159 +1
Lines 9557 9587 +30
Branches 2826 2834 +8
==========================================
+ Hits 9052 9082 +30
Misses 505 505 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
all looks good to me except regarding the handling of range request and deletion of the Content-Length
header from the response
src/helper/proxy/index.ts
Outdated
* * Content-Length | ||
* * Content-Range | ||
*/ | ||
const forceDeleteResponseHeaderNames = ['Content-Encoding', 'Content-Length', 'Content-Range'] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i'm not sure about Content-Length
and Content-Range
, as if the original request is for a byte range, the semantics is not preserved. a probable semantically correct implementation:
- if the original request contains
Range
header, fast-fail with 400 status code - strip
Accept-Range
from response header - indicate upstream error if response header contain
Content-Range
Content-Length
should be left alone in all case IMO
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you! This certainly needs fixing.
Content-Length
As you can see from the results below, in the case of a compressed response, Content-Length is the “size after compression”, so it cannot be returned as is.
% curl -I 'https://example.com/'
HTTP/2 200
accept-ranges: bytes
age: 526496
cache-control: max-age=604800
content-type: text/html; charset=UTF-8
date: Wed, 30 Oct 2024 21:13:42 GMT
etag: "3147526947"
expires: Wed, 06 Nov 2024 21:13:42 GMT
last-modified: Thu, 17 Oct 2019 07:18:26 GMT
server: ECAcc (lac/55F5)
x-cache: HIT
content-length: 1256
% curl --compressed --raw -i 'https://example.com/'
HTTP/2 200
content-encoding: gzip
age: 527039
cache-control: max-age=604800
content-type: text/html; charset=UTF-8
date: Wed, 30 Oct 2024 21:07:12 GMT
etag: "3147526947+gzip"
expires: Wed, 06 Nov 2024 21:07:12 GMT
last-modified: Thu, 17 Oct 2019 07:18:26 GMT
server: ECAcc (lac/55AA)
vary: Accept-Encoding
x-cache: HIT
content-length: 648
- Delete <- This is the current implementation
- Load the body and reset it <- There is an overhead because you have to clone() and load all the data
Some people may expect the latter, but since hono does not actively add Content-Length to other requests either, I think this is a reasonable response for hono.
However, if there is no Content-Encoding
in the first place, there is no need to delete the Content-Length
. I think this should be improved.
Content-Range
I think the current implementation is incorrect.
It seems that this will not change depending on Content-Encoding
(although in a real environment, it seems that the result of compression will almost never be returned in a request with Range), so I think it is correct to always return it as is.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Content-Length is the “size after compression”, so it cannot be returned as is.
Oh, in that case I am in favor of unconditionally deleting it.
It seems that this will not change depending on Content-Encoding (although in a real environment, it seems that the result of compression will almost never be returned in a request with Range), so I think it is correct to always return it as is.
I have no reason to believe the new implementation is incorrect and this seems to be a suitable strategy.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you!
Co-authored-by: Haochen M. Kotoi-Xie <[email protected]>
…ponse is uncompressed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me.
Hey @usualoma ! Thank you for the PR. I'll try to use this and comment later. |
@usualoma @yusukebe many thanks for your amazing works again! just fyi we ended up using a pre/process combo like the following to mitigate
we are only using this particular proxying method in dev server, so there might be other catches.. const response = await fetch(
new Request(target, {
body: c.req.raw.body,
method: c.req.raw.method,
headers: [...c.req.raw.headers.entries()].filter(
([k]) => !["connection", "accept-encoding"].includes(k.toLocaleLowerCase()),
),
}),
);
return new Response(response.body, {
...response,
status: response.status,
statusText: response.statusText,
headers: [...response.headers.entries()].filter(
([k]) =>
!["content-encoding", "transfer-encoding", "content-length"].includes(
k.toLocaleLowerCase(),
),
),
}); |
also, on node v22.11.0, it seems that |
@haochenx Thank you! Note: We probably need to delete “Hop-by-hop Headers." |
Looking forward to this one, we use |
@john-griffin Thanks for the information! We can refer to it. |
fixes #3518
Naming
The name
proxyFetch
seems a little redundant, but I rejected the other candidates for the following reasons.proxy
: The nameproxy
is simple but is avoided because it is confusing with the JavaScriptProxy
object.fetch
: The namefetch
is also good. Although it is in thehelper/proxy
namespace, so it can be distinguished, when it is used by being incorporated into the application, from the standpoint of reading the code, it looks likeglobalThis.fetch
is being called, so I decided to avoid it because of the cognitive load.Usage
The author should do the following, if applicable
bun run format:fix && bun run lint:fix
to format the code