Enable HTTP response compression for API and static assets

[lcharlick]

Brief summary

This introduces response compression using Express’s compression middleware.

Which issue is fixed?

N/A

In-depth Description

A response is only compressed if:

• Client sends Accept-Encoding: gzip (or br, or both)
• Response type is compressible (e.g. application/json, not image/jpeg)
• Response size is > 1kb (this is the library default, but I thought it better to be explicit)

Note that this also compresses static js/etc files served for the web app, not just API requests.

How have you tested this?

Manual testing by directly hitting the API and navigating around the web/mobile apps.

Some unscientific before/after numbers:

UI Page Loads

View	Uncompressed	Compressed	Reduction
Home tab	2.15 MB	1.42 MB	34% smaller
Library tab	2.43 MB	1.74 MB	28% smaller

API Endpoints

Endpoint	Page Size	Uncompressed	Compressed	Reduction
/api/libraries/<id>/items	50	130 KB	41 KB	68% smaller
/api/items/<id>	-	22 KB	3 KB	86% smaller

Screenshots

N/A

[Vito0912]

Is there any measurable speed or added resource consumption with bigger servers?
So can you please add loading times to the table to tell the whole truth (please also limit the CPUs that abs can take to simulate a low powered server as many run ABS on things like Raspberry PIs)

While I think the compression is awesome, if this adds measurable resource to the server I don't think this should be merged. Big libraries already suffer performance issues, saving a few mb (for an media server) won't make the difference. The 1kb threshold will be hit with nearly any request ABS does

Due to speed e.g. compression for downloads was already removed

But in general awesome idea

[nichwall]

Is this better than just enabling compression at the reverse proxy level? That will apply to all paths and other services running on a server.

[lcharlick]

Is there any measurable speed or added resource consumption with bigger servers? So can you please add loading times to the table to tell the whole truth (please also limit the CPUs that abs can take to simulate a low powered server as many run ABS on things like Raspberry PIs)
....

In terms of latency hitting the API it’s within noise levels (+/- 3ms per request on my potato NAS). Actual resources used may be tough to measure, unless there’s a node.js profiler we can use (I’m unfamiliar with the ecosystem, and a quick search brought up nothing).

It’s likely a trade-off. More resources spent compressing the response, but fewer sending it. The savings aren’t negligible though.

Is this better than just enabling compression at the reverse proxy level? That will apply to all paths and other services running on a server.

Same result, but not everyone runs a reverse proxy. I just use Tailscale, for example.

[Vito0912]

The savings aren’t negligible though.

If one uses an RP, they are (assuming the connection between the RP and ABS is not bottlenecked).
Depending on the RP and how it handles the headers, there might be two compressions applied (or did I miss something? I never really checked that). Edit: Ofc it won't do anything, but it would be not needed added latency again.

But I also see the use case in local networks when using a VPN where this could be helpful. ABS has a large payload problem, and with some third-party clients "caching" the whole library (items), this could significantly reduce traffic during setup, especially for apps that use the /me endpoint a lot.

I know too little about how this interacts with an existing RP, but if it only compresses the payload without an RP, I think it could be worth it. If it also compresses with an RP, I think it's not worth it. It's not much added latency (but we should not add things that are not broken, but fix the root cause, the data model), but the only real use case is a VPN.

[nichwall]

This probably needs more testing before it can be added if it also applies to any media streaming or downloading. There was an issue that looks to be caused by reverse proxy adding compression to downloads as well, but that probably needs some more investigation before saying this will cause the same problem. advplyr/audiobookshelf-app#1553
(edited for clarity)

[lcharlick]

I know too little about how this interacts with an existing RP, but if it only compresses the payload without an RP, I think it could be worth it. If it also compresses with an RP, I think it's not worth it.

It will still compress the response if the app is behind a RP. The RP won't (shouldn't?) attempt to compress it a second time, though. From the nginx docs, for example:

NGINX performs compression before sending responses to clients, but does not “double compress” responses that are already compressed (for example, by a proxied server).

---

This probably needs more testing before it can be added if it also applies to any media streaming or downloading

We should be good there, the middleware won't attempt to compress any non-text content. It uses compressible behind the scenes.