-
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Enable log buffering #4061
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
One way to test buffering is working as intended would be to enable log-level=info for the loadtests, in that case throughput should be more or less maintained. |
steve-chavez
added a commit
to steve-chavez/postgrest
that referenced
this issue
May 5, 2025
Benefits: - Removes `unsafePerformIO` from Logger - Reuses our cached time getter (`stateGetZTime`) instead of having another unnecessary one. See https://github.com/yesodweb/wai//blob/d50e1184600631a114e3d2ad119abdf0ef08834b/wai-extra/Network/Wai/Middleware/RequestLogger/Internal.hs#L22-L37. - Reuses Observation module, simplified logic - Will help with log buffering PostgREST#4061 TODO: - [ ] Currently we vendor `apacheLogStr`, but might be possible to expose this function upstream. If not, it's not a big problem since the function is small. - [ ] Time in logging differs a bit: ``` Old: postgrest_test_anonymous [04/May/2025:14:43:06 -0500] New: postgrest_test_anonymous [2025-05-04 22:43:01.28098247 -05] ``` + Otherwise all logging works the same. - [ ] Auth error logging relies on old logger middleware + We might need to put `observer` in Auth.hs + Or otherwise try to combine this with PostgREST#4059
3 tasks
steve-chavez
added a commit
to steve-chavez/postgrest
that referenced
this issue
May 5, 2025
Benefits: - Removes `unsafePerformIO` from Logger - Reuses our cached time getter (`stateGetZTime`) instead of having another unnecessary one. See https://github.com/yesodweb/wai//blob/d50e1184600631a114e3d2ad119abdf0ef08834b/wai-extra/Network/Wai/Middleware/RequestLogger/Internal.hs#L22-L37. - Reuses Observation module, simplified logic - Will help with log buffering PostgREST#4061 TODO: - [ ] Currently we vendor `apacheLogStr`, but might be possible to expose this function upstream. If not, it's not a big problem since the function is small. - [ ] Time in logging differs a bit: ``` Old: postgrest_test_anonymous [04/May/2025:14:43:06 -0500] New: postgrest_test_anonymous [2025-05-04 22:43:01.28098247 -05] ``` + Otherwise all logging works the same. - [ ] Auth error logging relies on old logger middleware + We might need to put `observer` in Auth.hs + Or otherwise try to combine this with PostgREST#4059
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Problem
Currently there's no log buffering, as noted in docs. This means that log levels higher than error cannot be enabled without reduced RPS.
Additionally, this might also cause other issues with the default log-level=error. For example on pool timeouts, the RPS will also suffer.
Solution
Enable log buffering, currently it's all set to line buffering on:
postgrest/main/Main.hs
Lines 15 to 21 in bf79766
This concern should be centralized on the
Logger.hs
module and possibly receive a config option as input.Notes
The logger middleware obfuscates this logic. Ideally we should reuse our Observation module as mentioned on:
postgrest/src/PostgREST/Logger.hs
Lines 58 to 70 in bf79766
The text was updated successfully, but these errors were encountered: