psp-10487 address major memory useage outliers. #4880
Conversation
| @@ -7,6 +7,8 @@ | |||
| <GenerateDocumentationFile>true</GenerateDocumentationFile> | |||
| <ProjectGuid>{16BC0468-78F6-4C91-87DA-7403C919E646}</ProjectGuid> | |||
| <TargetFramework>net8.0</TargetFramework> | |||
| <ServerGarbageCollection>true</ServerGarbageCollection> | |||
There was a problem hiding this comment.
The default GC setting is actually "workstation" which doesn't seem appropriate for our case. See here:
https://learn.microsoft.com/en-us/dotnet/standard/garbage-collection/workstation-server-gc
| @@ -257,8 +257,9 @@ public async Task<ExternalResponse<DocumentDetailModel>> TryUploadDocumentAsync( | |||
| string authenticationToken = await _authRepository.GetTokenAsync(); | |||
|
|
|||
| byte[] fileData; | |||
| using var byteReader = new BinaryReader(file.OpenReadStream()); | |||
| fileData = byteReader.ReadBytes((int)file.OpenReadStream().Length); | |||
There was a problem hiding this comment.
This opens an extra (untracked) readstream just to get the stream length.
| @@ -48,13 +48,13 @@ public async Task Invoke(HttpContext context) | |||
| { | |||
| context.Request.EnableBuffering(); | |||
| await using var requestStream = _recyclableMemoryStreamManager.GetStream(); | |||
| await context.Request.Body.CopyToAsync(requestStream); | |||
There was a problem hiding this comment.
The copy was happening before the max length check - as a result the entire copy operation was being persisted to memory - despite the using call (possibly due to the recycleable memory stream manager). This increased the size of the heap by the largest document uploaded.
| string body = null; | ||
| if (reader.BaseStream.Length < maxStreamLength) | ||
| if (context.Response.ContentLength < maxStreamLength) |
There was a problem hiding this comment.
similar story here - larger responses end up persisting in memory after the request is handled in these logging processes.
|
|
See CodeCov Report Here: https://app.codecov.io/github/bcgov/psp/pull/4880 |
No description provided.