EntityModel.ConvertToOpenApi method memory usage

[garrypopplewell]

When using IEdmModel.ConvertToOpenApi - some EDMX documents appear to trigger massive memory usage on some occasions it never seems to finish, I've left it for upwards of 30 minutes. I am trying to generate documentation for Microsoft Dynamics Business Central API's.

Assemblies affected

This is a C# WPF .NET7.0 app

• Microsoft.OpenApi V1.6.4
• Microsoft.OpenApi.OData V1.4.0
• Microsoft.OpenApi.Readers V1.6.4

Microsoft.OpenApi.OData.EdmModelOpenApiExtensions.ConvertToOpenApi()

Steps to reproduce

the below code in conjunction with the example EDMX examples replicates the issue

var EdmData = System.IO.File.ReadAllText(ExampleEdmxFilePath);
IEdmModel model = CsdlReader.Parse(XElement.Parse(EdmData).CreateReader());
var document = EntityModel.ConvertToOpenApi();

Expected result

The OpenApiDocument is created quickly and with reasonable memory usage

Actual result

The OpenApiDocument is not created/created very slowly and uses massive amount of memory

Additional detail

From the attached zip file

• EDMX 1 - this is a collection of ~15 custom API's - the OpenApiDocument takes around 5 minutes to be created, if I export this to a json OpenApiv3 document the resulting file is 3.5Gb and it generates ~22,000 paths, it's uses ~9Gb of memory according to Visual Studio resource profiler. If I import the OpenApiV3 Json document into Swagger it is not recognised so I suspect the 3.5Gb is just hitting some form of limit.
• EDMX 2 - a separate set of 20 or so custom api's, appears to work correctly, the OpenApiDocument takes a few seconds to generate and uses very little memory
• EDMX 3 - the collection of all standard Microsoft Dynamics Business Central API V2 API's, I've never got this to finish, I've left it for over 30 minutes and it's used up to 20Gb of memory.

EDMX examples.zip

[garrypopplewell]

Morning, is there any update on this? Do you need more information?

[irvinesunday]

Hey @garrypopplewell can you try to set the convert setting ExpandDerivedTypesNavigationProperties = false and revert on the progress?

[irvinesunday]

You can also disable EnableDollarCountPath by setting it to false. Currently, with EDMX 1 the number of paths generated is 22,383. Disabling that setting, the paths reduce to 16,794. One factor that may contribute to the high number of paths could be tied to the containment of some navigation properties (ContainsTarget="true"). This leads to over-expansion of navigation properties with some paths having 20+ segments. You may choose to identify which ones are invalid and remove them.

[garrypopplewell]

Hi Irvine, thanks for the suggestion, that doesn't seem to make any difference unfortunately.

Is there a way to strip out unwanted paths from the EDM model? the closest i've found is the below...
IODataPathProvider pathProvider = new ODataPathProvider(); var modelPaths = pathProvider.GetPaths(SourceEdmModel, convertSettings).Where(p => p.x = x)
but this doesn't allow me to use the ConvertToOpenApi method, it just allows me to parse the paths for myself and manually create my own API Model... which is difficult with all the internal classes in the Csdl namespace.

If you could show me a way to remove excess paths from the EDM model that would be perfect?

[irvinesunday]

Hi Irvine, thanks for the suggestion, that doesn't seem to make any difference unfortunately.

Is there a way to strip out unwanted paths from the EDM model? the closest i've found is the below... IODataPathProvider pathProvider = new ODataPathProvider(); var modelPaths = pathProvider.GetPaths(SourceEdmModel, convertSettings).Where(p => p.x = x) but this doesn't allow me to use the ConvertToOpenApi method, it just allows me to parse the paths for myself and manually create my own API Model... which is difficult with all the internal classes in the Csdl namespace.

If you could show me a way to remove excess paths from the EDM model that would be perfect?

Apologies for the long overdue reply @garrypopplewell

I have generated a list of paths for the EDMX1 file, EDMX1_OpenAPI Paths. If you scroll to the bottom of the file you'll see the number of paths per API. And a secondary file, EDMX1_OpenAPI Paths (No nav prop paths) which is similar to the above but without navigation property paths. These files are in the zipped folder. This is a conversion setting that can be toggled via Settings.EnableNavigationPropertyPath. Setting this to false disables generation of navigation property paths. From the two files, you can clearly see that a lot of the generated paths are as a result of the navigation property paths. If you can inspect these paths and identify the ones that are not supposed to be generated/expanded, then you should be able to remove their containments. This will help reduce the number of paths generated.
OpenAPI Paths.zip