Skip to content

optimization ConstantFolding memory consumption with periodic cleanup #31545

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

evkotov
Copy link
Contributor

@evkotov evkotov commented Jul 31, 2025

Details:

OpenVINO ConstantFolding transformation is consuming a lot of memory during model compilation, particularly for large models like GPT2. Memory profiling revealed that ConstantFolding alone consumed ~500MB during transformation pipeline,
contributing to peak memory usage of over 3GB.

Memory profiling identified that ConstantFolding creates large intermediate constant nodes. These nodes are freed by the transformation logic but memory is not immediately returned to the OS due to glibc malloc
behavior - freed memory is kept in internal pools for potential reuse.

  • Original ConstantFolding: ~500MB consumption (418.5MB + 81.93MB in major calls)
  • Peak memory without ConstantFolding: 1897.7MB
  • Peak memory with ConstantFolding: 3030.9MB
  • Memory pressure caused system-level performance degradation

Implemented periodic memory cleanup in ConstantFolding transformation using malloc_trim() to force return of unused memory to OS.

Tickets:

@evkotov evkotov requested a review from a team as a code owner July 31, 2025 12:05
@evkotov evkotov requested review from itikhono and removed request for a team July 31, 2025 12:05
@github-actions github-actions bot added category: Core OpenVINO Core (aka ngraph) category: transformations OpenVINO Runtime library - Transformations category: CPP API OpenVINO CPP API bindings labels Jul 31, 2025
@rkazants rkazants requested a review from CuriousPanCake July 31, 2025 12:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: Core OpenVINO Core (aka ngraph) category: CPP API OpenVINO CPP API bindings category: transformations OpenVINO Runtime library - Transformations do_not_merge do_not_review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant