Code for function is too large、High memory usage

[crowforkotlin]

"Code for function is too large" error and 1.6GB Memory Spike during JIT compilation of a 781KB Kotlin/Wasm module

Environment:

• Wasmtime version: v40.0.0 (Release Release Wasmtime 40.0.0 #12192)
• Host Machine (Build): Mac M2 (aarch64)
• Target Device (Runtime): Android (aarch64)
• Compiler: Kotlin/Wasm (WASI) via compileProductionExecutableKotlinWasmWasiOptimize
• Project Link: crowforkotlin/wasmline
• Source File: Plugin.kt

Description:
I am reporting a Compilation error: Code for function is too large that occurs during the JIT loading phase on Android.

Despite the generated .wasm binary being only 781.63 KB, the compilation fails when a single Kotlin function contains a large amount of logic/content. During the attempt to compile this module, I observed a massive RSS memory spike on the Android device, jumping from ~235 MB to over 1.6 GB in just 4 seconds, eventually resulting in the "function too large" error.

Wasmtime Configuration:
The engine is configured specifically for Android compatibility to avoid signal conflicts with ART and minimize VSS overhead:

/**
 * Creates and configures the Wasm Config object.
 *
 * Critical Android Settings:
 * 1. Signals-based traps DISABLED: To prevent conflicts with Android ART signal handlers (SIGSEGV).
 * 2. Memory Guard Size = 0: To prevent VSS (Virtual Set Size) OOM on 32-bit or limited devices.
 * 3. GC / Exceptions: Enabled for Kotlin/Wasm support.
 */
wasm_config_t *Engine::createConfig() {
    wasm_config_t *conf = wasm_config_new();

    // Feature Flags for Kotlin/Wasm support
    wasmtime_config_wasm_gc_set(conf, true);
    wasmtime_config_wasm_function_references_set(conf, true);
    wasmtime_config_wasm_exceptions_set(conf, true);

    // Optimization: Disable SIMD if not strictly needed (improves compatibility)
    wasmtime_config_wasm_simd_set(conf, false);
    wasmtime_config_wasm_relaxed_simd_set(conf, false);

    // [CRITICAL] Disable signal handlers to avoid crash conflicts with Android Runtime (ART)
    wasmtime_config_signals_based_traps_set(conf, false);

    // [CRITICAL] Set guard pages to 0 to minimize Virtual Memory usage (prevents OOM)
    wasmtime_config_memory_guard_size_set(conf, 0);

    // Set max stack size (512KB is usually sufficient for mobile logic)
    wasmtime_config_max_wasm_stack_set(conf, 512 * 1024);

    // Compiler Optimization Strategy: Optimize for Speed and Binary Size
    wasmtime_config_cranelift_opt_level_set(conf, WASMTIME_OPT_LEVEL_SPEED_AND_SIZE);
    wasmtime_config_cranelift_debug_verifier_set(conf, false);

    return conf;
}

Reproduction Steps:

1. Add a large amount of content/logic into the wasmWasiMain function in Plugin.kt.
2. Build the wasm file: ./gradlew wasmtime-core:plugin:compileProductionExecutableKotlinWasmWasiOptimize.
3. The resulting plugin.wasm is ~781 KB.
4. Load the module on an Android device using the configuration above.

Observed Behavior:

• Total Wasm Size: 781.63 KB (Small)
• Memory Usage: Spikes from 234MB to 1.6GB.
• Result: Compilation error: Code for function is too large.

Full Android Logs:

00:49:34.204  I  [MotionEvent] ViewRootImpl windowName 'crow.wasmtime.wasmline/crow.mordecai.wasmline.MainActivity', { action=ACTION_DOWN, id[0]=0, pointerCount=1, eventTime=51746485, downTime=51746485, phoneEventTime=00:49:34.191 } moveCount:0
00:49:34.206  D  getMiuiFreeformStackInfo mTmpFrames.miuiFreeFormStackInfo: null
00:49:34.217  I  CreateGraphicsPipeline pipeline cache hit. elpased time: 0.32 ms.
00:49:34.218  E  FrameInsert open fail: No such file or directory
00:49:34.230  I  This is non sticky GC, maxfree is 33554432 minfree is 8388608
00:49:34.227  W  type=1400 audit(0.0:2008818): avc:  denied  { getopt } for  path="/dev/socket/usap_pool_primary" scontext=u:r:untrusted_app:s0:c190,c257,c512,c768 tcontext=u:r:zygote:s0 tclass=unix_stream_socket permissive=0 app=crow.wasmtime.wasmline
00:49:34.322  I  [MotionEvent] ViewRootImpl windowName 'crow.wasmtime.wasmline/crow.mordecai.wasmline.MainActivity', { action=ACTION_UP, id[0]=0, pointerCount=1, eventTime=51746611, downTime=51746485, phoneEventTime=00:49:34.316 } moveCount:0
00:49:34.332  I  ==============================================
00:49:34.337  I  [Android] Wasm file : plugin.wasm    ||    wasm file exits : true    ||    wasm file path :  /data/user/0/crow.wasmtime.wasmline/cache/plugin.wasm
00:49:34.338  I  [Android] Cwasm cache file : plugin.cwasm    ||    cache file exits : false    ||    cache file path :  /data/user/0/crow.wasmtime.wasmline/cache/plugin.cwasm
00:49:34.345  I  [Wasmtime] Module --> Jit Compiling for /data/user/0/crow.wasmtime.wasmline/cache/plugin.wasm...
00:49:34.363  W  type=1400 audit(0.0:2008819): avc:  denied  { read } for  name="cpu.cfs_quota_us" dev="cgroup" ino=64 scontext=u:r:untrusted_app:s0:c190,c257,c512,c768 tcontext=u:object_r:cgroup:s0 tclass=file permissive=0 app=crow.wasmtime.wasmline
00:49:34.363  W  type=1400 audit(0.0:2008820): avc:  denied  { read } for  name="cpu.cfs_period_us" dev="cgroup" ino=65 scontext=u:r:untrusted_app:s0:c190,c257,c512,c768 tcontext=u:object_r:cgroup:s0 tclass=file permissive=0 app=crow.wasmtime.wasmline
00:49:34.363  W  type=1400 audit(0.0:2008821): avc:  denied  { read } for  name="cpu.cfs_quota_us" dev="cgroup" ino=10 scontext=u:r:untrusted_app:s0:c190,c257,c512,c768 tcontext=u:object_r:cgroup:s0 tclass=file permissive=0 app=crow.wasmtime.wasmline
00:49:34.363  W  type=1400 audit(0.0:2008822): avc:  denied  { read } for  name="cpu.cfs_period_us" dev="cgroup" ino=11 scontext=u:r:untrusted_app:s0:c190,c257,c512,c768 tcontext=u:object_r:cgroup:s0 tclass=file permissive=0 app=crow.wasmtime.wasmline
00:49:34.441  E  ->pid:19641,processName:"crow.wasmtime.wasmline",Rss Memory Size Change 234952kb -> 342100kb
00:49:34.494  E  ->pid:19641,processName:"crow.wasmtime.wasmline",Rss Memory Size Change 342100kb -> 428844kb
00:49:34.544  E  ->pid:19641,processName:"crow.wasmtime.wasmline",Rss Memory Size Change 428844kb -> 541612kb
00:49:34.602  E  ->pid:19641,processName:"crow.wasmtime.wasmline",Rss Memory Size Change 541612kb -> 657240kb
00:49:34.928  E  ->pid:19641,processName:"crow.wasmtime.wasmline",Rss Memory Size Change 657240kb -> 860432kb
00:49:35.761  E  ->pid:19641,processName:"crow.wasmtime.wasmline",Rss Memory Size Change 860432kb -> 959324kb
00:49:36.012  E  ->pid:19641,processName:"crow.wasmtime.wasmline",Rss Memory Size Change 959324kb -> 1075892kb
00:49:36.184  E  ->pid:19641,processName:"crow.wasmtime.wasmline",Rss Memory Size Change 1075892kb -> 1183820kb
00:49:37.835  E  ->pid:19641,processName:"crow.wasmtime.wasmline",Rss Memory Size Change 1183820kb -> 1306888kb
00:49:38.482  E  ->pid:19641,processName:"crow.wasmtime.wasmline",Rss Memory Size Change 1306888kb -> 1450064kb
00:49:38.639  E  ->pid:19641,processName:"crow.wasmtime.wasmline",Rss Memory Size Change 1450064kb -> 1612484kb
00:49:38.700  E  ->pid:19641,processName:"crow.wasmtime.wasmline",Rss Memory Size Change 1612484kb -> 1389664kb
00:49:38.700  E  ->pid:19641,processName:"crow.wasmtime.wasmline",Rss Memory Size Change 1389664kb -> 1011816kb
00:49:38.702  E  [Wasmtime] Module --> Error loading module /data/user/0/crow.wasmtime.wasmline/cache/plugin.wasm: Compilation error: Code for function is too large
00:49:38.708  I  [Wasmline] Load failure, because native load return false, file path is :  /data/user/0/crow.wasmtime.wasmline/cache/plugin.wasm
00:49:38.790  E  ->pid:19641,processName:"crow.wasmtime.wasmline",Rss Memory Size Change 1011816kb -> 727648kb

Artifacts:
Wasmline-wasmline-core-plugin.wasm.zip

• Wasm File Size (781.63 KB):
  
• Error Detail:
  
• Success state (after reducing content):
  

Analysis & Questions:

1. Why does a relatively small Wasm file (under 1MB) trigger a "Code too large" error? It seems the Kotlin/Wasm compiler is generating a single function with extremely high complexity.
2. The memory usage spike to 1.6GB suggests that Cranelift might be hitting a worst-case scenario in its e-graph optimization or register allocation phase.
3. Are there any specific Cranelift settings to mitigate this, or is this a fundamental limit on function complexity regardless of the total .wasm file size?

[crowforkotlin]

#11682 @cfallin

[crowforkotlin]

Wasmline-wasmline-core-plugin.wasm.zip

./wasmtime compile Wasmline-wasmline-core-plugin.wasm -o plugin.cwasm \
    --target aarch64-linux-android \
    -W gc=y \
    -W function-references=y \
    -W exceptions=y \
    -W simd=n \
    -W relaxed-simd=n \
    -O static-memory-guard-size=0 \
    -O dynamic-memory-guard-size=0 \
    -O signals-based-traps=n \
    -O opt-level=2 \
    -C cranelift-debug-verifier=no

[crowforkotlin]

wasm.zip

[cfallin]

Just an FYI -- I am on PTO and then at a conference, so I will be able to investigate this starting Mon, Jan 12. (Thanks for the report!)

[cfallin]

Actually, this was bothering me and I at least wanted to verify, so I double-checked with your provided Wasm module and the command

./wasmtime compile Wasmline-wasmline-core-plugin.wasm -o plugin.cwasm \
    --target aarch64-linux-android \
    -W gc=y \
    -W function-references=y \
    -W exceptions=y \
    -W simd=n \
    -W relaxed-simd=n \
    -O static-memory-guard-size=0 \
    -O dynamic-memory-guard-size=0 \
    -O signals-based-traps=n \
    -O opt-level=2 \
    -C cranelift-debug-verifier=no

However, both with a wasmtime built from main (b5272a5) and from just before the 40.0 branch was cut (ac3358b), running on macOS/aarch64 as you specify, I see the command complete just fine with minimal time taken, and definitely no "Code for function is too large" error -- e.g. from time,

target/release/wasmtime compile Wasmline-wasmline-core-plugin.wasm -o    -W    0.38s user 0.03s system 328% cpu 0.126 total

Am I missing some setting? I'm inclined to close as "cannot reproduce" if not but want to make sure I'm running exactly what you are.

[crowforkotlin]

Actually, this was bothering me and I at least wanted to verify, so I double-checked with your provided Wasm module and the command实际上，这让我感到困扰，至少我想验证一下，所以我用你提供的 Wasm 模块和命令进行了双重检查。实际上，这让我感到困扰，至少我想验证一下，所以我用你提供的 Wasm 模块和命令进行了双重检查。

./wasmtime compile Wasmline-wasmline-core-plugin.wasm -o plugin.cwasm \
    --target aarch64-linux-android \
    -W gc=y \
    -W function-references=y \
    -W exceptions=y \
    -W simd=n \
    -W relaxed-simd=n \
    -O static-memory-guard-size=0 \
    -O dynamic-memory-guard-size=0 \
    -O signals-based-traps=n \
    -O opt-level=2 \
    -C cranelift-debug-verifier=no

However, both with a wasmtime built from main (b5272a5) and from just before the 40.0 branch was cut (ac3358b), running on macOS/aarch64 as you specify, I see the command complete just fine with minimal time taken, and definitely no "Code for function is too large" error -- e.g. from time,然而，无论使用从 main ()构建的 wasmtime 还是从 40.0 分支之前构建的 wasmtime，在您指定的 macOS/aarch64 上运行时，命令都能正常完成，时间非常短，绝对没有“函数代码太长”的错误提示 - 例如，从 time。然而，无论使用从 main ()构建的 wasmtime 还是从 40.0 分支之前构建的 wasmtime，在您指定的 macOS/aarch64 上运行时，命令都能正常完成，时间非常短，绝对没有“函数代码太长”的错误提示 - 例如，从 time。

target/release/wasmtime compile Wasmline-wasmline-core-plugin.wasm -o    -W    0.38s user 0.03s system 328% cpu 0.126 total

Am I missing some setting? I'm inclined to close as "cannot reproduce" if not but want to make sure I'm running exactly what you are.我是不是漏掉了什么设置？如果不是的话，我倾向于关闭为“无法重现”，但还是想确保我正在运行的是你所要求的。我是不是漏掉了什么设置？如果不是的话，我倾向于关闭为“无法重现”，但还是想确保我正在运行的是你所要求的。

The file download was incorrect. You used a WASM file generated from a small amount of code; it was my oversight for sending it to the wrong location. While the CWASM generated from that file works fine, using the larger WASM file (700KB+) to generate a CWASM triggers an exception. I’m sending you the latest version now for you to check.文件下载不正确。您使用了一个来自少量代码生成的 WASM 文件；这是我的疏忽，将它发送到错误的位置。虽然从该文件生成的 CWASM 文件正常工作，但使用更大的 WASM 文件（700KB+）生成 CWASM 会触发异常。我现在将为您发送最新版本。

wasm.zip

The Kotlin/WASI build process uses Gradle tasks to generate WASM artifacts. The output ZIP includes 'kotlin' and 'optimize' folders, which seem to act as a size-optimization toggle similar to a release environment switch.[1] Yesterday, I inspected the WASM file with wasmtime-tools and discovered it contains over 22,000 global variables.[1] I realize this is not an ideal implementation, but I'm concerned that if a user implements extensive logic in the future, the resulting large file size might trigger exceptions during CWASM generation.

[cfallin]

Thanks for that. A few stats from your optimized/Wasmline-wasmline-core-plugin.wasm in your latest zip file, function 127 in the module:

• There are 4005519 (4 million) SSA values in the CLIF generated from that function body;
• There are 488081 (488k) basic blocks in that CLIF;
• There are 61796 Wasm bytecode operators in the function body;
• Those operators are distributed according to this histogram:

22433 global.get
16819 call
11238 i32.const
5612 global.set
5608 struct.new
  20 local.get
  12 struct.set
   9 end
   7 local.set
   7 drop
   6 ref.null
   6 ref.is_null
   6 local.tee
   6 if
   2 block
   1 unreachable
   1 try_table
   1 throw_ref
   1 br
   1 array.new_data

So, my thoughts:

• It's not surprising at all that feeding a CFG of 488k basic blocks into an optimizing compiler causes memory usage to grow, potentially to over a gigabyte. That's the nature of the beast: we build IR in memory and do passes over it. Just representing the IR itself, nevermind the analysis results, is going to take a large fraction of the usage you observed. Our optimizing compiler is not designed to have linear-time overhead (that would be impossible while still performing the kinds of optimizations we want to do), so any function this large is likely to blow up.
• struct.new in particular is expanding to a lot of IR, because of its inlined fast-path. I don't think that's the wrong choice though -- it makes sense to make allocation fast for the usual program structure that has maybe a few allocation sites per function, rather than try to optimize for code size for the uncommon case of a single function with 5608 struct.news (!).

The immediate issue is that lowering runs out of VRegs, for which we allow only 2M (2^21), because of bitpacking for efficiency. We could in theory look at increasing that. But at least we are returning a clean error ("function too large") rather than panic'ing.

On a broader scale, I don't think Cranelift (or any optimizing backend) is going to meet your requirements/expectations if that is the code that you need to compile. I would thus recommend either:

• Using Winch, once it supports exceptions and GC (it currently does not support either); or
• Modifying your codegen to not produce such a function, e.g. by splitting it up, using table-driven setup, using initializer expressions for the globals if the graph is acyclic, or something else.

cc @fitzgen for any thoughts on struct.new...

[crowforkotlin]

Interestingly, after refining my Kotlin/Wasm (WASI) implementation, a 700KB Wasm file successfully compiled into a 28MB CWASM via JIT without triggering the "Code for function is too large" error. I suspect the Kotlin/Wasm compiler might be performing aggressive inlining, potentially collapsing numerous function implementations into a single entry point (e.g., the main function). If this aggressive inlining could be disabled or tuned, the issue might be resolved. For context, I have successfully tested the build and execution on an Android app.
Additionally, I have a question regarding Wasmtime: is there a hard limit of 2 million virtual registers? I’m trying to understand if this issue stems from the generation of approximately 4 million (4M) virtual register values.
I am planning to report this to the Kotlin team with the relevant details. While I am not entirely certain about the underlying mechanism, the issue is consistently reproducible.

wasm-success-size28mb.zip

@cfallin