How to use custom build of clang (and modules)

[githubuser6000]

Hi all. I have a custom build of clang that I'd like to use with emcc. I set EMCC_LLVM_ROOT to the build directory of clang I compiled from source and emcc --check seems to return OK.

Now I am trying to use a testing version of clang compiler with C++ modules support. How do I continue using those features in my code but build a wasm target?

I seemingly am running into a bunch of compiler errors due to missing headers with a simple C++ module project.

[sbc100]

The version of clang that comes with emscripten/emsdk is basically tip-of-tree so you should not need to use a custom version just to get module support (i.e. the version of clang we ship should have all the latest module support).

However, emscripten itself still doesn't have integrated module support. See #21987 for example.

However if you do want to use your down version of clang when EM_LLVM_ROOT should certainly work. (Note that its EM_LLVM_ROOT not EMCC_LLVM_ROOT). Can you share exactly what errors you are seeing?

[sbc100]

If you really want to this bleeding edge C++23 support you could try updating the copy of libc++ in emscripten. We have a script for doing this called ./system/lib/update_libcxx.py which we use, but its not really designed for end users. You could also try just dropping in the missing header file into system/lib/libcxx. Once you have updated you can run ./embuilder build sysroot libcxx or you can run ./emcc --clear-cache to clear the cache and rebuild everything.

[githubuser6000]

I was able to compile things finally 😭. Thank you for all the help! This is so cool! 🔥

This is probably emscripten-glfw related, but felt like a emscripten-glue issue too. If not, please do let me know and I'll ask this over at emscripten-glfw repo. I am trying to build in "64-bit mode", so added -sMEMORY64 to my project (both compile and link flags), and manually edited a local copy of emscripten-glfw to add these flags to its build. Everything builds fine. I load it up in the browser with a python -m http.server, I get this

I couldn't find much info about this. I found this, but didn't seem related. I tried the -sWASM_BIGINT but that didn't seem to help either.

[sbc100]

It looks like maybe emscripten-glfw is not yet setup for -sMEMORY64. To enable that it would need to have __sig attributes in the library .js file. Maybe file an issue there regarding memory64 support.

[githubuser6000]

Just in case it helps anyone, I was not able to get ./system/lib/update_libcxx.py working with the latest libcxx, so I removed <flat_map> from the std.cppm file and kept adding / removing things till things started to work.

As for emscripten-glfw, I can confirm that window creation works when I don't use memory64. I created an issue on their repo. In the meantime, I found some threads discussing memory64 and pointer stuff and came across gen_sig_info.py, and the sigs file that seems to have sigs for the glfw that is bundled with emscripten. Can I reuse that file or use the script to generate a siginfo file that I can somehow add to the build process?

Another question: Is there a downside (performance or otherwise) to using MEMORY64? I thought it would be easier to do that than change my code to be "32-bit" tolerant.

[sbc100]

Regarding gen_sig_info.py, currently that only works for builtin libraries. For external libraries you still need to add/maintain the __sig attributes yourself (manually).

Regarding the overheads of memory64, yes there are some. Just like on native platforms switching to 64-bit has performance and memory cost due to increased sizes of your data structures. This used more memory and negatively effects cache locality. IIUC this is normally just a few percent slowdown. With WebAssembly there is also the additional cost of using memory64 which is that some of the tricks that are used to avoid bounds checks do not work with 64-bit memories, which will have a performance impact.. IIUC this is normally in the 5-10% range.

[ypujante]

@sbc100 Hi Sam. I was looking into this issue, and it seems that the only thing I need to do is to manually add a __sig entry to all the C functions I am implementing, something like this:

void emscripten_glfw3_context_init(emscripten::glfw3::Context *iContext,
                                   float iScale,
                                   ScaleChangeCallback, WindowResizeCallback, KeyboardCallback, ClipboardStringCallback, RequestFullscreen, ErrorHandler);

  emscripten_glfw3_context_init__deps: ['$specialHTMLTargets'],
  emscripten_glfw3_context_init__proxy: 'sync',
  emscripten_glfw3_context_init__sig: 'vpfpppppp', // <- add this?
  emscripten_glfw3_context_init: (context, scale, scaleChangeCallback, windowResizeCallback, keyboardCallback, clipboardCallback, requestFullscreen, errorHandler) => {

I assume I probably also need to add entries for all new glfwXXX API call that have been added in GLFW 4.0 (since the glfw3.h bundled with Emscripten is 3.3.8 so I assume the script only create signatures for these)?

Do I need the port file to also do something special with the settings.MEMORY64 setting? I checked all other ports and none seem to pass this setting along when compiling the port itself. Is it taken care of by Emscripten automatically? (for example the pthread settings is passed along...)

[sbc100]

Yes, any function that takes or returns a pointer value (or a size_t/intptr_t) value needs to have a __sig.

You can check src/lib/libsigs.js to see all the existing signatures that ship with emscripten. If a signature already exists there there is no need to add one.

No need to handle MEMORY64 in the port no.

[ypujante]

I have created PR that implements the changes discussed to support wasm64