My notes on setting up axmol - prebuild engine , VSCode and LUA debugging

[grif-on]

"I don't like to memorise everything , but i like to have proof that i am doing everything the same way as did before . It makes it easier to narrowing down problems ." - Grif_on
Please keep in mind that this is just a one particular way , which i have only tested on Windows 10 with VSCode , axmol 2.1.3 and targeting x64 via the MSVC compiler(toolchain) .
Be sure to read https://github.com/axmolengine/axmol/blob/dev/docs/DevSetup.md if you need to adapt this guide to other system/IDE/axmolVersion/arch/toolchain .

🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦

Engine :

1.1
This step is optional if you are updating the engine .
Delete "build_x64" folder from engine folder .
Delete "CMakeCache.txt" (or entire build folder) from project folders .
Also , use something like WinMerge to ensure that common LUA scripts are up to date between engine and your project (compare ".\extensions\scripting\lua-bindings\script" and ".\Content\src\axmol") .

1.2
Follow the steps in the "Common requirements" and "Prerequisites" chapters of https://github.com/axmolengine/axmol/blob/dev/docs/DevSetup.md (i.e. install PowerShell , install CMake , run "setup.ps1" , install C++ toolchain) .

1.3

cmake -B build_x64 -A x64 -DAX_BUILD_TESTS=OFF

1.4

cmake --build build_x64 --config Debug

1.5
Optionally if you have time .

cmake --build build_x64 --config Release

1.6
Check resulting prebuilts folder ("build_x64") - it should be ~3 GB in size (in case of default state of extensions)

🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦

Project :

2.1
Init from template .
Even if you are here for LUA but new to axmol/cocos2dx , look at the C++ template first .

axmol new -p dev.company.project_name -d ./ -l cpp project_name

After that , if you need LUA create new project and stick to the LUA template .

axmol new -p dev.company.project_name -d ./ -l lua project_name

Note - for simplicity i use same name of project "project_name" in commands , don't forget to change it .

2.2
If you are using git , create a .gitignore

# tmp folder for temp usage
/tmp
/temp

# Ignore vscode folder
/.vscode

# Ignore the CMake build folder (windows)
/build
/build_*

# Ignore the CMake build folder (ios-build/linux-build)
/*-build

2.3
Specify build and prebuild folders

cmake -B build -A x64 "-DAX_PREBUILT_DIR=build_x64"

2.4
First build in the terminal

cmake --build build --config Debug --target project_name

Note - if the prebuilts are configured correctly , only a few files should be compiled (not the whole engine !) .

Subsequent builds can be the same as 2.4 or via VSCode .

🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦

VSCode setup of project :

3.1
Make sure that cpp & cmake extensions are enabled

3.2
(Crtl+Shift+p) ">Cmake: configure" (be sure to select amd64 or a similar x64 toolkit when prompted or manually with ">CMake: Select a Kit")
After that PROJECT OUTLINE (in cmake panel) should have 1 exe and +-4 utility targets .

Note - if you have used CMake in your system for something else , then CMake may keep it generator/toolchain preferences , you can tell this is a case if you see that after 2.4 your PROJECT STATUS in VSCode has something rather than "[No Kit Selected]"/"[N/A - Select Kit]"/your_toolkit_name in it and/or PROCEJCT OUTLINE is bloated with engine tasks . In this case , delete the build folder , open VSCode preferences and change "CMake: Generator" to "Visual Studio 17 2022" or your installed toolchain , run ">CMake: Reset Cmake" , then redo 2.3 , 2.4 and 3.2 .

3.3
Pray that you will not stuck on 3.2 with some new obscure error .

3.4
Use the Build/Debug/Run buttons at the bottom (or F5) .
VSCode build output should be similar to 2.4 , i.e. only Source folder files should be built (for HelloWorld LUA template it should be only 2 files - AppDelegate.cpp and main.cpp) , if not then redo 3.2 (make sure PROJECT OUTLINE is correct) .

3.5
Optionally if you have some VSCode extensions that simplify the use of tasks .
Create tasks.json .

{
  "version": "2.0.0",
  "tasks": [
      {
          "label": "Run already compiled",
          "type": "shell",
          "command": "cd .\\build\\bin\\project_name\\Debug\\ && .\\project_name.exe",
          "presentation": {
            // Do not show terminal if everything ok
            "reveal": "silent"
          },
          "problemMatcher": []
      }
  ]
}

🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦

Debugging LUA axmol project with EmmyLua in VSCode :

4.1
Download EmmyLua by Tangzx extension from VSCode extensions panel .

4.2
Ensure that compiled executable runs normally (via F5 or manually) .

4.3
Add require "emmy_core" as the first line of code in ".\Content\src\main.lua" .

4.4
Go to "%USERPROFILE%\.vscode\extensions\tangzx.emmylua-0.7.4-win32-x64\debugger\emmy\windows\x64" (or similar) and copy these files into the folder that contains your project executable file (or use the alternative way described in the note) .
Run your executable again . In the axmol console you should see something like "[EMMY] version: 1, lua version: 54" .

Note - EmmyLua extension also has a command ">EmmyLua: Insert Emmy Debugger Code" which can automatically generate 3 lines of code .
First line appends full path to aforementioned files into package.cpath , second is same as in 4.3 and third executes tcpListen() function which instead of helping just messes up debugging :> , so we don't need this line (VSCode is by itself capable of setting its own port to match game one) .
Both ways are fragile in case of extension update (for obvious reasons) .
Full path guarantees that you will know that the extension has been updated and you need to update the path .
But imo copying is better as it is more source control friendly (since in case of full path way , your user name in the source code will be a bad thing , especially if you are developing the game in a team) .

4.5
Now open the "Run and Debug" panel in VSCode , then click on "create a launch.json file" and select "EmmyLua Launch Debug" . This should open this file and create an emmylua_launch template in it .
Here is an example of how to fill it

{
  "version": "0.2.0",
  "configurations": [
    {
      "type": "emmylua_launch",
      "request": "launch",
      "name": "EmmyDBG",
      "program": "D:\\git\\axmol_projects\\project_name\\build\\bin\\project_name\\Debug\\project_name.exe",
      "workingDir": "D:\\git\\axmol_projects\\project_name\\build\\bin\\project_name\\Debug\\",
      "arguments": [],
      "newWindow": false
    }
  ]
}

Unfortunately , i haven't figured out how to make relative paths work .

4.6
Set a breakpoint on the line with "require "config"" in ".\Content\src\main.lua" .
Now , press the green button in the "Run and Debug" panel (NOT F5 , because F5 will run C++ compilation) .
You can also use the button at the bottom of VSCode (if VSCode shows this additional button , again - not cmake one) .
Thats it , you should see a working debugger , with execution stopped at the breakpoint .
Step buttons , variables , watch , callstack , breakpoints - all usable .

🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦

Some EmmyLua debugger drawbacks :

EmmyLua can't change variables values on the fly .
Breakpoints on empty lines are broken .
It also seems that it is hard to debug both C++ and LUA at the same time . VScode allows multiple debugging , but i haven't managed to attach emmyLua to C++ debugged game .
But in other aspects it is well working LUA debugger .

🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦

Some EmmyLua debugger tips and facts :

Despite VScode says that conditional breakpoints are not supported , they do work .
"Callstack" entries are clickable and affect the scope of "Variables" entries (by default debugger selects the most top one i.e. current line scope) .
The most bottom "require 'main'" in the callstack is a call from C++ code (if you didn't changed that in AppDelegate.cpp) .
Breakpoints set on function end keyword are triggered in two different situations - one time when the function is defined and subsequently when the function is called .
Instead of setting a breakpoint you can use rightClick then use "Run to Line" , which will set a temporary breakpoint and resume execution of the code .
"Watch" entries are interactive , you can add code (even engine functions) in them and it will be executed when possible (for example , you can have a function that pretty prints you some game data as a string , and you can get such a string any time via Watch) . However , you can crash the game by adding nodes to the axmol scene tree via "Watch" , so be careful .

🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦

Rebuild the engine to add your own LUA bindings :

When you decide that this is the time to automatically bind your own C++ functions/classes in to LUA , use ".\tools\ci\genbindings.ps1" between 1.2 and 1.3 .

Note - you sure can perform each of the steps described here "https://github.com/axmolengine/axmol/blob/dev/tools/tolua/README.md" . The manual/intended way will take up less space on your hard drive than "genbindings.ps1" . But an automatic CI tool that downloads and configures EVERYTHING possible that the engine needs is much more reliable than manual actions imo .
Results of the auto bindings will be in ".\extensions\scripting\lua-bindings\auto" .
I haven't tried to add my own bindings yet (just played around with the existing ones) , but i ASUME that before running "genbindings.ps1" you will need to add/change ini files in ".\tools\tolua" like "ax_base.ini" and add ini files names in to "genbindings.py" .
Also , as a good idea - maybe it is best to learn how to set up your genbindings by examining some pull request in axmol repo that adds new extension .

🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦

Some additional VSCode LUA extensions and afterthoughts :

EmmyLuaCodeStyle by CppCXY .
Lua code formatter .
Its default code style is mostly similar to the axmol LUA template .
But if you want you can create ".editorconfig" and fill it with the command ">insert luaCodeStyle template" .
There are a lot of settings to tweak this formatter to fit your needs .

Lua by sumneko .
This extension has everything that you would ever need for LUA ... but doesn't have debugger .
It even has Cocos4.0 type definitions for syntax completion in VSCode ... but they drop compatibility with EmmyLua ... and they even deprecate built-in 3rd party type definitions (though these typedefs can still be found here https://github.com/LuaCATS/cocos4.0) .
I remember one time i managed to use these typedefs with this extension . It was been "eeeeehhhh ... ok" , but we definitely need to generate our own axmol typedefs , and preferably for EmmyLua .

🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦🟦

Oh , and feel free to use this guide as base for your own .
I'm posting it here so that it becomes public domain .
https://creativecommons.org/publicdomain/zero/1.0/