Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Step-by-step instructions so anyone can compile the source code at home on Windows and Linux. #344

Open
Spoiledpay opened this issue Jun 27, 2024 · 12 comments

Comments

@Spoiledpay
Copy link

Hello! Can you tell us which tools should be downloaded for compilation, such as Nodejs, Visual C++ 2022, Python. And how to generate it. Thanks.

@IsaacShelton
Copy link
Collaborator

For Windows, it is difficult to build.

You need to install:

  • Mingw-w64 compiler
  • CMake and Ninja

Then:

  • Obtain libCURL libraries/headers for windows https://curl.se/windows/dl-8.2.1_11/curl-8.2.1_11-win64-mingw.zip
  • Obtain zlib and zstd libraries/headers (they are dependencies for LLVM)
  • Download and build LLVM from source (they don't have prebuilt binaries for Windows), this takes 1-2 hours. I recommend version 16, 17, or 18. Currently we ship with LLVM 16 on Windows, but it should work with LLVM 18 as well.

Building LLVM... (you can add additional options and/or partially build if you like)

mkdir build
cmake -B build -DCMAKE_BUILD_TYPE="Release" -G Ninja
cmake --build build
  • Configure Adept CMake with correct values
    for your setup (you may have to tell CMake where dependencies are via environment variables), an example of the process:

    - name: Configure (Windows)
    if: ${{ matrix.os == 'windows-latest' }}
    run: |
    $env:Path = "C:/ProgramData/chocolatey/lib/mingw/tools/install/mingw64/bin;$env:Path"
    cmake -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}} -DCMAKE_PREFIX_PATH="${{github.workspace}}/llvm/llvm/build;C:\msys64\mingw64" -DCMAKE_LIBRARY_PATH=C:\msys64\mingw64\lib -DCURL_LIBRARY=curl -DCURL_LIBRARY_DIRS=${{github.workspace}}/libcurl/lib -DCURL_INCLUDE_DIR=libcurl/include -DADEPT_LINK_LLVM_STATIC=On -B ${{github.workspace}}/build -G Ninja
    env:
    LLVM_DIR: ${{github.workspace}}/llvm/llvm/build
    zstd_DIR: C:\msys64\mingw64
    zstd_LIBRARY: C:\msys64\mingw64\lib\libzstd.a
    ZLIB_INCLUDE_DIR: C:\msys64\mingw64\include
    ZLIB_LIBRARY: C:\msys64\mingw64\lib\libz.a

  • Build Adept using Ninja via CMake cmake --build build --config Release

  • Then, the adept binary will be in the build directory.

  • Move adept binary into a folder where you want it to be.

  • Clone github.com/AdeptLanguage/AdeptImport into the directory

  • Rename the cloned folder AdeptImport to import

  • Also, copy initial config file from res/necessities/adept.config into the directory

  • Lastly, copy all contents of res/necessities/windows into the directory

An example of the full build progress exists inside .github/workflows/remoteBuild.yml, which is the code that generates the nightly releases and installers.

For linux, it's a little easier,

You need to install developer versions of llvm, libcurl, libz, and zstd (most likely using your distro's package manager)

Then use CMake to configure and build Adept.

Lastly, put the resulting adept binary from the build directory into the folder where you want it to be.

  • Clone github.com/AdeptLanguage/AdeptImport into the directory
  • Rename the cloned folder AdeptImport to import
  • Lastly, copy initial config file from res/necessities/adept.config into the directory

This process is also documented inside .github/workflows/remoteBuild.yml, although it might be hard to follow since it's not just for linux.

You can then optionally add it to the path and/or symlink it.

Additionally if you want, there is always the option to fork this repo and customize .github/workflows/remoteBuild.yml which may be slightly easier. But this probably doesn't offer as much customization over the build environment as building locally would.

@Spoiledpay
Copy link
Author

Spoiledpay commented Jul 15, 2024

Hello, IsaacShelton!
If I understand, Linux makes it easier to compile the code and generate it for Linux and Windows at the same time.
In the case of Linux, can we do it in WSL using Ubuntu?

1 - we install sudo apt install build-essential.

after this,

2 - install llvm, libcurl, libz, and zstd all sources in a specific folder?

3 - Clone https://github.com/AdeptLanguage/Adept/
4 - Clone github.com/AdeptLanguage/AdeptImport into the directory // Here you say clone within Adept?
Rename the cloned folder AdeptImport to import
Lastly, copy initial config file from res/necessities/adept.config into the directory

Am I on the right path of understanding or are there a few more steps and which commands should we use to generate Adept.exe?

Can you help?

@IsaacShelton
Copy link
Collaborator

@Spoiledpay

Yes, if you are using WSL then it is a lot easier.

What you describe is correct

I just tested the below which works:

sudo add-apt-repository ppa:george-edison55/cmake-3.x   # (needed to install for cmake)
sudo apt install -y cmake
sudo apt install -y g++ # (need to install g++ if not already present)
sudo apt install -y libcurl4-openssl-dev
sudo apt install -y libzstd-dev
sudo apt install -y libz-dev
sudo apt install -y llvm-17-dev
git clone https://github.com/AdeptLanguage/Adept
cd Adept
mkdir build
cd build
cmake .. -G "Unix Makefiles"
make adept
# NOTE: CMake will automatically create adept.config and clone https://github.com/AdeptLanguage/AdeptImport
cp -R ../res/necessities/windows/* .
./adept ../tests/e2e/src/hello_world/main.adept -e

🎉

@Spoiledpay
Copy link
Author

Hello!
It worked perfect to generate a Linux bin.
With small modifications I managed to install LLVM 17.

I noticed that the procedure generates the Linux executable. What parameter is used to generate Windows bins?
And the package generated with complete functioning, how do you do it?
Separate everything and zip it up later?

I appreciate your support. And I saw that it is now in a new version in Rust, is there any reason?

@IsaacShelton
Copy link
Collaborator

IsaacShelton commented Jul 16, 2024

To build for Windows, it is a lot more painful.

You have to use mingw-w64 instead of gcc/g++.

It would be easiest to use something like https://www.msys2.org/ instead of WSL.

I can't test it on my VM, but it would be something like:

pacman -S mingw-w64-x86_64-toolchain
pacman -S cmake
pacman -S libcurl-devel
pacman -S libzstd-devel
pacman -S zlib-devel
pacman -S mingw-w64-x86_64-llvm
git clone https://github.com/AdeptLanguage/Adept
cd Adept
mkdir build
cd build
cmake .. -G "Unix Makefiles"
make adept
# NOTE: CMake will automatically create adept.config and clone https://github.com/AdeptLanguage/AdeptImport
cp -R ../res/necessities/windows/* .
./adept ../tests/e2e/src/hello_world/main.adept -e

For the standalone release, it is just a zipped version of the build (minus generated CMake files).

mkdir -p stage
mv build/adept.exe stage/adept.exe
mv res/necessities/windows/* stage/
mv AdeptWindowsInstaller/adeptX-X.bat stage/${{env.alternativeBinaryName}}.bat
mv AdeptConfig/adept.config stage/adept.config
mv AdeptImport stage/import
mv LICENSE stage/LICENSE

For the installer, we use Inno Setup with the installer template: https://github.com/IsaacShelton/AdeptWindowsInstaller

We use python3 AdeptWindowsInstaller/generate-guid.py to generate a random GUID for the release.

Then we perform substitutions of the values inside the template file. You would have to modify the below depending on where you build and stage the files. Also, each ${{ variable }}} should be replaced.

alternativeBinaryName is adept2-8
longVersion is 2.8.0
shortVersion is 2.8
fullName is Adept 2.8
steps.guid.outputs.guid is your generated GUID

sed -i AdeptWindowsInstaller/Adept-X.X-installer.iss -e 's/__STAGING_AREA__/..\/stage/g' -e 's/__OUTPUT_DIRECTORY__/./g' -e 's/__FAVICON_DIRECTORY__/..\/res/g' -e 's/adeptX-X/${{env.alternativeBinaryName}}/g' -e 's/X.X/${{env.shortVersion}}/g' -e 's/{7D143E16-9268-4F20-A0FA-8B30D8B14ECA}/${{steps.guid.outputs.guid}}/g'

Then we use Inno Setup to build the installer.

With regards to Adept 3.0 (the Rust version), it is in early development won't be ready for a couple of years.

It will be an improvement upon Adept 2.x, and fix many issues with the language.

It will:

  • Be way easier to build yourself (just cargo build)
  • Have better C interoperability
  • Have first-class support for C header/source files, so bindings won't be necessary (just include a header file)
  • Make some breaking changes in order to fix some language design issues
  • Have full support UTF-8 and Unicode support
  • Have better tools, including: language server, formatter, documentation generator, and more
  • Have new module and namespace system
  • Have better support for automatically managed values
  • Have improved compile times
  • And more

However, even after Adept 3.0 is released, Adept 2.x will still be maintained, and may eventually gain support in the new compiler as well.

@Spoiledpay
Copy link
Author

Another question: Is the platform you use to create MAC or Linux Arch?
When building bins for Windows, I thought there was something like the following to force compilation for windows:

PATH=$(echo "$PATH" | sed -e 's/:/mnt.*//g')
cd depends
make HOST=x86_64-w64-mingw32 // To generate the .exe for windows

1 - Thank you, I will test the issue in WSL and test if it works, your example here is to compile in the Linux environment to generate a native Adept.exe for Windows. After zipping, take it to the Windows environment.

@IsaacShelton
Copy link
Collaborator

IsaacShelton commented Jul 16, 2024

Yes I primarily use macOS for development.

I'm not very familiar with WSL, but that might work.

One thing that could be weird though, is that the generated exe would be linked against Ubuntu's LLVM libraries.

This means that the default target for the adept.exe would create linux binaries (which is probably not what you want).

I could be wrong though, you'd have to try it.

Currently the automated builds use MinGW-64, MSYS, and manually building LLVM. I've heard from others that building it with MSYS exclusively works well, they should have all the packages so nothing would need to be built from scratch. If WSL doesn't work then MSYS could also be a good option.

@Spoiledpay
Copy link
Author

Hello!
I imagine this is a lot to ask of you and due to the creation of a new language in version 3.

When you have special time, you can try to create a step-by-step guide for compiling on Windows. Software to be downloaded, similar to what was done for Linux or a way in WSL to cross compile and generate the .exe for Windows. It should help too.

I managed to compile Adept for Linux nbo WSL.

@IsaacShelton
Copy link
Collaborator

Yes I just added a step-by-step guide https://github.com/AdeptLanguage/Adept/blob/master/HOW_TO_BUILD.md

I recently discovered an easier way to build it with MSYS2, so that's what this guide uses.

Here is the excerpt on building on Windows:

Windows

MSYS2 Setup:

  • Download MSYS2
  • Install MSYS2
  • Open the MSYS2 MINGW64 Prompt
MSYS2 Mingw64 Command Prompt
  • pacman -S git --noconfirm
  • pacman -S mingw-w64-x86_64-cmake --noconfirm
  • pacman -S mingw-w64-x86_64-ninja --noconfirm
  • pacman -S mingw-w64-x86_64-gcc --noconfirm
  • pacman -S mingw-w64-x86_64-llvm --noconfirm
  • pacman -S mingw-w64-x86_64-zlib --noconfirm
  • git clone https://github.com/AdeptLanguage/Adept
  • cd Adept
  • mkdir build
  • /mingw64/bin/cmake -B build -DCMAKE_BUILD_TYPE="Release" -DCMAKE_C_FLAGS="-w" -G Ninja
  • /mingw64/bin/cmake --build build
  • build/adept tests/e2e/src/hello_world/main.adept -e

There is also information about how to link against LLVM statically if you wish:

https://github.com/AdeptLanguage/Adept/blob/master/HOW_TO_BUILD.md#linking-llvm-statically

@IsaacShelton
Copy link
Collaborator

Also sorry for taking so long to get back to you, I was away on vacation

@Spoiledpay
Copy link
Author

Hello!
Firstly, I think everyone deserves a vacation. This is part of life. I hope everything went well!

Second, thank you very much for your time in finding another way to compile on Windows. I managed to do it the way it was listed. And Adept.exe was generated. successfully!!

Now the question of the second step that makes the llvm dlls stay together in Adept.exe, this didn't work. Generating a complete package. There must be something more to be accomplished.

In view of this, it was a big step and I am grateful.
Sucesso

*Then I share the error in linking llvm dll and Adept

@IsaacShelton
Copy link
Collaborator

IsaacShelton commented Sep 8, 2024

The primary CMakeLists.txt has been recently updated to allow for this easier. HOW_TO_BUILD.md has also been updated to clarify how to link against LLVM statically on Windows.

Linking LLVM statically on Windows is actually supposed the default (I remembered when looking through the build file), but somehow it stopped working due to changes in CMake, which has been fixed in the most recent commit.


Since linking statically is now properly the default on Windows, you only need:

  • git pull (for newest commit)
  • pacman -S mingw-w64-x86_64-zstd --noconfirm (required when statically linking against LLVM)
  • /mingw64/bin/cmake -B build -DCMAKE_BUILD_TYPE="Release" -DCMAKE_C_FLAGS="-w" -G Ninja
  • /mingw64/bin/cmake --build build

HOW_TO_BUILD.md contains the fully updated process as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants