Windows Port of MOOSE & BISON

[dylanjm]

Reason

There is an activity for FY 2020 to get a working executable for MOOSE and BISON ported over to Windows 10. The primary goal is to provide this executable to end-users that can be downloaded and run out-of-box. This is not an effort to create a developer environment for the Windows platform.

Design

MOOSE and associated dependencies should be compiled with the latest Visual Studio compilers or in such a way that allows interoperability between programs.

Impact

This will open up accessibility to a group of new users in academia and industry.

Required Tools

This is my current tool-chain for accomplishing this task:

• Windows 10 Virtual Machine

  • Edition: Windows 10 Enterprise 64-bit
  • Version: 1903
  • OS build: 18362.535

• Cygwin Terminal Emulator

  • Version: 3.1.0

• Microsoft Visual Studio Community 2019

  • Version: 16.4.2

• Visual Studio Desktop C++ Workload

  • MSVC x64/x86 build tools
  • Windows 10 SDK
  • C++ CMake tools for Windows

When installing Cygwin you'll need to ensure you pick the following packages:

• git 2.21.0-1
• m4 1.4.18-1
• make 4.2.1-2
• mingw64-x86_64-gcc-g++
• python 2.7.16-1

Resources

• PETSc Installation on Windows
• Libmesh Installation Guide

Current Progress

• Build PETSc
• Build Libmesh
• Build MOOSE
• Build BISON

@dschwen

[dylanjm]

Current PETSc build successful:

$ ./configure --with-cc'win32fe cl' --with-fc=0 --with-cxx='win32fe cl' --with-mpi=0 --download-f2cblaslapack=1
$ make
$ make PETSC_DIR=<path/to/petsc> PETSC_ARCH=arch-mswin-c-debug check
Running test examples to verify correct installation
C/C++ example src/snes/examples/tutorials/ex19 run successfully with 1 MPI process
Completed test examples

[dylanjm]

To create the .exe though? Once it's created won't we just distribute that? Apologies, I'm not super familiar with what the end process will look like. I was using Cygwin for 2 reasons:

1. Quality of Life while building the tools
2. Libmesh instructions seem to encourage using an emulator to build on Windows.

[dschwen]

What emulator?! This is a posix compatibility layer but what you get is a native executable.

[dylanjm]

@dschwen Yes, I might've been using the wrong terminology. It's actually not running any Linux.

[dschwen]

@dylanjm please add setup instructions for the Visual Studio based build. Regarding errno.h I found the following

You need to install the Windows 10 SDK component of Visual Studio, as it contains the C runtime that provides errno.h.

[dylanjm]

Cygwin Installation & Setup

• Download the Cygwin 64-bit installer
• Go through the setup process and ensure you select the following packages to install:
  • git
  • m4
  • make
  • mingw64-x86_64-gcc-g++^**
  • python2 or python3
  • vim or emacs (optional)
• You should end up with a set of selected packages similar to the screenshot below:
  
• By default Cygwin comes with tools like diff & grep

**: The Mingw64 Gnu compiler is required by the libmesh windows installation instructions. The goal is to replace g++ with the MSVC compiler. If you desire to try and compile libmesh with g++ you will have to rename the program by running mv /usr/bin/mingw64-x86_64-gcc-g++ /usr/bin/g++

Visual Studio Installation & Setup

• Download Visual Studio Community 2019 at this link and launch the installer.

• Towards the end of the setup process, you'll be prompted to modify Visual Studio by installing a set of workflows or individual components. This is where you'll select the following workflows:

  • Desktop development with C++
  • Linux Development with C++
  • Python Development (optional if installed with cygwin)

• These workflows should translate to the following individual components (you can see individual components by clicking its respective tab at the top of the installer window).

  • .NET Framework 4.6.1 targeting pack
  • .NET Framework 4.8 SDK
  • Connectivity and publishing tools
  • Web Deploy
  • NuGet package manager
  • Text Template Transformation
  • C# and Visual Basic Roslyn compilers
  • C++ 2019 Redistributable Update
  • C++ Clang Compiler for Winows (9.0.0)
  • C++ Clang-cl for v142 build tools (x64/x86)
  • C++ CMake tools for Windows
  • C++ Universal Windows Platform support vor v142 build tools (ARM64)
  • C++ Windows XP Support for VS 2017 (v141) tools [Deprecated]
  • C++/CLI support for v142 builtd tools (14.24)
  • MSBuild
  • MSVC v142 - VS 2019 C++ ARM build tools (v14.24)
  • MSVC v142 - VS 2019 C++ ARM64 build tools (v14.24)
  • Python 3 64-bit (3.75)
  • Windows Universal CRT SDK
  • C++ AddressSanitizer (Experimental)
  • C++ profiling tools
  • JavaScript diagnostics
  • Just-In-Time debugger
  • Test Adapter for Boost.Test
  • Test Adapter for Google Test
  • C++ CMake tools for Linux
  • C++ core features
  • C++ for Linux Development
  • IntelligCode
  • JavaScript and TypeScript language support
  • Live Share
  • Pythong language support
  • Python miniconda
  • Python web support
  • Graphics debugger and GPU profiler for DirectX
  • C++ ATL for latest v142 build tools (x86 & x64)
  • TypeScript 3.7 SDK
  • Windows 10 SDK (10.0.18362.0)

• You should end up with a selection as shown in the screenshot below:
  

• After selecting these tools, finish the install.

• If you ever need to modify these tools, open up Visual Studio and select Tools > Get Tools and Features.... This will launch that same installer.

Getting all the build tools in PATH

• After getting Visual Studio installed, open up visual studio to a blank project.
• Go up to the top menu bar and select Tools > Command Line > Developer Command Prompt. This will launch a DOS command prompt.
• You now need to launch Cygwin from inside the DOS prompt. Run the following commands to do this:
  • cd /cygwin64/bin
  • mintty.exe
• This will open up a new Cygwin shell. You now need to run the following command (I'm not sure if this is actually required, but it's part of the directions in PETSc)
  • /usr/bin/bash --login
• The windows compilers should now be working. You can test this by running:

$ cl
Microsoft (R) C/C++ Optimizing Compiler Version 19.24.28314 for x86
Copyright (C) Microsoft Corporation. All rights reserved. 

usage: cl [ option... ] filename... [ /link linkoption... ]

You should now be ready to build PETSc and Libmesh. @dschwen

[dylanjm]

PETSc & Libmesh Configure Commands

PETSc

$ ./configure --with-cc'win32fe cl' --with-fc=0 --with-cxx='win32fe cl' --with-mpi=0 --download-f2cblaslapack=1
$ make
$ make PETSC_DIR=<path/to/petsc> PETSC_ARCH=arch-mswin-c-debug check
Running test examples to verify correct installation
C/C++ example src/snes/examples/tutorials/ex19 run successfully with 1 MPI process
Completed test examples

Libmesh

./configure --disable-metis --with-fparser=none --with-cxx="$PETSC_DIR/lib/petsc/bin/win32fe cl"
make
make install

Currently I'm getting compiling errors claiming a missing errno.h file. I was running into a few other configure errors. If your configure doesn't work, try using some of the following flags:

• --disable-netcdf
• --disable-netcdf4
• --disable-dap
• --disable-openmp

I think I need to tool around with my current env as it doesn't make sense that I should need to disable openmp as that should be an integral part of the build, I would think.

[dschwen]

Selecting just the two workflows mentioned above will only select a fraction of teh packages you listed

[dylanjm]

@dschwen I've updated the above comment. I've included a screenshot that should detail everything that should be selected.

[dylanjm]

MOOSE & BISON Windows Port

Tools & Software Required

• Microsoft Windows 10
• Visual Studio 2019
• Cygwin

Note: If using a virtual machine software such as VMware or Virtualbox, take a snapshot of the blank VM before starting this process. That way you can restart the process easily if you get stuck at a certain point.

Setting up Cygwin

Download

To begin, download Cygwin from its home page at www.cygwin.com. Insure that you download the 64-bit version named, setup-x86_64.exe. Once downloaded, launch the setup executable and go through the process of installation until you reach the package download section.

Do not delete setup-x86_64.exe, if you find out you need an extra package later, you will have to re-run the executable.

Package Install

Next, there will be a screen that looks like following the image. Ensure the Best option is selected in the top right-hand corner.

Using the search bar at the top, search for the following packages. Upon finding the correct package, select its respective version from the Skip dropdown menu in the New column.

make                         4.2.1-1
git                          2.21.0-1
python36                     3.6.8-1
python36-pip                 19.2.3-1
mingw64x86_64-gcc-g++        7.4.0-1
emacs                        26.3.2
m4                           1.4.18-1

After selecting these packages, click Next and finish the Cygwin install.

Verifying Cygwin Setup

Now to verify that everything installed properly, launch Cygwin from the Start menu. You should see a similar message the first time you launch Cygwin:

Copying skeleton files.
these files are for the users to personalise their cygwin experience.

They will never be overwritten nor automatically updated.
'./.bashrc’ -> '/home/MCDODJ//.bashrc’
'./.bash_profile’ -> '/home/MCDODJ//.bash_profile’
'./.inputre’ -> '/home/MCDODJ//.inputrc'
'./.profile’ -> '/home/MCDODJ//. profile’

You can then check that you have all the correct packages installed by running:

cygcheck -c > cyg_packages.txt

Which should output the following:

Cygwin Package Information
Package                                 Version               Status
_autorebase                             001007-1              OK
alternatives                            1.3.30c-10            OK
base-cygwin                             3.8-1                 OK
base-files                              4.3-2                 OK
bash                                    4.4.12-3              OK
binutils                                2.29-1                OK
bzip2                                   1.0.8-1               OK
ca-certificates                         2.32-1                OK
coreutils                               8.26-2                OK
crypto-policies                         20190218-1            OK
csih                                    0.9.11-1              OK
cygrunsrv                               1.62-1                OK
cygutils                                1.4.16-2              OK
cygwin                                  3.1.2-1               OK
dash                                    0.5.9.1-1             OK
desktop-file-utils                      0.23-1                OK
diffutils                               3.5-2                 OK
editrights                              1.03-1                OK
emacs                                   26.3-2                OK
emacs-common                            26.3-2                OK
file                                    5.32-1                OK
findutils                               4.6.0-1               OK
gamin                                   0.1.10-15             OK
gawk                                    5.0.1-1               OK
getent                                  2.18.90-4             OK
git                                     2.21.0-1              OK
grep                                    3.0-2                 OK
groff                                   1.22.4-1              OK
gsettings-desktop-schemas               3.24.1-1              OK
gzip                                    1.8-1                 OK
hostname                                3.13-1                OK
info                                    6.7-1                 OK
ipc-utils                               1.0-2                 OK
less                                    551-1                 OK
libargp                                 20110921-3            OK
libattr1                                2.4.48-2              OK
libblkid1                               2.33.1-1              OK
libbrotlicommon1                        1.0.7-1               OK
libbrotlidec1                           1.0.7-1               OK
libbz2_1                                1.0.8-1               OK
libcom_err2                             1.44.5-1              OK
libcrypt0                               2.1-1                 OK
libcrypt2                               4.4.4-1               OK
libcurl4                                7.66.0-1              OK
libdb5.3                                5.3.28-2              OK
libdbus1_3                              1.10.22-1             OK
libedit0                                20130712-1            OK
libexpat1                               2.2.6-1               OK
libfam0                                 0.1.10-15             OK
libfdisk1                               2.33.1-1              OK
libffi6                                 3.2.1-2               OK
libgcc1                                 7.4.0-1               OK
libgdbm4                                1.13-1                OK
libgdbm6                                1.18.1-1              OK
libgdbm_compat4                         1.18.1-1              OK
libglib2.0_0                            2.54.3-1              OK
libgmp10                                6.1.2-1               OK
libgnutls30                             3.6.9-1               OK
libgssapi_krb5_2                        1.15.2-2              OK
libguile17                              1.8.8-3               OK
libhogweed4                             3.4.1-1               OK
libiconv                                1.14-3                OK
libiconv2                               1.14-3                OK
libidn2_0                               2.2.0-1               OK
libintl8                                0.19.8.1-2            OK
libisl15                                0.16.1-1              OK
libk5crypto3                            1.15.2-2              OK
libkrb5_3                               1.15.2-2              OK
libkrb5support0                         1.15.2-2              OK
liblcms2_2                              2.9-1                 OK
libltdl7                                2.4.6-7               OK
liblzma5                                5.2.4-1               OK
libmailutils-sieve-extensions           3.5-1                 OK
libmailutils5                           3.5-1                 OK
libmpc3                                 1.1.0-1               OK
libmpfr6                                4.0.2-1               OK
libncursesw10                           6.1-1.20190727        OK
libnettle6                              3.4.1-1               OK
libnghttp2_14                           1.37.0-1              OK
libopenldap2_4_2                        2.4.48-1              OK
libp11-kit0                             0.23.15-1             OK
libpcre1                                8.43-1                OK
libpipeline1                            1.5.1-1               OK
libpkgconf3                             1.6.0-1               OK
libpopt-common                          1.16-2                OK
libpopt0                                1.16-2                OK
libpsl5                                 0.21.0-1              OK
libreadline7                            7.0.3-3               OK
libsasl2_3                              2.1.27-1              OK
libsigsegv2                             2.10-2                OK
libsmartcols1                           2.33.1-1              OK
libsqlite3_0                            3.30.0-1              OK
libssh-common                           0.8.7-1               OK
libssh4                                 0.8.7-1               OK
libssl1.0                               1.0.2t-1              OK
libssl1.1                               1.1.1d-1              OK
libstdc++6                              7.4.0-1               OK
libtasn1_6                              4.14-1                OK
libunistring2                           0.9.10-1              OK
libuuid-devel                           2.33.1-1              OK
libuuid1                                2.33.1-1              OK
libxml2                                 2.9.9-2               OK
login                                   1.13-1                OK
m4                                      1.4.18-1              OK
mailutils                               3.5-1                 OK
make                                    4.2.1-1               OK
man-db                                  2.7.6.1-1             OK
mingw64-x86_64-binutils                 2.29.1.787c9873-1     OK
mingw64-x86_64-gcc-core                 7.4.0-1               OK
mingw64-x86_64-gcc-g++                  7.4.0-1               OK
mingw64-x86_64-headers                  7.0.0-1               OK
mingw64-x86_64-runtime                  7.0.0-1               OK
mingw64-x86_64-windows-default-manifest 6.4-1                 OK
mingw64-x86_64-winpthreads              7.0.0-1               OK
mintty                                  3.1.0-1               OK
ncurses                                 6.1-1.20190727        OK
openssh                                 8.1p1-1               OK
openssl                                 1.1.1d-1              OK
p11-kit                                 0.23.15-1             OK
p11-kit-trust                           0.23.15-1             OK
perl                                    5.26.3-2              OK
perl-Error                              0.17028-1             OK
perl-Scalar-List-Utils                  1.53-1                OK
perl-TermReadKey                        2.38-1                OK
perl_autorebase                         5.26.3-2              OK
perl_base                               5.26.3-2              OK
pkg-config                              1.6.0-1               OK
pkgconf                                 1.6.0-1               OK
publicsuffix-list-dafsa                 20190717-1            OK
python-pip-wheel                        19.2.3-1              OK
python-setuptools-wheel                 41.2.0-1              OK
python36                                3.6.8-1               OK
python36-pip                            19.2.3-1              OK
python36-setuptools                     41.2.0-1              OK
rebase                                  4.4.4-1               OK
rsync                                   3.1.2-1               OK
run                                     1.3.4-2               OK
sed                                     4.4-1                 OK
shared-mime-info                        1.8-1                 OK
tar                                     1.29-1                OK
terminfo                                6.1-1.20190727        OK
terminfo-extra                          6.1-1.20190727        OK
tzcode                                  2019c-1               OK
tzdata                                  2019c-1               OK
util-linux                              2.33.1-1              OK
vim-minimal                             8.1.1772-1            OK
which                                   2.20-2                OK
xz                                      5.2.4-1               OK
zlib0                                   1.2.11-1              OK

You can confirm your output is the same by pasting the above output into a compare.txt file and running:

diff -y --suppress-common-lines cyg_packages.txt compare.txt

If both files match, there will be no output.

Setting up Visual Studio

Download and Install

To install Visual Studio 2019 Community Edition, visit its homepage at https://visualstudio.microsoft.com/downloads/. When launching vs_community__<version>.exe, a dialog will open, offering a set of workflows to install along side Visual Studio. Select the Desktop development with C++ workflow. The image below shows an example of what it should look like:

Then proceed with the install. Once the installation finishes, you will need to reboot your computer.

Verify Compilers

To verify that everything is working, open Visual Studio and select the continue without code option on the startup screen. Then head up to Tools » Command Line » Developer Command Prompt. This will a launch a DOS prompt where you’ll need to run:

/cygwin64/bin/mintty.exe

This will launch a new instance of a Cygwin prompt. In the Cygwin prompt, run the following bash commands:

/usr/bin/bash –-login
cl

This should result the following output if everything installed correctly:

Microsoft (R) C/C++ Optimizing Compiler Version 19.24.28315 for x86
Copyright (C) Microsoft Corporation.  All rights reserved.

usage: cl [ option... ] filename... [ /link linkoption... ]

It also may be a good idea to check the output of echo $PATH variable to make sure that all the Visual Studio compilers are in there.

Optional: Setting up Intel Compilers

CUSTOM_ID: sec:optional:intel

Register for trial version of Parallel Studio

Go to https://software.intel.com/en-us/parallel-studio.xe and click the Choose & Download button. On the next page select the Windows link under the Try it out! heading. After selecting Free Trial on the following page, follow the process of signing up for a free trial. The only information required is your email address.

Then follow the directions specified for Windows on this page:

https://software.intel.com/en-us/get-started-with-parallel-studio-xe

Optional: Setting up MS MPI

CUSTOM_ID: sec:optional:msmpi

The github page for Microsoft-MPI can be found at: https://github.com/microsoft/Microsoft-MPI

PETSc Build

The download and install instructions for PETSc can be found at https:mcs.anl.gov/petsc/documentation/installation.html. First, run the following git commands:

git clone -b maint https://gitlab.com/petsc/petsc.git $HOME/projects/petsc
cd $HOME/projects/petsc

Then configure PETSc by running the following commands:

./configure --with-cc='win32fe cl' --with-cxx='win32fe cl' --with-fc=0 --with-mpi=0 --download-f2cblaslapack=1

Once the configuration step has completed, follow the instructions at the end of the output and run:

(Make sure to replace <path/to/petsc> with the proper path on your computer.)

make PETSC_DIR=<path/to/petsc> PETSC_ARCH=arch-mswin-c-debug all

Once the build has completed, run all the tests to ensure it was built properly.

make test

If everything went smooth, you should see this output:

Running test examples to verify correct installation
Using PETSC_DIR=/home/MCDODJ/projects/petsc and PETSC_ARCH=mswin-c-debug
C/C++ example src/snes/examples/tutorials/ex19 run successfually with 1 MPI process
Completed test examples

PETSc is now built and working on Windows. Note: these instructions build PETSc without MPI capabilities. You’ll need to download either Intel or MS versions of MPI. Steps to do that are mentioned in Optional: Setting up Intel Compilers and Optional: Setting up MS MPI

libMesh Build

Instructions for building Libmesh on Windows can be found at https://libmesh.github.io/installation.html. Use git to clone libMesh and place it in the same directory that PETSc resides.

git clone git://github.com/libMesh/libmesh.git $HOME/projects/libmesh

# or

git clone https://github.com/libMesh/libmesh.git

cd $HOME/projects/libmesh
git submodule update --init contrib/timpi

Libmesh states on its website that

libMesh makes extensive use of the standard C++ library and increasing use of C++11 standard language and library features, so you will need a recent, standards-compliant compiler. We have tried very hard to make the code compiler-agnostic by avoiding questionable constructs. If you have a C++11-supporting compiler that won’t build libMesh code please let us know. You will also need a decent C compiler if you want to build some of the contributed packages that add functionality to the library.

libMesh Continuous Integration testing of all library modifications currently is done with gcc 6.2 and gcc 7.3. Less frequent testing is done with clang, Intel, and older gcc compilers.

Theortically, we should be able to build libMesh with the Microsoft Compiler. First configure the build by running:

./configure

NOTE: The only way I could successfully configure libmesh is by using gcc:

So first rename the mingw64 gcc compiler

mv /usr/bin/x86_64-w64-mingw32-gcc.exe /usr/bin/gcc.exe

Then configure libmesh

./configure --disable-metis \
  --with-fparser=none \
  --with-cc=gcc \
  --with-cxx=gcc

Then make sure PETSC_DIR and PETSC_ARCH are in your environment

export PETSC_DIR=<path/to/petsc>
export PETSC_ARCH="arch-mswin-c-debug all"

Then build libmesh

make

MOOSE Build

BISON Build

Tested & Failed Commands

PETSc

./configure --with-cc='win32fe cl' --with-cxx='win32fe cl' --with-fc=0 --with-mpi=0

libMesh

./configure --disable-metis \
  --with-fparser=none \
  --with-cc='../petsc/lib/petsc/bin/win32fe/win32fe.exe cl' \
  --with-cxx='../petsc/lib/petsc/bin/win32fe/win32fe.exe cl'\
  --with-mpi=none \
  --with-fc=none

configure error: ./configure failed for contrib/netcdf/v4

./configure --disable-metis \
  --with-fparser=none \
  --with-cc='../petsc/lib/petsc/bin/win32fe/win32fe.exe cl' \
  --with-cxx='../petsc/lib/petsc/bin/win32fe/win32fe.exe cl'\
  --with-mpi=none \
  --with-fc=none \
  --disable-netcdf

configure error: ./configure failed for contrib/netcdf/v4

./configure --disable-metis \
  --with-fparser=none \
  --with-cc=cl \
  --with-cxx=cl \
  --with-mpi=none \
  --with-fc=none \
  --disable-netcdf-4 \
  --disable-dap

configure: error: Can't find or link to the math library.
configure: error: ./configure failed for contrib/netcdf/v4

@dschwen