[Bug] build errors when trying to compile GRASS inside a sandbox

[hamarituc]

Describe the bug

When compiling GRASS GIS it tries do open some network connections when building HTML manual pages. This behavior was introduced in commit 432ceb7. When preparing packages for Linux distributions, the code is usually compiled in a sandbox which prevents network access. Furthermore accessing network resources during build leads to unreproducible results.

To Reproduce

Steps to reproduce the behavior:

1. Download GRASS source code and unpack it.
2. Write the following C program seccomp.c which will help us to build the source code inside a network sandbox without disabling networking on the whole system.

#include <stddef.h>
#include <stdio.h>
#include <errno.h>
#include <sys/prctl.h>
#include <linux/audit.h>
#include <linux/seccomp.h>
#include <linux/filter.h>
#include <linux/unistd.h>

__attribute__((constructor)) void configure_seccomp(void) {
  struct sock_filter filter [] = {
    BPF_STMT(BPF_LD + BPF_W + BPF_ABS, (offsetof(struct seccomp_data, arch))),
    BPF_JUMP(BPF_JMP + BPF_JEQ + BPF_K, AUDIT_ARCH_X86_64, 0, 3),
    BPF_STMT(BPF_LD + BPF_W + BPF_ABS, (offsetof(struct seccomp_data, nr))),
    BPF_JUMP(BPF_JMP + BPF_JEQ + BPF_K, __NR_connect, 0, 1),
    BPF_STMT(BPF_RET + BPF_K, SECCOMP_RET_ERRNO | (EACCES & SECCOMP_RET_DATA)),
    BPF_STMT(BPF_RET + BPF_K, SECCOMP_RET_ALLOW),
  };

  struct sock_fprog prog = {
       .len = (unsigned short)(sizeof(filter) / sizeof (filter[0])),
       .filter = filter,
  };

  prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0);
  prctl(PR_SET_SECCOMP, SECCOMP_MODE_FILTER, &prog);
}

3. Compile it: gcc -shared seccomp.c -o seccomp.so
4. Configure the source: ./configure
5. Compile the code inside the sandbox: LANG=C LD_PRELOAD="$(realpath seccomp.so)" make -j 4

You will see error messages like:

make[3]: Entering directory '/usr/local/src/grass-8.0.0/lib/vector'
if [ "" != "" ] ; then GISRC=/usr/local/src/grass-8.0.0/dist.x86_64-pc-linux-gnu/demolocation/.grassrc80 GISBASE=/usr/local/src/grass-8.0.0/dist.x86_64-pc-linux-gnu PATH="/usr/local/src/grass-8.0.0/dist.x86_64-pc-linux-gnu/bin:/usr/local/src/grass-8.0.0/dist.x86_64-pc-linux-gnu/bin:/usr/local/src/grass-8.0.0/dist.x86_64-pc-linux-gnu/scripts:$PATH" PYTHONPATH="/usr/local/src/grass-8.0.0/dist.x86_64-pc-linux-gnu/etc/python:/usr/local/src/grass-8.0.0/dist.x86_64-pc-linux-gnu/gui/wxpython:$PYTHONPATH" LD_LIBRARY_PATH="/usr/local/src/grass-8.0.0/dist.x86_64-pc-linux-gnu/bin:/usr/local/src/grass-8.0.0/dist.x86_64-pc-linux-gnu/bin:/usr/local/src/grass-8.0.0/dist.x86_64-pc-linux-gnu/scripts:/usr/local/src/grass-8.0.0/dist.x86_64-pc-linux-gnu/lib:/usr/local/src/grass-8.0.0/dist.x86_64-pc-linux-gnu/lib:" LC_ALL=C LANG=C LANGUAGE=C  --html-description < /dev/null | grep -v '</body>\|</html>' > vectorascii.tmp.html ; fi
VERSION_NUMBER=8.0.0 VERSION_DATE=2022 MODULE_TOPDIR=../.. \
        python3 /usr/local/src/grass-8.0.0/dist.x86_64-pc-linux-gnu/utils/mkhtml.py vectorascii > /usr/local/src/grass-8.0.0/dist.x86_64-pc-linux-gnu/docs/html/vectorascii.html
Traceback (most recent call last):
  File "/usr/lib/python3.9/urllib/request.py", line 1346, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/usr/lib/python3.9/http/client.py", line 1285, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1331, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1280, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1040, in _send_output
    self.send(msg)
  File "/usr/lib/python3.9/http/client.py", line 980, in send
    self.connect()
  File "/usr/lib/python3.9/http/client.py", line 1447, in connect
    super().connect()
  File "/usr/lib/python3.9/http/client.py", line 946, in connect
    self.sock = self._create_connection(
  File "/usr/lib/python3.9/socket.py", line 823, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
  File "/usr/lib/python3.9/socket.py", line 954, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/src/grass-8.0.0/dist.x86_64-pc-linux-gnu/utils/mkhtml.py", line 141, in download_git_commit
    response = urlopen(url, *args, **kwargs)
  File "/usr/local/src/grass-8.0.0/dist.x86_64-pc-linux-gnu/utils/mkhtml.py", line 113, in urlopen
    return urlrequest.urlopen(request, *args, **kwargs)
  File "/usr/lib/python3.9/urllib/request.py", line 214, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.9/urllib/request.py", line 517, in open
    response = self._open(req, data)
  File "/usr/lib/python3.9/urllib/request.py", line 534, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.9/urllib/request.py", line 1389, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "/usr/lib/python3.9/urllib/request.py", line 1349, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno -3] Temporary failure in name resolution>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/src/grass-8.0.0/dist.x86_64-pc-linux-gnu/utils/mkhtml.py", line 658, in <module>
    git_commit = get_last_git_commit(
  File "/usr/local/src/grass-8.0.0/dist.x86_64-pc-linux-gnu/utils/mkhtml.py", line 232, in get_last_git_commit
    response = download_git_commit(
  File "/usr/local/src/grass-8.0.0/dist.x86_64-pc-linux-gnu/utils/mkhtml.py", line 177, in download_git_commit
    gs.warning(
AttributeError: 'NoneType' object has no attribute 'warning'
make[3]: *** [../../include/Make/Html.make:7: /usr/local/src/grass-8.0.0/dist.x86_64-pc-linux-gnu/docs/html/vectorascii.html] Error 1
make[3]: Leaving directory '/usr/local/src/grass-8.0.0/lib/vector'

Expected behavior

The build system should not download data during build time. Preferable the source code archive should be self contained. At least there should be an option to disable this download feature in the configure script.

System description (please complete the following information):

• Operating System: Linux
• Linux Distribution: Gentoo
• GRASS GIS version 8.0.0

Additional context

There are actually 2 bugs:

• the download bug described above
• In mkhtml.py the error message of git is evaluated. This check may fail if a non-english locale is configured. This is the reason for LANG=C in the description above. You should ensure the proper local at all locations where program output is evaluated.

[hamarituc]

Maybe the same root cause as in #2137.

[nilason]

Maybe the same root cause as in #2137.

Most likely.

[wenzeslaus]

Perhaps an automated test build in GitHub Actions which somehow mimics the distribution build process (e.g., disables the network and works from a (non-Git) copy of the source code) would help identifying these issues sooner? The seccomp.c trick seems to me as something which is hard to maintain, but perhaps it is a start.

[hamarituc]

Perhaps an automated test build in GitHub Actions which somehow mimics the distribution build process (e.g., disables the network and works from a (non-Git) copy of the source code) would help identifying these issues sooner? The seccomp.c trick seems to me as something which is hard to maintain, but perhaps it is a start.

The seccomp-wrapper wasn't meant to be integrated permanently into an automated build CI toolchain, but merely a hint how to reproduce the issue without setting a up a dedicated sandbox.

[nilason]

Closing this as duplicate of #2137, addressed with #2139.