setup a cleaner build environment #665

rickysarraf · 2021-02-23T09:24:34Z

Multiple packages have run into build failures on our OBS. On closer
inspection, it was concluded to be issues with the locale settings.
These issues were never seen on the Debian buildd infrastructure because
there the locale setting LC_ALL=C.UTF-8 is in use.

This change keeps us closer to what is used in Debian buildds.

Signed-off-by: Ritesh Raj Sarraf [email protected]

rickysarraf · 2021-02-23T09:27:52Z

Multiple build failures, relevant to this change.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=982974
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981718
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981847

mlschroe · 2021-02-23T12:33:07Z

C.UTF-8 does not exist in older distros, so I'm not sure if it is a good idea to set it unconditionally.

coolo · 2021-02-23T12:38:21Z

https://unix.stackexchange.com/questions/597962/how-widespread-is-the-c-utf-8-locale suggests Arch Linux does not have it - so it's not about the age of the distro

mlschroe · 2021-02-23T12:41:13Z

True, but this pull request just patches the build-recipe-dsc file...

coolo · 2021-02-23T12:46:08Z

but it's about the distribution executing it, or do I get this wrong?

andrewshadura · 2021-02-26T12:51:04Z

The change only affects DSC_BUILD_CMD which runs in a Debian (or a Debian derivative) chroot, so I think what other distributions do isn’t relevant in this case.

emanueleaina · 2021-04-12T10:09:31Z

build-recipe-dsc

@@ -95,7 +95,7 @@ dsc_build() {
    # this allows the build environment to be manipulated
    # and alternate build commands can be used
    DSC_BUILD_CMD="$(queryconfig --dist "$BUILD_DIST" --archpath "$BUILD_ARCH" --configdir "$CONFIG_DIR" substitute dsc:build_cmd)"
-    test -z "$DSC_BUILD_CMD" && DSC_BUILD_CMD="dpkg-buildpackage -us -uc"
+    test -z "$DSC_BUILD_CMD" && DSC_BUILD_CMD="LC_ALL=C.UTF-8 dpkg-buildpackage -us -uc"


Is this the right place?

Shouldn't we change build-pkg-deb instead?

(but I am not sure who calls who to be honest)

Probably both need to be changed; also build-pkg-deb needs to consistently run deb_chroot.

Think this is OK now, right ?

andrewshadura · 2021-12-01T14:55:32Z

Any update on this?

Multiple packages have run into build failures on our OBS. On closer inspection, it was concluded to be issues with the locale settings. These issues were never seen on the Debian buildd infrastructure because there the locale setting LC_ALL=C.UTF-8 is in use. This change keeps us closer to what is used in Debian buildds. openSUSE#665 Signed-off-by: Emanuele Aina <[email protected]>

rickysarraf · 2022-02-01T13:35:37Z

Also added a couple of fixes we've been carrying in our setup for a while.

obbardc · 2025-03-18T13:39:56Z

@rickysarraf do you have any time to resolve this ? It would be nice to have this in OBS by default.

nickbroon · 2025-03-18T19:23:58Z

For additional context on Debian buildd setting LC_ALL to C.UTF-8 see: https://lists.debian.org/debian-devel/2024/06/msg00188.html

The lack of OBS also doing this also appears to be the cause of the recent breakage of installing fontconfig in OBS build environments reported here: https://lists.opensuse.org/archives/list/[email protected]/thread/C24P565DGJLSB4VDKHZVC7YK5KMQYUCP/

As such I would support the merging of this pull request.

Multiple packages have run into build failures on our OBS. On closer inspection, it was concluded to be issues with the locale settings. These issues were never seen on the Debian buildd infrastructure because there the locale setting LC_ALL=C.UTF-8 is in use. This change keeps us closer to what is used in Debian buildds. openSUSE#665 Signed-off-by: Emanuele Aina <[email protected]>

To make the build environment a bit more reproducible, ensure the local timezone is set to UTC when building packages. Signed-off-by: Emanuele Aina <[email protected]>

…ilds Our OBS workers run under a `screen` session, which gives them an actual `$TERM` variable. Some packages like `color.js` then enable colored output, which confuses some other packages like `node-grunt-legacy-log` which use it in their testsuite and do not expect colored output. To avoid that, unset `TERM`. Signed-off-by: Emanuele Aina <[email protected]>

spymemcached package is FTBFS because a test expects a reverse lookup of 127.0.0.1 to return localhost, but current /etc/hosts configuration returns the hostname instead. Let's move this `hostname/ip` entry to the end of /etc/hosts so the reverse lookup of 127.0.0.1 returns localhost, which makes sense to be the default case usually. Signed-off-by: Ariel D'Alessandro <[email protected]>

rickysarraf · 2025-03-20T08:25:45Z

@rickysarraf do you have any time to resolve this ? It would be nice to have this in OBS by default.

Thanks @obbardc
This is now rebased against the current HEAD

andrewshadura · 2025-03-20T08:27:40Z

@rickysarraf, thanks! There are a few more related changes in https://gitlab.collabora.com/obs/obs-build/-/commits/collabora/main, could you please pick them up too?

rickysarraf · 2025-03-20T10:49:54Z

@rickysarraf, thanks! There are a few more related changes in https://gitlab.collabora.com/obs/obs-build/-/commits/collabora/main, could you please pick them up too?

@andrewshadura Sure. I'm on it. Will check now and pick the ones relevant.

obbardc · 2025-03-20T11:02:55Z

I wonder if there is any scope to create some useful unit tests for building Debian packages (not as part of this pr of course, but if you're interested maybe we could open separate issue) ?

…ence Signed-off-by: Andrej Shadura <[email protected]> merge fallout Signed-off-by: Ritesh Raj Sarraf <[email protected]>

Signed-off-by: Andrej Shadura <[email protected]>

… builds Since "su -" cleans the environment, we need to tell it to keep variables we want to preserve. To make this easier, create a new wrapper su_deb_chroot that runs chroot and su and sets up the environment. Since DEBIAN_* variables are only needed when preparing the build chroot, don’t set them in su_deb_chroot. Signed-off-by: Andrej Shadura <[email protected]>

Using su - will set the path to a default value, which for a normal user doesn't include /sbin.. However some debian package do rely on sbin being in the path it seems. So explictly set the path to match the default set by debuild, including sbin Signed-off-by: Sjoerd Simons <[email protected]>

rickysarraf · 2025-03-20T15:04:49Z

@rickysarraf, thanks! There are a few more related changes in https://gitlab.collabora.com/obs/obs-build/-/commits/collabora/main, could you please pick them up too?

@andrewshadura Sure. I'm on it. Will check now and pick the ones relevant.

@andrewshadura I have now pushed all the cherry-picked commits as well. Given that this branch was rebased against upstream's master, some code had diverged and I had to resolve conflicts during the cherry-pick.

So, this PR could appreciate another round of quick review from you.

andrewshadura · 2025-03-20T15:05:30Z

I think it’s better to leave out 4e273c6. This was fixed in a different way upstream.

obbardc

Looks good to me.

I will test this in more detail with the failing Debian packages later, to see if it fixes the problem.

emanueleaina reviewed Apr 12, 2021

View reviewed changes

adrianschroeter force-pushed the master branch from 8b9ae5e to 7fc4617 Compare April 28, 2021 13:12

adrianschroeter force-pushed the master branch from e6ebce5 to 2a6beef Compare October 19, 2021 15:01

mlschroe force-pushed the master branch from f3ceef8 to efeece0 Compare January 12, 2022 13:16

rickysarraf force-pushed the obs-build-rickysarraf/master branch from 7fbc66c to a68beef Compare February 1, 2022 13:32

rickysarraf changed the title ~~Set LC_ALL to C.UTF-8~~ setup a cleaner build envronment Feb 1, 2022

rickysarraf changed the title ~~setup a cleaner build envronment~~ setup a cleaner build environment Feb 1, 2022

emanueleaina and others added 4 commits March 20, 2025 13:53

Force the local timezone to be UTC

d95202f

To make the build environment a bit more reproducible, ensure the local timezone is set to UTC when building packages. Signed-off-by: Emanuele Aina <[email protected]>

rickysarraf force-pushed the obs-build-rickysarraf/master branch from a68beef to 3a32943 Compare March 20, 2025 08:24

andrewshadura added 3 commits March 20, 2025 19:25

Consistently use deb_chroot instead of chroot where it makes a differ…

98f8dbf

…ence Signed-off-by: Andrej Shadura <[email protected]> merge fallout Signed-off-by: Ritesh Raj Sarraf <[email protected]>

Escape arguments properly to avoid leaking spaces

5af853e

Signed-off-by: Andrej Shadura <[email protected]>

rickysarraf force-pushed the obs-build-rickysarraf/master branch from 4e273c6 to 01e5688 Compare March 20, 2025 15:06

obbardc approved these changes Mar 21, 2025

View reviewed changes

setup a cleaner build environment #665

Are you sure you want to change the base?

setup a cleaner build environment #665

Uh oh!

Conversation

rickysarraf commented Feb 23, 2021

Uh oh!

rickysarraf commented Feb 23, 2021

Uh oh!

mlschroe commented Feb 23, 2021

Uh oh!

coolo commented Feb 23, 2021

Uh oh!

mlschroe commented Feb 23, 2021

Uh oh!

coolo commented Feb 23, 2021

Uh oh!

andrewshadura commented Feb 26, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

emanueleaina Apr 12, 2021

Choose a reason for hiding this comment

Uh oh!

andrewshadura Apr 12, 2021

Choose a reason for hiding this comment

Uh oh!

obbardc Mar 21, 2025

Choose a reason for hiding this comment

Uh oh!

andrewshadura commented Dec 1, 2021

Uh oh!

rickysarraf commented Feb 1, 2022

Uh oh!

obbardc commented Mar 18, 2025

Uh oh!

nickbroon commented Mar 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rickysarraf commented Mar 20, 2025

Uh oh!

andrewshadura commented Mar 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rickysarraf commented Mar 20, 2025

Uh oh!

obbardc commented Mar 20, 2025

Uh oh!

rickysarraf commented Mar 20, 2025

Uh oh!

andrewshadura commented Mar 20, 2025

Uh oh!

obbardc left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

andrewshadura commented Feb 26, 2021 •

edited

Loading

nickbroon commented Mar 18, 2025 •

edited

Loading

andrewshadura commented Mar 20, 2025 •

edited

Loading