Docker Compose Up does not work

[dneville-cvs]

Bug description

When I run docker compose up as I'm trying to follow this guide, I get the following error:

 => ERROR [superset-worker-beat dev 5/7] RUN apt-get update -qq     && apt-get install -yqq --no-install-recommends wget bzip2     && wget -q https://github.com/mozilla/geckodriver/release  3.8s
------
 > [superset-worker-beat dev 5/7] RUN apt-get update -qq     && apt-get install -yqq --no-install-recommends wget bzip2     && wget -q https://github.com/mozilla/geckodriver/releases/download/v0.34.0/geckodriver-v0.34.0-linux64.tar.gz -O - | tar xfz - -C /usr/local/bin     && wget -q https://download-installer.cdn.mozilla.net/pub/firefox/releases/125.0.3/linux-x86_64/en-US/firefox-125.0.3.tar.bz2 -O - | tar xfj - -C /opt     && ln -s /opt/firefox/firefox /usr/local/bin/firefox     && apt-get autoremove -yqq --purge wget bzip2 && rm -rf /var/[log,tmp]/* /tmp/* /var/lib/apt/lists/*:
2.653 debconf: delaying package configuration, since apt-utils is not installed
2.672 Selecting previously unselected package bzip2.
(Reading database ... 26168 files and directories currently installed.)
2.680 Preparing to unpack .../bzip2_1.0.8-5+b1_arm64.deb ...
2.680 Unpacking bzip2 (1.0.8-5+b1) ...
2.690 Selecting previously unselected package wget.
2.691 Preparing to unpack .../wget_1.21.3-1+b1_arm64.deb ...
2.691 Unpacking wget (1.21.3-1+b1) ...
2.738 Setting up wget (1.21.3-1+b1) ...
2.741 Setting up bzip2 (1.0.8-5+b1) ...
3.809 
3.809 bzip2: Compressed file ends unexpectedly;
3.809   perhaps it is corrupted?  *Possible* reason follows.
3.809 bzip2: Inappropriate ioctl for device
3.809   Input file = (stdin), output file = (stdout)
3.809 
3.809 It is possible that the compressed file(s) have become corrupted.
3.809 You can use the -tvv option to test integrity of such files.
3.809 
3.809 You can use the `bzip2recover' program to attempt to recover
3.809 data from undamaged sections of corrupted files.
3.809 
3.809 tar: Child returned status 2
3.809 tar: Error is not recoverable: exiting now
------
failed to solve: process "/bin/sh -c apt-get update -qq     && apt-get install -yqq --no-install-recommends wget bzip2     && wget -q https://github.com/mozilla/geckodriver/releases/download/${GECKODRIVER_VERSION}/geckodriver-${GECKODRIVER_VERSION}-linux64.tar.gz -O - | tar xfz - -C /usr/local/bin     && wget -q https://download-installer.cdn.mozilla.net/pub/firefox/releases/${FIREFOX_VERSION}/linux-x86_64/en-US/firefox-${FIREFOX_VERSION}.tar.bz2 -O - | tar xfj - -C /opt     && ln -s /opt/firefox/firefox /usr/local/bin/firefox     && apt-get autoremove -yqq --purge wget bzip2 && rm -rf /var/[log,tmp]/* /tmp/* /var/lib/apt/lists/*" did not complete successfully: exit code: 2

How to reproduce the bug

1. Clone the repo
2. run docker compose up

Screenshots/recordings

No response

Superset version

master / latest-dev

Python version

3.9

Node version

16

Browser

Chrome

Additional context

I am on an M2 Macbook Pro. I have made no modifications to the repo. I have made sure I'm on an up-to-date master branch. I have turned all the Docker resource settings to max.

Checklist

• I have searched Superset docs and Slack and didn't find a solution to my problem.
• I have searched the GitHub issue tracker and didn't find a similar bug report.
• I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

[dosubot]

Hey @dneville-cvs! I'm here to help you with your Docker Compose issue. I'm Dosu, a bot that can assist you in solving bugs, answering questions, and becoming a contributor. Let's work on resolving this together!

To resolve the error encountered when running docker compose up on your M2 Macbook Pro, you can try the following steps:

1. Set the environment variable BUILD_SUPERSET_FRONTEND_IN_DOCKER to false:
   This can help if the build process is slow or encountering issues due to resource constraints.

   export BUILD_SUPERSET_FRONTEND_IN_DOCKER=false

2. Run the frontend build locally:
   Execute the following commands in your terminal to build the frontend assets locally instead of inside the Docker container.

   npm install
   npm run dev

3. Run Docker Compose:
   After setting the environment variable and building the frontend locally, you can proceed with running Docker Compose.

   docker compose up

Additionally, ensure you are using the correct versions of geckodriver and firefox as specified in the Dockerfile:

• geckodriver: v0.29.0
• firefox: ESR (Extended Support Release)

This information is found in the following section of the Dockerfile:

FROM apache/superset:3.1.0

USER root

RUN apt-get update && \
    apt-get install --no-install-recommends -y firefox-esr

ENV GECKODRIVER_VERSION=0.29.0
RUN wget -q https://github.com/mozilla/geckodriver/releases/download/v${GECKODRIVER_VERSION}/geckodriver-v${GECKODRIVER_VERSION}-linux64.tar.gz && \
    tar -x geckodriver -zf geckodriver-v${GECKODRIVER_VERSION}-linux64.tar.gz -O > /usr/bin/geckodriver && \
    chmod 755 /usr/bin/geckodriver && \
    rm geckodriver-v${GECKODRIVER_VERSION}-linux64.tar.gz

RUN pip install --no-cache gevent psycopg2 redis

USER superset

These steps should help mitigate issues related to building assets within the Docker container, especially on resource-constrained devices like an M2 Macbook Pro ^[1][2].

To continue talking to Dosu, mention @dosu.

[rusackas]

The bot might be right about this being a resource constraing, but I haven't seen if fail at that step before, so I'll also CC/assign our resident Docker guru, @mistercrunch.

[dneville-cvs]

@dosu I have followed the recommended steps, but I'm still encountering the error.

superset/superset-frontend - (master) > echo $BUILD_SUPERSET_FRONTEND_IN_DOCKER
false

I ran npm install and npm run dev. The end output of npm run dev:

990 assets
14168 modules
webpack 5.89.0 compiled successfully in 3987 ms

I also confirmed geckodriver and firefox:

superset/superset-frontend - (master) > brew info geckodriver
==> geckodriver: stable 0.34.0 (bottled), HEAD
WebDriver <-> Marionette proxy
https://github.com/mozilla/geckodriver
Installed
/opt/homebrew/Cellar/geckodriver/0.34.0 (7 files, 6.3MB) *
  Poured from bottle using the formulae.brew.sh API on 2024-07-10 at 14:49:06
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/g/geckodriver.rb
License: MPL-2.0
==> Dependencies
Build: rust ✘
==> Options
--HEAD
        Install HEAD version
==> Analytics
install: 509 (30 days), 1,713 (90 days), 9,307 (365 days)
install-on-request: 509 (30 days), 1,711 (90 days), 9,253 (365 days)
build-error: 0 (30 days)
superset/superset-frontend - (master) > brew info firefox
==> firefox: 128.0 (auto_updates)
https://www.mozilla.org/firefox/
Installed
/opt/homebrew/Caskroom/firefox/128.0 (226B)
From: https://github.com/Homebrew/homebrew-cask/blob/HEAD/Casks/f/firefox.rb
==> Name
Mozilla Firefox
==> Description
Web browser
==> Languages
af, ar, be, bg, bn, ca, cs, de, en-CA, en-GB, en, eo, es-AR, es-CL, es-ES, fa, ff, fi, fr, gl, gn, gu, he, hi, in, it, ja, ka, ko, mr, my, ne, nl, pa-IN, pl, pt-BR, pt, ru, si, sq, sr, sv, ta, te, th, tl, tr, uk, ur, zh-TW, zh
==> Artifacts
Firefox.app (App)
/opt/homebrew/Caskroom/firefox/128.0/firefox.wrapper.sh -> firefox (Binary)
==> Analytics
install: 11,721 (30 days), 34,787 (90 days), 142,799 (365 days)

However, it was still this error:

 => ERROR [superset-worker dev 5/7] RUN apt-get update -qq     && apt-get install -yqq --no-install-recommends wget bzip2     && wget -q https://github.com/mozilla/geckodriver/releases/dow  4.4s
------
 > [superset-worker dev 5/7] RUN apt-get update -qq     && apt-get install -yqq --no-install-recommends wget bzip2     && wget -q https://github.com/mozilla/geckodriver/releases/download/v0.34.0/geckodriver-v0.34.0-linux64.tar.gz -O - | tar xfz - -C /usr/local/bin     && wget -q https://download-installer.cdn.mozilla.net/pub/firefox/releases/125.0.3/linux-x86_64/en-US/firefox-125.0.3.tar.bz2 -O - | tar xfj - -C /opt     && ln -s /opt/firefox/firefox /usr/local/bin/firefox     && apt-get autoremove -yqq --purge wget bzip2 && rm -rf /var/[log,tmp]/* /tmp/* /var/lib/apt/lists/*:
3.104 debconf: delaying package configuration, since apt-utils is not installed
3.131 Selecting previously unselected package bzip2.
(Reading database ... 26168 files and directories currently installed.)
3.220 Preparing to unpack .../bzip2_1.0.8-5+b1_arm64.deb ...
3.221 Unpacking bzip2 (1.0.8-5+b1) ...
3.236 Selecting previously unselected package wget.
3.237 Preparing to unpack .../wget_1.21.3-1+b1_arm64.deb ...
3.238 Unpacking wget (1.21.3-1+b1) ...
3.289 Setting up wget (1.21.3-1+b1) ...
3.292 Setting up bzip2 (1.0.8-5+b1) ...
4.356 
4.356 bzip2: Compressed file ends unexpectedly;
4.356   perhaps it is corrupted?  *Possible* reason follows.
4.356 bzip2: Inappropriate ioctl for device
4.356   Input file = (stdin), output file = (stdout)
4.356 
4.356 It is possible that the compressed file(s) have become corrupted.
4.356 You can use the -tvv option to test integrity of such files.
4.356 
4.356 You can use the `bzip2recover' program to attempt to recover
4.356 data from undamaged sections of corrupted files.
4.356 
4.357 tar: Child returned status 2
4.357 tar: Error is not recoverable: exiting now
------
failed to solve: process "/bin/sh -c apt-get update -qq     && apt-get install -yqq --no-install-recommends wget bzip2     && wget -q https://github.com/mozilla/geckodriver/releases/download/${GECKODRIVER_VERSION}/geckodriver-${GECKODRIVER_VERSION}-linux64.tar.gz -O - | tar xfz - -C /usr/local/bin     && wget -q https://download-installer.cdn.mozilla.net/pub/firefox/releases/${FIREFOX_VERSION}/linux-x86_64/en-US/firefox-${FIREFOX_VERSION}.tar.bz2 -O - | tar xfj - -C /opt     && ln -s /opt/firefox/firefox /usr/local/bin/firefox     && apt-get autoremove -yqq --purge wget bzip2 && rm -rf /var/[log,tmp]/* /tmp/* /var/lib/apt/lists/*" did not complete successfully: exit code: 2

[dosubot]

The error you're encountering suggests that the downloaded files might be corrupted. This can sometimes happen due to network issues or problems with the source files. Here are a few steps you can try to resolve this:

1. Verify Network Stability:
   Ensure your network connection is stable. Sometimes, network issues can cause incomplete or corrupted downloads.

2. Clear Docker Cache:
   Clear Docker's build cache to ensure that no corrupted files are being reused.

   docker builder prune

3. Manually Download and Verify Files:
   Manually download the geckodriver and firefox files to verify their integrity.

   wget https://github.com/mozilla/geckodriver/releases/download/v0.34.0/geckodriver-v0.34.0-linux64.tar.gz
   wget https://download-installer.cdn.mozilla.net/pub/firefox/releases/125.0.3/linux-x86_64/en-US/firefox-125.0.3.tar.bz2

   After downloading, you can test the integrity of these files:

   tar -tzf geckodriver-v0.34.0-linux64.tar.gz
   tar -tjf firefox-125.0.3.tar.bz2

4. Update Dockerfile:
   If the manual download works, you can update the Dockerfile to use the local files instead of downloading them during the build process. Place the downloaded files in a directory and update the Dockerfile to copy them:

   COPY geckodriver-v0.34.0-linux64.tar.gz /tmp/
   COPY firefox-125.0.3.tar.bz2 /tmp/
   
   RUN tar xfz /tmp/geckodriver-v0.34.0-linux64.tar.gz -C /usr/local/bin \
       && tar xfj /tmp/firefox-125.0.3.tar.bz2 -C /opt \
       && ln -s /opt/firefox/firefox /usr/local/bin/firefox \
       && apt-get autoremove -yqq --purge wget bzip2 && rm -rf /var/[log,tmp]/* /tmp/* /var/lib/apt/lists/*

These steps should help you bypass the issue with corrupted downloads during the Docker build process. If the issue persists, @mistercrunch, our resident Docker expert, might have additional insights.

To continue talking to Dosu, mention @dosu.

[dneville-cvs]

Well, following @dosu 's advice, I at least got a different error.

Docker builder prune, then manually download and verify files and edit the Dockerfile as instructed, and the result is below:

=> ERROR [superset dev 7/9] RUN tar xfz /tmp/geckodriver-v0.34.0-linux64.tar.gz -C /usr/local/bin     && tar xfj /tmp/firefox-125.0.3.tar.bz2 -C /opt     && ln -s /opt/firefox/firefox /us  0.1s
------
 > [superset dev 7/9] RUN tar xfz /tmp/geckodriver-v0.34.0-linux64.tar.gz -C /usr/local/bin     && tar xfj /tmp/firefox-125.0.3.tar.bz2 -C /opt     && ln -s /opt/firefox/firefox /usr/local/bin/firefox     && apt-get autoremove -yqq --purge wget bzip2 && rm -rf /var/[log,tmp]/* /tmp/* /var/lib/apt/lists/*:
0.083 
0.083 gzip: stdin: unexpected end of file
0.083 tar: Child returned status 1
0.083 tar: Error is not recoverable: exiting now
------
failed to solve: process "/bin/sh -c tar xfz /tmp/geckodriver-v0.34.0-linux64.tar.gz -C /usr/local/bin     && tar xfj /tmp/firefox-125.0.3.tar.bz2 -C /opt     && ln -s /opt/firefox/firefox /usr/local/bin/firefox     && apt-get autoremove -yqq --purge wget bzip2 && rm -rf /var/[log,tmp]/* /tmp/* /var/lib/apt/lists/*" did not complete successfully: exit code: 2

[dosubot]

To resolve the issue of the geckodriver file being corrupted or incomplete during the Docker build process, you can add a checksum verification step to ensure the integrity of the downloaded file. Here is an updated snippet of the Dockerfile with the checksum verification added:

# Install GeckoDriver WebDriver
ARG GECKODRIVER_VERSION=v0.34.0 \
    FIREFOX_VERSION=125.0.3

# Add the checksum for the geckodriver version you are downloading
ARG GECKODRIVER_CHECKSUM=your_geckodriver_checksum_here

RUN apt-get update -qq \
    && apt-get install -yqq --no-install-recommends wget bzip2 \
    && wget -q https://github.com/mozilla/geckodriver/releases/download/${GECKODRIVER_VERSION}/geckodriver-${GECKODRIVER_VERSION}-linux64.tar.gz -O geckodriver.tar.gz \
    && echo "${GECKODRIVER_CHECKSUM} geckodriver.tar.gz" | sha256sum -c - \
    && tar xfz geckodriver.tar.gz -C /usr/local/bin \
    && rm geckodriver.tar.gz \
    # Install Firefox
    && wget -q https://download-installer.cdn.mozilla.net/pub/firefox/releases/${FIREFOX_VERSION}/linux-x86_64/en-US/firefox-${FIREFOX_VERSION}.tar.bz2 -O firefox.tar.bz2 \
    && tar xfj firefox.tar.bz2 -C /opt \
    && ln -s /opt/firefox/firefox /usr/local/bin/firefox \
    && rm firefox.tar.bz2 \
    && apt-get autoremove -yqq --purge wget bzip2 && rm -rf /var/[log,tmp]/* /tmp/* /var/lib/apt/lists/*

Make sure to replace your_geckodriver_checksum_here with the actual checksum of the geckodriver version you are downloading. You can find the checksum on the Geckodriver releases page. This will ensure that the file is not corrupted or incomplete during the download process ^[1].

To continue talking to Dosu, mention @dosu.

[mistercrunch]

Are you on Windows/cygwin? In any case, the command that seems to fail is an apt-get command - and that has little to do with Superset. You're minimal repro seem like it'd be to strip the Dockerfile to that one command and would result in something like "I can't install some standard ubuntu library on my setup" which has nothing to do with Superset.

transcode-open/apt-cyg#130 ?

[dneville-cvs]

@mistercrunch I am on macOS 14.5 (23F79).

I created a minimal Dockerfile:

ARG PY_VER=3.10-slim-bookworm

FROM python:${PY_VER} AS dev

USER root

ARG GECKODRIVER_VERSION=v0.34.0 \
    FIREFOX_VERSION=125.0.3

RUN apt-get update -qq \
    && apt-get install -yqq --no-install-recommends wget bzip2 \
    && wget -q https://github.com/mozilla/geckodriver/releases/download/${GECKODRIVER_VERSION}/geckodriver-${GECKODRIVER_VERSION}-linux64.tar.gz -O - | tar xfz - -C /usr/local/bin \
    # Install Firefox
    && wget -q https://download-installer.cdn.mozilla.net/pub/firefox/releases/${FIREFOX_VERSION}/linux-x86_64/en-US/firefox-${FIREFOX_VERSION}.tar.bz2 -O - | tar xfj - -C /opt \
    && ln -s /opt/firefox/firefox /usr/local/bin/firefox \
    && apt-get autoremove -yqq --purge wget bzip2 && rm -rf /var/[log,tmp]/* /tmp/* /var/lib/apt/lists/*

I ran this command to build: docker build -t bs_test -f test_dockerfile --platform linux/amd64 .

I got the same results:

 => [internal] load build definition from test_dockerfile                                                                                                                                     0.0s
 => => transferring dockerfile: 1.16kB                                                                                                                                                        0.0s
 => [internal] load metadata for docker.io/library/python:3.10-slim-bookworm                                                                                                                  0.4s
 => [internal] load .dockerignore                                                                                                                                                             0.0s
 => => transferring context: 1.30kB                                                                                                                                                           0.0s
 => CACHED [1/2] FROM docker.io/library/python:3.10-slim-bookworm@sha256:3be54aca807a43b5a1fa2133b1cbb4b58a018d6ebb1588cf1050b7cbebf15d55                                                     0.0s
 => ERROR [2/2] RUN apt-get update -qq     && apt-get install -yqq --no-install-recommends wget bzip2     && wget -q https://github.com/mozilla/geckodriver/releases/download/v0.34.0/geckod  7.9s
------                                                                                                                                                                                             
 > [2/2] RUN apt-get update -qq     && apt-get install -yqq --no-install-recommends wget bzip2     && wget -q https://github.com/mozilla/geckodriver/releases/download/v0.34.0/geckodriver-v0.34.0-linux64.tar.gz -O - | tar xfz - -C /usr/local/bin     && wget -q https://download-installer.cdn.mozilla.net/pub/firefox/releases/125.0.3/linux-x86_64/en-US/firefox-125.0.3.tar.bz2 -O - | tar xfj - -C /opt     && ln -s /opt/firefox/firefox /usr/local/bin/firefox     && apt-get autoremove -yqq --purge wget bzip2 && rm -rf /var/[log,tmp]/* /tmp/* /var/lib/apt/lists/*:                       
6.220 debconf: delaying package configuration, since apt-utils is not installed                                                                                                                    
6.332 Selecting previously unselected package bzip2.                                                                                                                                               
(Reading database ... 6696 files and directories currently installed.)
6.336 Preparing to unpack .../bzip2_1.0.8-5+b1_amd64.deb ...
6.339 Unpacking bzip2 (1.0.8-5+b1) ...
6.420 Selecting previously unselected package libpsl5:amd64.
6.421 Preparing to unpack .../libpsl5_0.21.2-1_amd64.deb ...
6.422 Unpacking libpsl5:amd64 (0.21.2-1) ...
6.503 Selecting previously unselected package wget.
6.504 Preparing to unpack .../wget_1.21.3-1+b2_amd64.deb ...
6.505 Unpacking wget (1.21.3-1+b2) ...
6.609 Setting up libpsl5:amd64 (0.21.2-1) ...
6.612 Setting up wget (1.21.3-1+b2) ...
6.616 Setting up bzip2 (1.0.8-5+b1) ...
6.618 Processing triggers for libc-bin (2.36-9+deb12u7) ...
7.882 
7.882 bzip2: Compressed file ends unexpectedly;
7.882   perhaps it is corrupted?  *Possible* reason follows.
7.882 bzip2: Inappropriate ioctl for device
7.882   Input file = (stdin), output file = (stdout)
7.883 
7.883 It is possible that the compressed file(s) have become corrupted.
7.883 You can use the -tvv option to test integrity of such files.
7.883 
7.883 You can use the `bzip2recover' program to attempt to recover
7.883 data from undamaged sections of corrupted files.
7.883 
7.886 tar: Child returned status 2
7.887 tar: Error is not recoverable: exiting now
------
test_dockerfile:18
--------------------
  17 |     
  18 | >>> RUN apt-get update -qq \
  19 | >>>     && apt-get install -yqq --no-install-recommends wget bzip2 \
  20 | >>>     && wget -q https://github.com/mozilla/geckodriver/releases/download/${GECKODRIVER_VERSION}/geckodriver-${GECKODRIVER_VERSION}-linux64.tar.gz -O - | tar xfz - -C /usr/local/bin \
  21 | >>>     # Install Firefox
  22 | >>>     && wget -q https://download-installer.cdn.mozilla.net/pub/firefox/releases/${FIREFOX_VERSION}/linux-x86_64/en-US/firefox-${FIREFOX_VERSION}.tar.bz2 -O - | tar xfj - -C /opt \
  23 | >>>     && ln -s /opt/firefox/firefox /usr/local/bin/firefox \
  24 | >>>     && apt-get autoremove -yqq --purge wget bzip2 && rm -rf /var/[log,tmp]/* /tmp/* /var/lib/apt/lists/*
  25 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c apt-get update -qq     && apt-get install -yqq --no-install-recommends wget bzip2     && wget -q https://github.com/mozilla/geckodriver/releases/download/${GECKODRIVER_VERSION}/geckodriver-${GECKODRIVER_VERSION}-linux64.tar.gz -O - | tar xfz - -C /usr/local/bin     && wget -q https://download-installer.cdn.mozilla.net/pub/firefox/releases/${FIREFOX_VERSION}/linux-x86_64/en-US/firefox-${FIREFOX_VERSION}.tar.bz2 -O - | tar xfj - -C /opt     && ln -s /opt/firefox/firefox /usr/local/bin/firefox     && apt-get autoremove -yqq --purge wget bzip2 && rm -rf /var/[log,tmp]/* /tmp/* /var/lib/apt/lists/*" did not complete successfully: exit code: 2

I'm curious if you get the same error.

[dneville-cvs]

ok, here's my true minimum repro:

ARG PY_VER=3.10-slim-bookworm

FROM python:${PY_VER} AS dev

USER root

ARG FIREFOX_VERSION=125.0.3

RUN apt-get update -qq
RUN apt-get install -yqq --no-install-recommends wget bzip2
RUN wget -q https://download-installer.cdn.mozilla.net/pub/firefox/releases/${FIREFOX_VERSION}/linux-x86_64/en-US/firefox-${FIREFOX_VERSION}.tar.bz2 -O - | tar xfj - -C /opt

scratch/superset - (master) > docker build -t bs_test -f test_dockerfile --platform linux/amd64 .                                                                        
[+] Building 7.3s (7/7) FINISHED                                                                                                                                            docker:desktop-linux
 => [internal] load build definition from test_dockerfile                                                                                                                                   0.0s
 => => transferring dockerfile: 402B                                                                                                                                                        0.0s
 => [internal] load metadata for docker.io/library/python:3.10-slim-bookworm                                                                                                                0.8s
 => [internal] load .dockerignore                                                                                                                                                           0.0s
 => => transferring context: 1.30kB                                                                                                                                                         0.0s
 => CACHED [1/4] FROM docker.io/library/python:3.10-slim-bookworm@sha256:3be54aca807a43b5a1fa2133b1cbb4b58a018d6ebb1588cf1050b7cbebf15d55                                                   0.0s
 => [2/4] RUN apt-get update -qq                                                                                                                                                            2.8s
 => [3/4] RUN apt-get install -yqq --no-install-recommends wget bzip2                                                                                                                       3.3s
 => ERROR [4/4] RUN wget -q https://download-installer.cdn.mozilla.net/pub/firefox/releases/125.0.3/linux-x86_64/en-US/firefox-125.0.3.tar.bz2 -O - | tar xfj - -C /opt                     0.4s
------                                                                                                                                                                                           
 > [4/4] RUN wget -q https://download-installer.cdn.mozilla.net/pub/firefox/releases/125.0.3/linux-x86_64/en-US/firefox-125.0.3.tar.bz2 -O - | tar xfj - -C /opt:                                
0.376                                                                                                                                                                                            
0.376 bzip2: Compressed file ends unexpectedly;                                                                                                                                                  
0.376   perhaps it is corrupted?  *Possible* reason follows.                                                                                                                                     
0.376 bzip2: Inappropriate ioctl for device
0.376   Input file = (stdin), output file = (stdout)
0.376 
0.376 It is possible that the compressed file(s) have become corrupted.
0.376 You can use the -tvv option to test integrity of such files.
0.376 
0.376 You can use the `bzip2recover' program to attempt to recover
0.376 data from undamaged sections of corrupted files.
0.376 
0.380 tar: Child returned status 2
0.380 tar: Error is not recoverable: exiting now
------
test_dockerfile:11
--------------------
   9 |     RUN apt-get update -qq
  10 |     RUN apt-get install -yqq --no-install-recommends wget bzip2
  11 | >>> RUN wget -q https://download-installer.cdn.mozilla.net/pub/firefox/releases/${FIREFOX_VERSION}/linux-x86_64/en-US/firefox-${FIREFOX_VERSION}.tar.bz2 -O - | tar xfj - -C /opt
  12 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c wget -q https://download-installer.cdn.mozilla.net/pub/firefox/releases/${FIREFOX_VERSION}/linux-x86_64/en-US/firefox-${FIREFOX_VERSION}.tar.bz2 -O - | tar xfj - -C /opt" did not complete successfully: exit code: 2

View build details: docker-desktop://dashboard/build/desktop-linux/desktop-linux/68ldam0gbqlt2ln5r6ssfuat5```

[mistercrunch]

Maybe a dumb question, but does the host have internet access? Can you wget anything else? Or maybe that particular domain is restricted somewhere in your network stack?

[dneville-cvs]

@mistercrunch definitely not a dumb question - I also tested just using wget directly on the terminal and it worked just fine with this command: wget -q https://download-installer.cdn.mozilla.net/pub/firefox/releases/125.0.3/linux-x86_64/en-US/firefox-125.0.3.tar.bz2 -O - | tar xfj -

But also, the wget for geckodriver works just fine in the Docker container, it is only Firefox that fails.

[mistercrunch]

Oh right, and it's pretty clear that its the tar command that fails saying bz2 is corrupted. Unclear what's happening here. I just tried to rerun that dockerfile above with the same build command and it succeeds on my machine.

$ docker build -t bs_test --platform linux/amd64 .
[+] Building 25.3s (9/9) FINISHED                                                                                                                                        docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                                                                     0.0s
 => => transferring dockerfile: 434B                                                                                                                                                     0.0s
 => [internal] load metadata for docker.io/library/python:3.10-slim-bookworm                                                                                                             2.1s
 => [auth] library/python:pull token for registry-1.docker.io                                                                                                                            0.0s
 => [internal] load .dockerignore                                                                                                                                                        0.0s
 => => transferring context: 1.30kB                                                                                                                                                      0.0s
 => [1/4] FROM docker.io/library/python:3.10-slim-bookworm@sha256:3be54aca807a43b5a1fa2133b1cbb4b58a018d6ebb1588cf1050b7cbebf15d55                                                       2.8s
 => => resolve docker.io/library/python:3.10-slim-bookworm@sha256:3be54aca807a43b5a1fa2133b1cbb4b58a018d6ebb1588cf1050b7cbebf15d55                                                       0.0s
 => => sha256:3be54aca807a43b5a1fa2133b1cbb4b58a018d6ebb1588cf1050b7cbebf15d55 9.13kB / 9.13kB                                                                                           0.0s
 => => sha256:8244f266195f442379acd0e7d8918b1ef3967050ebf49c40291e89d0a1ba8f94 1.94kB / 1.94kB                                                                                           0.0s
 => => sha256:de7ad0443e19e374770dc29d7c7677a2f9e2162918003747266fd3b4f983cd85 6.92kB / 6.92kB                                                                                           0.0s
 => => sha256:f11c1adaa26e078479ccdd45312ea3b88476441b91be0ec898a7e07bfd05badc 29.13MB / 29.13MB                                                                                         1.3s
 => => sha256:e9e504b3ee6224b21b006ddf334064b7764c2aadcc9d8be012cf44640f045483 3.51MB / 3.51MB                                                                                           0.6s
 => => sha256:455661270fd3aeac7e92f2eabcb2b9fd03c0f97dcc26aef6889555cb9ebc23a4 12.38MB / 12.38MB                                                                                         1.5s
 => => sha256:8c0cd2ed72c6f636fe88f580adf3f0c5f5efd9254dc3feece396ac26fe3b8eaf 231B / 231B                                                                                               0.8s
 => => sha256:3d2f440aae987a194992068ec8c982c0295ac62537989c70aeccce6fa1548fd6 3.16MB / 3.16MB                                                                                           1.2s
 => => extracting sha256:f11c1adaa26e078479ccdd45312ea3b88476441b91be0ec898a7e07bfd05badc                                                                                                0.8s
 => => extracting sha256:e9e504b3ee6224b21b006ddf334064b7764c2aadcc9d8be012cf44640f045483                                                                                                0.1s
 => => extracting sha256:455661270fd3aeac7e92f2eabcb2b9fd03c0f97dcc26aef6889555cb9ebc23a4                                                                                                0.3s
 => => extracting sha256:8c0cd2ed72c6f636fe88f580adf3f0c5f5efd9254dc3feece396ac26fe3b8eaf                                                                                                0.0s
 => => extracting sha256:3d2f440aae987a194992068ec8c982c0295ac62537989c70aeccce6fa1548fd6                                                                                                0.2s
 => [2/4] RUN apt-get update -qq                                                                                                                                                         9.3s
 => [3/4] RUN apt-get install -yqq --no-install-recommends wget bzip2                                                                                                                    2.2s
 => [4/4] RUN wget -q https://download-installer.cdn.mozilla.net/pub/firefox/releases/125.0.3/linux-x86_64/en-US/firefox-125.0.3.tar.bz2 -O - | tar xfj - -C /opt                        8.4s
 => exporting to image                                                                                                                                                                   0.3s
 => => exporting layers                                                                                                                                                                  0.3s
 => => writing image sha256:2346ca1c902c7fd2bf8ef74ad462f51e44669bb8f96b5f0c721e00b23c1a9c18                                                                                             0.0s
 => => naming to docker.io/library/bs_test                                                                                                                                               0.0s

If you can repro consistently and that's not just a cosmic ray type thing there's probably something in your network stack or local cdn.mozilla.net that's corrupted. Actually that's probably the best explanation, your particular regional instance of the mozilla.cdn has this one file corrupted. For the sake of this issue it could be good to share which region you're in so that we can check this hypothesis.

[dneville-cvs]

Ok, I think you are correct about it being something in my network stack... I just tried a vpn instead of my regular network and was unable to repro the issue there. I'll close this issue and open one with the network folks at work. Thanks for your help!