Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: warnings when building the Dockerfile #228

Open
HarikrishnanBalagopal opened this issue Jul 3, 2024 · 1 comment
Open

bug: warnings when building the Dockerfile #228

HarikrishnanBalagopal opened this issue Jul 3, 2024 · 1 comment

Comments

@HarikrishnanBalagopal
Copy link
Contributor

HarikrishnanBalagopal commented Jul 3, 2024

Overview

The following line produces warnings because the environment variables are not defined.

ENV CUDA_HOME="/usr/local/cuda" \
PATH="/usr/local/nvidia/bin:${CUDA_HOME}/bin:${PATH}" \
LD_LIBRARY_PATH="/usr/local/nvidia/lib:/usr/local/nvidia/lib64:$CUDA_HOME/lib64:$CUDA_HOME/extras/CUPTI/lib64:${LD_LIBRARY_PATH}"

Steps to reproduce

  1. Build the image according to the command in the build/README.md
$ docker build . -t sft-trainer:mytag -f build/Dockerfile
[+] Building 0.5s (29/29) FINISHED                                                                                                                                             docker:default
 => [internal] load build definition from Dockerfile                                                                                                                                     0.0s
 => => transferring dockerfile: 6.24kB                                                                                                                                                   0.0s
 => WARN: FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 24)                                                                                                          0.0s
 => WARN: FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 46)                                                                                                          0.0s
 => WARN: FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 53)                                                                                                          0.0s
 => WARN: FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 77)                                                                                                          0.0s
 => WARN: FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 101)                                                                                                         0.0s
 => WARN: FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 138)                                                                                                         0.0s
 => [internal] load metadata for registry.access.redhat.com/ubi9/ubi:latest                                                                                                              0.2s
 => [internal] load .dockerignore                                                                                                                                                        0.0s
 => => transferring context: 2B                                                                                                                                                          0.0s
 => [internal] settings cache mount permissions                                                                                                                                          0.0s
 => [internal] load build context                                                                                                                                                        0.1s
 => => transferring context: 95.82kB                                                                                                                                                     0.1s
 => [base 1/3] FROM registry.access.redhat.com/ubi9/ubi:latest@sha256:081c96d1b1c7cd1855722d01f1ca53360510443737b1eb33284c6c4c330e537c                                                   0.0s
 => CACHED [base 2/3] RUN dnf remove -y --disableplugin=subscription-manager         subscription-manager     && dnf install -y python3.11 procps     && ln -s /usr/bin/python3.11 /bin  0.0s
 => CACHED [base 3/3] RUN useradd -u 1000 tuning -m -g 0 --system &&     chmod g+rx /home/tuning                                                                                         0.0s
 => CACHED [release-base 1/1] RUN rpm -e $(dnf repoquery python3-* -q --installed) dnf python3 yum crypto-policies-scripts                                                               0.0s
 => CACHED [release 1/9] RUN mkdir -p /licenses                                                                                                                                          0.0s
 => CACHED [release 2/9] COPY LICENSE /licenses/                                                                                                                                         0.0s
 => CACHED [release 3/9] RUN mkdir /app &&     chown -R tuning:0 /app /tmp &&     chmod -R g+rwX /app /tmp                                                                               0.0s
 => CACHED [release 4/9] COPY build/accelerate_launch.py fixtures/accelerate_fsdp_defaults.yaml /app/                                                                                    0.0s
 => CACHED [release 5/9] COPY build/utils.py /app/build/                                                                                                                                 0.0s
 => CACHED [release 6/9] RUN chmod +x /app/accelerate_launch.py                                                                                                                          0.0s
 => CACHED [release 7/9] RUN touch /.aim_profile &&     chmod -R 777 /.aim_profile &&     mkdir /.cache &&     chmod -R 777 /.cache                                                      0.0s
 => CACHED [release 8/9] WORKDIR /app                                                                                                                                                    0.0s
 => CACHED [cuda-base 1/1] RUN dnf config-manager        --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel9/x86_64/cuda-rhel9.repo     && dnf install -y          0.0s
 => CACHED [cuda-devel 1/1] RUN dnf config-manager        --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel9/x86_64/cuda-rhel9.repo     && dnf install -y         0.0s
 => CACHED [python-installations 1/8] RUN dnf install -y git &&     rm -f /usr/share/doc/perl-Net-SSLeay/examples/server_key.pem &&     dnf clean all                                    0.0s
 => CACHED [python-installations 2/8] WORKDIR /tmp                                                                                                                                       0.0s
 => CACHED [python-installations 3/8] RUN --mount=type=cache,target=/home/tuning/.cache/pip,uid=1000     python -m pip install --user build                                              0.0s
 => CACHED [python-installations 4/8] COPY --chown=tuning:root tuning tuning                                                                                                             0.0s
 => CACHED [python-installations 5/8] COPY .git .git                                                                                                                                     0.0s
 => CACHED [python-installations 6/8] COPY pyproject.toml pyproject.toml                                                                                                                 0.0s
 => CACHED [python-installations 7/8] RUN if [[ -z "" ]];     then python -m build --wheel --outdir /tmp;     else pip download fms-hf-tuning== --dest /tmp --only-binary=:all: --no-de  0.0s
 => CACHED [python-installations 8/8] RUN --mount=type=cache,target=/home/tuning/.cache/pip,uid=1000     python -m pip install --user wheel &&     python -m pip install --user "$(head  0.0s
 => CACHED [release 9/9] COPY --from=python-installations /home/tuning/.local /home/tuning/.local                                                                                        0.0s
 => exporting to image                                                                                                                                                                   0.0s
 => => exporting layers                                                                                                                                                                  0.0s
 => => writing image sha256:ab460be904959c3fe3573dbd8a8790f4d12ccfca710baf5fbbb1990d538bc868                                                                                             0.0s
 => => naming to docker.io/library/sft-trainer:mytag                                                                                                                                     0.0s

 9 warnings found (use --debug to expand):
 - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 24)
 - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 46)
 - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 53)
 - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 77)
 - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 101)
 - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 138)
 - UndefinedVar: Usage of undefined variable '$CUDA_HOME' (line 72)
 - UndefinedVar: Usage of undefined variable '$CUDA_HOME' (line 72)
 - UndefinedVar: Usage of undefined variable '$LD_LIBRARY_PATH' (line 72)
$ echo $?
0
@HarikrishnanBalagopal
Copy link
Contributor Author

These have been fixed

 - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 24)
 - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 46)
 - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 53)
 - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 77)
 - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 101)
 - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 138)

These still remain

 - UndefinedVar: Usage of undefined variable '$CUDA_HOME' (line 72)
 - UndefinedVar: Usage of undefined variable '$CUDA_HOME' (line 72)
 - UndefinedVar: Usage of undefined variable '$LD_LIBRARY_PATH' (line 72)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant