Problems Building NodeJS for AppEngine Flex

[dbburgess]

Problem

This is the root of the problem:

failed to fetch builder image 'gcr.io/gae-runtimes/buildpacks/google-gae-22/nodejs/builder:latest': image 'gcr.io/gae-runtimes/buildpacks/google-gae-22/nodejs/builder:latest' does not exist on the daemon: not found

Context

I have a build for AppEngine Flex Environment which started failing sometime between 2024-10-21T19:05:46.332616026Z and 2024-10-22T13:05:10.305881767Z. It seems to be failing to find the buildpack image gcr.io/gae-runtimes/buildpacks/google-gae-22/nodejs/builder:latest

This approach was informed by App Engine and Cloud Function Builders and Buildpacks in the README.md of this repository and the CloudBuild docs guide to Deploying to App Engine.

CloudBuild Config

This is what our cloudbuild.yaml file looks like:

options:
  logging: CLOUD_LOGGING_ONLY
steps:
  - name: gcr.io/k8s-skaffold/pack
    entrypoint: pack
    args:
      - build
      - us-east4-docker.pkg.dev/$PROJECT_ID/$PROJECT_ID-docker-images/[APP_NAME]
      - --network
      - cloudbuild
      - --builder
      - gcr.io/gae-runtimes/buildpacks/google-gae-22/nodejs/builder
      - --tag
      - us-east4-docker.pkg.dev/$PROJECT_ID/$PROJECT_ID-docker-images/[APP_NAME]:$TAG_NAME
      - --tag
      - us-east4-docker.pkg.dev/$PROJECT_ID/$PROJECT_ID-docker-images/[APP_NAME]:latest
      - --cache-image
      - us-east4-docker.pkg.dev/$PROJECT_ID/$PROJECT_ID-docker-images/[APP_NAME]:cached
      - --publish

CloudBuild Log

starting build "[redacted]"
FETCHSOURCE
Fetching storage object: gs:/[redacted]
Copying gs://[redacted].125466-3b822a8e4e4d42549783b9f9787f517f.tgz#1729602300167080...
/ [0 files][ 0.0 B/512.4 KiB] / [1 files][512.4 KiB/512.4 KiB]
Operation completed over 1 objects/512.4 KiB.
BUILD
Pulling image: gcr.io/k8s-skaffold/pack
Using default tag: latest
latest: Pulling from k8s-skaffold/pack
396c31837116: Pulling fs layer
9776b10d5c8c: Pulling fs layer
52cb9ac3197f: Pulling fs layer
9776b10d5c8c: Verifying Checksum
9776b10d5c8c: Download complete
396c31837116: Verifying Checksum
396c31837116: Download complete
52cb9ac3197f: Verifying Checksum
52cb9ac3197f: Download complete
396c31837116: Pull complete
9776b10d5c8c: Pull complete
52cb9ac3197f: Pull complete
Digest: sha256:[redacted]
Status: Downloaded newer image for gcr.io/k8s-skaffold/pack:latest
gcr.io/k8s-skaffold/pack:latest
ERROR: failed to build: failed to fetch builder image 'gcr.io/gae-runtimes/buildpacks/google-gae-22/nodejs/builder:latest': image 'gcr.io/gae-runtimes/buildpacks/google-gae-22/nodejs/builder:latest' does not exist on the daemon: not found
ERROR
ERROR: build step 0 "gcr.io/k8s-skaffold/pack" failed: step exited with non-zero status: 1

This same build was successful just yesterday:

2024-10-21T19:05:46.332616026Z
Status: Downloaded newer image for gcr.io/gae-runtimes/buildpacks/google-gae-22/nodejs/builder:latest

Can anyone provide insight on why this broke? Is this a temporary issue, or a change in the buildpacks that we need to adapt to?

[dbburgess]

This buildpack is still listed on the Builders page, but the repository lists the builder with a deprecated tag as of two days ago, which seems like probably why our build stopped working.

I can't find any details on the release notes or anywhere else. Are these builders going away?

[kennethye1]

We've renamed it to gcr.io/serverless-runtimes/google-22-full/builder/nodejs. I'll file a bug to update the documentation.

[dbburgess]

Ah, that is very helpful, thank you!