Lighthouse User Flow doens't work in Docker

[FabianSchoenfeld]

FAQ

• Yes, my issue is not about variability or throttling.
• Yes, my issue is not about a specific accessibility audit (file with axe-core instead).

URL

https://wikipedia.org

What happened?

The user flow doesn't work in docker image:

File to reproduce:

package.json

  "name": "lighthouse-userflow",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "type": "module",
  "dependencies": {
    "chrome-launcher": "^1.1.0",
    "cron": "^2.3.0",
    "express": "^4.18.2",
    "lighthouse": "^11.4.0",
    "prom-client": "^14.2.0",
    "puppeteer": "^21.7.0"
  },
  "scripts": {
    "start": "node index.js"
  }
}

index.js

import { LighthouseMetricsExporter } from "./lighthouseMetricsExporter.js";

// WEB VITALS and more
const newLighthouseMetrics = new LighthouseMetricsExporter();
await newLighthouseMetrics.init();

lighthouseMetricsExporter.js

import { startFlow } from "lighthouse";
import puppeteer, { KnownDevices } from "puppeteer";
import express from "express";

const config = [
  {
    url: "https://wikipedia.org",
    label: "startpage",
    devices: ["mobile"],
    click: ".lang-list-button-text",
    wait: "#js-lang-lists",
  },
];
export class LighthouseMetricsExporter {
  isRunning = false;
  gaugeOne;
  cronJob;
  server;

  constructor() {
    this.fireTick = this.fireTick.bind(this);
    this.onTick = this.onTick.bind(this);
    this.onComplete = this.onComplete.bind(this);
  }

  async init() {
    const labelNames = [
      "audit",
      "score",
      "page",
      "group",
      "country",
      "device",
      "asset_url",
      "url",
    ];

    this.server = express();
    this.server.get("/", (_, res) => {
      res.status(200).end("ok");
    });

    this.server.get("/metrics", async (req, res) => {
      try {
        res.set("Content-Type", register.contentType);
        res.end(await register.metrics());
      } catch (ex) {
        console.log("Error: ", ex);
        res.status(500).end(ex.message);
      }
    });
    this.server.listen(3000);
    this.fireTick(); // starts immediately
  }

  async fireTick() {
    if (this.isRunning) {
      console.warn("This CRON is already in execution");
      return;
    }
    this.isRunning = true;
    try {
      await this.onTick.bind(this)();
    } catch (exception) {
      console.error("Tick stopped due to an error");
      console.error(exception);
    }
    this.isRunning = false;
  }

  async onTick() {
    const browser = await puppeteer.launch({
      headless: "new",
      executablePath: process.env.CHROME_PATH
        ? process.env.CHROME_PATH
        : undefined,
      args: [
        "--no-sandbox",
        "--disable-setuid-sandbox",
        "--disable-dev-shm-usage",
        "--disable-accelerated-2d-canvas",
        "--no-first-run",
        "--no-zygote",
      ],
      timeout: 10_000, // 10 seconds
      protocolTimeout: 30_000, // 30 seconds
    });

    const flagsShared = {
      logLevel: "error",
      output: ["html"],
      onlyCategories: ["performance"],
      disableFullPageScreenshot: true,
    };

    const iPhone = KnownDevices["iPhone 6"];

    for (const { url, label, devices, click, wait } of config) {
      for (const device of devices) {
        const page = await browser.newPage();

        await page.emulate(iPhone);

        await page.goto(url);

        const flow = await startFlow(page, flagsShared);

        await flow.startTimespan();

        await page.click(click);
        await page.waitForSelector(wait);

        await flow.endTimespan();

        const flowResult = await flow.createFlowResult();

        this.printLighthouseMetric(flowResult.steps[0].lhr, label, device);
      }
    }
    await browser.close();
  }

  printLighthouseMetric(lighthouseReport, page, device) {
    const standardLabels = { page, device };
    const auditedItems = Object.keys(lighthouseReport.audits);
    auditedItems.forEach((auditItem) => {
      if (
        typeof lighthouseReport.audits[auditItem]?.numericValue === "number"
      ) {
        console.log(
          { audit: auditItem, ...standardLabels },
          lighthouseReport.audits[auditItem].numericValue
        );
      }
    });
    this.onComplete();
  }

  onComplete() {
    console.log("done");
    process.exit();
  }
}

Dockerfile

# Docker now supports running x86-64 (Intel) binaries on Apple silicon with Rosetta 2.
# https://www.docker.com/blog/docker-desktop-4-25/
#
# on arm you have to build for linux/amd64, e.g.: 
# docker build --platform linux/amd64 -f Dockerfile -t lighthouse-metrics-exporter .
# docker run --rm -p 3000:3000 --platform linux/amd64 -it --cap-add=SYS_ADMIN lighthouse-metrics-exporter:latest
#
# https://developer.chrome.com/blog/chrome-for-testing?hl=de
# https://github.com/puppeteer/puppeteer/blob/main/docker/Dockerfile

FROM node:21-bookworm-slim

ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD true
ENV CHROME_DEBUG_PORT=9222
ENV CHROME_PATH=/usr/bin/google-chrome

RUN apt-get update && apt-get install gnupg wget -y && \
  wget --quiet --output-document=- https://dl-ssl.google.com/linux/linux_signing_key.pub | gpg --dearmor > /etc/apt/trusted.gpg.d/google-archive.gpg && \
  sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' && \
  apt-get update && \
  apt-get install google-chrome-stable -y --no-install-recommends && \
  rm -rf /var/lib/apt/lists/*

WORKDIR /usr/src/app

COPY index.js lighthouseMetricsExporter.js package.json package-lock.json ./

RUN npm install

EXPOSE 3000

CMD ["node", "index.js"]

What did you expect?

Working like "node index.js". A complete user flow, with report, but at point await flow.startTimespan(); it stuck

What have you tried?

• ChromeLauncher
• different docker images
• different configurations for chromeLauncher, puppeteer

How were you running Lighthouse?

node, Other

Lighthouse Version

10.x

Chrome Version

latest

Node Version

21

OS

docker / debian

Relevant log output

*nodejs:*
> node index.js
{ audit: 'total-blocking-time', page: 'startpage', device: 'mobile' } 0
{
  audit: 'cumulative-layout-shift',
  page: 'startpage',
  device: 'mobile'
} 0
{
  audit: 'interaction-to-next-paint',
  page: 'startpage',
  device: 'mobile'
} 88
{
  audit: 'mainthread-work-breakdown',
  page: 'startpage',
  device: 'mobile'
} 178.5600000000001
{ audit: 'bootup-time', page: 'startpage', device: 'mobile' } 5.212
{ audit: 'uses-long-cache-ttl', page: 'startpage', device: 'mobile' } 0
{ audit: 'total-byte-weight', page: 'startpage', device: 'mobile' } 0
done
^C
> node index.js
{ audit: 'total-blocking-time', page: 'startpage', device: 'mobile' } 0
{
  audit: 'cumulative-layout-shift',
  page: 'startpage',
  device: 'mobile'
} 0
{
  audit: 'interaction-to-next-paint',
  page: 'startpage',
  device: 'mobile'
} 88
{
  audit: 'mainthread-work-breakdown',
  page: 'startpage',
  device: 'mobile'
} 202.50999999999993
{ audit: 'bootup-time', page: 'startpage', device: 'mobile' } 6.053
{ audit: 'uses-long-cache-ttl', page: 'startpage', device: 'mobile' } 0
{ audit: 'total-byte-weight', page: 'startpage', device: 'mobile' } 0
done

*docker:*
Tick stopped due to an error
ProtocolError: Runtime.evaluate timed out. Increase the 'protocolTimeout' setting in launch/connect calls for a higher timeout if needed.
    at <instance_members_initializer> (file:///usr/src/app/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:92:14)
    at new Callback (file:///usr/src/app/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:96:16)
    at CallbackRegistry.create (file:///usr/src/app/node_modules/puppeteer-core/lib/esm/puppeteer/common/CallbackRegistry.js:19:26)
    at Connection._rawSend (file:///usr/src/app/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/Connection.js:77:26)
    at CdpCDPSession.send (file:///usr/src/app/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/CDPSession.js:63:33)
    at #evaluate (file:///usr/src/app/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:178:18)
    at ExecutionContext.evaluateHandle (file:///usr/src/app/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:166:36)
    at file:///usr/src/app/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/ExecutionContext.js:55:29
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async LazyArg.get (file:///usr/src/app/node_modules/puppeteer-core/lib/esm/puppeteer/common/LazyArg.js:20:16)

[connorjclark]

I'd like a few things clarified before looking further into this.

Lighthouse Version
10.x

1. Your package.json mentions 11.4 - is this a typo?

Node Version
21

2. We only test on Node 18 currently. Does using 18 change the results for you?

3. Does this script work outside docker, specifically does it work directly on Linux?

4. I also wonder about the arm/x64 difference here. The official chrome package last I checked only distributes as x64. I noted your comment up top the Dockerfile about Rosetta, but all the same I'm not 100% sure this will work and it'd be good to rule it out. Do you happen to have an Intel mac to try the same setup? Even if it does work, Rosetta in my experience drastically reduces Lighthouse accuracy to the point of being useless cli: throw error if running x64 node on mac arm64 #14288

In general we don't support Docker, and specifically we advise against it as we expect it to result in less accurate results do to overhead in execution. That said, if you can answer these questions it would assist in us digging in further by ruling out some possible issues upfront.

[connorjclark]

Since your script is timing out, I do suspect the slowdown issues mentioned in point 4 above as being the culprit. Either from Rosetta (expected...) or Docker (but I wouldn't expect that much slowdown from Docker...) You could test this theory by increasing the global timeout here

[FabianSchoenfeld]

Thanks so for for your shared knowledge!

to 1.) Sorry, you are right. I use 11.4.x

to 2.) Node 18 doesn't change anything

to 3 & 4.) unfortunately no linux test so far. I'll looking for a older intel macos.

That could be an issue...

[ZengTianShengZ]

Very similar to my problem, I also got stuck running Lighthouse in Alpine Linux environment. This is the issue I raised, which can be used as additional information.

#15793

[FabianSchoenfeld]

With more power for the deployment (k8s) and node:18.bookworm-slim it runs at least in k8s cluster.