Engine stops working between version 7.2.4 and 8.0.0

[LVerneyEC]

Hi,

I've noticed a bug on my end when updating the engine past version 7.2.4.

Version 7.2.4 works perfectly fine across our VLOPSs tracking declarations: https://code.europa.eu/dsa/terms-and-conditions-database/vlops-and-vloses/vlop-vlose-declarations.

When updating to version 8.0.0, it crashes/hangs, usually on the "Apple App Store" policies (sometimes to the point that even a Ctrl+C would not kill the engine, and one has to force kill it). Last log lines would be:

[...]
2025-11-12T12:04:12+00:00 info  Amazon Store — Data Access for Vetted Researchers                           No changes after filtering, did not record version
2025-11-12T12:04:13+00:00 info  Amazon Store — Marketplace Sellers Conditions                               Recorded version with id 13c5e016809d3c7a0c889ee02807e122918d8db0
2025-11-12T12:04:13+00:00 info  Apple App Store — Apple Developer Agreement                                 No changes after filtering, did not record version
2025-11-12T12:04:14+00:00 info  Apple App Store — Claims of Infringement                                    Recorded version with id c597b2c42b13315c91e187b0d6b93747b42da08f
2025-11-12T12:04:14+00:00 info  Apple App Store — Developer Program License Agreement                       Recorded version with id 93bda281b8223a097d12c324f6342a51f70ebd6f
2025-11-12T12:04:15+00:00 info  Apple App Store — Media Services Terms and Conditions                       No changes after filtering, did not record version
2025-11-12T12:04:15+00:00 info  Apple App Store — Redress Rights                                            No changes after filtering, did not record version

(not an exact match, sometimes it happens slightly before, sometimes slightly after, but always around the Apple App Store policies).

I am using Node v20.19.5, which should be fully supported.

Tests done:

• Running on single service (including "Apple App Store") works fine.
• Updating to the latest version (9.2.0) still exhibits the same behavior.
• Internet uplink should not be an issue here, and anyways it should crash with a proper timeout/error/socket hang up and not block the scraping process.

Steps to reproduce:

$ git clone https://github.com/OpenTermsArchive/engine && git checkout v8.0.0
$ nvm use lts/iron
$ node -v
v20.19.5
$ npm -v
10.8.2
$ npm ci
npm warn deprecated inflight@1.0.6: This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful.
npm warn deprecated lodash.get@4.4.2: This package is deprecated. Use the optional chaining (?.) operator instead.
npm warn deprecated lodash.isequal@4.5.0: This package is deprecated. Use require('node:util').isDeepStrictEqual instead.
npm warn deprecated @humanwhocodes/config-array@0.13.0: Use @eslint/config-array instead
npm warn deprecated rimraf@3.0.2: Rimraf versions prior to v4 are no longer supported
npm warn deprecated @humanwhocodes/object-schema@2.0.3: Use @eslint/object-schema instead
npm warn deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported
npm warn deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported
npm warn deprecated glob@7.1.6: Glob versions prior to v9 are no longer supported
npm warn deprecated formidable@1.2.6: Please upgrade to latest, formidable@v2 or formidable@v3! Check these notes: https://bit.ly/2ZEqIau
npm warn deprecated querystring@0.2.0: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
npm warn deprecated node-domexception@1.0.0: Use your platform's native DOMException instead
npm warn deprecated text-encoding@0.7.0: no longer maintained
npm warn deprecated superagent@3.7.0: Please upgrade to superagent v10.2.2+, see release notes at https://github.com/forwardemail/superagent/releases/tag/v10.2.2 - maintenance is supported by Forward Email @ https://forwardemail.net
npm warn deprecated @accordproject/concerto-core@3.19.8: This version of the package is deprecated
npm warn deprecated @accordproject/markdown-pdf@0.16.25: Not maintained
npm warn deprecated eslint@8.57.1: This version is no longer supported. Please see https://eslint.org/version-support for other options.
npm warn deprecated sib-api-v3-sdk@8.5.0: Package no longer supported. Contact Support at https://www.npmjs.com/support for more info.

added 946 packages in 10s

200 packages are looking for funding
  run `npm fund` for details

$ npx ota track
[... - crash as above]

The latter being run with our own set of declarations (note that you need to run the ./build.sh bash script to generate JSON declarations) and an empty data/{snapshots,versions} folder.

I don't see anything specific around the terms where it is crashing (could be PDF handling, but terms with PDF is not the blocking one and always processed ; could be fullDom/htmlOnly, but both are already in terms before ; could be a specific website or URL, but it's semi-random and it's only happening above a specific version of the engine).

Thanks,
Best

[Ndpnt]

Hi @LVerneyEC,

Thanks for the detailed report.

I followed your reproduction steps but wasn’t able to reproduce the issue, it runs fine on my local machine. Could you confirm whether this happens only on your server or also on your local machine?

Also, could you share how much RAM is available on the system where it fails? We’ve seen similar hangs before when the process was terminated by the system due to high memory usage.

You might try running the engine with a higher memory limit, for example (adjusting 1536 based on your system’s available memory):

node --max-old-space-size=1536 ./bin/ota.js track

[LVerneyEC]

@Ndpnt did you try from scratch or from a full clone of the current state across the 3 repositories (snapshots / declarations / versions)?

My system has plenty of free RAM (39GB) and uplink is not supposed to be an issue in this situation (no corporate proxy or such).

EDIT: I notice this is also happening for me in the initial refiltering step, so it might be state-dependent and happening based on current content of the snapshots.

EDIT2: When this is happening the ota-track.js script is running at 100% CPU for up to 10 minutes and still blocking. Only a SIGKILL can kill it.

[LVerneyEC]

Sorry for the double posting, but I managed to narrow it down to a reproducible set.

First, I switched the concurrency manually in https://github.com/OpenTermsArchive/engine/blob/main/src/archivist/index.js#L23 to have max 1 concurrent thread. Then, the error was happening always at the same location and I pinpointed it to

npx ota track -s Booking.com -t 'Animal Welfare Standards'

which is this declaration. It works perfectly fine with engine version 7.2.4 and crashes for engine version 8.0.0 and above.

If I clear the snapshot/versions cache, it succeeds in fetching the snapshot but is stuck as well on extraction of the version.

I tried to disable any filter and use the simplest selector (select: body) to avoid any complex processing issue, but this issue still arises.

It crashes on this line:

engine/src/archivist/extract/dom.js

Lines 4 to 7 in aec5133

	const { document } = new jsdom.JSDOM(content, {
	url: location,
	virtualConsole: new jsdom.VirtualConsole(),
	}).window;

Seems very similar to jsdom/jsdom#2176 but no precise solution.

EDIT: I tried to use latest engine version 9.2.0 but pinning old dependencies from 7.2.4 but no success:

$ cat package.json
{
  "type": "module",
  "dependencies": {
    "@opentermsarchive/engine": "^9.2.0",
        "@stylistic/eslint-plugin-js": "^1.4.1",
    "abort-controller": "^3.0.0",
    "ajv": "^8.17.1",
    "ajv-formats": "^3.0.1",
    "archiver": "^5.3.0",
    "async": "^3.2.2",
    "chai": "^4.3.4",
    "chai-as-promised": "^7.1.1",
    "chai-exclude": "^2.1.0",
    "commander": "^9.4.1",
    "config": "^4.1.1",
    "croner": "^8.0.2",
    "cronstrue": "^2.50.0",
    "cross-env": "^7.0.3",
    "datauri": "^4.1.0",
    "dotenv": "^10.0.0",
    "eslint": "^8.53.0",
    "eslint-config-airbnb-base": "^15.0.0",
    "eslint-plugin-chai-friendly": "^0.7.2",
    "eslint-plugin-import": "^2.29.0",
    "eslint-plugin-jsdoc": "^50.6.9",
    "eslint-plugin-json-format": "^2.0.1",
    "eslint-plugin-no-only-tests": "^3.1.0",
    "express": "^4.19.2",
    "express-async-errors": "^3.1.1",
    "fetch-charset-detection": "^1.0.1",
    "fs-extra": "^10.0.0",
    "helmet": "^6.0.1",
    "http-proxy-agent": "^5.0.0",
    "https": "^1.0.0",
    "https-proxy-agent": "^5.0.0",
    "iconv-lite": "^0.6.3",
    "joplin-turndown-plugin-gfm": "^1.0.12",
    "js-yaml": "^4.1.0",
    "jsdom": "^18.1.0",
    "json-source-map": "^0.6.1",
    "lodash": "^4.17.21",
    "mime": "^2.5.2",
    "mocha": "^9.1.3",
    "mongodb": "^4.9.0",
    "morgan": "^1.10.0",
    "node-fetch": "^3.1.0",
    "octokit": "2.0.2",
    "pdfjs-dist": "^2.9.359",
    "puppeteer": "^22.8.1",
    "puppeteer-extra": "^3.3.6",
    "puppeteer-extra-plugin-stealth": "^2.11.2",
    "sib-api-v3-sdk": "^8.2.1",
    "simple-git": "^3.27.0",
    "swagger-jsdoc": "^6.2.8",
    "swagger-ui-express": "^5.0.0",
    "winston": "^3.9.0",
    "winston-mail": "^2.0.0"
  }
}

(also with only jsdom pinned to 18.1.1 - checked in node_modules as well)

[LVerneyEC]

And last debugging bits, I think we are close to the root cause here:

• Install from a separate package.json crashes. My package.json is:

{
  "type": "module",
  "dependencies": {
    "@opentermsarchive/engine": "^9.2.0"
  }
}

• Install from git clone https://github.com/OpenTermsArchive/engine; cd engine; npm ci or git clone https://github.com/OpenTermsArchive/engine; cd engine; npm i works.

• Install from git clone https://github.com/OpenTermsArchive/engine; cd engine; rm -rf node_modules package-lock.json; npm i breaks the same as first point above.