chore(ci): migrate ai-platform/snippets via ESM

[davidcavazos]

Based from #4100

Experiment to migrate CJS to ESM, adding "type": "module" to package.json.

cd ai-platform/snippets
npx eslint-cjs-to-esm "**/*.js" --fix
npx lebab --replace . --transform commonjs

• Please merge this PR for me once it is approved

Note: Any check with (dev), (experimental), or (legacy) can be ignored and should not block your PR from merging (see CI testing).

[gemini-code-assist]

Hello @davidcavazos, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

Summary of Changes

Hello team! Gemini here to provide a summary of this pull request.

This PR focuses on migrating the ai-platform/snippets directory from CommonJS (CJS) to ECMAScript Modules (ESM). This is an experimental change as noted in the description, utilizing tools like eslint-cjs-to-esm and lebab to automate the conversion process.

In addition to the module system migration, the PR also updates several package dependencies and development dependencies within the ai-platform/snippets package. A new CI configuration file is added, and the CI test configurations are updated to reflect that the ai-platform/snippets tests are now expected to pass in the main CI run.

Highlights

• ESM Migration: The core change is the migration of JavaScript files in ai-platform/snippets from CommonJS (require/module.exports) to ECMAScript Modules (import/export). This involves changes across numerous snippet and test files.
• Dependency Updates: Several dependencies and devDependencies in ai-platform/snippets/package.json have been updated to newer versions, including @google-cloud/aiplatform, @google-cloud/bigquery, chai, mocha, uuid, and sinon.
• CI Configuration Changes: The CI configuration is updated to remove ai-platform/snippets from the list of failing tests in the main CI and add it to the list of tests skipped in the dev CI (as they are expected to pass in prod). A new ci-setup.json file is added for the directory.

Changelog

Click here to see the changelog

• .github/config/nodejs-dev.jsonc
  • Added ai-platform/snippets to the list of directories skipped in dev CI (line 80).
• .github/config/nodejs.jsonc
  • Removed ai-platform/snippets from the list of directories with failing tests in main CI (line 78).
• ai-platform/snippets/ci-setup.json
  • Added new CI setup file with timeout and secret configuration (lines 1-6).
• ai-platform/snippets/expensive-test/create-data-labeling-job-image.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha, uuid) (lines 19-23).
• ai-platform/snippets/expensive-test/create-data-labeling-job-video.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha, uuid) (lines 19-23).
• ai-platform/snippets/expensive-test/create-data-labeling-job.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha, uuid) (lines 19-23).
• ai-platform/snippets/expensive-test/get-training-pipeline.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha) (lines 19-22).
• ai-platform/snippets/expensive-test/import-data-text-entity-extraction.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha) (lines 19-22).
• ai-platform/snippets/expensive-test/import-data-text-sentiment-analysis.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha) (lines 19-22).
• ai-platform/snippets/expensive-test/import-data-video-object-tracking.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha) (lines 19-22).
• ai-platform/snippets/gemma2PredictGpu.js
  • Converted module.exports to export default (line 68).
• ai-platform/snippets/gemma2PredictTpu.js
  • Converted module.exports to export default (line 71).
• ai-platform/snippets/imagen-edit-image-inpainting-insert-mask.js
  • Converted require('fs') to require('node:fs') and require('util') to require('node:util') (lines 47-48).
• ai-platform/snippets/imagen-edit-image-inpainting-remove-mask.js
  • Converted require('fs') to require('node:fs') and require('util') to require('node:util') (lines 47-48).
• ai-platform/snippets/imagen-edit-image-mask-free.js
  • Converted require('fs') to require('node:fs') and require('util') to require('node:util') (lines 46-47).
• ai-platform/snippets/imagen-edit-image-outpainting-mask.js
  • Converted require('fs') to require('node:fs') and require('util') to require('node:util') (lines 47-48).
• ai-platform/snippets/imagen-generate-image.js
  • Converted require('fs') to require('node:fs') and require('util') to require('node:util') (lines 45-46).
• ai-platform/snippets/imagen-get-short-form-image-captions.js
  • Converted require('fs') to require('node:fs') (line 45).
• ai-platform/snippets/imagen-get-short-form-image-responses.js
  • Converted require('fs') to require('node:fs') (line 47).
• ai-platform/snippets/package.json
  • Added "type": "module" to enable ESM (line 6).
  • Updated test script to include --reporter list (line 14).
  • Updated @google-cloud/aiplatform dependency from ^3.0.0 to ^4.0.0 (line 17).
  • Updated @google-cloud/bigquery dependency from ^7.0.0 to ^8.0.0 (line 18).
  • Updated chai devDependency from ^4.5.0 to ^5.0.0 (line 23).
  • Updated mocha devDependency from ^10.0.0 to ^11.0.0 (line 24).
  • Updated uuid devDependency from ^10.0.0 to ^11.0.0 (line 25).
  • Updated sinon devDependency from ^18.0.0 to ^20.0.0 (line 26).
• ai-platform/snippets/predict-custom-trained-model.js
  • Converted require('util') to require('node:util') and require('fs') to require('node:fs') (lines 30-31).
• ai-platform/snippets/predict-image-classification.js
  • Converted require('fs') to require('node:fs') (line 55).
• ai-platform/snippets/predict-image-from-image-and-text.js
  • Converted require('fs') to require('node:fs') (line 57).
  • Converted exports.predictImageFromImageAndText = main to export {main as predictImageFromImageAndText} (line 98).
• ai-platform/snippets/predict-image-object-detection.js
  • Converted require('fs') to require('node:fs') (line 55).
• ai-platform/snippets/test/batch-prediction-gemini.test.js
  • Converted require statements to import statements for external libraries (chai, mocha, @google-cloud/aiplatform) and built-in module (child_process) (lines 19-22).
  • Aliased imported execSync from node:child_process to _execSync (line 21).
• ai-platform/snippets/test/create-batch-prediction-job-text-classification.test.js
  • Converted require statements to import statements for external libraries (chai, mocha, uuid, @google-cloud/aiplatform) and built-in module (child_process) (lines 19-25).
  • Aliased imported execSync from node:child_process to _execSync (line 22).
• ai-platform/snippets/test/create-batch-prediction-job-text-entity-extraction.test.js
  • Converted require statements to import statements for external libraries (chai, mocha, uuid, @google-cloud/aiplatform) and built-in module (child_process) (lines 19-25).
• ai-platform/snippets/test/create-batch-prediction-job-text-sentiment-analysis.test.js
  • Converted require statements to import statements for external libraries (chai, mocha, uuid, @google-cloud/aiplatform) and built-in module (child_process) (lines 19-25).
• ai-platform/snippets/test/create-batch-prediction-job-video-classification.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha, uuid, @google-cloud/aiplatform) (lines 19-27).
• ai-platform/snippets/test/create-batch-prediction-job-video-object-tracking.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha, uuid, @google-cloud/aiplatform) (lines 19-27).
• ai-platform/snippets/test/create-custom-job.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha, uuid) (lines 19-23).
• ai-platform/snippets/test/create-dataset-image.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha, uuid) (lines 19-23).
• ai-platform/snippets/test/create-dataset-tabular-bigquery.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha, uuid) (lines 19-23).
• ai-platform/snippets/test/create-dataset-tabular-gcs.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha, uuid) (lines 19-23).
• ai-platform/snippets/test/create-dataset-text.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha, uuid) (lines 19-23).
• ai-platform/snippets/test/create-dataset-video.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha, uuid) (lines 19-23).
• ai-platform/snippets/test/create-dataset.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha, uuid) (lines 19-23).
• ai-platform/snippets/test/create-endpoint.test.js
  • Converted require statements to import statements for built-in module (child_process) and external libraries (chai, mocha, uuid) (lines 19-23).
• ai-platform/snippets/test/create-featurestore-fixed-nodes-sample.test.js
  • Converted require statements to import statements for built-in module (child_process) and external libraries (chai, mocha, uuid) (lines 19-23).
• ai-platform/snippets/test/create-featurestore-sample.test.js
  • Converted require statements to import statements for built-in module (child_process) and external libraries (chai, mocha, uuid) (lines 19-23).
• ai-platform/snippets/test/create-hyperparameter-tuning-job.test.js
  • Converted require statements to import statements for built-in module (child_process) and external libraries (chai, mocha, uuid, @google-cloud/aiplatform) (lines 19-26).
• ai-platform/snippets/test/create-training-pipeline-image-classification.test.js
  • Converted require statements to import statements for built-in module (child_process) and external libraries (chai, mocha, uuid, @google-cloud/aiplatform) (lines 19-26).
• ai-platform/snippets/test/create-training-pipeline-image-object-detection.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha, uuid, @google-cloud/aiplatform) (lines 19-28).
• ai-platform/snippets/test/create-training-pipeline-tabular-classification.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha, uuid, @google-cloud/aiplatform) (lines 19-28).
• ai-platform/snippets/test/create-training-pipeline-tabular-regression.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha, uuid, @google-cloud/aiplatform) (lines 19-28).
• ai-platform/snippets/test/create-training-pipeline-text-entity-extraction.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha, uuid, @google-cloud/aiplatform) (lines 19-28).
• ai-platform/snippets/test/create-training-pipeline-text-sentiment-analysis.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha, uuid, @google-cloud/aiplatform) (lines 19-28).
• ai-platform/snippets/test/create-training-pipeline-video-action-recognition.test.js
  • Converted require statements to import statements for built-in module (child_process) and external libraries (chai, mocha, uuid, @google-cloud/aiplatform) (lines 19-26).
• ai-platform/snippets/test/create-training-pipeline-video-classification.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha, uuid, @google-cloud/aiplatform) (lines 19-28).
• ai-platform/snippets/test/create-training-pipeline-video-object-tracking.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha, uuid, @google-cloud/aiplatform) (lines 19-28).
• ai-platform/snippets/test/deploy-model.test.js
  • Converted require statements to import statements for built-in module (child_process) and external libraries (chai, mocha, uuid) (lines 19-23).
• ai-platform/snippets/test/embedding-model-tuning.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha, @google-cloud/aiplatform) (lines 19-23).
• ai-platform/snippets/test/entity-type-samples.test.js
  • Converted require statements to import statements for built-in module (child_process) and external libraries (chai, mocha, uuid) (lines 19-23).
• ai-platform/snippets/test/export-model-tabular-classification.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha) (lines 19-22).
• ai-platform/snippets/test/feature-samples.test.js
  • Converted require statements to import statements for built-in module (child_process) and external libraries (chai, mocha, uuid) (lines 19-23).
• ai-platform/snippets/test/feature-values-samples.test.js
  • Converted require statements to import statements for built-in module (child_process) and external libraries (@google-cloud/bigquery, chai, mocha, uuid) (lines 19-24).
• ai-platform/snippets/test/featurestore-samples.test.js
  • Converted require statements to import statements for built-in module (child_process) and external libraries (chai, mocha, uuid) (lines 19-23).
• ai-platform/snippets/test/gemma2Prediction.test.js
  • Converted require statements to import statements for external libraries (chai, mocha, sinon) and local files (../gemma2PredictGpu.js, ../gemma2PredictTpu.js) (lines 19-23).
• ai-platform/snippets/test/get-custom-job.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha) (lines 19-22).
• ai-platform/snippets/test/get-hyperparameter-tuning-job.test.js
  • Converted require statements to import statements for built-in module (child_process) and external libraries (chai, mocha) (lines 19-21).
• ai-platform/snippets/test/get-model-evaluation-slice.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha) (lines 19-22).
• ai-platform/snippets/test/get-model-evaluation-tabular-classification.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha) (lines 19-22).
• ai-platform/snippets/test/get-model-evaluation-tabular-regression.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha) (lines 19-22).
• ai-platform/snippets/test/get-model-evaluation-video-action-recognition.test.js
  • Converted require statements to import statements for built-in module (child_process) and external libraries (chai, mocha) (lines 19-21).
• ai-platform/snippets/test/get-model-evaluation-video-classification.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha) (lines 19-22).
• ai-platform/snippets/test/get-model-evaluation-video-object-tracking.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha) (lines 19-22).
• ai-platform/snippets/test/get-model.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha) (lines 19-22).
• ai-platform/snippets/test/get-training-pipeline.test.js
  • Converted require statements to import statements for built-in module (child_process) and external libraries (chai, mocha) (lines 19-21).
• ai-platform/snippets/test/imagen.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha) (lines 19-22).
• ai-platform/snippets/test/import-data-video-classification.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha) (lines 19-22).
• ai-platform/snippets/test/list-endpoints.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha) (lines 18-21).
• ai-platform/snippets/test/list-model-evaluation-slices.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha) (lines 19-22).
• ai-platform/snippets/test/predict-image-from-image-and-text.test.js
  • Converted require statements to import statements for external libraries (chai, mocha, sinon) and local file (../predict-image-from-image-and-text) (lines 19-22).
• ai-platform/snippets/test/predict-image-object-detection.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha) (lines 19-22).
• ai-platform/snippets/test/predict-tabular-classification.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha) (lines 19-22).
• ai-platform/snippets/test/predict-tabular-regression.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha) (lines 19-22).
• ai-platform/snippets/test/predict-text-classification.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha) (lines 19-22).
• ai-platform/snippets/test/predict-text-embeddings.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha) (lines 19-22).
• ai-platform/snippets/test/predict-text-entity-extraction.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha) (lines 19-22).
• ai-platform/snippets/test/upload-model.test.js
  • Converted require statements to import statements for built-in modules (path, child_process) and external libraries (chai, mocha, uuid) (lines 19-23).

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

---

From CJS to ESM,
A module journey starts,
Imports now lead the way,
Changing code's old parts,
A modern Node.js heart.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This PR initiates the migration of ai-platform/snippets to ES Modules. While the intention is good, the migration appears incomplete, with issues related to __dirname usage in ESM contexts and inconsistent module imports. Addressing these is crucial for the snippets to function correctly.

Summary of Findings

• Usage of __dirname in ES Modules: __dirname is not available in ES modules by default, causing ReferenceError in test files. Update these using import.meta.url and fileURLToPath.
• Incomplete CJS to ESM Conversion (require statements): Several files (gemma2Predict*.js, featurestore tests, and snippet files like imagen-*.js) still contain require() calls, particularly for @google-cloud/aiplatform and Node.js built-in modules. Convert these to import statements.
• Inconsistent Import Styles: The import style for @google-cloud/aiplatform (default vs. named v1 import) varies across test files. Standardize this.
• Dependency Version Bumps: Major versions of dependencies (e.g., @google-cloud/aiplatform, chai, mocha) have been bumped. Ensure all snippets are re-tested with these new versions to catch breaking changes.

Merge Readiness

This pull request has issues related to the incomplete ESM migration (usage of __dirname and remaining require statements). These issues will likely cause runtime errors and test failures. Therefore, I recommend that these changes not be merged until these issues are addressed. After the fixes, thorough testing of all snippets would be advisable, especially given the major dependency version upgrades. I am not authorized to approve pull requests.

[gemini-code-assist]

The __dirname variable is not available by default in ES modules, leading to a ReferenceError at runtime. Refactor to determine the current directory path in an ESM context.

This issue applies to many other test files using __dirname:

• ai-platform/snippets/expensive-test/create-data-labeling-job-video.test.js
• ai-platform/snippets/expensive-test/create-data-labeling-job.test.js
• ai-platform/snippets/test/create-batch-prediction-job-video-classification.test.js
• ...and others.

Update these files to use the ESM-compatible way of getting the directory name?

Suggested change

	const cwd = path.join(__dirname, '..');
	import { fileURLToPath } from 'node:url';
	import { dirname } from 'node:path';
	const __filename = fileURLToPath(import.meta.url);
	const __dirname = dirname(__filename);
	const cwd = path.join(__dirname, '..');

[gemini-code-assist]

The require('@google-cloud/aiplatform').v1 statement for FeaturestoreServiceClient was not converted to an ES import. In an ESM context, this will likely cause an error.

This issue is also present in other feature store related test files:

• ai-platform/snippets/test/create-featurestore-sample.test.js
• ai-platform/snippets/test/entity-type-samples.test.js
• ai-platform/snippets/test/feature-samples.test.js
• ai-platform/snippets/test/feature-values-samples.test.js (which also has a require for @google-cloud/bigquery that needs conversion).

Update these to use import syntax?

Suggested change

	const {FeaturestoreServiceClient} = require('@google-cloud/aiplatform').v1;
	import { FeaturestoreServiceClient } from '@google-cloud/aiplatform';
	import { v1 } from '@google-cloud/aiplatform';

[gemini-code-assist]

These require calls for fs and util are inside a function within an ES module. Use top-level import statements for better ESM compatibility. This also applies to the require('@google-cloud/aiplatform') on line 30.

This pattern is present in several other imagen-*.js, predict-custom-trained-model.js, predict-image-classification.js, and predict-image-object-detection.js files. Refactor to use top-level imports?

import fs from 'node:fs';
import util from 'node:util';

[gemini-code-assist]

The import import aiplatform from '@google-cloud/aiplatform'; is a default import. Later, aiplatform.v1.JobServiceClient is used.

Other files (e.g., create-batch-prediction-job-text-classification.test.js) use import { v1 } from '@google-cloud/aiplatform'; followed by new v1.JobServiceClient(). Standardize the import style for @google-cloud/aiplatform across test files for consistency. Check the recommended import style for @google-cloud/aiplatform v4 and apply it consistently?

[gemini-code-assist]

Similar to other snippet files, this require('node:fs') is inside a function in an ES module. Move this to a top-level import fs from 'node:fs'; statement.

import fs from 'node:fs';

[davidcavazos]

There are still a lot of manual edits that would need to be done. Many imports are requires inside the function body, which would need to be transformed to imports, bubbled outside the function, and bubble up the region tags outside the function to include the imports.