Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(glue): timeout and worker type validation for Ray jobs #32119

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

badmintoncryer
Copy link
Contributor

Issue # (if applicable)

Closes #29612.

Reason for this change

AWS Glue Ray job has some restriction.

  • must use Z.2X worker type
CREATE_FAILED [...] Worker type cannot be null and only [Z.2X] worker types are supported for glueray jobs
  • must not specify timeout
UPDATE_FAILED [...] Timeout not supported for Ray jobs

Description of changes

Add validation for above restriction.

    if (executable.type.name === JobType.RAY.name) {
      if (props.workerType !== WorkerType.Z_2X) {
        throw new Error(`WorkerType must be Z_2X for Ray jobs, got: ${props.workerType}`);
      }
      if (props.timeout !== undefined) {
        throw new Error('Timeout cannot be set for Ray jobs');
      }
    }

Description of how you validated changes

Add unit test.

Checklist


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@aws-cdk-automation aws-cdk-automation requested a review from a team November 14, 2024 00:36
@github-actions github-actions bot added bug This issue is a bug. effort/medium Medium work item – several days of effort p2 distinguished-contributor [Pilot] contributed 50+ PRs to the CDK labels Nov 14, 2024
@badmintoncryer badmintoncryer marked this pull request as draft November 14, 2024 00:38
@badmintoncryer badmintoncryer changed the title chore(glue): add validation for ray job chore(glue): timeout and worker type validation for ray job Nov 14, 2024
Copy link

codecov bot commented Nov 14, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.29%. Comparing base (588f1f6) to head (fa9de80).
Report is 23 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #32119   +/-   ##
=======================================
  Coverage   77.29%   77.29%           
=======================================
  Files         114      114           
  Lines        7627     7627           
  Branches     1360     1360           
=======================================
  Hits         5895     5895           
  Misses       1550     1550           
  Partials      182      182           
Flag Coverage Δ
suite.unit 77.29% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
packages/aws-cdk 77.29% <ø> (ø)

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: fa9de80
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@badmintoncryer badmintoncryer marked this pull request as ready for review November 14, 2024 06:43
@badmintoncryer badmintoncryer changed the title chore(glue): timeout and worker type validation for ray job chore(glue): timeout and worker type validation for Ray jobs Nov 14, 2024
@aws-cdk-automation aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Nov 14, 2024
Copy link
Contributor

@mazyu36 mazyu36 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
However, this fix will not be reflected once #30833 is merged.
I'll leave the decision on how to proceed to the maintainers.

@aws-cdk-automation aws-cdk-automation added pr/needs-maintainer-review This PR needs a review from a Core Team Member and removed pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. labels Nov 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug. distinguished-contributor [Pilot] contributed 50+ PRs to the CDK effort/medium Medium work item – several days of effort p2 pr/needs-maintainer-review This PR needs a review from a Core Team Member
Projects
None yet
Development

Successfully merging this pull request may close these issues.

(glue): Timeout and workerType missing validation for Ray job types
3 participants