Skip to content

Conversation

reingart
Copy link

Description

In verify job stanzas, do not copy container commands or arguments if not specified in skaffold.yaml

If not, original command/argument defined in the job will be overwritten with nil.

Previous "bogus" behavior could be reproduced using empty string command or args, for example:

verify:
- name: test-happy-path
  container:
    name: test
    image: test
    args: []
  executionMode:
    kubernetesCluster:
      jobManifestPath: tests/job.yml

UT comming soon

Do not copy container commands or arguments if not specified in skaffold.yaml

If not, original command/argument defined in the job will be overwritten with nil.
@reingart reingart requested a review from a team as a code owner September 19, 2025 00:30
@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @reingart, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a critical bug in Skaffold's verify functionality that could lead to unexpected behavior in Kubernetes jobs. Previously, if container commands or arguments were omitted from the skaffold.yaml configuration for a verify step, they would inadvertently clear out any pre-defined commands or arguments in the underlying Kubernetes job manifest. The fix introduces conditional logic to prevent this overwriting, ensuring that only explicitly provided values are applied. Additionally, the change enhances the observability of the verification process by integrating more detailed debug and trace logging.

Highlights

  • Bug Fix: Container Command/Args Overwrite: Resolved an issue where verify jobs would incorrectly overwrite existing container commands or arguments in Kubernetes job manifests with nil if they were not explicitly defined in the skaffold.yaml.
  • Conditional Patching Logic: The patchToK8sContainer function now conditionally applies Command and Args to a Kubernetes container only if these fields are present in the Skaffold configuration, preventing unintended clearing of values.
  • Enhanced Debug Logging: Added several debug and trace log statements to provide better visibility into the lifecycle of verify jobs, including creation, pod status (succeeded/failed), cleanup, and container patching details.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@reingart reingart marked this pull request as draft September 19, 2025 00:31
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request correctly addresses an issue where container command and args from a job manifest were being overwritten. The conditional logic in patchToK8sContainer is a good fix. The addition of extensive debug and trace logging is also a welcome improvement for maintainability. I have a few suggestions to refine the implementation. The primary concern is ensuring the fix fully covers the scenario described in the PR (using an empty slice []). Additionally, the new logging statements should consistently use the available function context instead of context.TODO() to preserve trace information and enable proper cancellation.

Comment on lines +423 to +428
if container.Command != nil {
dst.Command = container.Command
}
if container.Args != nil {
dst.Args = container.Args
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

This fix correctly prevents overwriting the command/args when they are not specified in the skaffold.yaml. However, the PR description mentions that the 'bogus' behavior can be reproduced with args: []. With the current check (container.Args != nil), an empty slice [] is not nil, so it would still overwrite the original arguments from the manifest with an empty list. To fully address the case described, it would be better to check if the slice has any elements before overwriting. This applies to both Command and Args.

Suggested change
if container.Command != nil {
dst.Command = container.Command
}
if container.Args != nil {
dst.Args = container.Args
}
if len(container.Command) > 0 {
dst.Command = container.Command
}
if len(container.Args) > 0 {
dst.Args = container.Args
}

// This is because the k8s API server can be unresponsive when hit with a large
// intitial set of Job CREATE requests
if waitErr := wait.Poll(100*time.Millisecond, 30*time.Second, func() (bool, error) {
olog.Entry(context.TODO()).Debugf("Creating verify job in cluster: %+v\n", job)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

It's great that you're adding more logging. However, using context.TODO() should be avoided when a more specific context is available. In this function, ctx is passed as an argument and should be used here instead. This ensures that trace information and other context-scoped values are propagated correctly.

This same feedback applies to the other new logging calls in this file where ctx is available (e.g., in watchJob and Cleanup). For the logs in createJobFromManifestPath, which doesn't currently accept a context, consider refactoring that function to accept a context from its caller.

Suggested change
olog.Entry(context.TODO()).Debugf("Creating verify job in cluster: %+v\n", job)
olog.Entry(ctx).Debugf("Creating verify job in cluster: %+v\n", job)

job.Name = jobName
job.Labels["skaffold.dev/run-id"] = v.labeller.GetRunID()
var original corev1.Container
olog.Entry(context.TODO()).Tracef("Lookging for container %s in %+v\n", container.Name, job.Spec.Template.Spec.Containers)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

There's a typo here: 'Lookging' should be 'Looking'.

Suggested change
olog.Entry(context.TODO()).Tracef("Lookging for container %s in %+v\n", container.Name, job.Spec.Template.Spec.Containers)
olog.Entry(context.TODO()).Tracef("Looking for container %s in %+v\n", container.Name, job.Spec.Template.Spec.Containers)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant