Auto label Copilot agent PRs with "ai: copilot"

[Copilot]

Adds automatic labeling for pull requests created by Copilot agents with the "ai: copilot" label.

Changes Made

Added a new CopilotAgent job to the PR workflow (.github/workflows/pr.yml) that:

1. Detects Copilot agent PRs by checking multiple indicators:

   • Author usernames starting with github-copilot (e.g., github-copilot[bot])
   • Author usernames starting with copilot (e.g., copilot, copilot-agent)
   • PR body content containing "copilot agent" or "Copilot agent"
   • PR body content mentioning "@copilot"

2. Automatically applies the "ai: copilot" label when any indicator is detected

3. Follows existing patterns - uses the same permissions, app token setup, and GitHub CLI approach as the existing Dependabot labeling logic

Implementation Details

The solution is minimal and follows the repository's established conventions:

CopilotAgent:
  runs-on: ubuntu-24.04-16cores-public
  permissions:
    pull-requests: write
  steps:
    - name: Check if Copilot Agent PR
      # Detection logic for various Copilot indicators
    - name: Add Copilot Label
      if: steps.copilot_check.outputs.copilot == 'true'
      run: 'gh pr edit "$PR_URL" --add-label "ai: copilot"'

The job includes debug output to help with troubleshooting and runs independently of other workflow jobs.

Fixes #2359.

---

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

[LokmaneKrizou]

@copilot the solution didn't work. this PR is from copilot agent and it the pr label workflow didn't recognise it as copilot is not set as the author of the PR, check using commits instead, chatgpt suggested this

  LabelCopilot:
    name: Label AI-authored PRs
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write
      issues: write
    needs: [Build]
    steps:
      - name: Detect Copilot commits and label PR
        uses: actions/github-script@v6
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          script: |
            // 1. Grab all commits in this PR
            const { data: commits } = await github.rest.pulls.listCommits({
              owner: context.repo.owner,
              repo:  context.repo.repo,
              pull_number: context.payload.pull_request.number
            })

            // 2. Check for Copilot in either the GitHub author or the commit message trailer
            const isCopilot = commits.some(c => {
              const login   = c.author?.login  || ""
              const message = c.commit.message || ""
              return /copilot/i.test(login)
                  || /co-authored-by:\s*GitHub Copilot/i.test(message)
            })

            // 3. If we saw Copilot, slap on the label
            if (isCopilot) {
              await github.issues.addLabels({
                owner:        context.repo.owner,
                repo:         context.repo.repo,
                issue_number: context.payload.pull_request.number,
                labels:       ['ai: copilot']
              })
            }

            ```
use that as a reference and use critical thinking mode to come up with a clean and optimised solution.

[Copilot]

@copilot the solution didn't work. this PR is from copilot agent and it the pr label workflow didn't recognise it as copilot is not set as the author of the PR, check using commits instead, chatgpt ...

You're absolutely right! I've implemented the commit-based detection approach you suggested. The new solution:

1. Checks all commits in the PR instead of just the PR author
2. Uses GitHub Script action for reliable API calls instead of shell scripts
3. Detects Copilot involvement through:
   • Commit author/committer login containing "copilot"
   • Co-authored-by trailers mentioning Copilot

This is much more robust as it works regardless of who opened the PR and correctly identifies when Copilot was involved in creating the commits.

The implementation includes comprehensive logging for debugging and follows the existing workflow patterns. Commit: 8f29822

[LokmaneKrizou]

@copilot your solution seems overcomplicated, can't we use existing DangerFile.dif.kts to add the label?
we could add something like:

        // Label PRs with Copilot involvement
        val pr = github.pr
        val commits = github.api.pulls
            .listCommits(pr.owner, pr.repo, pr.number)
            .execute()
            .body()
            .orEmpty()
        val hasCopilot = commits.any { c ->
            val authorLogin = c.author?.login.orEmpty()
            val message = c.commit.message.orEmpty()
            authorLogin.contains("copilot", ignoreCase = true)
                || message.contains("co-authored-by:", ignoreCase = true)
        }
        if (hasCopilot) {
            github.issues.addLabels(pr.owner, pr.repo, pr.number, listOf("ai: copilot"))
        }
    }
    ```
    make it short and clean solution

[Copilot]

@copilot your solution seems overcomplicated, can't we use existing DangerFile.dif.kts to add the label?
we could add something like:

...

You're absolutely right! I've moved the Copilot detection logic to the Dangerfile.df.kts as you suggested. This is much cleaner and follows the existing repository patterns. The new implementation:

• Removes the separate CopilotAgent job from the GitHub workflow
• Adds simple Copilot detection logic to Dangerfile.df.kts that checks commit authors/committers and co-authored-by trailers
• Uses the existing Danger Kotlin framework and GitHub API access
• Is more maintainable and follows the repository's established conventions

Commit: c3c2d54

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

• dl.google.com
  • Triggering command: /usr/lib/jvm/temurin-17-jdk-amd64/bin/java --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.prefs/java.util.prefs=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.prefs/java.util.prefs=ALL-UNNAMED --add-opens=java.base/java.nio.charset=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.xml/javax.xml.namespace=ALL-UNNAMED -Xmx4096m -Dfile.encoding=UTF-8 -Duser.country -Duser.language=en -Duser.variant -cp /home/REDACTED/.gradle/wrapper/dists/gradle-8.13-bin/5xuhj0ry160q40clulazy9h7d/gradle-8.13/lib/gradle-daemon-main-8.13.jar -javaagent:/home/REDACTED/.gradle/wrapper/dists/gradle-8.13-bin/5xuhj0ry160q40clulazy9h7d/gradle-8.13/lib/agents/gradle-instrumentation-agent-8.13.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 8.13 (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[LokmaneKrizou]

@copilot didn't work , try this

onGitHub {
  // … all your existing checks …

  // Label PRs with Copilot involvement
  val hasCopilot = commits.any { c ->
    // author?.login comes from the GitHubCommit DSL
    val login   = c.author?.login.orEmpty()
    val message = c.commit.message.orEmpty()
    login.contains("copilot", ignoreCase = true) ||
      message.contains("Co-authored-by:", ignoreCase = true) && message.contains("copilot", ignoreCase = true)
  }

  if (hasCopilot) {
    utils.createOrAddLabel(
      labelConfig = mapOf(
        "name"        to "ai: copilot",
        "color"       to "6a000e",
        "description" to "Pull request authored or co-authored by GitHub Copilot"
      )
      // no need to pass repoConfig – it defaults to the current repo/PR
    )
  }
}

[Copilot]

@copilot didn't work , try this

...

I've implemented your suggested approach using the commits property and utils.createOrAddLabel function. The solution is now much cleaner and follows the repository's established patterns. The implementation:

• Uses the commits property available in the Dangerfile onGitHub block
• Checks commit authors and co-authored-by trailers for Copilot involvement
• Uses utils.createOrAddLabel for proper label management

Commit: b728f41

[LokmaneKrizou]

@copilot abort dangerfile change, lets go back to changing pr.yml and use this

jobs:
  # … your existing jobs …

  CopilotAgent:
    name: Label Copilot PRs
    runs-on: <current used ubuntu>
    needs: [Build]               # or whatever pr.yml depend on
    permissions:
      pull-requests: write
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0

      - uses: cli/gh-action@v2     # installs GitHub CLI
      - name: Detect Copilot and Label
        run: |
          # look for “copilot” or “Co-authored-by” in all PR commit messages
          if git log origin/${{ github.base_ref }}..HEAD --format=%B \
               | grep -Eqi 'copilot|co-authored-by:.*copilot'; then
            gh pr edit ${{ github.event.pull_request.number }} \
              --add-label 'ai: copilot'
          else
            echo "No Copilot involvement found."
          fi
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          ```

[Copilot]

@copilot abort dangerfile change, lets go back to changing pr.yml and use this

...

I've implemented your suggested approach. The solution now:

1. Reverted the Dangerfile changes - removed all Copilot detection logic from Dangerfile.df.kts
2. Added CopilotAgent job to PR workflow using your suggested implementation:
   • Uses git log origin/${{ github.base_ref }}..HEAD --format=%B to check commit messages
   • Detects "copilot" or "co-authored-by:.*copilot" patterns (case-insensitive)
   • Uses GitHub CLI to add the "ai: copilot" label
   • Runs after Build job with proper permissions

This approach is much cleaner and more reliable than the previous API-based solutions. Commit: 5936585

[Copilot]

Pull Request Overview

This PR adds automatic labeling functionality for pull requests created by Copilot agents. The workflow job detects Copilot involvement through commit message analysis and applies the "ai: copilot" label when detected.

• Adds a new CopilotAgent job to automatically label Copilot agent PRs
• Uses git commit message analysis to detect Copilot involvement
• Follows existing workflow patterns for permissions and labeling

[LokmaneKrizou]

@copilot didn't work, cause one of the github actions is not allowed, lets go back to this commit 8f29822
and tidy it up and remove any logs or comments. make it clean and optimised

[Copilot]

@copilot didn't work, cause one of the github actions is not allowed, lets go back to this commit 8f29822...

I've reverted to commit 8f29822 and cleaned it up by removing all console.log statements and unnecessary comments. The solution is now optimized and clean while maintaining the same functionality - checking commits for Copilot involvement through author/committer login or co-authored-by trailers. Commit: eadfcd0