Skip to content

Extraction-check: generate SARIF file #310

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

byronantak
Copy link
Collaborator

@byronantak byronantak commented Jun 3, 2025

Add logic to generate sarif.json file to extraction-check command

Technical

  • Every check adds unique rule ids per possible failure (except AI checker, it's generic)
  • Update cli checker to include a breakdown of a failure per source text unit
  • Update tests
    • Simplify some tests
  • Fix GlossaryCaseChecker side-effect and extra new line problems

@byronantak byronantak self-assigned this Jun 3, 2025
@byronantak byronantak changed the title Bantak/generate sarif file Extraction-check: generate SARIF file Jun 3, 2025
@byronantak byronantak force-pushed the bantak/generate-sarif-file branch from b31efda to 9360117 Compare June 3, 2025 14:57
String fileName = "i18n-checks.sarif.json";
try {
Path filePath = Paths.get(".", fileName);
String sarifThing = objectMapper.writeValueAsString(sarif);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
String sarifThing = objectMapper.writeValueAsString(sarif);
String sarifData = objectMapper.writeValueAsString(sarif);

for (Map.Entry<String, List<AICheckResult>> failure : failureMap.entrySet()) {
List<String> suggestedFixes =
failure.getValue().stream().map(AICheckResult::getSuggestedFix).toList();
String suggestFixEntry = String.join(System.lineSeparator(), suggestedFixes);
Copy link
Member

@mattwilshire mattwilshire Jun 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could simplify this by doing the join on the map directly:

String suggestFixEntry = failure.getValue().stream()
        .map(AICheckResult::getSuggestedFix)
        .collect(Collectors.joining(System.lineSeparator()));

sb.append(
" is spelt incorrectly. Suggested correct spellings are: ");
sb.append(String.join(", ", suggestions));
sb.append(System.lineSeparator());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we use method chaining here instead for the builder ?
Something like the following:

sb.append(BULLET_POINT)
     .append(QUOTE_MARKER)
     .append(misspeltWord)
     .append(QUOTE_MARKER)
     .append(" is spelt incorrectly. Suggested correct spellings are: ")
     .append(String.join(", ", suggestions))
     .append(System.lineSeparator());

for (CliCheckResult checkFailure : cliCheckerFailures) {
ResultLevel resultLevel = checkFailure.isHardFail() ? ResultLevel.ERROR : ResultLevel.WARNING;
sarifBuilder.addRun(
checkFailure.getCheckName(), "http://pinch.pinadmin.com/glow-string-checks");
Copy link
Member

@mattwilshire mattwilshire Jun 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove / change this URL string ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, this url should be configurable

Integer startLineNumber = Integer.parseInt(usage.substring(colonIndex + 1));
return new Location(fileUri, startLineNumber);
} catch (NumberFormatException e) {
logger.info(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
logger.info(
logger.error(

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or logger.warn

public class CliCheckResult {

public record CheckFailure(String ruleId, String failureMessage) {}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to use an enum here for the ruleId ?
We could then convert it to a string when it is needed, something like the following:

public enum RULE_ID {
    AGGREGATE_SPELLING_SUGGESTIONS,
    EQUAL_CONTEXT_AND_COMMENT_STRINGS,
    EMPTY_CONTEXT_AND_COMMENT_STRINGS,
    EMPTY_COMMENT_STRING;
    ...

    @Override
    public String toString() {
        return name();
    }
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a CliCheckerType Enum for each of the CLI checks which could be re-used here as the rule id

failureText = "Context string is empty.";
} else if (isBlank(comment)) {
ruleId = "EMPTY_COMMENT_STRING";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see why you can't re-use the CliCheckerType enum as some checks have multiple rule ids, moving these to their own enum will centralise our rule ids and make them easier to add/track in future

failureText = "Comment string is empty.";
}

return failureText;
if (ruleId == null) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What will happen if we return a CliCheckResult with it's rule id field set to null? I feel that's a bit cleaner

Copy link
Collaborator Author

@byronantak byronantak Jun 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer a null object instead of an object of all null fields (in this instance) 🤷‍♂️
It also keeps to the same pattern this method was already following

for (CliCheckResult checkFailure : cliCheckerFailures) {
ResultLevel resultLevel = checkFailure.isHardFail() ? ResultLevel.ERROR : ResultLevel.WARNING;
sarifBuilder.addRun(
checkFailure.getCheckName(), "http://pinch.pinadmin.com/glow-string-checks");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, this url should be configurable

@byronantak byronantak force-pushed the bantak/generate-sarif-file branch from ad6cbed to 9f34fd2 Compare July 30, 2025 13:06
@byronantak byronantak marked this pull request as ready for review July 31, 2025 11:53
@byronantak byronantak requested a review from a team as a code owner July 31, 2025 11:53
@byronantak byronantak requested a review from garionpin July 31, 2025 11:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants