Skip to content

[bug] Same code scan on linux vs Mac reports differently #249

@MadaManu

Description

@MadaManu

🐞Bug Report

Bug Description

On Linux it is reported as:
NOT COVERED CONDITIONALS_NEGATION at utils.go:14:9

On Mac it is reported as:
LIVED CONDITIONALS_NEGATION at utils.go:14:9

Therefore the aggregated results are different:

On Linux:

Test efficacy: 100.00%
Mutator coverage: 95.24%

On Mac:

Test efficacy: 95.24%
Mutator coverage: 100.00%

Which makes it hard to

To Reproduce

Steps to reproduce the behavior:

  1. Have a utils.go file with:
// other logic
if err != nil {
 // handling of error
}
  1. Do not cover this case for CONDITIONALS_NEGATION
  2. Unleash the gremlins on it in Mac
  3. Unleash the gremlins on the same code in Linux
  4. Observe the different reporting

Found behaviour

Discrepancy between the same failure between OSes.

Expected behaviour

The report and the status of the finding should be the same so that consistent thresholds can be set, regardless of the OS that executes the unleash.
This would make things consistent between local runs of development (MacOS) and the build agents, which are linux.

Operating System

  • OS: Oracle Linux Server 8.10 and MacOS 14.7

Metadata

Metadata

Assignees

Labels

c/bugCategory: A issue or PR related to a bug

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions