Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(gnovm): add fuzz and fuzz CLI #3446

Open
wants to merge 27 commits into
base: master
Choose a base branch
from
Open

feat(gnovm): add fuzz and fuzz CLI #3446

wants to merge 27 commits into from

Conversation

rlaau
Copy link

@rlaau rlaau commented Jan 6, 2025

This PR introduces fuzzing functionality and a corresponding CLI for the GnoVM. The fuzzing can be executed using the following command:

cmd: gno test [filename] -fuzz [function_name] -i [iterations] -v [verbose]

Key Points to Note:

  1. Coverage Implementation Pending:
    The current fuzzing implementation does not include coverage-related functions. To achieve complete fuzzing functionality, coverage support must be added in future updates.

  2. Iteration-Based Fuzzing:
    Unlike Go's fuzzing mechanism, which supports a timeout parameter, Gno's fuzzer uses -i to specify the number of iterations.

  3. Function Signature Requirement:
    The function passed to f.Fuzz in Gno must strictly follow the signature:
    func(*testing.T, args ...interface{}).

Fuzzing Input/ Output example:

  1. Input example:
    cmd: gno test mock_test.gno -fuzz Fuzz
  2. output example:
    bash
    --Failing input: ["/xeb", 560]

Example Usage Image:

image

Example Usage Code:

package testing

import (
	"testing"
	"unicode/utf8"
)

func TestMock(t *testing.T) {
	t.Errorf(`New("abc") == New("abc")`)
	println("hello")
}

func FuzzMock(f *testing.F) {
	f.Add("apple hello", int(400002131323))
	f.Add("rainy day", int(98401132231331))
	f.Add("winter comes", int(12349123123))
	f.Fuzz(func(t *testing.T, orig ...interface{}) {
		v, ok := orig[0].(string)
		if !ok {
			panic("dont match")
		}
		i, ok2 := orig[1].(int)
		if !ok2 {
			panic("dont match")
		}
		rev := testing.Reverse1(v)
		doubleRev := testing.Reverse1(rev)
		if v != doubleRev && i > 300 && i < 500 {
			t.Errorf("Before: %q, after: %q", orig, doubleRev)
		}
		if utf8.ValidString(v) && !utf8.ValidString(rev) && i > 300 && i < 1000 {
			t.Errorf("Reverse produced invalid UTF-8 string %q", rev)
		}
	})
}

@github-actions github-actions bot added the 📦 🤖 gnovm Issues or PRs gnovm related label Jan 6, 2025
@Gno2D2
Copy link
Collaborator

Gno2D2 commented Jan 6, 2025

🛠 PR Checks Summary

All Automated Checks passed. ✅

Manual Checks (for Reviewers):
  • IGNORE the bot requirements for this PR (force green CI check)
  • The pull request description provides enough details
Read More

🤖 This bot helps streamline PR reviews by verifying automated checks and providing guidance for contributors and reviewers.

✅ Automated Checks (for Contributors):

🟢 Maintainers must be able to edit this pull request (more info)

☑️ Contributor Actions:
  1. Fix any issues flagged by automated checks.
  2. Follow the Contributor Checklist to ensure your PR is ready for review.
    • Add new tests, or document why they are unnecessary.
    • Provide clear examples/screenshots, if necessary.
    • Update documentation, if required.
    • Ensure no breaking changes, or include BREAKING CHANGE notes.
    • Link related issues/PRs, where applicable.
☑️ Reviewer Actions:
  1. Complete manual checks for the PR, including the guidelines and additional checks if applicable.
📚 Resources:
Debug
Automated Checks
Maintainers must be able to edit this pull request (more info)

If

🟢 Condition met
└── 🟢 The pull request was created from a fork (head branch repo: rlaau/gno)

Then

🟢 Requirement satisfied
└── 🟢 Maintainer can modify this pull request

Manual Checks
**IGNORE** the bot requirements for this PR (force green CI check)

If

🟢 Condition met
└── 🟢 On every pull request

Can be checked by

  • Any user with comment edit permission
The pull request description provides enough details

If

🟢 Condition met
└── 🟢 Not (🔴 Pull request author is a member of the team: core-contributors)

Can be checked by

  • team core-contributors

@notJoon notJoon added the review/triage-pending PRs opened by external contributors that are waiting for the 1st review label Jan 6, 2025
@notJoon
Copy link
Member

notJoon commented Jan 6, 2025

It seems you commit the wrong files in pkg/mod directory. please remove those files.

@rlaau
Copy link
Author

rlaau commented Jan 7, 2025

It seems you commit the wrong files in pkg/mod directory. please remove those files.

Thank you for letting me know, I deleted the file.

Copy link
Member

@notJoon notJoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll have to give it some more time and check it out again slowly.

gnovm/stdlibs/testing/crash_logger.gno Outdated Show resolved Hide resolved
gnovm/stdlibs/testing/data_structures.gno Outdated Show resolved Hide resolved
gnovm/stdlibs/testing/data_structures.gno Outdated Show resolved Hide resolved
gnovm/stdlibs/testing/data_structures.gno Outdated Show resolved Hide resolved
gnovm/stdlibs/testing/data_structures.gno Outdated Show resolved Hide resolved
gnovm/stdlibs/testing/get_coverage.gno Outdated Show resolved Hide resolved
gnovm/stdlibs/testing/hash_machine.gno Outdated Show resolved Hide resolved
gnovm/stdlibs/testing/hash_machine.gno Outdated Show resolved Hide resolved
gnovm/stdlibs/testing/random_test.gno Outdated Show resolved Hide resolved
gnovm/pkg/test/test.go Outdated Show resolved Hide resolved
@notJoon
Copy link
Member

notJoon commented Jan 7, 2025

Also, if this PR is still in progress, please convert the status to Draft.

@rlaau rlaau marked this pull request as draft January 7, 2025 09:04
@rlaau
Copy link
Author

rlaau commented Jan 7, 2025

Also, if this PR is still in progress, please convert the status to Draft.

I just checked that there were replies. I will re-open the pr after solving everything.

@jefft0 jefft0 removed the review/triage-pending PRs opened by external contributors that are waiting for the 1st review label Jan 9, 2025
@jefft0
Copy link
Contributor

jefft0 commented Jan 9, 2025

Removed the review/triage-pending label while this PR is Draft.

@rlaau rlaau marked this pull request as ready for review January 9, 2025 10:49
@rlaau
Copy link
Author

rlaau commented Jan 9, 2025

Removed the review/triage-pending label while this PR is Draft.

I modified the annotation and coding convention to suit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
📦 🤖 gnovm Issues or PRs gnovm related
Projects
Status: In Review
Development

Successfully merging this pull request may close these issues.

4 participants