Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow user to specify N retries #2043

Open
ClarkKentIsSuperman opened this issue Oct 15, 2024 · 0 comments
Open

Allow user to specify N retries #2043

ClarkKentIsSuperman opened this issue Oct 15, 2024 · 0 comments
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@ClarkKentIsSuperman
Copy link

🚀 Feature Request

Allow for N (user defined) number of retries of the same question.

🔈 Motivation

In some cases, asking the exact same question 5+ times causes the jailbreak to actually work. Sometimes injection only happens after N retries, like llama3 - I had to insert/ask the same question 5 times and finally it jailbroke and told me how to make a bomb or something like that. If I was testing with Giskard, it may just test once and call it a pass.

🛰 Alternatives

None

📎 Additional context

None

@ClarkKentIsSuperman ClarkKentIsSuperman added the enhancement New feature or request label Oct 15, 2024
@henchaves henchaves added the good first issue Good for newcomers label Oct 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Development

No branches or pull requests

2 participants