-
Notifications
You must be signed in to change notification settings - Fork 206
fix PR experiment additional arg parsing conflict #1217
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
/gcbrun exp -n wentao -m vertex_ai_gemini-2-5-pro-chat -b quick-test -ns 1 -- -csv -ag |
|
Requested GKE Job: ofg-pr-1217-wentao JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1217-wentao |
|
/gcbrun exp -n wentao -m vertex_ai_gemini-2-5-pro-chat -b quick-test -ns 1 -- -csv -ag |
|
Requested GKE Job: ofg-pr-1217-wentao JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1217-wentao |
|
Hi @DonggeLiu, I checked the additional arg parsing conflict we discussed before. I am proposing to add an additional argument splitter in docker_run.py to parse and extract flags exclusive to web.py (e.g., -csv, -gs). Only the rest of the additional args will be piped to run_all_experiment.py My example run added two additional args: |
|
Thanks @wenta0g! Feel free to do this in this PR or a new PR. I don' think the bug relates to this PR. |
|
Thanks, Dongge! I removed the quoted lines under this PR. |
Thanks! Shall we run another experiment at your convenience to test report generation? |
|
/gcbrun exp -n wentao -m vertex_ai_gemini-2-5-pro-chat -b quick-test -ns 1 -- -csv -ag |
|
Requested GKE Job: ofg-pr-1217-wentao JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1217-wentao |
|
Hi @DonggeLiu, the public-facing report looks good to me. Let me know if there's any GKE issue that I'm not aware of. |
Nope, this should be good to merge. BTW, this is the correct report URL: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2026-01-20-1217-wentao-quick-test/index.html There is a known date mismatch issue ( You can find the correct URLs at: |
This PR fix the problem when issuing
/gcbruncommand with additional args-- additional_args. The additional_args may flow toweb.pyorrun_all_experiment.py, causingweb.pyto fail when the args are exclusively torun_all_experiment.py