Skip to content

Nejumi leaderboard 4 upstage #345

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
alex-woosuk-kwon wants to merge 23 commits into
base: dev
Choose a base branch
from
Open

Nejumi leaderboard 4 upstage #345

alex-woosuk-kwon wants to merge 23 commits into from

Conversation

@alex-woosuk-kwon
Copy link

@alex-woosuk-kwon alex-woosuk-kwon commented Oct 1, 2025

Main changes:

  • 2 Solar model configuration creation (high AND minimal reasoning)
  • Updated SUPPORTED_MODELS.md to map Upstage Handler to Prompting Mode
  • Added Upstage Handler mapping in model_config.py
  • Modified existing Upstage Handler (upstage.py) to successfully call Solar-Pro2 API while passing the correct configuration data from model configuration yaml file
  • Modify base upstage api url to be generic in llm_inference_adapter.py

Woo Suk Kwon added 23 commits September 30, 2025 08:42
Add Solar Pro2 configuration for Upstage API evaluation
7b4f018 
- Add config-solar-pro2-250909.yaml with Upstage API settings
- Configure for testmode and function calling support
- Ready for EC2 evaluation run
Added model configuration + Ready to test existing upstage handler
876be46
modified name to solar-pro2 (lower-case)
0ef1f68
Change model configuration base_model value
f48824f
change modelstyle value
a6edc98
Print out temperature
601f615
Add generator in config
3b0fc24
Pass base+model configuration to handler
d1c3103
add reasoning under generator
437717d
Add configuration for non-bfcl benchmark
17c052e
change file name
169fba3
Increase interval time
40bfd48
Add debug statement to llm_adapter
a9f7f2a
clean up code
7b401f1
modify project name
a4491ec
Only run mtbench
00b2e5e
Add quotation
48c84d3
Debug run_eval.py
b747c75
Resolve conflict
52a05b8
Add more debug prints to run_eval.py
f635d24
Comment out debug print
6d5af07
Finish test mode
805789d
changed swebench config
13fe6ae
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@alex-woosuk-kwon