Skip to content

Commit d69453a

Browse files
kcz358kcz358
committed
2 parents e483c0b + 381dd57 commit d69453a

File tree

282 files changed

+24307
-8083
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

282 files changed

+24307
-8083
lines changed

‎.github/ISSUE_TEMPLATE/1-bug-report.yml

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
name: 🐞 Bug report
2+
description: Create a report to help us reproduce and fix the bug
3+
title: "[Bug] "
4+
labels: ['Bug']
5+
6+
body:
7+
- type: checkboxes
8+
attributes:
9+
label: Checklist
10+
options:
11+
- label: 1. I have searched related issues but cannot get the expected help.
12+
- label: 2. The bug has not been fixed in the latest version.
13+
- label: 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
14+
- label: 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
15+
- label: 5. Please use English, otherwise it will be closed.
16+
- type: textarea
17+
attributes:
18+
label: Describe the bug
19+
description: A clear and concise description of what the bug is.
20+
validations:
21+
required: true
22+
- type: textarea
23+
attributes:
24+
label: Reproduction
25+
description: |
26+
What command or script did you run? Which **model** are you using?
27+
placeholder: |
28+
A placeholder for the command.
29+
validations:
30+
required: true
31+
- type: textarea
32+
attributes:
33+
label: Environment
34+
description: |
35+
Please provide necessary environment information here with `python3 -m sglang.check_env`. Otherwise the issue will be closed.
36+
placeholder: Environment here.
37+
validations:
38+
required: true

‎.github/ISSUE_TEMPLATE/2-feature-request.yml

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
name: 🚀 Feature request
2+
description: Suggest an idea for this project
3+
title: "[Feature] "
4+
5+
body:
6+
- type: checkboxes
7+
attributes:
8+
label: Checklist
9+
options:
10+
- label: 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
11+
- label: 2. Please use English, otherwise it will be closed.
12+
- type: textarea
13+
attributes:
14+
label: Motivation
15+
description: |
16+
A clear and concise description of the motivation of the feature.
17+
validations:
18+
required: true
19+
- type: textarea
20+
attributes:
21+
label: Related resources
22+
description: |
23+
If there is an official code release or third-party implementations, please also provide the information here, which would be very helpful.

‎.github/pull_request_template.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
<!-- Thank you for your contribution! We appreciate it. The following guidelines will help improve your pull request and facilitate feedback. If anything is unclear, don't hesitate to submit your pull request and ask the maintainers for assistance. -->
2+
3+
## Motivation
4+
5+
<!-- Explain the purpose of this PR and the goals it aims to achieve. -->
6+
7+
## Modifications
8+
9+
<!-- Describe the changes made in this PR. -->
10+
11+
## Checklist
12+
13+
- [ ] Format your code according to the [Contributor Guide](https://github.com/sgl-project/sglang/blob/main/docs/en/contributor_guide.md).
14+
- [ ] Add unit tests as outlined in the [Contributor Guide](https://github.com/sgl-project/sglang/blob/main/docs/en/contributor_guide.md).
15+
- [ ] Update documentation as needed, including docstrings or example tutorials.

‎.github/workflows/accuracy-test.yml

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
name: Accuracy Test
2+
3+
on:
4+
push:
5+
branches: [ main ]
6+
paths:
7+
- "python/sglang/**"
8+
- "test/**"
9+
pull_request:
10+
branches: [ main ]
11+
paths:
12+
- "python/sglang/**"
13+
- "test/**"
14+
workflow_dispatch:
15+
16+
concurrency:
17+
group: accuracy-test-${{ github.ref }}
18+
cancel-in-progress: true
19+
20+
jobs:
21+
accuracy-test:
22+
if: github.repository == 'sgl-project/sglang' || github.event_name == 'pull_request'
23+
runs-on: 1-gpu-runner
24+
25+
steps:
26+
- name: Checkout code
27+
uses: actions/checkout@v3
28+
29+
- name: Install dependencies
30+
run: |
31+
pip install --upgrade pip
32+
pip install -e "python[all]"
33+
pip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.4/ --force-reinstall
34+
35+
git clone https://github.com/merrymercy/human-eval.git
36+
cd human-eval
37+
pip install -e .
38+
39+
- name: Evaluate Accuracy
40+
timeout-minutes: 20
41+
run: |
42+
cd test/srt
43+
python3 test_eval_accuracy_large.py

‎.github/workflows/cache-purge.yml

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
name: Weekly Cache Purge
2+
3+
on:
4+
schedule:
5+
- cron: '0 0 * * 0' # Every Sunday at 00:00
6+
workflow_dispatch:
7+
8+
jobs:
9+
purge-cache:
10+
if: github.repository == 'sgl-project/sglang'
11+
runs-on: self-hosted
12+
13+
steps:
14+
- name: Checkout code
15+
uses: actions/checkout@v3
16+
17+
- name: Purge pip cache
18+
run: |
19+
source $HOME/venv/bin/activate
20+
echo "$HOME/venv/bin" >> $GITHUB_PATH
21+
pip cache purge
22+
23+
- name: Update dependencies
24+
run: |
25+
pip install --upgrade pip
26+
pip install -e "python[all]"
27+
pip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.4/ --force-reinstall

‎.github/workflows/cancel-pr-workflow.yml

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
name: Cancel PR Workflows on Merge
2+
3+
on:
4+
pull_request_target:
5+
types:
6+
- closed
7+
8+
permissions:
9+
actions: write
10+
11+
jobs:
12+
cancel:
13+
if: github.event.pull_request.merged == true
14+
runs-on: ubuntu-latest
15+
steps:
16+
- name: Cancel Previous Runs
17+
uses: styfle/[email protected]
18+
with:
19+
workflow_id: all
20+
access_token: ${{ secrets.GITHUB_TOKEN }}
21+
ignore_sha: true
22+
pr_number: ${{ github.event.pull_request.number }}

‎.github/workflows/close-inactive-issues.yml

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
name: Close Inactive Issues
2+
3+
on:
4+
schedule:
5+
- cron: '0 0 * * *'
6+
workflow_dispatch:
7+
8+
permissions:
9+
issues: write
10+
contents: read
11+
12+
jobs:
13+
close-inactive-issues:
14+
if: github.repository == 'sgl-project/sglang'
15+
runs-on: ubuntu-latest
16+
steps:
17+
- name: Check and close inactive issues
18+
uses: actions/github-script@v6
19+
with:
20+
github-token: ${{secrets.GITHUB_TOKEN}}
21+
script: |
22+
const sixtyDaysAgo = new Date(Date.now() - 60 * 24 * 60 * 60 * 1000);
23+
24+
const [owner, repo] = process.env.GITHUB_REPOSITORY.split('/');
25+
console.log(`Owner: ${owner}, Repo: ${repo}`);
26+
27+
async function fetchIssues(page = 1) {
28+
console.log(`Fetching issues for ${owner}/${repo}, page ${page}`);
29+
return await github.rest.issues.listForRepo({
30+
owner,
31+
repo,
32+
state: 'open',
33+
sort: 'updated',
34+
direction: 'asc',
35+
per_page: 100,
36+
page: page
37+
});
38+
}
39+
40+
async function processIssues() {
41+
console.log('Starting to process issues');
42+
console.log(`Repository: ${owner}/${repo}`);
43+
44+
let page = 1;
45+
let hasMoreIssues = true;
46+
while (hasMoreIssues) {
47+
try {
48+
const issues = await fetchIssues(page);
49+
console.log(`Fetched ${issues.data.length} issues on page ${page}`);
50+
51+
if (issues.data.length === 0) {
52+
hasMoreIssues = false;
53+
break;
54+
}
55+
56+
for (const issue of issues.data) {
57+
if (new Date(issue.updated_at) < sixtyDaysAgo) {
58+
try {
59+
await github.rest.issues.update({
60+
owner,
61+
repo,
62+
issue_number: issue.number,
63+
state: 'closed',
64+
labels: [...issue.labels.map(l => l.name), 'inactive']
65+
});
66+
await github.rest.issues.createComment({
67+
owner,
68+
repo,
69+
issue_number: issue.number,
70+
body: 'This issue has been automatically closed due to inactivity. Please feel free to reopen it if needed.'
71+
});
72+
console.log(`Closed issue #${issue.number} due to inactivity.`);
73+
} catch (error) {
74+
console.error(`Failed to close issue #${issue.number}: ${error.message}`);
75+
}
76+
} else {
77+
console.log(`Issue #${issue.number} is still active. Stopping processing.`);
78+
hasMoreIssues = false;
79+
break;
80+
}
81+
}
82+
page += 1;
83+
} catch (error) {
84+
console.error(`Error fetching issues on page ${page}: ${error.message}`);
85+
hasMoreIssues = false;
86+
}
87+
}
88+
console.log('Finished processing issues');
89+
}
90+
91+
await processIssues();

‎.github/workflows/e2e-test.yml

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
name: E2E Test
2+
3+
on:
4+
push:
5+
branches: [ main ]
6+
paths:
7+
- "python/sglang/**"
8+
- "test/**"
9+
pull_request:
10+
branches: [ main ]
11+
paths:
12+
- "python/sglang/**"
13+
- "test/**"
14+
workflow_dispatch:
15+
16+
concurrency:
17+
group: e2e-test-${{ github.ref }}
18+
cancel-in-progress: true
19+
20+
jobs:
21+
e2e-test:
22+
if: github.repository == 'sgl-project/sglang' || github.event_name == 'pull_request'
23+
runs-on: 1-gpu-runner
24+
25+
steps:
26+
- name: Checkout code
27+
uses: actions/checkout@v3
28+
29+
- name: Install dependencies
30+
run: |
31+
pip install --upgrade pip
32+
pip install -e "python[all]"
33+
pip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.4/ --force-reinstall
34+
35+
- name: Benchmark Serving Throughput
36+
timeout-minutes: 10
37+
run: |
38+
cd test/srt
39+
python3 -m unittest test_serving_throughput.TestServingThroughput.test_default
40+
41+
- name: Benchmark Serving Latency
42+
timeout-minutes: 10
43+
run: |
44+
python3 -m sglang.bench_latency --model meta-llama/Meta-Llama-3.1-8B-Instruct --batch-size 1 --input 128 --output 8
45+
46+
- name: Benchmark Serving Throughput (w/o RadixAttention)
47+
timeout-minutes: 10
48+
run: |
49+
cd test/srt
50+
python3 -m unittest test_serving_throughput.TestServingThroughput.test_default_without_radix_cache
51+
52+
- name: Benchmark Serving Throughput (w/o ChunkedPrefill)
53+
timeout-minutes: 10
54+
run: |
55+
cd test/srt
56+
python3 -m unittest test_serving_throughput.TestServingThroughput.test_default_without_chunked_prefill

‎.github/workflows/lint.yml

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
name: Lint
2+
3+
on: [push, pull_request]
4+
5+
jobs:
6+
lint:
7+
runs-on: ubuntu-20.04
8+
steps:
9+
- uses: actions/checkout@v2
10+
- name: Set up Python 3.8
11+
uses: actions/setup-python@v2
12+
with:
13+
python-version: 3.8
14+
- name: Install pre-commit hook
15+
run: |
16+
python -m pip install pre-commit
17+
pre-commit install
18+
- name: Linting
19+
run: pre-commit run --all-files

‎.github/workflows/moe-test.yml

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
name: MoE Test
2+
3+
on:
4+
push:
5+
branches: [ main ]
6+
paths:
7+
- "python/sglang/**"
8+
- "test/**"
9+
pull_request:
10+
branches: [ main ]
11+
paths:
12+
- "python/sglang/**"
13+
- "test/**"
14+
workflow_dispatch:
15+
16+
concurrency:
17+
group: moe-test-${{ github.ref }}
18+
cancel-in-progress: true
19+
20+
jobs:
21+
moe-test:
22+
if: github.repository == 'sgl-project/sglang' || github.event_name == 'pull_request'
23+
runs-on: 2-gpu-runner
24+
25+
steps:
26+
- name: Checkout code
27+
uses: actions/checkout@v3
28+
29+
- name: Install dependencies
30+
run: |
31+
pip install --upgrade pip
32+
pip install -e "python[all]"
33+
pip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.4/ --force-reinstall
34+
35+
- name: Benchmark MoE Serving Throughput
36+
timeout-minutes: 10
37+
run: |
38+
cd test/srt
39+
python3 -m unittest test_moe_serving_throughput.TestServingThroughput.test_default
40+
41+
- name: Benchmark MoE Serving Throughput (w/o RadixAttention)
42+
timeout-minutes: 10
43+
run: |
44+
cd test/srt
45+
python3 -m unittest test_moe_serving_throughput.TestServingThroughput.test_default_without_radix_cache

0 commit comments

Comments
 (0)