Skip to content

feat: add support for ModelScope API #1490

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

yrk111222
Copy link

@yrk111222 yrk111222 commented Jun 26, 2025

实现功能

实现对魔搭社区ModelScope API的接入,可以通过修改配置应用魔搭社区丰富的模型。获取魔搭社区的免费API-Key具体可以查看文档

注意

记得在使用之前绑定阿里云账号,具体绑定操作可以查看链接

好的,这是翻译成中文的 pull request 总结:

Sourcery 总结

添加对 ModelScope API 的支持,通过引入一个新的适配器和配置,将其注册到 LLMPresetAdaptersPlugin 中,并处理带有工具调用解析的流式和非流式聊天补全。

新特性:

  • 在 LLMPresetAdaptersPlugin 中注册 ModelScopeAdapter 和 ModelScopeConfig
  • 实现 ModelScopeAdapter 以将聊天请求发送到 ModelScope API
  • 支持带有工具调用解析的流式和非流式聊天补全流程
Original summary in English

Summary by Sourcery

Add support for ModelScope API by introducing a new adapter and configuration, registering it in the LLMPresetAdaptersPlugin, and handling both streaming and non-streaming chat completions with tool call parsing.

New Features:

  • Register ModelScopeAdapter and ModelScopeConfig in the LLMPresetAdaptersPlugin
  • Implement ModelScopeAdapter to send chat requests to ModelScope API
  • Support both streaming and non-streaming chat completion flows with tool call parsing

Copy link
Contributor

sourcery-ai bot commented Jun 26, 2025

## 审查者指南

此 PR 通过在 LLM 注册表中注册一个新的适配器并实现一个专用的 ModelScopeAdapter(带有其自己的配置),从而添加了端到端的 ModelScope API 支持,该适配器完整地实现了自定义请求构建、流式/非流式逻辑和响应解析。

#### ModelScopeAdapter 集成的类图

```mermaid
classDiagram
    class OpenAIConfig {
    }
    class ModelScopeConfig {
        api_base: str = "https://api-inference.modelscope.cn/v1"
    }
    OpenAIConfig <|-- ModelScopeConfig

    class OpenAIAdapterChatBase {
    }
    class ModelScopeAdapter {
        +chat(req: LLMChatRequest) LLMChatResponse
        +_handle_streaming_request(api_url: str, headers: dict, model: str, req: LLMChatRequest) LLMChatResponse
        +_handle_non_streaming_request(api_url: str, headers: dict, data: dict, model: str) LLMChatResponse
        +_parse_response(response_data: dict, model: str) LLMChatResponse
    }
    OpenAIAdapterChatBase <|-- ModelScopeAdapter

    class LLMChatRequest {
    }
    class LLMChatResponse {
    }
    class Usage {
    }
    class Message {
    }
    class LLMChatContentPartType {
    }
    class LLMChatTextContent {
    }
    class LLMToolCallContent {
    }

    ModelScopeAdapter --> ModelScopeConfig
    ModelScopeAdapter --> LLMChatRequest
    ModelScopeAdapter --> LLMChatResponse
    LLMChatResponse --> Usage
    LLMChatResponse --> Message
    Message --> LLMChatContentPartType
    LLMChatContentPartType <|-- LLMChatTextContent
    LLMChatContentPartType <|-- LLMToolCallContent

文件级别变更

变更 详情 文件
在 LLM 注册表中注册 ModelScope 适配器
  • 为 ModelScopeAdapter 和 ModelScopeConfig 添加注册表条目
  • 在注册后记录适配器加载
kirara_ai/plugins/llm_preset_adapters/__init__.py
实现 ModelScopeAdapter 和配置
  • 引入带有自定义 api_base 的 ModelScopeConfig 子类
  • 扩展 OpenAIAdapterChatBase 以覆盖 chat(),构建和过滤 API 有效负载
  • 添加带有 SSE 解析、内容聚合和工具调用提取的流式处理程序 (_handle_streaming_request)
  • 添加非流式处理程序 (_handle_non_streaming_request) 和响应解析器 (_parse_response) 以进行常规补全
kirara_ai/plugins/llm_preset_adapters/modelscope_adapter.py

提示和命令

与 Sourcery 互动

  • 触发新的审查: 在 pull request 上评论 @sourcery-ai review
  • 继续讨论: 直接回复 Sourcery 的审查评论。
  • 从审查评论生成 GitHub issue: 通过回复审查评论,要求 Sourcery 从审查评论创建一个 issue。 您也可以回复审查评论并使用 @sourcery-ai issue 从中创建一个 issue。
  • 生成 pull request 标题: 在 pull request 标题中的任何位置写入 @sourcery-ai 以随时生成标题。 您也可以在 pull request 上评论 @sourcery-ai title 以随时(重新)生成标题。
  • 生成 pull request 摘要: 在 pull request 正文中的任何位置写入 @sourcery-ai summary 以随时在您想要的位置生成 PR 摘要。 您也可以在 pull request 上评论 @sourcery-ai summary 以随时(重新)生成摘要。
  • 生成审查者指南: 在 pull request 上评论 @sourcery-ai guide 以随时(重新)生成审查者指南。
  • 解决所有 Sourcery 评论: 在 pull request 上评论 @sourcery-ai resolve 以解决所有 Sourcery 评论。 如果您已经解决了所有评论并且不想再看到它们,这将非常有用。
  • 驳回所有 Sourcery 审查: 在 pull request 上评论 @sourcery-ai dismiss 以驳回所有现有的 Sourcery 审查。 如果您想从新的审查开始,这将特别有用 - 不要忘记评论 @sourcery-ai review 以触发新的审查!

自定义您的体验

访问您的 仪表板 以:

  • 启用或禁用审查功能,例如 Sourcery 生成的 pull request 摘要、审查者指南等。
  • 更改审查语言。
  • 添加、删除或编辑自定义审查说明。
  • 调整其他审查设置。

获得帮助

```
Original review guide in English

Reviewer's Guide

This PR adds end-to-end ModelScope API support by registering a new adapter in the LLM registry and implementing a dedicated ModelScopeAdapter (with its own config), complete with custom request construction, streaming/non-streaming logic, and response parsing.

Class diagram for ModelScopeAdapter integration

classDiagram
    class OpenAIConfig {
    }
    class ModelScopeConfig {
        api_base: str = "https://api-inference.modelscope.cn/v1"
    }
    OpenAIConfig <|-- ModelScopeConfig

    class OpenAIAdapterChatBase {
    }
    class ModelScopeAdapter {
        +chat(req: LLMChatRequest) LLMChatResponse
        +_handle_streaming_request(api_url: str, headers: dict, model: str, req: LLMChatRequest) LLMChatResponse
        +_handle_non_streaming_request(api_url: str, headers: dict, data: dict, model: str) LLMChatResponse
        +_parse_response(response_data: dict, model: str) LLMChatResponse
    }
    OpenAIAdapterChatBase <|-- ModelScopeAdapter

    class LLMChatRequest {
    }
    class LLMChatResponse {
    }
    class Usage {
    }
    class Message {
    }
    class LLMChatContentPartType {
    }
    class LLMChatTextContent {
    }
    class LLMToolCallContent {
    }

    ModelScopeAdapter --> ModelScopeConfig
    ModelScopeAdapter --> LLMChatRequest
    ModelScopeAdapter --> LLMChatResponse
    LLMChatResponse --> Usage
    LLMChatResponse --> Message
    Message --> LLMChatContentPartType
    LLMChatContentPartType <|-- LLMChatTextContent
    LLMChatContentPartType <|-- LLMToolCallContent
Loading

File-Level Changes

Change Details Files
Register ModelScope adapter in LLM registry
  • Add registry entry for ModelScopeAdapter and ModelScopeConfig
  • Log adapter loading after registration
kirara_ai/plugins/llm_preset_adapters/__init__.py
Implement ModelScopeAdapter and configuration
  • Introduce ModelScopeConfig subclass with custom api_base
  • Extend OpenAIAdapterChatBase to override chat(), build and filter API payloads
  • Add streaming handler (_handle_streaming_request) with SSE parsing, content aggregation, and tool‐call extraction
  • Add non-streaming handler (_handle_non_streaming_request) and response parser (_parse_response) for regular completions
kirara_ai/plugins/llm_preset_adapters/modelscope_adapter.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @yrk111222 -

AI Agents
:
## 

### 1
<location> `kirara_ai/plugins/llm_preset_adapters/modelscope_adapter.py:45` </location>
<code_context>
+        }
+        
+        # 
+        data = {
+            "messages": asyncio.run(convert_llm_chat_message_to_openai_message(req.messages, self.media_manager)),
+            "model": req.model,
</code_context>

<issue_to_address>
asyncio.run

asyncio.run,Event LoopWeb Server Notebookasyncio.run
</issue_to_address>

### 2
<location> `kirara_ai/plugins/llm_preset_adapters/modelscope_adapter.py:97` </location>
<code_context>
+        role: Optional[str] = None
+        usage = {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0}
+        
+        for line in response.iter_lines():
+            if not line:
+                continue
+                
</code_context>

<issue_to_address>


,
</issue_to_address>

<suggested_fix>
<<<<<<< SEARCH
        response = requests.post(api_url, json=data, headers=headers, stream=True)
        response.raise_for_status()

        # 
        full_content = ""
        tool_calls = []
        finish_reason: Optional[str] = None
        role: Optional[str] = None
        usage = {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0}

        for line in response.iter_lines():
            if not line:
                continue
=======
        import socket
        from requests.exceptions import Timeout, RequestException

        response = requests.post(api_url, json=data, headers=headers, stream=True, timeout=30)
        response.raise_for_status()

        # 
        full_content = ""
        tool_calls = []
        finish_reason: Optional[str] = None
        role: Optional[str] = None
        usage = {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0}

        max_iterations = 10000  # 
        iteration = 0

        try:
            for line in response.iter_lines():
                iteration += 1
                if iteration > max_iterations:
                    logger.error("Streaming response exceeded maximum allowed iterations (%d), aborting.", max_iterations)
                    break
                if not line:
                    continue
        except (Timeout, RequestException, socket.timeout) as e:
            logger.error("Error occurred during streaming response: %s", str(e))
            # 
            raise
>>>>>>> REPLACE

</suggested_fix>

### 3
<location> `kirara_ai/plugins/llm_preset_adapters/modelscope_adapter.py:172` </location>
<code_context>
+
+    def _handle_non_streaming_request(self, api_url: str, headers: dict, data: dict, model: str) -> LLMChatResponse:
+        """""""
+        response = requests.post(api_url, json=data, headers=headers)
+        
+        try:
</code_context>

<issue_to_address>
requests.post

API
</issue_to_address>

<suggested_fix>
<<<<<<< SEARCH
        response = requests.post(api_url, json=data, headers=headers)
=======
        response = requests.post(api_url, json=data, headers=headers, timeout=10)
>>>>>>> REPLACE

</suggested_fix>

### 4
<location> `kirara_ai/plugins/llm_preset_adapters/modelscope_adapter.py:77` </location>
<code_context>
+        else:
+            return self._handle_streaming_request(api_url, headers, req.model, req)
+    
+    def _handle_streaming_request(self, api_url: str, headers: dict, model: str, req:LLMChatRequest) -> LLMChatResponse:
+        """Qwen/QwQ-32B"""
+        data = {
</code_context>

<issue_to_address>
response-building logichelper methods

SSE/response‐building logicDRY

1) **SSE→JSON**  
```python
def _sse_parser(self, resp) -> Iterator[dict]:
    for line in resp.iter_lines():
        if not line or not line.startswith(b"data: "):
            continue
        data = line[6:].decode("utf-8")
        if data == "[DONE]":
            return
        try:
            yield json.loads(data)
        except json.JSONDecodeError:
            logger.warning(f"Bad SSE JSON: {data}")
```

In `_handle_streaming_request`
```python
for chunk in self._sse_parser(response):
    # existing “choices = chunk.get…” logic
```

2) **tool-call → LLMToolCallContent list**  
```python
def _make_tool_contents(self, raw_calls: List[dict]) -> List[LLMToolCallContent]:
    return [
        LLMToolCallContent(
            id=c["id"],
            name=c["function"]["name"],
            parameters=json.loads(c["function"].get("arguments", "{}"))
        )
        for c in raw_calls
    ]
```

```python
content = [
    LLMToolCallContent(…)
    for call in tool_calls
]
```
with
```python
content = self._make_tool_contents(tool_calls)
```

3) **LLMChatResponse builder**  
```python
def _build_response(
    self,
    model: str,
    usage: dict,
    content_parts: List[LLMChatContentPartType],
    role: str,
    finish_reason: str
) -> LLMChatResponse:
    return LLMChatResponse(
        model=model,
        usage=Usage(**{
            k: usage.get(k, 0) for k in ("prompt_tokens","completion_tokens","total_tokens")
        }),
        message=Message(
            content=content_parts,
            role=role,
            tool_calls=pick_tool_calls(content_parts),
            finish_reason=finish_reason
        )
    )
```
`_handle_streaming_request``_parse_response`
```python
return self._build_response(model, usage, content_parts, role or "assistant", finish_reason or "")
```

—80%
</issue_to_address>

Sourcery ✨
!👍👎
Original comment in English

Hey @yrk111222 - I've reviewed your changes and they look great!

Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments

### Comment 1
<location> `kirara_ai/plugins/llm_preset_adapters/modelscope_adapter.py:45` </location>
<code_context>
+        }
+        
+        # 构建请求数据
+        data = {
+            "messages": asyncio.run(convert_llm_chat_message_to_openai_message(req.messages, self.media_manager)),
+            "model": req.model,
</code_context>

<issue_to_address>
Potential blocking on asyncio.run in synchronous context.

Refactor to avoid asyncio.run, as it can fail if an event loop is already running (e.g., in web servers or notebooks). Ensure this method is only called when no event loop exists.
</issue_to_address>

### Comment 2
<location> `kirara_ai/plugins/llm_preset_adapters/modelscope_adapter.py:97` </location>
<code_context>
+        role: Optional[str] = None
+        usage = {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0}
+        
+        for line in response.iter_lines():
+            if not line:
+                continue
+                
</code_context>

<issue_to_address>
No timeout or error handling for streaming response.

Without timeout or error handling, the loop may hang if the server becomes unresponsive. Please add timeout handling or limit the number of iterations.
</issue_to_address>

<suggested_fix>
<<<<<<< SEARCH
        response = requests.post(api_url, json=data, headers=headers, stream=True)
        response.raise_for_status()

        # 处理流式响应
        full_content = ""
        tool_calls = []
        finish_reason: Optional[str] = None
        role: Optional[str] = None
        usage = {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0}

        for line in response.iter_lines():
            if not line:
                continue
=======
        import socket
        from requests.exceptions import Timeout, RequestException

        response = requests.post(api_url, json=data, headers=headers, stream=True, timeout=30)
        response.raise_for_status()

        # 处理流式响应
        full_content = ""
        tool_calls = []
        finish_reason: Optional[str] = None
        role: Optional[str] = None
        usage = {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0}

        max_iterations = 10000  # 防止死循环
        iteration = 0

        try:
            for line in response.iter_lines():
                iteration += 1
                if iteration > max_iterations:
                    logger.error("Streaming response exceeded maximum allowed iterations (%d), aborting.", max_iterations)
                    break
                if not line:
                    continue
        except (Timeout, RequestException, socket.timeout) as e:
            logger.error("Error occurred during streaming response: %s", str(e))
            # 可以根据需要抛出异常或返回部分内容
            raise
>>>>>>> REPLACE

</suggested_fix>

### Comment 3
<location> `kirara_ai/plugins/llm_preset_adapters/modelscope_adapter.py:172` </location>
<code_context>
+
+    def _handle_non_streaming_request(self, api_url: str, headers: dict, data: dict, model: str) -> LLMChatResponse:
+        """处理非流式请求"""
+        response = requests.post(api_url, json=data, headers=headers)
+        
+        try:
</code_context>

<issue_to_address>
No timeout specified for requests.post.

Setting a timeout prevents the request from hanging indefinitely if the API does not respond.
</issue_to_address>

<suggested_fix>
<<<<<<< SEARCH
        response = requests.post(api_url, json=data, headers=headers)
=======
        response = requests.post(api_url, json=data, headers=headers, timeout=10)
>>>>>>> REPLACE

</suggested_fix>

### Comment 4
<location> `kirara_ai/plugins/llm_preset_adapters/modelscope_adapter.py:77` </location>
<code_context>
+        else:
+            return self._handle_streaming_request(api_url, headers, req.model, req)
+    
+    def _handle_streaming_request(self, api_url: str, headers: dict, model: str, req:LLMChatRequest) -> LLMChatResponse:
+        """处理流式请求,当模型为Qwen/QwQ-32B时"""
+        data = {
</code_context>

<issue_to_address>
Consider extracting repeated parsing and response-building logic into helper methods to reduce code duplication.

Here are three small, focused extractions you can make to collapse the duplicated parsing/response‐building logic. None of these changes revert the behavior—you’ll still parse SSE vs. JSON the same way—but you DRY out:

1) **Extract SSE→JSON chunks into a helper**  
```python
def _sse_parser(self, resp) -> Iterator[dict]:
    for line in resp.iter_lines():
        if not line or not line.startswith(b"data: "):
            continue
        data = line[6:].decode("utf-8")
        if data == "[DONE]":
            return
        try:
            yield json.loads(data)
        except json.JSONDecodeError:
            logger.warning(f"Bad SSE JSON: {data}")
```

In `_handle_streaming_request` replace your loop with:  
```python
for chunk in self._sse_parser(response):
    # existing “choices = chunk.get…” logic
```

2) **Extract tool-call → LLMToolCallContent list**  
```python
def _make_tool_contents(self, raw_calls: List[dict]) -> List[LLMToolCallContent]:
    return [
        LLMToolCallContent(
            id=c["id"],
            name=c["function"]["name"],
            parameters=json.loads(c["function"].get("arguments", "{}"))
        )
        for c in raw_calls
    ]
```
Then in both methods replace
```python
content = [
    LLMToolCallContent(…)
    for call in tool_calls
]
```
with
```python
content = self._make_tool_contents(tool_calls)
```

3) **Pull out common LLMChatResponse builder**  
```python
def _build_response(
    self,
    model: str,
    usage: dict,
    content_parts: List[LLMChatContentPartType],
    role: str,
    finish_reason: str
) -> LLMChatResponse:
    return LLMChatResponse(
        model=model,
        usage=Usage(**{
            k: usage.get(k, 0) for k in ("prompt_tokens","completion_tokens","total_tokens")
        }),
        message=Message(
            content=content_parts,
            role=role,
            tool_calls=pick_tool_calls(content_parts),
            finish_reason=finish_reason
        )
    )
```
Then at end of both `_handle_streaming_request` and `_parse_response` you simply do:
```python
return self._build_response(model, usage, content_parts, role or "assistant", finish_reason or "")
```

—these three helpers remove > 80% of your copy-paste while preserving every feature.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

}

# 构建请求数据
data = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (bug_risk): asyncio.run

asyncio.run,

asyncio.run,Web Server Notebook

Original comment in English

issue (bug_risk): Potential blocking on asyncio.run in synchronous context.

Refactor to avoid asyncio.run, as it can fail if an event loop is already running (e.g., in web servers or notebooks). Ensure this method is only called when no event loop exists.

Comment on lines 87 to 99
response = requests.post(api_url, json=data, headers=headers, stream=True)
response.raise_for_status()

# 处理流式响应
full_content = ""
tool_calls = []
finish_reason: Optional[str] = None
role: Optional[str] = None
usage = {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0}

for line in response.iter_lines():
if not line:
continue
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (bug_risk):

Suggested change
response = requests.post(api_url, json=data, headers=headers, stream=True)
response.raise_for_status()
# 处理流式响应
full_content = ""
tool_calls = []
finish_reason: Optional[str] = None
role: Optional[str] = None
usage = {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0}
for line in response.iter_lines():
if not line:
continue
import socket
from requests.exceptions import Timeout, RequestException
response = requests.post(api_url, json=data, headers=headers, stream=True, timeout=30)
response.raise_for_status()
#
full_content = ""
tool_calls = []
finish_reason: Optional[str] = None
role: Optional[str] = None
usage = {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0}
max_iterations = 10000 #
iteration = 0
try:
for line in response.iter_lines():
iteration += 1
if iteration > max_iterations:
logger.error("Streaming response exceeded maximum allowed iterations (%d), aborting.", max_iterations)
break
if not line:
continue
except (Timeout, RequestException, socket.timeout) as e:
logger.error("Error occurred during streaming response: %s", str(e))
#
raise
Original comment in English

suggestion (bug_risk): No timeout or error handling for streaming response.

Without timeout or error handling, the loop may hang if the server becomes unresponsive. Please add timeout handling or limit the number of iterations.

Suggested change
response = requests.post(api_url, json=data, headers=headers, stream=True)
response.raise_for_status()
# 处理流式响应
full_content = ""
tool_calls = []
finish_reason: Optional[str] = None
role: Optional[str] = None
usage = {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0}
for line in response.iter_lines():
if not line:
continue
import socket
from requests.exceptions import Timeout, RequestException
response = requests.post(api_url, json=data, headers=headers, stream=True, timeout=30)
response.raise_for_status()
# 处理流式响应
full_content = ""
tool_calls = []
finish_reason: Optional[str] = None
role: Optional[str] = None
usage = {"prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0}
max_iterations = 10000 # 防止死循环
iteration = 0
try:
for line in response.iter_lines():
iteration += 1
if iteration > max_iterations:
logger.error("Streaming response exceeded maximum allowed iterations (%d), aborting.", max_iterations)
break
if not line:
continue
except (Timeout, RequestException, socket.timeout) as e:
logger.error("Error occurred during streaming response: %s", str(e))
# 可以根据需要抛出异常或返回部分内容
raise


def _handle_non_streaming_request(self, api_url: str, headers: dict, data: dict, model: str) -> LLMChatResponse:
"""处理非流式请求"""
response = requests.post(api_url, json=data, headers=headers)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (bug_risk): requests.post

API

Suggested change
response = requests.post(api_url, json=data, headers=headers)
response = requests.post(api_url, json=data, headers=headers, timeout=10)
Original comment in English

suggestion (bug_risk): No timeout specified for requests.post.

Setting a timeout prevents the request from hanging indefinitely if the API does not respond.

Suggested change
response = requests.post(api_url, json=data, headers=headers)
response = requests.post(api_url, json=data, headers=headers, timeout=10)

else:
return self._handle_streaming_request(api_url, headers, req.model, req)

def _handle_streaming_request(self, api_url: str, headers: dict, model: str, req:LLMChatRequest) -> LLMChatResponse:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (complexity):

/response-building logichelper methods.

SSE/response‐building logic

  1. SSE→JSON
def _sse_parser(self, resp) -> Iterator[dict]:
    for line in resp.iter_lines():
        if not line or not line.startswith(b"data: "):
            continue
        data = line[6:].decode("utf-8")
        if data == "[DONE]":
            return
        try:
            yield json.loads(data)
        except json.JSONDecodeError:
            logger.warning(f"Bad SSE JSON: {data}")

In _handle_streaming_request

for chunk in self._sse_parser(response):
    # existing “choices = chunk.get…” logic
  1. tool-call → LLMToolCallContent list
def _make_tool_contents(self, raw_calls: List[dict]) -> List[LLMToolCallContent]:
    return [
        LLMToolCallContent(
            id=c["id"],
            name=c["function"]["name"],
            parameters=json.loads(c["function"].get("arguments", "{}"))
        )
        for c in raw_calls
    ]
content = [
    LLMToolCallContent(…)
    for call in tool_calls
]

with

content = self._make_tool_contents(tool_calls)
  1. LLMChatResponse builder
def _build_response(
    self,
    model: str,
    usage: dict,
    content_parts: List[LLMChatContentPartType],
    role: str,
    finish_reason: str
) -> LLMChatResponse:
    return LLMChatResponse(
        model=model,
        usage=Usage(**{
            k: usage.get(k, 0) for k in ("prompt_tokens","completion_tokens","total_tokens")
        }),
        message=Message(
            content=content_parts,
            role=role,
            tool_calls=pick_tool_calls(content_parts),
            finish_reason=finish_reason
        )
    )

_handle_streaming_request``_parse_response

return self._build_response(model, usage, content_parts, role or "assistant", finish_reason or "")

—80%

Original comment in English

issue (complexity): Consider extracting repeated parsing and response-building logic into helper methods to reduce code duplication.

Here are three small, focused extractions you can make to collapse the duplicated parsing/response‐building logic. None of these changes revert the behavior—you’ll still parse SSE vs. JSON the same way—but you DRY out:

  1. Extract SSE→JSON chunks into a helper
def _sse_parser(self, resp) -> Iterator[dict]:
    for line in resp.iter_lines():
        if not line or not line.startswith(b"data: "):
            continue
        data = line[6:].decode("utf-8")
        if data == "[DONE]":
            return
        try:
            yield json.loads(data)
        except json.JSONDecodeError:
            logger.warning(f"Bad SSE JSON: {data}")

In _handle_streaming_request replace your loop with:

for chunk in self._sse_parser(response):
    # existing “choices = chunk.get…” logic
  1. Extract tool-call → LLMToolCallContent list
def _make_tool_contents(self, raw_calls: List[dict]) -> List[LLMToolCallContent]:
    return [
        LLMToolCallContent(
            id=c["id"],
            name=c["function"]["name"],
            parameters=json.loads(c["function"].get("arguments", "{}"))
        )
        for c in raw_calls
    ]

Then in both methods replace

content = [
    LLMToolCallContent(…)
    for call in tool_calls
]

with

content = self._make_tool_contents(tool_calls)
  1. Pull out common LLMChatResponse builder
def _build_response(
    self,
    model: str,
    usage: dict,
    content_parts: List[LLMChatContentPartType],
    role: str,
    finish_reason: str
) -> LLMChatResponse:
    return LLMChatResponse(
        model=model,
        usage=Usage(**{
            k: usage.get(k, 0) for k in ("prompt_tokens","completion_tokens","total_tokens")
        }),
        message=Message(
            content=content_parts,
            role=role,
            tool_calls=pick_tool_calls(content_parts),
            finish_reason=finish_reason
        )
    )

Then at end of both _handle_streaming_request and _parse_response you simply do:

return self._build_response(model, usage, content_parts, role or "assistant", finish_reason or "")

—these three helpers remove > 80% of your copy-paste while preserving every feature.


# 处理工具调用或普通响应
content: list[LLMChatContentPartType] = []
if tool_calls := message.get("tool_calls", None):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (code-quality): Replace dict.get(x, None) with dict.get(x) (remove-none-from-default-get)

Suggested change
if tool_calls := message.get("tool_calls", None):
if tool_calls := message.get("tool_calls"):


get
keyNone``None

Original comment in English

suggestion (code-quality): Replace dict.get(x, None) with dict.get(x) (remove-none-from-default-get)

Suggested change
if tool_calls := message.get("tool_calls", None):
if tool_calls := message.get("tool_calls"):


ExplanationWhen using a dictionary's get method you can specify a default to return if
the key is not found. This defaults to None, so it is unnecessary to specify
None if this is the required behaviour. Removing the unnecessary argument
makes the code slightly shorter and clearer.

}

# 构建请求数据
data = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (code-quality): (merge-dict-assign)

Original comment in English

issue (code-quality): Merge dictionary assignment with declaration (merge-dict-assign)

else:
return self._handle_streaming_request(api_url, headers, req.model, req)

def _handle_streaming_request(self, api_url: str, headers: dict, model: str, req:LLMChatRequest) -> LLMChatResponse:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (code-quality):



25%。

  • 10
Original comment in English

issue (code-quality): We've found these issues:


Explanation
The quality score for this function is below the quality threshold of 25%.
This score is a combination of the method length, cognitive complexity and working memory.

How can you solve this?

It might be worth refactoring this function to make it shorter and more readable.

  • Reduce the function length by extracting pieces of functionality out into
    their own functions. This is the most important thing you can do - ideally a
    function should be less than 10 lines.
  • Reduce nesting, perhaps by introducing guard clauses to return early.
  • Ensure that variables are tightly scoped, so that code using related concepts
    sits together within the function rather than being scattered.

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型检查结果 ❌

在 PR 修改的代码行中发现了 5 个类型问题,需要修复。

已对修改的代码行创建了 5 个行级评论。

logger.debug(f"Sending request to ModelScope API: {api_url}")
logger.debug(f"Request data: {json.dumps(data, indent=2, ensure_ascii=False)}")
if req.model not in ["Qwen/QwQ-32B","deepseek-ai/DeepSeek-R1-0528"]:
return self._handle_non_streaming_request(api_url, headers, data, req.model)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型错误: Argument 4 to "_handle_non_streaming_request" of "ModelScopeAdapter" has incompatible type "str | None"; expected "str" (arg-type)

详细信息请参考 mypy 文档

"stream": True,
"messages": [{
"role": "user",
"content": req.messages[0].content[0].text

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型错误: Item "LLMChatImageContent" of "LLMChatTextContent | LLMChatImageContent | LLMToolCallContent | LLMToolResultContent" has no attribute "text" (union-attr)

详细信息请参考 mypy 文档

"stream": True,
"messages": [{
"role": "user",
"content": req.messages[0].content[0].text

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型错误: Item "LLMToolCallContent" of "LLMChatTextContent | LLMChatImageContent | LLMToolCallContent | LLMToolResultContent" has no attribute "text" (union-attr)

详细信息请参考 mypy 文档

"stream": True,
"messages": [{
"role": "user",
"content": req.messages[0].content[0].text

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型错误: Item "LLMToolResultContent" of "LLMChatTextContent | LLMChatImageContent | LLMToolCallContent | LLMToolResultContent" has no attribute "text" (union-attr)

详细信息请参考 mypy 文档

),
message=Message(
content=content_parts,
role=role or "assistant",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型错误: Argument "role" to "Message" has incompatible type "str"; expected "Literal['user', 'assistant', 'system', 'tool']" (arg-type)

详细信息请参考 mypy 文档

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型检查结果 ❌

在 PR 修改的代码行中发现了 5 个类型问题,需要修复。

已对修改的代码行创建了 5 个行级评论。

logger.debug(f"Sending request to ModelScope API: {api_url}")
logger.debug(f"Request data: {json.dumps(data, indent=2, ensure_ascii=False)}")
if req.model not in ["Qwen/QwQ-32B","deepseek-ai/DeepSeek-R1-0528"]:
return self._handle_non_streaming_request(api_url, headers, data, req.model)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型错误: Argument 4 to "_handle_non_streaming_request" of "ModelScopeAdapter" has incompatible type "str | None"; expected "str" (arg-type)

详细信息请参考 mypy 文档

"stream": True,
"messages": [{
"role": "user",
"content": req.messages[0].content[0].text

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型错误: Item "LLMChatImageContent" of "LLMChatTextContent | LLMChatImageContent | LLMToolCallContent | LLMToolResultContent" has no attribute "text" (union-attr)

详细信息请参考 mypy 文档

"stream": True,
"messages": [{
"role": "user",
"content": req.messages[0].content[0].text

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型错误: Item "LLMToolCallContent" of "LLMChatTextContent | LLMChatImageContent | LLMToolCallContent | LLMToolResultContent" has no attribute "text" (union-attr)

详细信息请参考 mypy 文档

"stream": True,
"messages": [{
"role": "user",
"content": req.messages[0].content[0].text

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型错误: Item "LLMToolResultContent" of "LLMChatTextContent | LLMChatImageContent | LLMToolCallContent | LLMToolResultContent" has no attribute "text" (union-attr)

详细信息请参考 mypy 文档

),
message=Message(
content=content_parts,
role=role or "assistant",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型错误: Argument "role" to "Message" has incompatible type "str"; expected "Literal['user', 'assistant', 'system', 'tool']" (arg-type)

详细信息请参考 mypy 文档

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型检查结果 ❌

在 PR 修改的代码行中发现了 5 个类型问题,需要修复。

已对修改的代码行创建了 5 个行级评论。

logger.debug(f"Sending request to ModelScope API: {api_url}")
logger.debug(f"Request data: {json.dumps(data, indent=2, ensure_ascii=False)}")
if req.model not in ["Qwen/QwQ-32B","deepseek-ai/DeepSeek-R1-0528"]:
return self._handle_non_streaming_request(api_url, headers, data, req.model)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型错误: Argument 4 to "_handle_non_streaming_request" of "ModelScopeAdapter" has incompatible type "str | None"; expected "str" (arg-type)

详细信息请参考 mypy 文档

"stream": True,
"messages": [{
"role": "user",
"content": req.messages[0].content[0].text

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型错误: Item "LLMChatImageContent" of "LLMChatTextContent | LLMChatImageContent | LLMToolCallContent | LLMToolResultContent" has no attribute "text" (union-attr)

详细信息请参考 mypy 文档

"stream": True,
"messages": [{
"role": "user",
"content": req.messages[0].content[0].text

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型错误: Item "LLMToolCallContent" of "LLMChatTextContent | LLMChatImageContent | LLMToolCallContent | LLMToolResultContent" has no attribute "text" (union-attr)

详细信息请参考 mypy 文档

"stream": True,
"messages": [{
"role": "user",
"content": req.messages[0].content[0].text

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型错误: Item "LLMToolResultContent" of "LLMChatTextContent | LLMChatImageContent | LLMToolCallContent | LLMToolResultContent" has no attribute "text" (union-attr)

详细信息请参考 mypy 文档

),
message=Message(
content=content_parts,
role=data["messages"][0].get("role", "assistant"),

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型错误: Value of type "object" is not indexable (index)

详细信息请参考 mypy 文档

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型检查结果 ❌

在 PR 修改的代码行中发现了 4 个类型问题,需要修复。

已对修改的代码行创建了 4 个行级评论。

logger.debug(f"Sending request to ModelScope API: {api_url}")
logger.debug(f"Request data: {json.dumps(data, indent=2, ensure_ascii=False)}")
if req.model not in ["Qwen/QwQ-32B","deepseek-ai/DeepSeek-R1-0528"]:
return self._handle_non_streaming_request(api_url, headers, data, req.model)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型错误: Argument 4 to "_handle_non_streaming_request" of "ModelScopeAdapter" has incompatible type "str | None"; expected "str" (arg-type)

详细信息请参考 mypy 文档

# 只返回最终内容
content_parts = [LLMChatTextContent(text=full_content)]

first_message = data["messages"][0] if data.get("messages") and len(data["messages"]) > 0 else {}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型错误: Value of type "object" is not indexable (index)

详细信息请参考 mypy 文档

# 只返回最终内容
content_parts = [LLMChatTextContent(text=full_content)]

first_message = data["messages"][0] if data.get("messages") and len(data["messages"]) > 0 else {}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型错误: Argument 1 to "len" has incompatible type "object"; expected "Sized" (arg-type)

详细信息请参考 mypy 文档

),
message=Message(
content=content_parts,
role=data["messages"][0].get("role", "assistant"),

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型错误: Value of type "object" is not indexable (index)

详细信息请参考 mypy 文档

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型检查结果 ❌

在 PR 修改的代码行中发现了 1 个类型问题,需要修复。

已对修改的代码行创建了 1 个行级评论。

),
message=Message(
content=content_parts,
role=role_value,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型错误: Argument "role" to "Message" has incompatible type "str"; expected "Literal['user', 'assistant', 'system', 'tool']" (arg-type)

详细信息请参考 mypy 文档

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型检查结果 ❌

在 PR 修改的代码行中发现了 2 个类型问题,需要修复。

已对修改的代码行创建了 2 个行级评论。

if "role" in delta and delta["role"]:
role_str = delta["role"]
if role_str in {'user', 'assistant', 'system', 'tool'}:
role_value = cast(Literal['user', 'assistant', 'system', 'tool'], role_str)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型错误: Name "Literal" is not defined (name-defined)

详细信息请参考 mypy 文档

content_parts = [LLMChatTextContent(text=full_content)]

# 构建响应时确保使用有效的角色
valid_role: Literal['user', 'assistant', 'system', 'tool'] = role_value or "assistant"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MyPy 类型错误: Name "Literal" is not defined (name-defined)

详细信息请参考 mypy 文档

Copy link

MyPy 类型检查通过 ✅

PR 修改的代码行通过了类型检查。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant