Skip to content

Commit abbf036

Browse files
Harold-lkkHarold-lkk
authored
[Enchance] update chat format (InternLM#634)
1 parent 4281caf commit abbf036

File tree

3 files changed

+170
-170
lines changed

3 files changed

+170
-170
lines changed

‎chat/chat_format.md

Lines changed: 83 additions & 83 deletions
Original file line numberDiff line numberDiff line change
@@ -9,24 +9,24 @@ InternLM2-Chat adopts a new chat format to flexibly support a wider range of app
99
The regular chat structure usually contains three roles: `system`, `user`, and `assistant`, formatted as follows for multi-turn dialogues:
1010

1111
```
12-
[UNUSED_TOKEN_146]system
13-
You are InternLM2-Chat, a harmless AI assistant[UNUSED_TOKEN_145]
14-
[UNUSED_TOKEN_146]user
15-
Hello[UNUSED_TOKEN_145]
16-
[UNUSED_TOKEN_146]assistant
17-
Hello, I am InternLM2-Chat, how can I assist you?[UNUSED_TOKEN_145]
12+
<|im_start|>system
13+
You are InternLM2-Chat, a harmless AI assistant<|im_end|>
14+
<|im_start|>user
15+
Hello<|im_end|>
16+
<|im_start|>assistant
17+
Hello, I am InternLM2-Chat, how can I assist you?<|im_end|>
1818
```
1919

20-
Here, `[UNUSED_TOKEN_146]` acts as the start token for each turn of dialogue, and `[UNUSED_TOKEN_145]` as the end token. Each turn of dialogue typically starts with `[UNUSED_TOKEN_146]role` and ends with the model's output `[UNUSED_TOKEN_145]`, where role represents `system`, `user`, `assistant`, and `environment`. You may refer to the [code in huggingface](https://huggingface.co/internlm/internlm2-chat-7b/blob/main/modeling_internlm2.py#L1138) to see how the chat history is organized.
20+
Here, `<|im_start|>` acts as the start token for each turn of dialogue, and `<|im_end|>` as the end token. Each turn of dialogue typically starts with `<|im_start|>role` and ends with the model's output `<|im_end|>`, where role represents `system`, `user`, `assistant`, and `environment`. You may refer to the [code in huggingface](https://huggingface.co/internlm/internlm2-chat-7b/blob/main/modeling_internlm2.py#L1138) to see how the chat history is organized.
2121

2222
Currently, the InternLM2-Chat model's vocabulary maintains the following mappings to support full functionalities:
2323

24-
- `[UNUSED_TOKEN_146]`: Start token for each role's dialogue, the token ID is **92543**
25-
- `[UNUSED_TOKEN_145]`: End token for each role's dialogue, the token ID is **92542**
26-
- `[UNUSED_TOKEN_144]`: Start token for invoking external plugins, the token ID is **92541**
27-
- `[UNUSED_TOKEN_143]`: End token for invoking external plugins, the token ID is **92540**
28-
- `[UNUSED_TOKEN_142]`: Code interpreter, the token ID is **92539**
29-
- `[UNUSED_TOKEN_141]`: External plugins, regular tools, the token ID is **92538**
24+
- `<|im_start|>`: Start token for each role's dialogue, the token ID is **92543**
25+
- `<|im_end|>`: End token for each role's dialogue, the token ID is **92542**
26+
- `<|action_start|>`: Start token for invoking external tools, like interpreter or plugin, the token ID is **92541**
27+
- `<|action_end|>`: End token for invoking external plugins, the token ID is **92540**
28+
- `<|interpreter|>`: Code interpreter, the token ID is **92539**
29+
- `<|plugin|>`: External plugins, regular tools, the token ID is **92538**
3030

3131
## Complete Structure
3232

@@ -37,9 +37,9 @@ The complete chat format of InternLM2-Chat, based on the basic structure, also i
3737
InternLM2-Chat support multiple formats (e.g., ReAct) to conduct function call, especially **json format** to ease downstream applications。An example of complete function call is shown below.
3838

3939
```
40-
[UNUSED_TOKEN_146]system
41-
You are InternLM2-Chat, a harmless AI assistant[UNUSED_TOKEN_145]
42-
[UNUSED_TOKEN_146]system name=[UNUSED_TOKEN_141]
40+
<|im_start|>system
41+
You are InternLM2-Chat, a harmless AI assistant<|im_end|>
42+
<|im_start|>system name=<|plugin|>
4343
[
4444
{
4545
"name": "get_current_weather",
@@ -57,46 +57,46 @@ You are InternLM2-Chat, a harmless AI assistant[UNUSED_TOKEN_145]
5757
},
5858
}
5959
]
60-
[UNUSED_TOKEN_145]
61-
[UNUSED_TOKEN_146]user
62-
I want to know today's weather in Shanghai[UNUSED_TOKEN_145]
63-
[UNUSED_TOKEN_146]assistant
64-
Sure, I will search for the weather of Shanghai.[UNUSED_TOKEN_144][UNUSED_TOKEN_141]
65-
{"name": "get_current_weather", "parameters": {"location": "Shanghai"}}[UNUSED_TOKEN_143][UNUSED_TOKEN_145]
66-
[UNUSED_TOKEN_146]environment name=[UNUSED_TOKEN_141]
67-
{"temperature": 22}[UNUSED_TOKEN_145]
68-
[UNUSED_TOKEN_146]assistant
69-
The weather in Shanghai is 22 celsius[UNUSED_TOKEN_145]
60+
<|im_end|>
61+
<|im_start|>user
62+
I want to know today's weather in Shanghai<|im_end|>
63+
<|im_start|>assistant
64+
Sure, I will search for the weather of Shanghai.<|action_start|><|plugin|>
65+
{"name": "get_current_weather", "parameters": {"location": "Shanghai"}}<|action_end|><|im_end|>
66+
<|im_start|>environment name=<|plugin|>
67+
{"temperature": 22}<|im_end|>
68+
<|im_start|>assistant
69+
The weather in Shanghai is 22 celsius<|im_end|>
7070
```
7171

72-
1. First，there will be a new system prompt that describe the protocol of tools in json format. The content starts with `[UNUSED_TOKEN_146]system name=[UNUSED_TOKEN_141]\n` and ends with `[UNUSED_TOKEN_145]`. `name=[UNUSED_TOKEN_141]` indicates the system prompt comes from tools. InternLM2-Chat supports and follows multiple system prompts in the chat history, so we can also see the system prompt ask the model to be helpful assistant.
73-
2. Model will call the tools in a streaming format after receving the user prompt, i.e., it will naturally speak something (thoughts, helpful response) then call the tools. The model will output `[UNUSED_TOKEN_144][UNUSED_TOKEN_141]`, where `[UNUSED_TOKEN_144]` indicates the model needs to call extensions and `[UNUSED_TOKEN_141]` indicates it want to use common tools. Then the model will output the calling parameters in json format and end it with `[UNUSED_TOKEN_143]`.
74-
3. The environment returns the function calling outputs, starts with `[UNUSED_TOKEN_146]environment name=[UNUSED_TOKEN_141]`, indicating it comes from the function calling in the environment. The content is in json format, `{"temperature": 22}`, ends with `[UNUSED_TOKEN_145]`.
72+
1. First，there will be a new system prompt that describe the protocol of tools in json format. The content starts with `<|im_start|>system name=<|plugin|>\n` and ends with `<|im_end|>`. `name=<|plugin|>` indicates the system prompt comes from tools. InternLM2-Chat supports and follows multiple system prompts in the chat history, so we can also see the system prompt ask the model to be helpful assistant.
73+
2. Model will call the tools in a streaming format after receving the user prompt, i.e., it will naturally speak something (thoughts, helpful response) then call the tools. The model will output `<|action_start|><|plugin|>`, where `<|action_start|>` indicates the model needs to call extensions and `<|plugin|>` indicates it want to use common tools. Then the model will output the calling parameters in json format and end it with `<|action_end|>`.
74+
3. The environment returns the function calling outputs, starts with `<|im_start|>environment name=<|plugin|>`, indicating it comes from the function calling in the environment. The content is in json format, `{"temperature": 22}`, ends with `<|im_end|>`.
7575

7676
### Code Interpreter
7777

7878
InternLM2-Chat support to use code interpreter to solve complex math problems and data analysis. The whole process is similar to the function calling, shown as below.
7979

8080
```
81-
[UNUSED_TOKEN_146]system
82-
You are InternLM2-Chat, a harmless AI assistant[UNUSED_TOKEN_145]
83-
[UNUSED_TOKEN_146]system name=[UNUSED_TOKEN_142]
81+
<|im_start|>system
82+
You are InternLM2-Chat, a harmless AI assistant<|im_end|>
83+
<|im_start|>system name=<|interpreter|>
8484
You now have access to a Jupyter notebook environment supporting Python code execution. Just send code to python to run in this stateful environment. This feature is suitable for:
8585
- Data analysis or processing (such as data manipulation and graphic creation)
8686
- Complex calculations (such as math and physics problems)
8787
- Programming examples (for understanding programming concepts or language features)
8888
- Text processing and analysis (including text analysis and natural language processing)
8989
- Machine learning and data science (model training and data visualization)
90-
- File operations and data import (handling CSV, JSON, etc. formats)[UNUSED_TOKEN_145]
91-
[UNUSED_TOKEN_146]user
90+
- File operations and data import (handling CSV, JSON, etc. formats)<|im_end|>
91+
<|im_start|>user
9292
Please help me process and visualize this dataset.
93-
[UNUSED_TOKEN_145]
94-
[UNUSED_TOKEN_146]user name=file
95-
[{"path": "data.csv", size='10K'}][UNUSED_TOKEN_145]
96-
[UNUSED_TOKEN_146]assistant
93+
<|im_end|>
94+
<|im_start|>user name=file
95+
[{"path": "data.csv", size='10K'}]<|im_end|>
96+
<|im_start|>assistant
9797
I have processed the data and visualized it for you.
9898
99-
[UNUSED_TOKEN_144][UNUSED_TOKEN_142]
99+
<|action_start|><|interpreter|>
100100
```python
101101
import plotly.express as px
102102
import pandas as pd
@@ -118,40 +118,40 @@ fig.update_traces(hovertemplate='<b>Date: %{text}</b><br>Wind Direction 9am: %{x
118118
119119
# Show the plot
120120
fig.show()
121-
```[UNUSED_TOKEN_143]
122-
[UNUSED_TOKEN_145]
123-
[UNUSED_TOKEN_146]environment name=[UNUSED_TOKEN_142]
121+
```<|action_end|>
122+
<|im_end|>
123+
<|im_start|>environment name=<|interpreter|>
124124
![image](xxx.png)
125-
[UNUSED_TOKEN_145]
126-
[UNUSED_TOKEN_146]assistant
127-
Since the code output is not included here, I cannot provide specific chart content. However, if the code executed correctly, it should display a polar plot with two filled areas representing the relationship between wind direction at 9 am and rainfall, and between wind direction at 3 pm and rainfall, respectively. The values for each direction are based on the average rainfall calculated from the provided dataset. The chart should have a clear title, a legend, and be intuitive for comparing rainfall with different wind directions. Given the use of a dark theme, the overall appearance of the chart should be bright lines and filled areas on a dark background.[UNUSED_TOKEN_145]
128-
[UNUSED_TOKEN_146]user
129-
Can you explain the code?[UNUSED_TOKEN_145]
130-
[UNUSED_TOKEN_146]assistant
125+
<|im_end|>
126+
<|im_start|>assistant
127+
Since the code output is not included here, I cannot provide specific chart content. However, if the code executed correctly, it should display a polar plot with two filled areas representing the relationship between wind direction at 9 am and rainfall, and between wind direction at 3 pm and rainfall, respectively. The values for each direction are based on the average rainfall calculated from the provided dataset. The chart should have a clear title, a legend, and be intuitive for comparing rainfall with different wind directions. Given the use of a dark theme, the overall appearance of the chart should be bright lines and filled areas on a dark background.<|im_end|>
128+
<|im_start|>user
129+
Can you explain the code?<|im_end|>
130+
<|im_start|>assistant
131131
The code uses Python's Plotly library to create a polar plot that shows the relationship between wind direction and rainfall. It starts by creating a Plotly Figure object. Then, I added two `Scatterpolar` traces. The first trace shows the relationship between wind direction at 9 am and average rainfall, and the second trace shows the same for 3 pm. In these traces, the `r` parameter represents the average rainfall, and `theta` represents the wind direction. `fill='toself'` means the shape is filled to its own boundary. Next, I updated the chart layout, hid the radial axis, added a legend, set the title and its font, chose a dark theme, and set the chart's background color, width, and height. Finally, the chart was displayed using `fig.show()`.
132-
[UNUSED_TOKEN_145]
132+
<|im_end|>
133133
```
134134

135-
1. First，there will be a new system prompt that describe the usages of code interpreter. The content starts with `[UNUSED_TOKEN_146]system name=[UNUSED_TOKEN_142]\n` and ends with `[UNUSED_TOKEN_145]`. `name=[UNUSED_TOKEN_142]` indicates the system prompt comes from code interpreter. InternLM2-Chat supports and follows multiple system prompts in the chat history, so we can also see the system prompt ask the model to be helpful assistant.
136-
2. Model will call the tools in a streaming format after receving the user prompt, i.e., it will naturally speak something (thoughts, helpful response) then call the code interpreter. The model will output `[UNUSED_TOKEN_144][UNUSED_TOKEN_142]`, where `[UNUSED_TOKEN_144]` indicates the model needs to call extensions and `[UNUSED_TOKEN_142]` indicates it want to use code interpreter. Then the model will output the code in a markdown-style code block and end it with `[UNUSED_TOKEN_143]`.
137-
3. The environment will execute the code and return outputs, starts with `[UNUSED_TOKEN_146]environment name=[UNUSED_TOKEN_142]`, indicating it comes from the code interpreter in the environment, and the content ends with `[UNUSED_TOKEN_145]`.
135+
1. First，there will be a new system prompt that describe the usages of code interpreter. The content starts with `<|im_start|>system name=<|interpreter|>\n` and ends with `<|im_end|>`. `name=<|interpreter|>` indicates the system prompt comes from code interpreter. InternLM2-Chat supports and follows multiple system prompts in the chat history, so we can also see the system prompt ask the model to be helpful assistant.
136+
2. Model will call the tools in a streaming format after receving the user prompt, i.e., it will naturally speak something (thoughts, helpful response) then call the code interpreter. The model will output `<|action_start|><|interpreter|>`, where `<|action_start|>` indicates the model needs to call extensions and `<|interpreter|>` indicates it want to use code interpreter. Then the model will output the code in a markdown-style code block and end it with `<|action_end|>`.
137+
3. The environment will execute the code and return outputs, starts with `<|im_start|>environment name=<|interpreter|>`, indicating it comes from the code interpreter in the environment, and the content ends with `<|im_end|>`.
138138

139139
### Function Call + Code Interpreter
140140

141141
InternLM2-Chat can choose to use tools or code interpreter automatically. We only need to send the system prompt of both to the model respectively. An example is shown as below
142142

143143
```
144-
[UNUSED_TOKEN_146]system
145-
You are InternLM2-Chat, a harmless AI assistant[UNUSED_TOKEN_145]
146-
[UNUSED_TOKEN_146]system name=[UNUSED_TOKEN_142]
144+
<|im_start|>system
145+
You are InternLM2-Chat, a harmless AI assistant<|im_end|>
146+
<|im_start|>system name=<|interpreter|>
147147
You now have access to a Jupyter notebook environment supporting Python code execution. Just send code to python to run in this stateful environment. This feature is suitable for:
148148
- Data analysis or processing (such as data manipulation and graphic creation)
149149
- Complex calculations (such as math and physics problems)
150150
- Programming examples (for understanding programming concepts or language features)
151151
- Text processing and analysis (including text analysis and natural language processing)
152152
- Machine learning and data science (model training and data visualization)
153-
- File operations and data import (handling CSV, JSON, etc. formats)[UNUSED_TOKEN_145]
154-
[UNUSED_TOKEN_146]system name=[UNUSED_TOKEN_141]
153+
- File operations and data import (handling CSV, JSON, etc. formats)<|im_end|>
154+
<|im_start|>system name=<|plugin|>
155155
[
156156
{
157157
"name": "get_current_weather",
@@ -169,16 +169,16 @@ You now have access to a Jupyter notebook environment supporting Python code exe
169169
},
170170
}
171171
]
172-
[UNUSED_TOKEN_145]
173-
[UNUSED_TOKEN_146]user
172+
<|im_end|>
173+
<|im_start|>user
174174
Please help me process and visualize this dataset.
175-
[UNUSED_TOKEN_145]
176-
[UNUSED_TOKEN_146]user name=file
177-
[{"path": "data.csv", size='10K'}][UNUSED_TOKEN_145]
178-
[UNUSED_TOKEN_146]assistant
175+
<|im_end|>
176+
<|im_start|>user name=file
177+
[{"path": "data.csv", size='10K'}]<|im_end|>
178+
<|im_start|>assistant
179179
I have processed the data and visualized it for you.
180180
181-
[UNUSED_TOKEN_144][UNUSED_TOKEN_142]
181+
<|action_start|><|interpreter|>
182182
```python
183183
import plotly.express as px
184184
import pandas as pd
@@ -200,25 +200,25 @@ fig.update_traces(hovertemplate='<b>Date: %{text}</b><br>Wind Direction 9am: %{x
200200
201201
# Show the plot
202202
fig.show()
203-
```[UNUSED_TOKEN_143]
204-
[UNUSED_TOKEN_145]
205-
[UNUSED_TOKEN_146]environment name=[UNUSED_TOKEN_142]
203+
```<|action_end|>
204+
<|im_end|>
205+
<|im_start|>environment name=<|interpreter|>
206206
![image](xxx.png)
207-
[UNUSED_TOKEN_145]
208-
[UNUSED_TOKEN_146]assistant
209-
Since the code output is not included here, I cannot provide specific chart content. However, if the code executed correctly, it should display a polar plot with two filled areas representing the relationship between wind direction at 9 am and rainfall, and between wind direction at 3 pm and rainfall, respectively. The values for each direction are based on the average rainfall calculated from the provided dataset. The chart should have a clear title, a legend, and be intuitive for comparing rainfall with different wind directions. Given the use of a dark theme, the overall appearance of the chart should be bright lines and filled areas on a dark background.[UNUSED_TOKEN_145]
210-
[UNUSED_TOKEN_146]user
211-
Can you explain the code?[UNUSED_TOKEN_145]
212-
[UNUSED_TOKEN_146]assistant
207+
<|im_end|>
208+
<|im_start|>assistant
209+
Since the code output is not included here, I cannot provide specific chart content. However, if the code executed correctly, it should display a polar plot with two filled areas representing the relationship between wind direction at 9 am and rainfall, and between wind direction at 3 pm and rainfall, respectively. The values for each direction are based on the average rainfall calculated from the provided dataset. The chart should have a clear title, a legend, and be intuitive for comparing rainfall with different wind directions. Given the use of a dark theme, the overall appearance of the chart should be bright lines and filled areas on a dark background.<|im_end|>
210+
<|im_start|>user
211+
Can you explain the code?<|im_end|>
212+
<|im_start|>assistant
213213
The code uses Python's Plotly library to create a polar plot that shows the relationship between wind direction and rainfall. It starts by creating a Plotly Figure object. Then, I added two `Scatterpolar` traces. The first trace shows the relationship between wind direction at 9 am and average rainfall, and the second trace shows the same for 3 pm. In these traces, the `r` parameter represents the average rainfall, and `theta` represents the wind direction. `fill='toself'` means the shape is filled to its own boundary. Next, I updated the chart layout, hid the radial axis, added a legend, set the title and its font, chose a dark theme, and set the chart's background color, width, and height. Finally, the chart was displayed using `fig.show()`.
214-
[UNUSED_TOKEN_145]
215-
[UNUSED_TOKEN_146]user
216-
I want to know today's weather in Shanghai[UNUSED_TOKEN_145]
217-
[UNUSED_TOKEN_146]assistant
218-
Sure, I will search for the weather of Shanghai.[UNUSED_TOKEN_144][UNUSED_TOKEN_141]
219-
{"name": "get_current_weather", "parameters": {"location": "Shanghai"}}[UNUSED_TOKEN_143][UNUSED_TOKEN_145]
220-
[UNUSED_TOKEN_146]environment name=[UNUSED_TOKEN_141]
221-
{"temperature": 22}[UNUSED_TOKEN_145]
222-
[UNUSED_TOKEN_146]assistant
223-
The weather in Shanghai is 22 celsius[UNUSED_TOKEN_145]
214+
<|im_end|>
215+
<|im_start|>user
216+
I want to know today's weather in Shanghai<|im_end|>
217+
<|im_start|>assistant
218+
Sure, I will search for the weather of Shanghai.<|action_start|><|plugin|>
219+
{"name": "get_current_weather", "parameters": {"location": "Shanghai"}}<|action_end|><|im_end|>
220+
<|im_start|>environment name=<|plugin|>
221+
{"temperature": 22}<|im_end|>
222+
<|im_start|>assistant
223+
The weather in Shanghai is 22 celsius<|im_end|>
224224
```

0 commit comments

Comments
 (0)