Skip to content

Commit

Permalink
[Enchance] update chat format (InternLM#634)
Browse files Browse the repository at this point in the history
  • Loading branch information
@Harold-lkk
Harold-lkk authored Jan 19, 2024
1 parent 4281caf commit abbf036
Show file tree
Hide file tree
Showing 3 changed files with 170 additions and 170 deletions.
166 changes: 83 additions & 83 deletions chat/chat_format.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,24 +9,24 @@ InternLM2-Chat adopts a new chat format to flexibly support a wider range of app
The regular chat structure usually contains three roles: `system`, `user`, and `assistant`, formatted as follows for multi-turn dialogues:

```
[UNUSED_TOKEN_146]system
You are InternLM2-Chat, a harmless AI assistant[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]user
Hello[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]assistant
Hello, I am InternLM2-Chat, how can I assist you?[UNUSED_TOKEN_145]
<|im_start|>system
You are InternLM2-Chat, a harmless AI assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
Hello, I am InternLM2-Chat, how can I assist you?<|im_end|>
```

Here, `[UNUSED_TOKEN_146]` acts as the start token for each turn of dialogue, and `[UNUSED_TOKEN_145]` as the end token. Each turn of dialogue typically starts with `[UNUSED_TOKEN_146]role` and ends with the model's output `[UNUSED_TOKEN_145]`, where role represents `system`, `user`, `assistant`, and `environment`. You may refer to the [code in huggingface](https://huggingface.co/internlm/internlm2-chat-7b/blob/main/modeling_internlm2.py#L1138) to see how the chat history is organized.
Here, `<|im_start|>` acts as the start token for each turn of dialogue, and `<|im_end|>` as the end token. Each turn of dialogue typically starts with `<|im_start|>role` and ends with the model's output `<|im_end|>`, where role represents `system`, `user`, `assistant`, and `environment`. You may refer to the [code in huggingface](https://huggingface.co/internlm/internlm2-chat-7b/blob/main/modeling_internlm2.py#L1138) to see how the chat history is organized.

Currently, the InternLM2-Chat model's vocabulary maintains the following mappings to support full functionalities:

- `[UNUSED_TOKEN_146]`: Start token for each role's dialogue, the token ID is **92543**
- `[UNUSED_TOKEN_145]`: End token for each role's dialogue, the token ID is **92542**
- `[UNUSED_TOKEN_144]`: Start token for invoking external plugins, the token ID is **92541**
- `[UNUSED_TOKEN_143]`: End token for invoking external plugins, the token ID is **92540**
- `[UNUSED_TOKEN_142]`: Code interpreter, the token ID is **92539**
- `[UNUSED_TOKEN_141]`: External plugins, regular tools, the token ID is **92538**
- `<|im_start|>`: Start token for each role's dialogue, the token ID is **92543**
- `<|im_end|>`: End token for each role's dialogue, the token ID is **92542**
- `<|action_start|>`: Start token for invoking external tools, like interpreter or plugin, the token ID is **92541**
- `<|action_end|>`: End token for invoking external plugins, the token ID is **92540**
- `<|interpreter|>`: Code interpreter, the token ID is **92539**
- `<|plugin|>`: External plugins, regular tools, the token ID is **92538**

## Complete Structure

Expand All @@ -37,9 +37,9 @@ The complete chat format of InternLM2-Chat, based on the basic structure, also i
InternLM2-Chat support multiple formats (e.g., ReAct) to conduct function call, especially **json format** to ease downstream applications。An example of complete function call is shown below.

```
[UNUSED_TOKEN_146]system
You are InternLM2-Chat, a harmless AI assistant[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]system name=[UNUSED_TOKEN_141]
<|im_start|>system
You are InternLM2-Chat, a harmless AI assistant<|im_end|>
<|im_start|>system name=<|plugin|>
[
{
"name": "get_current_weather",
Expand All @@ -57,46 +57,46 @@ You are InternLM2-Chat, a harmless AI assistant[UNUSED_TOKEN_145]
},
}
]
[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]user
I want to know today's weather in Shanghai[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]assistant
Sure, I will search for the weather of Shanghai.[UNUSED_TOKEN_144][UNUSED_TOKEN_141]
{"name": "get_current_weather", "parameters": {"location": "Shanghai"}}[UNUSED_TOKEN_143][UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]environment name=[UNUSED_TOKEN_141]
{"temperature": 22}[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]assistant
The weather in Shanghai is 22 celsius[UNUSED_TOKEN_145]
<|im_end|>
<|im_start|>user
I want to know today's weather in Shanghai<|im_end|>
<|im_start|>assistant
Sure, I will search for the weather of Shanghai.<|action_start|><|plugin|>
{"name": "get_current_weather", "parameters": {"location": "Shanghai"}}<|action_end|><|im_end|>
<|im_start|>environment name=<|plugin|>
{"temperature": 22}<|im_end|>
<|im_start|>assistant
The weather in Shanghai is 22 celsius<|im_end|>
```

1. First，there will be a new system prompt that describe the protocol of tools in json format. The content starts with `[UNUSED_TOKEN_146]system name=[UNUSED_TOKEN_141]\n` and ends with `[UNUSED_TOKEN_145]`. `name=[UNUSED_TOKEN_141]` indicates the system prompt comes from tools. InternLM2-Chat supports and follows multiple system prompts in the chat history, so we can also see the system prompt ask the model to be helpful assistant.
2. Model will call the tools in a streaming format after receving the user prompt, i.e., it will naturally speak something (thoughts, helpful response) then call the tools. The model will output `[UNUSED_TOKEN_144][UNUSED_TOKEN_141]`, where `[UNUSED_TOKEN_144]` indicates the model needs to call extensions and `[UNUSED_TOKEN_141]` indicates it want to use common tools. Then the model will output the calling parameters in json format and end it with `[UNUSED_TOKEN_143]`.
3. The environment returns the function calling outputs, starts with `[UNUSED_TOKEN_146]environment name=[UNUSED_TOKEN_141]`, indicating it comes from the function calling in the environment. The content is in json format, `{"temperature": 22}`, ends with `[UNUSED_TOKEN_145]`.
1. First，there will be a new system prompt that describe the protocol of tools in json format. The content starts with `<|im_start|>system name=<|plugin|>\n` and ends with `<|im_end|>`. `name=<|plugin|>` indicates the system prompt comes from tools. InternLM2-Chat supports and follows multiple system prompts in the chat history, so we can also see the system prompt ask the model to be helpful assistant.
2. Model will call the tools in a streaming format after receving the user prompt, i.e., it will naturally speak something (thoughts, helpful response) then call the tools. The model will output `<|action_start|><|plugin|>`, where `<|action_start|>` indicates the model needs to call extensions and `<|plugin|>` indicates it want to use common tools. Then the model will output the calling parameters in json format and end it with `<|action_end|>`.
3. The environment returns the function calling outputs, starts with `<|im_start|>environment name=<|plugin|>`, indicating it comes from the function calling in the environment. The content is in json format, `{"temperature": 22}`, ends with `<|im_end|>`.

### Code Interpreter

InternLM2-Chat support to use code interpreter to solve complex math problems and data analysis. The whole process is similar to the function calling, shown as below.

```
[UNUSED_TOKEN_146]system
You are InternLM2-Chat, a harmless AI assistant[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]system name=[UNUSED_TOKEN_142]
<|im_start|>system
You are InternLM2-Chat, a harmless AI assistant<|im_end|>
<|im_start|>system name=<|interpreter|>
You now have access to a Jupyter notebook environment supporting Python code execution. Just send code to python to run in this stateful environment. This feature is suitable for:
- Data analysis or processing (such as data manipulation and graphic creation)
- Complex calculations (such as math and physics problems)
- Programming examples (for understanding programming concepts or language features)
- Text processing and analysis (including text analysis and natural language processing)
- Machine learning and data science (model training and data visualization)
- File operations and data import (handling CSV, JSON, etc. formats)[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]user
- File operations and data import (handling CSV, JSON, etc. formats)<|im_end|>
<|im_start|>user
Please help me process and visualize this dataset.
[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]user name=file
[{"path": "data.csv", size='10K'}][UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]assistant
<|im_end|>
<|im_start|>user name=file
[{"path": "data.csv", size='10K'}]<|im_end|>
<|im_start|>assistant
I have processed the data and visualized it for you.
[UNUSED_TOKEN_144][UNUSED_TOKEN_142]
<|action_start|><|interpreter|>
```python
import plotly.express as px
import pandas as pd
Expand All @@ -118,40 +118,40 @@ fig.update_traces(hovertemplate='<b>Date: %{text}</b><br>Wind Direction 9am: %{x
# Show the plot
fig.show()
```[UNUSED_TOKEN_143]
[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]environment name=[UNUSED_TOKEN_142]
```<|action_end|>
<|im_end|>
<|im_start|>environment name=<|interpreter|>
![image](xxx.png)
[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]assistant
Since the code output is not included here, I cannot provide specific chart content. However, if the code executed correctly, it should display a polar plot with two filled areas representing the relationship between wind direction at 9 am and rainfall, and between wind direction at 3 pm and rainfall, respectively. The values for each direction are based on the average rainfall calculated from the provided dataset. The chart should have a clear title, a legend, and be intuitive for comparing rainfall with different wind directions. Given the use of a dark theme, the overall appearance of the chart should be bright lines and filled areas on a dark background.[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]user
Can you explain the code?[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]assistant
<|im_end|>
<|im_start|>assistant
Since the code output is not included here, I cannot provide specific chart content. However, if the code executed correctly, it should display a polar plot with two filled areas representing the relationship between wind direction at 9 am and rainfall, and between wind direction at 3 pm and rainfall, respectively. The values for each direction are based on the average rainfall calculated from the provided dataset. The chart should have a clear title, a legend, and be intuitive for comparing rainfall with different wind directions. Given the use of a dark theme, the overall appearance of the chart should be bright lines and filled areas on a dark background.<|im_end|>
<|im_start|>user
Can you explain the code?<|im_end|>
<|im_start|>assistant
The code uses Python's Plotly library to create a polar plot that shows the relationship between wind direction and rainfall. It starts by creating a Plotly Figure object. Then, I added two `Scatterpolar` traces. The first trace shows the relationship between wind direction at 9 am and average rainfall, and the second trace shows the same for 3 pm. In these traces, the `r` parameter represents the average rainfall, and `theta` represents the wind direction. `fill='toself'` means the shape is filled to its own boundary. Next, I updated the chart layout, hid the radial axis, added a legend, set the title and its font, chose a dark theme, and set the chart's background color, width, and height. Finally, the chart was displayed using `fig.show()`.
[UNUSED_TOKEN_145]
<|im_end|>
```

1. First，there will be a new system prompt that describe the usages of code interpreter. The content starts with `[UNUSED_TOKEN_146]system name=[UNUSED_TOKEN_142]\n` and ends with `[UNUSED_TOKEN_145]`. `name=[UNUSED_TOKEN_142]` indicates the system prompt comes from code interpreter. InternLM2-Chat supports and follows multiple system prompts in the chat history, so we can also see the system prompt ask the model to be helpful assistant.
2. Model will call the tools in a streaming format after receving the user prompt, i.e., it will naturally speak something (thoughts, helpful response) then call the code interpreter. The model will output `[UNUSED_TOKEN_144][UNUSED_TOKEN_142]`, where `[UNUSED_TOKEN_144]` indicates the model needs to call extensions and `[UNUSED_TOKEN_142]` indicates it want to use code interpreter. Then the model will output the code in a markdown-style code block and end it with `[UNUSED_TOKEN_143]`.
3. The environment will execute the code and return outputs, starts with `[UNUSED_TOKEN_146]environment name=[UNUSED_TOKEN_142]`, indicating it comes from the code interpreter in the environment, and the content ends with `[UNUSED_TOKEN_145]`.
1. First，there will be a new system prompt that describe the usages of code interpreter. The content starts with `<|im_start|>system name=<|interpreter|>\n` and ends with `<|im_end|>`. `name=<|interpreter|>` indicates the system prompt comes from code interpreter. InternLM2-Chat supports and follows multiple system prompts in the chat history, so we can also see the system prompt ask the model to be helpful assistant.
2. Model will call the tools in a streaming format after receving the user prompt, i.e., it will naturally speak something (thoughts, helpful response) then call the code interpreter. The model will output `<|action_start|><|interpreter|>`, where `<|action_start|>` indicates the model needs to call extensions and `<|interpreter|>` indicates it want to use code interpreter. Then the model will output the code in a markdown-style code block and end it with `<|action_end|>`.
3. The environment will execute the code and return outputs, starts with `<|im_start|>environment name=<|interpreter|>`, indicating it comes from the code interpreter in the environment, and the content ends with `<|im_end|>`.

### Function Call + Code Interpreter

InternLM2-Chat can choose to use tools or code interpreter automatically. We only need to send the system prompt of both to the model respectively. An example is shown as below

```
[UNUSED_TOKEN_146]system
You are InternLM2-Chat, a harmless AI assistant[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]system name=[UNUSED_TOKEN_142]
<|im_start|>system
You are InternLM2-Chat, a harmless AI assistant<|im_end|>
<|im_start|>system name=<|interpreter|>
You now have access to a Jupyter notebook environment supporting Python code execution. Just send code to python to run in this stateful environment. This feature is suitable for:
- Data analysis or processing (such as data manipulation and graphic creation)
- Complex calculations (such as math and physics problems)
- Programming examples (for understanding programming concepts or language features)
- Text processing and analysis (including text analysis and natural language processing)
- Machine learning and data science (model training and data visualization)
- File operations and data import (handling CSV, JSON, etc. formats)[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]system name=[UNUSED_TOKEN_141]
- File operations and data import (handling CSV, JSON, etc. formats)<|im_end|>
<|im_start|>system name=<|plugin|>
[
{
"name": "get_current_weather",
Expand All @@ -169,16 +169,16 @@ You now have access to a Jupyter notebook environment supporting Python code exe
},
}
]
[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]user
<|im_end|>
<|im_start|>user
Please help me process and visualize this dataset.
[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]user name=file
[{"path": "data.csv", size='10K'}][UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]assistant
<|im_end|>
<|im_start|>user name=file
[{"path": "data.csv", size='10K'}]<|im_end|>
<|im_start|>assistant
I have processed the data and visualized it for you.
[UNUSED_TOKEN_144][UNUSED_TOKEN_142]
<|action_start|><|interpreter|>
```python
import plotly.express as px
import pandas as pd
Expand All @@ -200,25 +200,25 @@ fig.update_traces(hovertemplate='<b>Date: %{text}</b><br>Wind Direction 9am: %{x
# Show the plot
fig.show()
```[UNUSED_TOKEN_143]
[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]environment name=[UNUSED_TOKEN_142]
```<|action_end|>
<|im_end|>
<|im_start|>environment name=<|interpreter|>
![image](xxx.png)
[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]assistant
Since the code output is not included here, I cannot provide specific chart content. However, if the code executed correctly, it should display a polar plot with two filled areas representing the relationship between wind direction at 9 am and rainfall, and between wind direction at 3 pm and rainfall, respectively. The values for each direction are based on the average rainfall calculated from the provided dataset. The chart should have a clear title, a legend, and be intuitive for comparing rainfall with different wind directions. Given the use of a dark theme, the overall appearance of the chart should be bright lines and filled areas on a dark background.[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]user
Can you explain the code?[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]assistant
<|im_end|>
<|im_start|>assistant
Since the code output is not included here, I cannot provide specific chart content. However, if the code executed correctly, it should display a polar plot with two filled areas representing the relationship between wind direction at 9 am and rainfall, and between wind direction at 3 pm and rainfall, respectively. The values for each direction are based on the average rainfall calculated from the provided dataset. The chart should have a clear title, a legend, and be intuitive for comparing rainfall with different wind directions. Given the use of a dark theme, the overall appearance of the chart should be bright lines and filled areas on a dark background.<|im_end|>
<|im_start|>user
Can you explain the code?<|im_end|>
<|im_start|>assistant
The code uses Python's Plotly library to create a polar plot that shows the relationship between wind direction and rainfall. It starts by creating a Plotly Figure object. Then, I added two `Scatterpolar` traces. The first trace shows the relationship between wind direction at 9 am and average rainfall, and the second trace shows the same for 3 pm. In these traces, the `r` parameter represents the average rainfall, and `theta` represents the wind direction. `fill='toself'` means the shape is filled to its own boundary. Next, I updated the chart layout, hid the radial axis, added a legend, set the title and its font, chose a dark theme, and set the chart's background color, width, and height. Finally, the chart was displayed using `fig.show()`.
[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]user
I want to know today's weather in Shanghai[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]assistant
Sure, I will search for the weather of Shanghai.[UNUSED_TOKEN_144][UNUSED_TOKEN_141]
{"name": "get_current_weather", "parameters": {"location": "Shanghai"}}[UNUSED_TOKEN_143][UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]environment name=[UNUSED_TOKEN_141]
{"temperature": 22}[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]assistant
The weather in Shanghai is 22 celsius[UNUSED_TOKEN_145]
<|im_end|>
<|im_start|>user
I want to know today's weather in Shanghai<|im_end|>
<|im_start|>assistant
Sure, I will search for the weather of Shanghai.<|action_start|><|plugin|>
{"name": "get_current_weather", "parameters": {"location": "Shanghai"}}<|action_end|><|im_end|>
<|im_start|>environment name=<|plugin|>
{"temperature": 22}<|im_end|>
<|im_start|>assistant
The weather in Shanghai is 22 celsius<|im_end|>
```
Loading

0 comments on commit abbf036

Please sign in to comment.