update readme

Author: Yuanbo Li

README.md

@@ -5,7 +5,7 @@
 
 - 前端界面
 
-  ![console](./console.png)
+  ![console](./readme_1.png)
 
 - 部署方式
   - 参考 [workshop](https://catalog.us-east-1.prod.workshops.aws/workshops/158a2497-7cbe-4ba4-8bee-2307cb01c08a/en-US)
@@ -62,65 +62,6 @@
 
 - 知识库构建
 
-  + 构建Opensearch Index
-    其中**doc_type**可以为以下四个值**['Question','Paragraph','Sentence','Abstract']**
-    注意："dimension": 768 这个参数需要根据实际使用的向量模型输出纬度进行修改  
-    ```shell
-    PUT chatbot-index
-    {
-        "settings" : {
-            "index":{
-                "number_of_shards" : 1,
-                "number_of_replicas" : 0,
-                "knn": "true",
-                "knn.algo_param.ef_search": 32
-            }
-        },
-        "mappings": {
-            "properties": {
-                "publish_date" : {
-                    "type": "date",
-                    "format": "yyyy-MM-dd HH:mm:ss"
-                },
-                "idx" : {
-                    "type": "integer"
-                },
-                "doc_type" : {
-                    "type" : "keyword"
-                },
-                "doc": {
-                    "type": "text",
-                    "analyzer": "ik_max_word",
-                    "search_analyzer": "ik_smart"
-                },
-                "content": {
-                    "type": "text",
-                    "analyzer": "ik_max_word",
-                    "search_analyzer": "ik_smart"
-                },
-                "doc_title": {
-                    "type": "keyword"
-                },
-                "doc_author": {
-                    "type": "keyword"
-                },
-                "doc_category": {
-                    "type": "keyword"
-                },
-                "embedding": {
-                    "type": "knn_vector",
-                    "dimension": 768,
-                    "method": {
-                        "name": "hnsw",
-                        "space_type": "cosinesimil",
-                        "engine": "nmslib",
-                        "parameters": {
-                            "ef_construction": 512,
-                            "m": 32
-                        }
-                    }            
-                }
-            }
-        }
-    }
-    ```
+  参考[README.md](https://github.com/aws-samples/private-llm-qa-bot/blob/main/code/offline_process/aos_schema.md)构建知识库, 构建了知识库以后才能导入知识文件。构建完毕后，可以从前端页面导入知识。导入成功以后，能够在文档库中找到对应的知识文件
+  
+  ![console](./readme_2.png)

doc_preprocess/README.md

@@ -1,3 +1,46 @@
+
+
+### 知识 - 标准中间格式
+在知识构造的过程中，所有原始格式的文档，都需要考虑转换到一种统一的知识格式，方便后续进行知识的注入和召回优化。
+
+```json
+# schema
+{
+    "page_content":"{will include original document string and sliced document string}",
+    "metadata":{
+        "content_type":"paragraph", # paragraph/table/QA
+        "heading_hierarchy":{
+        },
+        "figure_list":[
+        ],
+        "chunk_id":"",
+        "file_path":"",
+        "keywords":[
+        ],
+        "summary":""
+    }
+}
+
+# FAQ 例子
+{
+    "page_content":"{Question}=>{Answer}",
+    "metadata":{
+        "content_type":"QA",
+        "heading_hierarchy":{
+        },
+        "figure_list":[
+        ],
+        "chunk_id":"",
+        "file_path":"",
+        "keywords":[
+        ],
+        "summary":""
+    }
+}
+```
+
+
+
 ### 从长文档提取FAQ
 
 - 适应场景
@@ -51,7 +94,7 @@
   + ./common-stock-fs.pdf
 
 - Usage
-    
+  
     输出为输入的同名目录
 
     ```shell