Skip to content

feat: Add material upload script #1516

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .env.local
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,6 @@ SQL_HOST=localhost
SQL_PORT=3306
SQL_USER=root
SQL_PASSWORD=admin
SQL_DATABASE=tiny_engine
SQL_DATABASE=tiny_engine

backend_url=http://localhost:9090/material-center/api/component/bundle/create
51 changes: 51 additions & 0 deletions docs/api/backend-api/material-center.md
Original file line number Diff line number Diff line change
Expand Up @@ -327,3 +327,54 @@ tenant 通过前端配置统一 x-lowcode-org 请求头传递,接口中不用
</tbody>
</table>

## 物料同步接口

<a id=物料同步接口> </a>

### 基本信息

**Path:** /material-center/api/component/bundle/create

**Method:** POST

**接口描述:**

<p>处理物料同步上传bundle.json文件创建或更新组件</p>


### 请求参数

**Headers**

| 参数名称 | 参数值 | 是否必须 | 示例 | 备注 |
| ------------ | ---------------- | -------- | ---- | ---- |
| Content-Type | multipart/form-data | 是 | | |

**路径参数**

| 参数名称 | 示例 | 备注 |
| -------- | ---- | ------ |
| file | bundle.json | bundle.json文件 |

**Body**

<table>
<thead class="ant-table-thead">
<tr>
<th key=name>名称</th><th key=type>类型</th><th key=required>是否必须</th><th key=default>默认值</th><th key=desc>备注</th><th key=sub>其他信息</th>
</tr>
</thead><tbody className="ant-table-tbody">
</tbody>
</table>


### 返回数据

<table>
<thead class="ant-table-thead">
<tr>
<th key=name>名称</th><th key=type>类型</th><th key=required>是否必须</th><th key=default>默认值</th><th key=desc>备注</th><th key=sub>其他信息</th>
</tr>
</thead><tbody className="ant-table-tbody"><tr key=0-0><td key=0><span style="padding-left: 0px"><span style="color: #8c8a8a"></span> data</span></td><td key=1><span>object</span></td><td key=2>非必须</td><td key=3></td><td key=4><span style="white-space: pre-wrap"></span></td><td key=5></td></tr><tr key=0-0-0><td key=0><span style="padding-left: 20px"><span style="color: #8c8a8a">├─</span> insertNum</span></td><td key=1><span>number</span></td><td key=2>非必须</td><td key=3></td><td key=4><span style="white-space: pre-wrap">新增组件数</span></td><td key=5></td></tr> <tr key=0-0-0><td key=0><span style="padding-left: 20px"><span style="color: #8c8a8a">├─</span> updateNum</span></td><td key=1><span>number</span></td><td key=2>非必须</td><td key=3></td><td key=4><span style="white-space: pre-wrap">更新组件数</span></td><td key=5></td></tr>
</tbody>
</table>
Binary file added docs/solutions/imgs/bundle_upload.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/solutions/imgs/component_create_code.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
23 changes: 19 additions & 4 deletions docs/solutions/material-sync-solution.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,24 @@
## 物料同步方案

### 方案一 (推荐使用)
通过前端脚本上传编辑好的bundle.json文件。
```shell
pnpm uploadMaterials
```

#### 注意事项
1. bundle.json文件内组件版本version字段为必须字段,缺少的需要补充不然上传接口报错
2. bundle.json文件内组件版本version与数据库t_component内version不一致时为新增,一致为更新,也可根据自身需求更改相关代码逻辑
3. .env.local中配置访问后端接口的路径,默认java后端路径

Comment on lines +9 to +13
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Specify the exact env key & give an example

Bullet 3 just says “在 .env.local 中配置访问后端接口的路径” but never names the key. The accompanying code and backend doc call it backend_url, yet that string never appears here.

Add the key name and a minimal example to prevent misconfiguration:

-3. .env.local中配置访问后端接口的路径,默认java后端路径
+3. 在 `.env.local` 中配置后端接口地址,键名为 **BACKEND_URL**(若无则新增)。默认指向 Java 后端:
+
+```dotenv
+BACKEND_URL=https://your-backend.example.com
+```
🤖 Prompt for AI Agents
In docs/solutions/material-sync-solution.md around lines 9 to 13, the third
bullet point mentions configuring the backend API path in .env.local but does
not specify the exact environment variable key. Update this bullet to explicitly
name the key as BACKEND_URL and provide a minimal example line like
BACKEND_URL=https://your-backend.example.com to clarify the configuration and
prevent misconfiguration.

![注意事项图](./imgs/component_create_code.png '注意事项图')

### 方案二
由于当前情况下物料无法通过页面交互维护,提供一个临时方案,由前端托管物料。
在本地运行时提供nodejs脚本维护物料,执行脚本保证mockServer工程和前端工程物料的同步。如果启动了后端服务,还可以连接数据库,保证本地json文件和数据库的一致性。

![物料同步数据流转图](./imgs/synchronize-materials.png '物料同步数据流转图')
### 拆分脚本
#### 拆分脚本

```shell
pnpm splitMaterials
Expand All @@ -13,7 +27,7 @@ pnpm splitMaterials
之前对物料的修改要先从大文件bundle.json中找到对应组件,然后进行修改调试,很容易出现找错组件或难以对比历史的情况。
执行该脚本可以将物料资产包拆分为单个组件或区块,再结合构建的脚本,可以将修改限制在相对较小的文件中,便于维护物料和对比变更。

### 构建脚本
#### 构建脚本

```shell
pnpm buildMaterials
Expand All @@ -26,14 +40,15 @@ pnpm buildMaterials
- 连接数据库,初始化组件表,将组件数据写入数据库
- 监听组件文件变化,重新构建物料资产包和应用 mock 数据,新增或更新数据库中组件数据

### 注意事项
#### 注意事项

1. 本地还没有执行过拆分脚本的,先执行一次,后续可以不用再执行,只需要对单个组件进行维护
2. 单个组件文件需要遵循物料协议,更新文件后保存即可触发重新构建物料资产包,刷新页面后生效
3. 连接数据库失败,不影响构建本地物料资产包即bundle.json
4. 连接数据库需要的配置项在`.env.local`中
5. 数据关联关系可能没有入库,如果查询不到新添加的组件,需要手动在数据库表r_material_component、r_material_history_component添加关联关系数据

### 部署建议
#### 部署建议

需要将设计器进行部署时,关于物料的部署方案,以下供参考:

Expand Down
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@
"setup": "node ./scripts/setup.js",
"splitMaterials": "node ./scripts/splitMaterials.mjs",
"buildMaterials": "node ./scripts/buildMaterials.mjs",
"updateTemplate": "node ./scripts/updateTemplate.mjs"
"updateTemplate": "node ./scripts/updateTemplate.mjs",
"uploadMaterials": "node scripts/uploadMaterials.mjs"
},
"devDependencies": {
"@eslint/js": "^8.57.1",
Expand Down
40 changes: 40 additions & 0 deletions scripts/uploadMaterials.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
import { Buffer } from 'buffer'
import dotenv from 'dotenv'
import fs from 'fs-extra'
import path from 'node:path'
import Logger from './logger.mjs'

const logger = new Logger('uploadMaterials')

// 先构造出.env*文件的绝对路径
const appDirectory = fs.realpathSync(process.cwd())
const resolveApp = (relativePath) => path.resolve(appDirectory, relativePath)
const pathsDotenv = resolveApp('.env')
dotenv.config({ path: `${pathsDotenv}.local` })
const { backend_url } = process.env

const bundlePath = path.join(process.cwd(), '/designer-demo/public/mock/bundle.json')
const bundle = fs.readJSONSync(bundlePath)
Comment on lines +16 to +17
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Make the bundle path configurable and add validation.

The hardcoded path to bundle.json reduces flexibility and could break if the project structure changes. Additionally, there's no validation to ensure the file exists or is readable.

Apply this diff to make the path configurable and add validation:

-const bundlePath = path.join(process.cwd(), '/designer-demo/public/mock/bundle.json')
-const bundle = fs.readJSONSync(bundlePath)
+const bundlePath = process.env.BUNDLE_PATH || path.join(process.cwd(), 'designer-demo/public/mock/bundle.json')
+
+if (!fs.existsSync(bundlePath)) {
+  logger.error(`Bundle file not found at: ${bundlePath}`)
+  process.exit(1)
+}
+
+let bundle
+try {
+  bundle = fs.readJSONSync(bundlePath)
+} catch (error) {
+  logger.error(`Failed to read bundle file: ${error.message}`)
+  process.exit(1)
+}

Also add BUNDLE_PATH to .env.local if you want to make it configurable.

Committable suggestion skipped: line range outside the PR's diff.

🤖 Prompt for AI Agents
In scripts/uploadMaterials.mjs around lines 16 to 17, the bundle.json path is
hardcoded, reducing flexibility and lacking validation. Modify the code to read
the bundle path from an environment variable (e.g., BUNDLE_PATH) with a fallback
default, then add checks to verify the file exists and is readable before
attempting to read it. This ensures configurability and prevents runtime errors
due to missing or inaccessible files.

const jsonBuffer = Buffer.from(JSON.stringify(bundle))
const boundary = '----WebKitFormBoundary7MA4YWxkTrZu0gW'
const formHeaders = {
'Content-Type': `multipart/form-data; boundary=${boundary}`,
}

let body = `--${boundary}\r\n`
body += 'Content-Disposition: form-data; name="file"; filename="bundle.json"\r\n'
body += 'Content-Type: application/json\r\n\r\n'
body += jsonBuffer.toString() + `\r\n--${boundary}--`
Comment on lines +18 to +27
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Use FormData API instead of manual multipart construction.

Manually constructing multipart/form-data is error-prone and hard to maintain. The current implementation uses a fixed boundary string which could potentially cause conflicts.

Apply this diff to use the built-in FormData API:

-const jsonBuffer = Buffer.from(JSON.stringify(bundle))
-const boundary = '----WebKitFormBoundary7MA4YWxkTrZu0gW'
-const formHeaders = {
-  'Content-Type': `multipart/form-data; boundary=${boundary}`,
-}
-
-let body = `--${boundary}\r\n`
-body += 'Content-Disposition: form-data; name="file"; filename="bundle.json"\r\n'
-body += 'Content-Type: application/json\r\n\r\n'
-body += jsonBuffer.toString() + `\r\n--${boundary}--`
+const formData = new FormData()
+const blob = new Blob([JSON.stringify(bundle)], { type: 'application/json' })
+formData.append('file', blob, 'bundle.json')

And update the fetch call:

-fetch(backend_url, {
+fetch(BACKEND_URL, {
   method: 'POST',
-  headers: formHeaders,
-  body: body,
+  body: formData,
 })
🤖 Prompt for AI Agents
In scripts/uploadMaterials.mjs around lines 18 to 27, the multipart/form-data
body is manually constructed with a fixed boundary string, which is error-prone
and hard to maintain. Replace this manual construction by using the built-in
FormData API to create the form data and append the JSON file. Then update the
fetch call to use formData as the body and set headers from
formData.getHeaders() or equivalent to ensure correct content-type with
boundary.


fetch(backend_url, {
method: 'POST',
headers: formHeaders,
body: body,
})
.then(response => response.json())
.then(data => {
logger.success('File uploaded successfully:', data)
})
.catch(error => {
logger.error('Error uploading file:', error)
})
Comment on lines +29 to +40
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Improve error handling and add input validation.

The current error handling is basic and doesn't provide proper exit codes for script automation. Additionally, there's no validation of the backend URL.

Apply this diff to improve error handling:

+// Validate backend URL
+if (!BACKEND_URL) {
+  logger.error('BACKEND_URL environment variable is not set')
+  process.exit(1)
+}
+
+try {
+  new URL(BACKEND_URL)
+} catch (error) {
+  logger.error(`Invalid backend URL: ${BACKEND_URL}`)
+  process.exit(1)
+}
+
-fetch(backend_url, {
+fetch(BACKEND_URL, {
   method: 'POST',
   body: formData,
 })
-  .then(response => response.json())
+  .then(async response => {
+    if (!response.ok) {
+      throw new Error(`HTTP error! status: ${response.status}`)
+    }
+    return response.json()
+  })
   .then(data => {
     logger.success('File uploaded successfully:', data)
+    process.exit(0)
   })
   .catch(error => {
     logger.error('Error uploading file:', error)
+    process.exit(1)
   })

Committable suggestion skipped: line range outside the PR's diff.

🤖 Prompt for AI Agents
In scripts/uploadMaterials.mjs around lines 29 to 40, improve error handling by
validating the backend_url before making the fetch call to ensure it is a valid
URL. Modify the catch block to log the error and exit the process with a
non-zero exit code to support script automation. This will make the script more
robust and provide clear failure signals.