Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: 分块上传优化 #2813 #2816

Open
wants to merge 31 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
8642550
feat: 分块上传优化 #2813
zzdjx Dec 3, 2024
5cbe04d
fix: 分块上传优化-问题修复 #2813
zzdjx Dec 3, 2024
613b9f8
Merge remote-tracking branch 'upstream/master' into dev_kurt
zzdjx Dec 6, 2024
a80c07a
fix: 分块上传优化-代码调整+单测 #2813
zzdjx Dec 6, 2024
d65bc84
fix: 分块上传优化-代码调整 #2813
zzdjx Dec 6, 2024
8009fcf
Merge remote-tracking branch 'upstream/master' into dev_kurt
zzdjx Dec 10, 2024
c21d488
fix: NodeAttribute抽离公共放到metadata #2813
zzdjx Dec 10, 2024
318f65a
fix: 新增Generic下Block事件监听 #2813
zzdjx Dec 11, 2024
ba35f7f
fix: 代码规范 #2813
zzdjx Dec 11, 2024
89a7905
fix: 错误修复 #2813
zzdjx Dec 11, 2024
b71c81a
feat: 逻辑重构 #2813
zzdjx Dec 19, 2024
be06fec
Merge remote-tracking branch 'upstream/master' into issue_2813
zzdjx Dec 19, 2024
8e0f621
fix: 代码规范 #2813
zzdjx Dec 19, 2024
11f5525
fix: 代码格式及部分逻辑优化 #2813
zzdjx Dec 25, 2024
b940e8c
fix: 修复isSeparateUpload uploadId null异常 #2813
zzdjx Dec 26, 2024
1ca4460
feat: 分块上传并发场景下优化 #2813
zzdjx Jan 6, 2025
2804cbb
fix: 代码优化 #2813
zzdjx Jan 6, 2025
020d69b
fix: test方法 #2813
zzdjx Jan 6, 2025
71ac504
fix: 修复问题 #2813
zzdjx Jan 6, 2025
b7c980d
fix: 修复问题 #2813
zzdjx Jan 6, 2025
3ae6dac
fix: 文档更新 #2813
zzdjx Jan 6, 2025
f33e49c
fix: 增加部分log #2813
zzdjx Jan 6, 2025
637fac2
fix: 调试日志 #2813
zzdjx Jan 6, 2025
5cbba61
fix: 调试日志 #2813
zzdjx Jan 6, 2025
6469506
fix: 问题修复 #2813
zzdjx Jan 6, 2025
28ba5d7
fix: 问题修复 #2813
zzdjx Jan 7, 2025
7b53435
feat: blocks 新增过期逻辑 #2813
zzdjx Jan 9, 2025
c3fb6fa
feat: 过期时间改为配置项 #2813
zzdjx Jan 9, 2025
dbd0fe9
fix: 优化一下细节问题 #2813
zzdjx Jan 13, 2025
ce19623
fix: 优化一下细节问题 #2813
zzdjx Jan 13, 2025
e4735c0
fix: 优化一下细节问题 #2813
zzdjx Jan 13, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
205 changes: 205 additions & 0 deletions docs/apidoc-user/generic/SeparateBlock.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,205 @@
# Generic通用制品仓库分块文件操作

[toc]

## 初始化分块上传

- API: POST /generic/separate/block/{project}/{repo}/{path}
- API 名称: start_block_upload
- 功能说明:
- 中文:初始化分块上传
- English:start block upload

- 请求体
此接口请求体为空

- 请求字段说明

|字段|类型|是否必须|默认值|说明|Description|
|---|---|---|---|---|---|
|project|string|是|无|项目名称|project name|
|repo|string|是|无|仓库名称|repo name|
|path|string|是|无|完整路径|full path|

- 请求头

|字段|类型|是否必须|默认值|说明|Description|
|---|---|---|---|---|---|
- |X-BKREPO-SIZE|string|是|0L|文件大小|上传文件的总大小|
|X-BKREPO-MD5|string|否|无|文件md5|file md5|
|X-BKREPO-OVERWRITE|boolean|否|false|是否覆盖已存在文件|overwrite exist file|

- 响应体

``` json
{
"code" : 0,
"message" : null,
"data" : null,
"traceId" : null
}
```

## 上传分块文件

- API: PUT /generic/separate/{project}/{repo}/{path}
- API 名称: block_upload
- 功能说明:
- 中文:分块上传通用制品文件
- English:upload generic artifact file block

- 请求体
[文件流]

- 请求字段说明

|字段|类型|是否必须|默认值|说明|Description|
|---|---|---|---|---|---|
|project|string|是|无|项目名称|project name|
|repo|string|是|无|仓库名称|repo name|
|path|string|是|无|完整路径|full path|

- 请求头

|字段|类型| 是否必须 |默认值| 说明 | Description |
|---|---|-|---|------|---------|
|X-BKREPO-SEQUENCE|int|是|无| 分块序号(从1开始), SEQUENCE 和 OFFSET 二者不可同时为空 |block sequence(start from 1) |
- |X-BKREPO-OFFSET|int|是|无| 分块偏移量,SEQUENCE 和 OFFSET 二者不可同时为空| block offset(start from 0)|
|X-BKREPO-SHA256|string|否|无| 文件sha256|file sha256|
|X-BKREPO-MD5|string|否|无| 文件md5| file md5|

- 响应体

``` json
{
"code" : 0,
"message" : null,
"data" : null,
"traceId" : null
}
```

## 完成分块上传

- API: PUT /generic/separate/block/{project}/{repo}/{path}
- API 名称: complete_block_upload
- 功能说明:
- 中文:完成化分块上传
- English:complete block upload

- 请求体
此接口请求体为空

- 请求字段说明

|字段|类型|是否必须|默认值|说明|Description|
|---|---|---|---|---|---|
|project|string|是|无|项目名称|project name|
|repo|string|是|无|仓库名称|repo name|
|path|string|是|无|完整路径|full path|

- 请求头

- 响应体

``` json
{
"code" : 0,
"message" : null,
"data" : null,
"traceId" : ""
}
```

## 终止(取消)分块上传

- API: DELETE /generic/separate/block/{project}/{repo}/{path}
- API 名称: abort_block_upload
- 功能说明:
- 中文:终止(取消)分块上传
- English:abort block upload

- 请求体
此接口请求体为空

- 请求字段说明

|字段|类型|是否必须|默认值|说明|Description|
|---|---|---|---|---|---|
|project|string|是|无|项目名称|project name|
|repo|string|是|无|仓库名称|repo name|
|path|string|是|无|完整路径|full path|

- 请求头

- 响应体

``` json
{
"code" : 0,
"message" : null,
"data" : null,
"traceId" : null
}
```

## 查询已上传的分块列表

- API: GET /generic/separate/block/{project}/{repo}/{path}
- API 名称: list_upload_block
- 功能说明:
- 中文:查询已上传的分块列表
- English:list upload block

- 请求体
此接口请求体为空

- 请求字段说明

|字段|类型|是否必须|默认值|说明|Description|
|---|---|---|---|---|---|
|project|string|是|无|项目名称|project name|
|repo|string|是|无|仓库名称|repo name|
|path|string|是|无|完整路径|full path|

- 请求头

|字段|类型|是否必须|默认值|说明|Description|
|---|---|---|---|---|---|
|Authorization|string|否|无|Basic Auth认证头,Basic base64(username:password)|Basic Auth header|

- 响应体

``` json
{
"code" : 0,
"message" : null,
"data" : [ {
"size" : 10240,
"sha256" : "d17f25ecfbcc7857f7bebea469308be0b2580943e96d13a3ad98a13675c4bfc2",
"startPos" : xxx
}, {
"size" : 10240,
"sha256" : "cc399d73903f06ee694032ab0538f05634ff7e1ce5e8e50ac330a871484f34cf",
"startPos" : xxx
} ],
"traceId" : null
}
```

- 响应字段说明

|字段|类型|说明|Description|
|---|---|---|---|
|code|bool|错误编码。 0表示success,>0表示失败错误|0:success, other: failure|
|message|string|错误消息|the failure message|
|data|list|分块列表|block list|
|traceId|string|请求跟踪id|trace id|

- 分块信息字段说明

|字段| 类型 | 说明 |Description|
|---|-----|----|---|
|size| long |分块大小|block size|
|sha256| string| 分块sha256|block sha256 checksum|
|startPos| long| 分块起始位置|block sequence|
Loading
Loading