上传大文件
在部署的上传限制(默认 200 MB)以内,文件可以通过 POST /v1/cards 的单个请求上传。文件更大,或者网络条件下长请求容易中断时,请分段上传文件:
- 创建上传:告诉 Sursa 文件的名称、类型和大小。
- 发送各个分段,顺序不限,也可以并行发送。失败的分段单独重新发送。
- 用整个文件的 SHA-256 完成上传。Sursa 组装文件并进行检查。
- 用上传 ID 代替文件:用于卡片、压缩包或自定义数据源的文档。
| 限制项 | 值 |
|---|---|
| 文件大小 | 最大 5 GB |
| 分段大小 | 由 Sursa 在 part_size 中选定,5 MiB–100 MiB;最后一个分段可以更小 |
| 分段数量 | 最多 1000 个 |
| 完成上传的时限 | 创建上传后 24 小时 |
| 使用已完成上传的时限 | 24 小时 |
1. 创建上传
POST /v1/uploads 接受文件的名称、类型、大小及其用途:
FILE=training-video-transcripts.zip
SIZE=$(wc -c < "$FILE" | tr -d ' ')
curl -X POST https://api.sursa.com/v1/uploads \
-H "Authorization: Bearer $SURSA_API_KEY" \
-H "Idempotency-Key: $(uuidgen)" \
-H "Content-Type: application/json" \
-d "{\"filename\": \"$FILE\", \"content_type\": \"application/zip\", \"size\": $SIZE, \"purpose\": \"archive\"}"{
"id": "upl_5d3c2b1a0f9e8d7c6b5a493827161504",
"status": "pending",
"part_size": 8388608,
"part_count": 23,
"expires_at": "2026-10-25T10:00:00Z"
}
purpose | 上传的用途 |
|---|---|
card_file | 卡片:POST /v1/cards。 |
archive | 文档压缩包:POST /v1/cards/archive。 |
import_attachment | 自定义数据源中某个文档的文件,放在 upload_id 或 attachments 中。 |
密钥需要具有上传去向所对应的权限范围:card_file 和 archive 需要 cards:write,import_attachment 需要 import:write。
Sursa 选择 part_size,并告诉您要发送多少个分段。如果事先知道文件的 SHA-256,也可以在这里以 sha256 传入。
2. 发送分段
把文件切成 part_count 段,每段 part_size 字节(最后一段是剩余部分),每段用 PUT /v1/uploads/{upload_id}/parts/{n} 发送。分段从 1 开始编号。请求体是该分段的原始字节。
UPLOAD=upl_5d3c2b1a0f9e8d7c6b5a493827161504
split -b 8388608 -a 4 "$FILE" part-
n=1
for part in part-*; do
curl -X PUT "https://api.sursa.com/v1/uploads/$UPLOAD/parts/$n" \
-H "Authorization: Bearer $SURSA_API_KEY" \
-H "Content-Type: application/octet-stream" \
--data-binary "@$part" --retry 5 --fail
n=$((n + 1))
done再次发送某个分段会替换它,因此失败或超时的分段只需重新发送。通常最多并行发送 4 个分段,就能占满高速连接的带宽。
检查分段
每个分段都经由 API 发送:没有直接通往存储的链接。分段的响应带有 ETag 头,其值是加了引号的该分段的 SHA-256,JSON 正文的 etag 中也是同一个值。GET /v1/uploads/{upload_id} 在 parts 中列出目前已收到的分段,因此中断后只需发送缺失的分段。
3. 完成上传
POST /v1/uploads/{upload_id}/complete 接受整个文件的 SHA-256(64 个十六进制字符),还可以在 parts 中传入每个分段的 etag,以便同时检查各个分段:
SHA=$(shasum -a 256 "$FILE" | cut -d' ' -f1)
curl -X POST "https://api.sursa.com/v1/uploads/$UPLOAD/complete" \
-H "Authorization: Bearer $SURSA_API_KEY" \
-H "Idempotency-Key: complete-$UPLOAD" \
-H "Content-Type: application/json" \
-d "{\"sha256\": \"$SHA\"}"Sursa 组装各个分段,比较哈希值,并像检查任何上传一样检查文件:检查格式,启用病毒扫描时还会进行扫描。检查完成时响应为 200,检查仍在进行时为 202。上传的状态从 processing 变为 ready,或者变为 rejected,原因在 reject_reason 中。用 GET /v1/uploads/{upload_id} 读取状态。
4. 使用上传
在原本发送文件的地方传入 upload_id。每个上传只能使用一次:使用后,其状态为 consumed。
用于卡片和压缩包时,请在 JSON 请求体中发送 upload_id,而不是使用 multipart/form-data 表单。这种请求体在 collection 中按名称指定合集。
# A card from the uploaded file
curl -X POST https://api.sursa.com/v1/cards \
-H "Authorization: Bearer $SURSA_API_KEY" \
-H "Idempotency-Key: card-from-$UPLOAD" \
-H "Content-Type: application/json" \
-d "{\"upload_id\": \"$UPLOAD\", \"collection\": \"support\", \"async\": true}"
# An archive of documents from the uploaded ZIP
curl -X POST https://api.sursa.com/v1/cards/archive \
-H "Authorization: Bearer $SURSA_API_KEY" \
-H "Idempotency-Key: archive-from-$UPLOAD" \
-H "Content-Type: application/json" \
-d "{\"upload_id\": \"$UPLOAD\", \"collection\": \"support\"}"完整的上传程序
import { createHash } from "node:crypto";
import { createReadStream } from "node:fs";
import { open, stat } from "node:fs/promises";
import { basename } from "node:path";
const BASE = "https://api.sursa.com";
const auth = { Authorization: `Bearer ${process.env.SURSA_API_KEY}` };
async function call(method, path, body, headers = {}) {
const isJson = body !== undefined && !(body instanceof Uint8Array);
const response = await fetch(BASE + path, {
method,
headers: { ...auth, ...(isJson ? { "Content-Type": "application/json" } : {}), ...headers },
body: isJson ? JSON.stringify(body) : body,
});
if (!response.ok) throw new Error(`${method} ${path}: HTTP ${response.status} ${await response.text()}`);
return response.json();
}
export async function uploadLargeFile(file, contentType, purpose, parallel = 4) {
const { size } = await stat(file);
const upload = await call("POST", "/v1/uploads",
{ filename: basename(file), content_type: contentType, size, purpose },
{ "Idempotency-Key": crypto.randomUUID() });
const hash = createHash("sha256");
for await (const block of createReadStream(file)) hash.update(block);
const handle = await open(file);
let next = 1;
async function worker() {
for (let n = next++; n <= upload.part_count; n = next++) {
const buffer = new Uint8Array(Math.min(upload.part_size, size - (n - 1) * upload.part_size));
await handle.read(buffer, 0, buffer.length, (n - 1) * upload.part_size);
for (let attempt = 1; ; attempt++) {
try {
await call("PUT", `/v1/uploads/${upload.id}/parts/${n}`, buffer, { "Content-Type": "application/octet-stream" });
break;
} catch (error) {
if (attempt === 5) throw error;
await new Promise((resolve) => setTimeout(resolve, 1000 * 2 ** attempt));
}
}
}
}
await Promise.all(Array.from({ length: parallel }, worker));
await handle.close();
await call("POST", `/v1/uploads/${upload.id}/complete`, { sha256: hash.digest("hex") },
{ "Idempotency-Key": `complete-${upload.id}` });
for (;;) {
const state = await call("GET", `/v1/uploads/${upload.id}`);
if (state.status === "ready") return upload.id;
if (state.status === "rejected") throw new Error(`upload rejected: ${state.reject_reason}`);
await new Promise((resolve) => setTimeout(resolve, 2000));
}
}错误
| 错误码 | 状态 | 原因 |
|---|---|---|
uploads.too_large | 413 | 文件大于 5 GB,或大于组织套餐的全部存储空间。 |
limits.storage | 402 | 组织剩余的可用存储空间不足以存放该文件。 |
uploads.part_invalid | 400 | 分段编号超出范围,或分段大小不在允许范围内。 |
uploads.checksum_mismatch | 422 | 组装后文件的 SHA-256 与您发送的不一致。某个分段已损坏或缺失:重新发送各分段,再次完成上传。 |
uploads.not_ready | 409 | 上传尚未完成和检查。请等待 ready。 |
uploads.already_used | 409 | 上传已被使用。每个上传只能使用一次。 |
uploads.expired | 410 | 已超过 24 小时。请创建新的上传。 |
uploads.rejected | 422 | 文件未通过检查,例如格式与文件名不符。错误消息会说明原因。 |
uploads.not_found | 404 | 组织中没有这个上传。 |