2026-06-11 18:34:55 +08:00
|
|
|
|
# AI Labeling 实验工作区
|
|
|
|
|
|
|
|
|
|
|
|
## 用途
|
|
|
|
|
|
为《军事科技》栏目 AI 自动打标做模型选型实验。
|
|
|
|
|
|
基准答案集 = 10 期已审稿的标准答案。
|
|
|
|
|
|
|
|
|
|
|
|
## 目录结构
|
|
|
|
|
|
- `benchmark-set/transcripts/` 10 期文稿
|
|
|
|
|
|
- `benchmark-set/ground-truth.json` 10 期的标准答案
|
|
|
|
|
|
- `prompts/` prompt 模板
|
|
|
|
|
|
- `scripts/` 实验脚本(含 import_transcripts.py)
|
|
|
|
|
|
- `experiments/` 每次跑出来的结果(不入 git)
|
|
|
|
|
|
|
|
|
|
|
|
## 使用步骤
|
|
|
|
|
|
|
|
|
|
|
|
### 1. 安装依赖
|
|
|
|
|
|
```bash
|
|
|
|
|
|
pip install -r scripts/requirements.txt
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
### 2. 配置 API Key
|
|
|
|
|
|
```bash
|
|
|
|
|
|
cp .env.example .env
|
2026-06-24 16:29:18 +08:00
|
|
|
|
# 填入真实 MIMO_API_KEY
|
2026-06-11 18:34:55 +08:00
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
### 3. 导入文稿
|
|
|
|
|
|
```bash
|
|
|
|
|
|
python scripts/import_transcripts.py
|
|
|
|
|
|
```
|
|
|
|
|
|
> 将源目录(刘瑞桦收集)的 10 期 docx 文稿解析并清洗,落入 `benchmark-set/transcripts/`。
|
|
|
|
|
|
|
|
|
|
|
|
### 4. 跑单期打标
|
|
|
|
|
|
```bash
|
2026-06-24 16:29:18 +08:00
|
|
|
|
python scripts/run_labeling.py --ep 4 --model mimo-v2.5-pro
|
2026-06-11 18:34:55 +08:00
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
## 当前版本
|
|
|
|
|
|
- v0.1: Prompt 2(叙事结构)含 3 示例 + 10 期 ground-truth
|
2026-06-24 16:29:18 +08:00
|
|
|
|
- 生产模型: mimo-v2.5-pro / 备用: DeepSeek V4 Pro
|
2026-06-11 18:34:55 +08:00
|
|
|
|
|
|
|
|
|
|
## ground-truth 版本
|
|
|
|
|
|
- `v0.1` (2026-06-11): 锁定 10 期 `narrative_structure` + `opening_hook` 标准答案
|
|
|
|
|
|
- 其他字段(program_format / equipment_domain / scene_tags / tech_tags)待 Prompt 1 阶段补
|