feat: CCA v6 腾讯云部署 + 审稿台(含查找替换)
- deploy/cca_route.py: Flask 蓝图(6个API端点),WAV自动转MP3 - deploy/cca.html: 4步单页流程(上传→处理→审稿→下载),查找替换(Ctrl+H) - src/term_normalizer.py: 新增正则层(同音字/引号/书名号/小数点/波浪号) - src/ai_proofreader.py: speaker角色识别+专家段增强Prompt+的地得加强 - src/ai_line_breaker.py: 引号不跨屏+极短行合并+短句合并间隔放宽 - cca_pipeline.py: Step 2.5 校对后二次正则兜底 - 已部署至 http://101.42.29.217/cca.html Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
+176
-65
@@ -7,11 +7,13 @@ AI 校对器 — ASR 稿与 A 稿比对 + 上下文纠错
|
||||
- 军事术语规范化("f15j"→"F-15J")
|
||||
- 的/地/得纠错
|
||||
- 去除口语填充词("嗯""那个""就是说")
|
||||
- 专家采访段落强化去口头语
|
||||
|
||||
策略:
|
||||
- 将 ASR 全文 + A 稿全文一起发给 DeepSeek
|
||||
- AI 结合节目主题和上下文做纠错
|
||||
- 返回修正后的句子列表 + 修改说明
|
||||
- 专家采访段落用增强版 Prompt,更严格地删除口头语
|
||||
"""
|
||||
|
||||
import json
|
||||
@@ -37,21 +39,68 @@ PROOFREAD_SYSTEM_PROMPT = """你是电视军事节目《军事科技》的字幕
|
||||
|
||||
**铁律(违反任何一条都算失败):**
|
||||
- ASR稿是已经录好的音频的转写,内容不能改——**绝不润色语句、绝不调整语序、绝不增删实词**
|
||||
- 只修三类问题:① 错别字/同音字 ② 术语格式 ③ 口语填充词
|
||||
- 除这三类外的一切文字,原封不动照抄,一个字都不能动
|
||||
- A稿只用来判断"这个词在本期节目的语境下应该是哪个字",不能把ASR稿往A稿的措辞上靠
|
||||
- 只修下列允许的几类问题,除此之外一个字都不能动
|
||||
- **A稿与ASR内容冲突时ASR优先**(配音员可能改过措辞),但专有名词的正确写法/格式按A稿
|
||||
- **数字表达照抄ASR原文**:不要参考A稿调整数字的位置、格式或表述方式。ASR说"马赫数0.9"就保持"马赫数0.9",不要改成A稿的"0.9马赫"
|
||||
|
||||
**允许修的三类:**
|
||||
1. **同音字/错别字**(ASR听错的字):如"建制"→"舰只"、"舰手"→"舰艏"、"继承"→"击沉"、"空花弹"→"滑翔弹"
|
||||
2. **术语格式**:英文型号大小写+连字符("f15j"→"F-15J"、"v22"→"V-22"、"rq四"→"RQ-4")
|
||||
3. **口语填充词删除**:只删"嗯""呃""唉""啊""呢""那个""就是说""这个"这类纯填充词。如果"这个"后面紧跟名词作指示代词("这个导弹"),保留不删
|
||||
**允许修的类别:**
|
||||
1. **同音字/错别字**(ASR听错的字):如"建制"→"舰只"、"舰手"→"舰艏"、"继承"→"击沉"、"空花弹"→"滑翔弹"、"沉默"→"沉没"(指船只)
|
||||
2. **代词纠错**:武器装备/导弹/飞机/舰艇等的代词应为"它"而非"他"。注意:指代国家时不改(国家口语中用"他"是可接受的)。只纠正明确指代物件(武器、军舰、飞机、导弹)的情况
|
||||
3. **的/地/得纠错**(重要!ASR无法区分三个"de",你必须逐句检查并修正):
|
||||
- **"的"用在名词前**(形容词/名词 + 的 + 名词):强大的性能、日本的军备、重要的舰只
|
||||
- **"地"用在动词前**(副词 + 地 + 动词):不断地进行、持续地推动、快速地发展、正式地把、大规模地改装、积极地推进、不断地扩大、明确地表示
|
||||
- **"得"用在补语前**(动词 + 得 + 补语):发展得很快、做得很好、打得很准
|
||||
- 判断方法:看"de"后面跟的是名词还是动词——跟动词就用"地",跟名词就用"的",是评价/程度补语就用"得"
|
||||
- 常见错误模式:"不断的进行"→"不断地进行"、"持续的推动"→"持续地推动"、"正式的把"→"正式地把"、"大力的发展"→"大力地发展"
|
||||
4. **术语格式**:英文型号大小写+连字符("f15j"→"F-15J"、"v22"→"V-22"、"rq四"→"RQ-4")
|
||||
5. **中文数字保留**:ASR可能把"数十"转成"数10"、"几百"转成"几100"——必须改回中文写法
|
||||
6. **武器昵称引号**:如A稿中武器有引号昵称("鱼鹰""战斧""全球鹰"),ASR中同一武器无引号时补上中文双引号
|
||||
7. **口语填充词删除**:只删"嗯""呃""唉""那个""就是说"这类纯填充词。"这个"后面紧跟名词作指示代词("这个导弹")时保留
|
||||
|
||||
**绝对不许做的(哪怕你觉得改了更好也不许):**
|
||||
- 不许调整语序("它在性质上就是"不许改成"它本质上就是")
|
||||
- 不许替换实词("不是那么特别的顺利"不许改成"不太顺利")
|
||||
- 不许参考A稿的数字表达方式来改ASR的数字写法
|
||||
- 不许增删标点来改变句子结构
|
||||
- 不许把口语化表达改成书面语
|
||||
- 不许根据A稿的措辞替换ASR中意思相同但用词不同的表达
|
||||
- 不许根据A稿的措辞替换ASR中意思相同但用词不同的表达(如A稿"陆续订购",ASR说"先后采购"→保持"先后采购")
|
||||
|
||||
**输出格式:**
|
||||
JSON数组,每个元素:{"id": 编号, "original": "原文", "corrected": "修正后", "changes": "修改说明(无修改写空字符串)"}
|
||||
只输出JSON,不要其他内容。"""
|
||||
|
||||
|
||||
PROOFREAD_EXPERT_SYSTEM_PROMPT = """你是电视军事节目《军事科技》的字幕校对专家。你将收到两份材料:
|
||||
1. **ASR稿**:语音识别的转写结果,带有时间编号,是字幕的基础。**本批全部来自专家采访段落**
|
||||
2. **A稿**:编导写的节目文稿(仅包含解说词,不包含专家采访内容——专家说的话A稿里没有)
|
||||
|
||||
你的任务是校对 ASR 稿中的**语音识别错误**,同时**严格清除专家的口头语**。
|
||||
|
||||
**铁律(违反任何一条都算失败):**
|
||||
- ASR稿是已经录好的音频的转写,内容不能改——**绝不润色语句、绝不调整语序、绝不增删实词**
|
||||
- 只修下列允许的几类问题,除此之外一个字都不能动
|
||||
- 由于是专家采访,A稿中没有对应内容,所以**不要用A稿措辞替换专家的话**,A稿只用于确认专有名词写法
|
||||
|
||||
**允许修的类别:**
|
||||
1. **同音字/错别字**(ASR听错的字):如"建制"→"舰只"、"舰手"→"舰艏"、"继承"→"击沉"、"沉默"→"沉没"(指船只)
|
||||
2. **代词纠错**:武器装备/导弹/飞机/舰艇等的代词应为"它"而非"他"。指代国家时不改
|
||||
3. **的/地/得纠错**(重要!ASR无法区分三个"de",你必须逐句检查并修正):
|
||||
- **"的"用在名词前**(形容词/名词 + 的 + 名词)
|
||||
- **"地"用在动词前**(副词 + 地 + 动词):不断地进行、持续地推动、正式地把、大规模地改装
|
||||
- **"得"用在补语前**(动词 + 得 + 补语):发展得很快
|
||||
- 常见错误:"不断的进行"→"不断地进行"、"持续的推动"→"持续地推动"
|
||||
4. **术语格式**:英文型号大小写+连字符
|
||||
5. **口语填充词删除(专家采访重点!必须严格执行)**:
|
||||
- **必删**:嗯、呃、唉、啊(句首或句中作语气词时)、那个、这个(非指示代词时)、那么(非表示程度时)、就是说、应该说、可以说、怎么说呢、相对来讲、相对来说
|
||||
- **判断"这个/那个"**:紧跟具体名词="指示代词"保留("这个导弹");单独出现或后面是虚词/停顿=口头语删除("这个呢它是"→删"这个"、"发展这个日向级"→删"这个")
|
||||
- **判断"啊"**:句首"啊射程""啊这个"=口头语删除;"啊"在感叹句末尾=保留(极少出现在专家采访中)
|
||||
- **判断"那么"**:"那么大""那么快"=程度副词保留;"那么它就是"=口头语删除
|
||||
6. **数字表达照抄ASR原文**,不参考A稿
|
||||
|
||||
**绝对不许做的:**
|
||||
- 不许调整语序、替换实词、把口语化改书面语
|
||||
- 不许用A稿的措辞替换专家的话(专家说的内容A稿没有,不存在"参考"关系)
|
||||
- 不许删除有意义的词(只删纯口头语填充词)
|
||||
|
||||
**输出格式:**
|
||||
JSON数组,每个元素:{"id": 编号, "original": "原文", "corrected": "修正后", "changes": "修改说明(无修改写空字符串)"}
|
||||
@@ -76,6 +125,62 @@ def _create_client():
|
||||
)
|
||||
|
||||
|
||||
def identify_speakers(
|
||||
sentences: List[Tuple[int, int, str, int]],
|
||||
) -> Dict[int, str]:
|
||||
"""
|
||||
识别每个 speaker_id 的角色。
|
||||
|
||||
规则(基于《军事科技》节目结构):
|
||||
- 找到说"各位观众你们好"或"欢迎收看军事科技"的 speaker → 主持人(也是解说配音员)
|
||||
- 导视段(最早出现的)speaker 如果和主持人不同 → 也是解说(录音环境不同导致分裂)
|
||||
- 剩余的 speaker → 专家/其他(统一按"专家采访"对待,加强去口头语)
|
||||
|
||||
返回: {speaker_id: "narration"|"host"|"expert"}
|
||||
"""
|
||||
if not sentences:
|
||||
return {}
|
||||
|
||||
speaker_texts: Dict[int, str] = {}
|
||||
speaker_first_appear: Dict[int, int] = {}
|
||||
for i, (bg, ed, text, spk) in enumerate(sentences):
|
||||
if spk not in speaker_texts:
|
||||
speaker_texts[spk] = ""
|
||||
speaker_first_appear[spk] = i
|
||||
speaker_texts[spk] += text
|
||||
|
||||
roles: Dict[int, str] = {}
|
||||
|
||||
# 找主持人:说过"各位观众你们好"或"欢迎收看军事科技"
|
||||
host_spk = None
|
||||
for spk, text in speaker_texts.items():
|
||||
if "各位观众" in text or "欢迎收看" in text or "主持人" in text:
|
||||
host_spk = spk
|
||||
roles[spk] = "host"
|
||||
break
|
||||
|
||||
# 最早出现的 speaker 是解说(导视段配音员)
|
||||
earliest_spk = min(speaker_first_appear, key=speaker_first_appear.get)
|
||||
if earliest_spk not in roles:
|
||||
roles[earliest_spk] = "narration"
|
||||
|
||||
# 如果主持人和解说是不同 speaker,两个都标记
|
||||
# 如果相同,那就是同一个人(标为 narration 即可)
|
||||
if host_spk is not None and host_spk == earliest_spk:
|
||||
roles[host_spk] = "narration"
|
||||
|
||||
# 剩余的全部标为专家/其他
|
||||
for spk in speaker_texts:
|
||||
if spk not in roles:
|
||||
roles[spk] = "expert"
|
||||
|
||||
role_summary = {spk: f"{role}({len([s for s in sentences if s[3]==spk])}句)"
|
||||
for spk, role in roles.items()}
|
||||
print(f"[校对] Speaker 角色识别: {role_summary}")
|
||||
|
||||
return roles
|
||||
|
||||
|
||||
def proofread_batch(
|
||||
asr_sentences: List[Tuple[int, int, str, int]],
|
||||
script_text: str,
|
||||
@@ -83,81 +188,87 @@ def proofread_batch(
|
||||
) -> List[Tuple[int, int, str, int]]:
|
||||
"""
|
||||
对 ASR 句子列表做 AI 校对。
|
||||
|
||||
输入:
|
||||
asr_sentences: [(start_ms, end_ms, text, speaker_id), ...]
|
||||
script_text: A稿全文
|
||||
batch_size: 每批处理的句子数
|
||||
|
||||
返回:
|
||||
校对后的句子列表,格式同输入
|
||||
专家采访段落使用增强版 Prompt(更严格的口头语清除)。
|
||||
"""
|
||||
if not asr_sentences:
|
||||
return []
|
||||
|
||||
client = _create_client()
|
||||
|
||||
# A稿截取(太长的话截前8000字,够提供上下文了)
|
||||
script_truncated = script_text[:8000] if len(script_text) > 8000 else script_text
|
||||
|
||||
corrected_sentences = list(asr_sentences) # 浅拷贝
|
||||
# 识别说话人角色
|
||||
speaker_roles = identify_speakers(asr_sentences)
|
||||
|
||||
corrected_sentences = list(asr_sentences)
|
||||
total_changes = 0
|
||||
|
||||
for batch_start in range(0, len(asr_sentences), batch_size):
|
||||
batch = asr_sentences[batch_start:batch_start + batch_size]
|
||||
batch_end = batch_start + len(batch)
|
||||
# 按角色分组处理:专家用增强 Prompt,其余用标准 Prompt
|
||||
expert_indices = []
|
||||
normal_indices = []
|
||||
for i, (bg, ed, text, spk) in enumerate(asr_sentences):
|
||||
if speaker_roles.get(spk) == "expert":
|
||||
expert_indices.append(i)
|
||||
else:
|
||||
normal_indices.append(i)
|
||||
|
||||
# 构建 ASR 文本(带编号)
|
||||
asr_lines = []
|
||||
for i, (bg, ed, text, spk) in enumerate(batch):
|
||||
asr_lines.append(f"[{i+1}] {text}")
|
||||
asr_text = "\n".join(asr_lines)
|
||||
print(f"[校对] 解说/主持 {len(normal_indices)} 句, 专家采访 {len(expert_indices)} 句")
|
||||
|
||||
print(f"[校对] 处理第 {batch_start+1}-{batch_end} 句...")
|
||||
def _process_batch(indices, system_prompt, label):
|
||||
nonlocal total_changes
|
||||
for batch_start in range(0, len(indices), batch_size):
|
||||
batch_idx = indices[batch_start:batch_start + batch_size]
|
||||
|
||||
try:
|
||||
resp = client.chat.completions.create(
|
||||
model=os.environ.get("DEEPSEEK_MODEL", "deepseek-chat"),
|
||||
messages=[
|
||||
{"role": "system", "content": PROOFREAD_SYSTEM_PROMPT},
|
||||
{"role": "user", "content": PROOFREAD_USER_TEMPLATE.format(
|
||||
script_text=script_truncated,
|
||||
asr_text=asr_text,
|
||||
)},
|
||||
],
|
||||
temperature=0.1,
|
||||
max_tokens=4000,
|
||||
)
|
||||
asr_lines = []
|
||||
for seq, idx in enumerate(batch_idx):
|
||||
asr_lines.append(f"[{seq+1}] {asr_sentences[idx][2]}")
|
||||
asr_text = "\n".join(asr_lines)
|
||||
|
||||
result_text = resp.choices[0].message.content.strip()
|
||||
print(f"[校对-{label}] 处理第 {batch_start+1}-{batch_start+len(batch_idx)} 句...")
|
||||
|
||||
# 尝试解析 JSON
|
||||
# 去掉可能的 markdown 代码块标记
|
||||
if result_text.startswith("```"):
|
||||
result_text = result_text.split("\n", 1)[1]
|
||||
if result_text.endswith("```"):
|
||||
result_text = result_text[:-3]
|
||||
result_text = result_text.strip()
|
||||
try:
|
||||
resp = client.chat.completions.create(
|
||||
model=os.environ.get("DEEPSEEK_MODEL", "deepseek-chat"),
|
||||
messages=[
|
||||
{"role": "system", "content": system_prompt},
|
||||
{"role": "user", "content": PROOFREAD_USER_TEMPLATE.format(
|
||||
script_text=script_truncated,
|
||||
asr_text=asr_text,
|
||||
)},
|
||||
],
|
||||
temperature=0.1,
|
||||
max_tokens=4000,
|
||||
)
|
||||
|
||||
corrections = json.loads(result_text)
|
||||
result_text = resp.choices[0].message.content.strip()
|
||||
if result_text.startswith("```"):
|
||||
result_text = result_text.split("\n", 1)[1]
|
||||
if result_text.endswith("```"):
|
||||
result_text = result_text[:-3]
|
||||
result_text = result_text.strip()
|
||||
|
||||
# 应用修正
|
||||
for item in corrections:
|
||||
idx = item.get("id", 0) - 1 # 编号从1开始
|
||||
corrected = item.get("corrected", "")
|
||||
changes = item.get("changes", "")
|
||||
corrections = json.loads(result_text)
|
||||
|
||||
if 0 <= idx < len(batch) and corrected and changes:
|
||||
original_idx = batch_start + idx
|
||||
bg, ed, _, spk = corrected_sentences[original_idx]
|
||||
corrected_sentences[original_idx] = (bg, ed, corrected, spk)
|
||||
total_changes += 1
|
||||
print(f" 修正: '{item.get('original','')}' → '{corrected}' ({changes})")
|
||||
for item in corrections:
|
||||
seq = item.get("id", 0) - 1
|
||||
corrected = item.get("corrected", "")
|
||||
changes = item.get("changes", "")
|
||||
|
||||
except json.JSONDecodeError as e:
|
||||
print(f"[校对] JSON解析失败,跳过本批: {e}", file=sys.stderr)
|
||||
except Exception as e:
|
||||
print(f"[校对] 出错: {e}", file=sys.stderr)
|
||||
if 0 <= seq < len(batch_idx) and corrected and changes:
|
||||
original_idx = batch_idx[seq]
|
||||
bg, ed, _, spk = corrected_sentences[original_idx]
|
||||
corrected_sentences[original_idx] = (bg, ed, corrected, spk)
|
||||
total_changes += 1
|
||||
print(f" 修正: '{item.get('original','')}' → '{corrected}' ({changes})")
|
||||
|
||||
except json.JSONDecodeError as e:
|
||||
print(f"[校对-{label}] JSON解析失败,跳过本批: {e}", file=sys.stderr)
|
||||
except Exception as e:
|
||||
print(f"[校对-{label}] 出错: {e}", file=sys.stderr)
|
||||
|
||||
if normal_indices:
|
||||
_process_batch(normal_indices, PROOFREAD_SYSTEM_PROMPT, "解说")
|
||||
if expert_indices:
|
||||
_process_batch(expert_indices, PROOFREAD_EXPERT_SYSTEM_PROMPT, "专家")
|
||||
|
||||
print(f"[校对] 完成,共修正 {total_changes} 处")
|
||||
return corrected_sentences
|
||||
|
||||
Reference in New Issue
Block a user