【游戏视频汉化 #1】 AI语音识别软件 Whisper编译

shane007

多年之前，就有为不带字幕的游戏视频配上字幕的想法。
: ^0 v( D5 E+ q 但是当时条件不成熟，但是目前来看，条件似乎成熟了
0 {7 t+ w+ a8 l
  `( k# o3 C) ^- c Whisper是openAI的开源语音识别软件。
它有一个.net的版本，在这个版本的基础上进行少量修改，就能将游戏视频对应的字幕识别成srt格式。
之后，对这个srt文件再进行在线批量翻译之后，进行少量调整之后，汉化工作就完成了。
9 r: k- ~* M9 n( E) M, E5 l% s2 f
地址如下
- p, _7 q! O+ M6 v2 V9 h8 V" }9 P https://github.com/sandrohanea/whisper.net
/ {6 [6 R: \/ c' x1 K

5 O' S, P5 G* y& F: c* A  b8 k/ S; I1 l: _ 编译最好使用vs2022编译，否则在.net sdk版本上会出很多问题。
7 g' I5 M& a4 i9 D4 v) [
" |2 w2 n5 h+ t( I; F6 c 编译好之后，有几个注意点
: A! m& U, @( ^4 h8 V8 f
" C4 u3 h. C3 Z% G/ } <0>使用的模型文件修改为大模型，ggml-large.bin，用这个模型效果比较好。
+ y" q! X  ~( Q    当然，所有时间也会比较多，估计转换一批文件需要几个甚至几十个小时。
' d: w: M* I/ e% Y! T' T( O) Z: H
* T: c4 O7 k3 D" Q9 ]# r <1>Language要设定为"english"。

3 s+ v9 Z8 t% z( D0 ^2 i& _

1. /*    var builder = factory.CreateBuilder()
   % g1 q/ g, Y8 _4 o; A
2.         .WithLanguage(opt.Language);*/
   
3.     var builder = factory.CreateBuilder()
   
4.     .WithLanguage("english");

复制代码

<2>缺省好像只支持Wav格式，而且是要16K采样率的，需要实现转换成这种格式，否则会出错。

* |' ]! U# m/ M2 S6 t <3>缺省只提供了一个例子wav文件的转换，需要改为批量形式。
3 f+ H6 N, ], R3 f; }% o  u( k   （遍历某个目录中的所有文件）
* r* Y3 ^, A  {
<4>输出的文件，需要稍加整理，以符合srt格式

4 Y# `6 f! F7 i  G1 [/ V   以下是一个Wav文件的控制台输出（幽魂开场动画）
: C2 ?6 a# K1 z

1. 
   
2. whisper_init_from_file_no_state: loading model from 'ggml-large.bin'
     ]$ [) s% k2 T* s
3. whisper_model_load: loading model
   
4. whisper_model_load: n_vocab       = 51865
   
5. whisper_model_load: n_audio_ctx   = 1500
   
6. whisper_model_load: n_audio_state = 1280
   ' J- w: I' C4 b, z: l3 ~! d  J
7. whisper_model_load: n_audio_head  = 20
   
8. whisper_model_load: n_audio_layer = 32
   
9. whisper_model_load: n_text_ctx    = 448
     p! n8 ?1 t* n! f" l; Y9 K& W
10. whisper_model_load: n_text_state  = 1280
    
11. whisper_model_load: n_text_head   = 20
    
12. whisper_model_load: n_text_layer  = 32
    3 w3 J1 \/ W  V- ]4 W9 Q! J4 z- O$ M
13. whisper_model_load: n_mels        = 80
    
14. whisper_model_load: ftype         = 1
    
15. whisper_model_load: qntvr         = 0
    
16. whisper_model_load: type          = 5
    
17. whisper_model_load: mem required  = 3557.00 MB (+   71.00 MB per decoder)
    
18. whisper_model_load: adding 1608 extra tokens
    ; H$ B1 R$ d/ \. z- }
19. whisper_model_load: model ctx     = 2951.27 MB
    
20. whisper_model_load: model size    = 2950.66 MB
    6 {( N. m3 T) v
21. whisper_init_state: kv self size  =   70.00 MB
    ' F5 `- G1 t/ f. ~1 s
22. whisper_init_state: kv cross size =  234.38 MB
    ' c9 s  ~! V  d! ?$ d/ m! o, }
23. New Segment: 00:00:00 ==> 00:00:02.7600000 :  (birds chirping)
    8 h: @8 C, Q0 i# @" `' k9 U
24. New Segment: 00:00:03.6600000 ==> 00:00:05.9000000 :  (exhaling)
    , \6 b) n% H1 T2 q! ]
25. New Segment: 00:00:05.9000000 ==> 00:00:08.6600000 :  (birds chirping)
    ; g- x. ~# B3 W' s3 l- o
26. New Segment: 00:00:08.6600000 ==> 00:00:35.1200000 :  (gun firing)
    
27. New Segment: 00:00:36.1200000 ==> 00:00:38.5400000 :  (gun firing)
    8 a0 @- r4 J- T1 V+ o) z4 B6 m4 M4 q
28. New Segment: 00:00:39.0600000 ==> 00:00:41.4800000 :  (gun firing)
    " T9 }# {* o2 v* \- n
29. New Segment: 00:00:41.4800000 ==> 00:00:49.4000000 :  (tires screeching)
    ' M; q$ H( q& ^8 Q
30. New Segment: 00:00:49.4000000 ==> 00:00:58.5800000 :  (glass shattering)
    
31. New Segment: 00:00:58.5800000 ==> 00:01:07.7400000 :  (singing in foreign language)
    
32. New Segment: 00:01:07.7400000 ==> 00:01:11.5800000 :  (singing in foreign language)
    
33. New Segment: 00:01:11.5800000 ==> 00:01:17 :  (tires screeching)
    ! |+ _0 m9 l9 j2 M3 g4 L) d& v- l
34. New Segment: 00:01:17 ==> 00:01:24.8400000 :  (singing in foreign language)
    
35. New Segment: 00:01:24.8400000 ==> 00:01:28.6400000 :  (panting)
    7 O. I0 y- r, S4 R/ _
36. New Segment: 00:01:36.7800000 ==> 00:01:39.2000000 :  (gun firing)
    3 F! H( o6 q0 a9 a& C
37. New Segment: 00:01:39.2000000 ==> 00:01:43.4600000 :  - Adrian.
    2 A4 Z" g% G- B, X, Q# W1 W
38. New Segment: 00:01:43.4600000 ==> 00:01:45.6200000 :  - Oh God.
    ( P8 H. y' G1 o/ {& m; x
39. New Segment: 00:01:45.6200000 ==> 00:01:48.2000000 :  - What's the matter sweetheart?
    
40. New Segment: 00:01:48.2000000 ==> 00:01:50.4200000 :  Oh.
    3 [3 Y9 n* r9 Q, @: E; Y* Y
41. New Segment: 00:01:50.4200000 ==> 00:01:53.4600000 :  - Oh it's horrible.
    
42. New Segment: 00:01:53.4600000 ==> 00:01:55.3000000 :  - Shh.
    4 z/ Q, Y5 Y" x+ h
43. New Segment: 00:01:55.3000000 ==> 00:02:02.3400000 :  It was just a bad dream.
    
44. New Segment: 00:02:05.4200000 ==> 00:02:09.8800000 :  - You don't ever have to be afraid of anything.
    
45. New Segment: 00:02:09.8800000 ==> 00:02:12.8000000 :  I'll always be here to protect you.
    
46. New Segment: 00:02:12.9200000 ==> 00:02:15.5000000 :  (gentle music)
    6 Y3 k  u  L4 G# j; r% l  V# `- H
47. New Segment: 00:02:16.4800000 ==> 00:02:19.0600000 :  (gentle music)
    
48. New Segment: 00:02:19.0600000 ==> 00:02:21.6400000 :  (gentle music)
    1 N9 l4 H, r1 e8 |% ~# k
49. New Segment: 00:02:21.6400000 ==> 00:02:24.2200000 :  (gentle music)
    
50. New Segment: 00:02:24.5400000 ==> 00:02:27.1200000 :  (gentle music)
    7 u6 C/ {& U/ k! Z: S
51. New Segment: 00:02:27.1200000 ==> 00:02:29.7000000 :  (gentle music)
    & Z, \* e% C. A: R9 b# m
52. New Segment: 00:02:29.7000000 ==> 00:02:33.1800000 :  [Music]
    

复制代码

0 ]$ h' A9 r) r$ {- ]