【游戏视频汉化 #1】 AI语音识别软件 Whisper编译

shane007

多年之前，就有为不带字幕的游戏视频配上字幕的想法。
- }& b  u  C" Q 但是当时条件不成熟，但是目前来看，条件似乎成熟了

Whisper是openAI的开源语音识别软件。
- @% Q2 o$ t' o' S  ~ 它有一个.net的版本，在这个版本的基础上进行少量修改，就能将游戏视频对应的字幕识别成srt格式。
1 ~% E4 s; V8 A0 ?2 l1 Q 之后，对这个srt文件再进行在线批量翻译之后，进行少量调整之后，汉化工作就完成了。

地址如下
0 d# C; D1 r3 G) i, |5 a https://github.com/sandrohanea/whisper.net


" J4 P+ X) B/ B* l/ N 编译最好使用vs2022编译，否则在.net sdk版本上会出很多问题。

2 D, h4 O# Q% W; `0 L, E# p 编译好之后，有几个注意点
4 S! G, E% w% Y( y1 a8 z- Z
<0>使用的模型文件修改为大模型，ggml-large.bin，用这个模型效果比较好。
# k/ k- [' L  E2 |    当然，所有时间也会比较多，估计转换一批文件需要几个甚至几十个小时。

% W) p  P5 m0 R# P7 `9 j2 r <1>Language要设定为"english"。

1. /*    var builder = factory.CreateBuilder()
   
2.         .WithLanguage(opt.Language);*/
     S8 H* v; [4 m6 m1 j
3.     var builder = factory.CreateBuilder()
   5 n4 u# w4 @* |- y
4.     .WithLanguage("english");

复制代码

<2>缺省好像只支持Wav格式，而且是要16K采样率的，需要实现转换成这种格式，否则会出错。
/ D3 g# @$ m4 L+ T' `
& k' a0 x- `: w6 t; g0 u <3>缺省只提供了一个例子wav文件的转换，需要改为批量形式。
3 K4 K# R% _0 H0 Z   （遍历某个目录中的所有文件）
5 }$ g& e  {& O" O
<4>输出的文件，需要稍加整理，以符合srt格式
- _9 n0 ]0 I) [2 B! [6 W9 n3 \- @2 C
   以下是一个Wav文件的控制台输出（幽魂开场动画）

: s1 p5 m5 w- B3 k7 W3 v

1. 
   
2. whisper_init_from_file_no_state: loading model from 'ggml-large.bin'
   
3. whisper_model_load: loading model
   # h; ]. f3 H" T
4. whisper_model_load: n_vocab       = 51865
   
5. whisper_model_load: n_audio_ctx   = 1500
   
6. whisper_model_load: n_audio_state = 1280
   4 s$ G( Q* N1 i3 |2 \) F
7. whisper_model_load: n_audio_head  = 20
   5 o$ U+ p/ r: b, K/ u: |
8. whisper_model_load: n_audio_layer = 32
   
9. whisper_model_load: n_text_ctx    = 448
   0 N3 G6 u7 D( E3 |& t" j" G
10. whisper_model_load: n_text_state  = 1280
    5 Q% H7 ?7 r. C- q0 `
11. whisper_model_load: n_text_head   = 20
    
12. whisper_model_load: n_text_layer  = 32
    
13. whisper_model_load: n_mels        = 80
    
14. whisper_model_load: ftype         = 1
    , ]2 }' L1 t) W3 \& i
15. whisper_model_load: qntvr         = 0
    
16. whisper_model_load: type          = 5
    / T& p0 ?9 N, u5 v5 M9 R# ~
17. whisper_model_load: mem required  = 3557.00 MB (+   71.00 MB per decoder)
    7 F. L4 y0 |9 \5 b+ D* N' g8 D
18. whisper_model_load: adding 1608 extra tokens
    3 f/ y* _+ U% R) |" _( f
19. whisper_model_load: model ctx     = 2951.27 MB
      l  E* [8 f2 Y) b) |/ q/ K+ k: a9 X) k
20. whisper_model_load: model size    = 2950.66 MB
    
21. whisper_init_state: kv self size  =   70.00 MB
    
22. whisper_init_state: kv cross size =  234.38 MB
    
23. New Segment: 00:00:00 ==> 00:00:02.7600000 :  (birds chirping)
    % U* [: j6 j9 n, Z- S+ p
24. New Segment: 00:00:03.6600000 ==> 00:00:05.9000000 :  (exhaling)
    5 y0 g8 k6 {$ [' r% K. C' C+ k
25. New Segment: 00:00:05.9000000 ==> 00:00:08.6600000 :  (birds chirping)
    / i* E( C" K; |
26. New Segment: 00:00:08.6600000 ==> 00:00:35.1200000 :  (gun firing)
      A: S! z7 ~$ ^6 p) X" Q4 U
27. New Segment: 00:00:36.1200000 ==> 00:00:38.5400000 :  (gun firing)
    % }7 T4 k3 z5 Q; J( s3 b
28. New Segment: 00:00:39.0600000 ==> 00:00:41.4800000 :  (gun firing)
    ! ^1 i: ^1 A0 E! [+ ^+ E4 @6 r, A( ~
29. New Segment: 00:00:41.4800000 ==> 00:00:49.4000000 :  (tires screeching)
    
30. New Segment: 00:00:49.4000000 ==> 00:00:58.5800000 :  (glass shattering)
    
31. New Segment: 00:00:58.5800000 ==> 00:01:07.7400000 :  (singing in foreign language)
    & T, l: T7 @9 S
32. New Segment: 00:01:07.7400000 ==> 00:01:11.5800000 :  (singing in foreign language)
    
33. New Segment: 00:01:11.5800000 ==> 00:01:17 :  (tires screeching)
    
34. New Segment: 00:01:17 ==> 00:01:24.8400000 :  (singing in foreign language)
      o  D6 j% K* S9 w
35. New Segment: 00:01:24.8400000 ==> 00:01:28.6400000 :  (panting)
    8 i/ X5 k6 t  F; J4 u) |5 |4 c
36. New Segment: 00:01:36.7800000 ==> 00:01:39.2000000 :  (gun firing)
    
37. New Segment: 00:01:39.2000000 ==> 00:01:43.4600000 :  - Adrian.
    6 x- A- ?7 _( c% f8 E
38. New Segment: 00:01:43.4600000 ==> 00:01:45.6200000 :  - Oh God.
    
39. New Segment: 00:01:45.6200000 ==> 00:01:48.2000000 :  - What's the matter sweetheart?
    
40. New Segment: 00:01:48.2000000 ==> 00:01:50.4200000 :  Oh.
    
41. New Segment: 00:01:50.4200000 ==> 00:01:53.4600000 :  - Oh it's horrible.
    ! Q' ~( H: d& ?" v, [! E
42. New Segment: 00:01:53.4600000 ==> 00:01:55.3000000 :  - Shh.
    $ {# X& r) w$ u2 S
43. New Segment: 00:01:55.3000000 ==> 00:02:02.3400000 :  It was just a bad dream.
    1 S+ W, K# ]' f, q
44. New Segment: 00:02:05.4200000 ==> 00:02:09.8800000 :  - You don't ever have to be afraid of anything.
    
45. New Segment: 00:02:09.8800000 ==> 00:02:12.8000000 :  I'll always be here to protect you.
    
46. New Segment: 00:02:12.9200000 ==> 00:02:15.5000000 :  (gentle music)
    
47. New Segment: 00:02:16.4800000 ==> 00:02:19.0600000 :  (gentle music)
    0 h3 K) s; C, ]
48. New Segment: 00:02:19.0600000 ==> 00:02:21.6400000 :  (gentle music)
    7 S, S4 c& N* g3 o: [: U: V
49. New Segment: 00:02:21.6400000 ==> 00:02:24.2200000 :  (gentle music)
    + }1 H) ~" L6 M4 C: [3 V
50. New Segment: 00:02:24.5400000 ==> 00:02:27.1200000 :  (gentle music)
    
51. New Segment: 00:02:27.1200000 ==> 00:02:29.7000000 :  (gentle music)
    ' x$ W9 r6 G" I  a
52. New Segment: 00:02:29.7000000 ==> 00:02:33.1800000 :  [Music]
    

复制代码

3 G* f- @: i7 r$ A8 k7 v