【游戏视频汉化 #1】 AI语音识别软件 Whisper编译

shane007

多年之前，就有为不带字幕的游戏视频配上字幕的想法。
但是当时条件不成熟，但是目前来看，条件似乎成熟了

0 ?! I, X3 G* ~  a) {3 j Whisper是openAI的开源语音识别软件。
: h: n# L" N$ `" T 它有一个.net的版本，在这个版本的基础上进行少量修改，就能将游戏视频对应的字幕识别成srt格式。
之后，对这个srt文件再进行在线批量翻译之后，进行少量调整之后，汉化工作就完成了。
- i0 s0 ^. Y7 A
地址如下
https://github.com/sandrohanea/whisper.net

9 X  K7 F. Y8 Y( T* o, b! ]' ?
6 b, ?0 ?, ?$ ^7 M; q$ A 编译最好使用vs2022编译，否则在.net sdk版本上会出很多问题。

编译好之后，有几个注意点

) H; F: F; @- d2 p' V3 i/ O <0>使用的模型文件修改为大模型，ggml-large.bin，用这个模型效果比较好。
0 c0 a+ b7 k' @    当然，所有时间也会比较多，估计转换一批文件需要几个甚至几十个小时。
) N. ~/ B2 e" Q
<1>Language要设定为"english"。
6 c4 y2 u; k* |& y. @

1. /*    var builder = factory.CreateBuilder()
   
2.         .WithLanguage(opt.Language);*/
   " R( d6 d& J) `
3.     var builder = factory.CreateBuilder()
   
4.     .WithLanguage("english");

复制代码

1 D  u# F! ]  D3 @ <2>缺省好像只支持Wav格式，而且是要16K采样率的，需要实现转换成这种格式，否则会出错。

<3>缺省只提供了一个例子wav文件的转换，需要改为批量形式。
/ S/ ]4 |- L, P0 V9 b( K8 o+ P   （遍历某个目录中的所有文件）

<4>输出的文件，需要稍加整理，以符合srt格式

   以下是一个Wav文件的控制台输出（幽魂开场动画）
; \% W& K1 B7 F

1. 
   / ]: y& k6 {# ?
2. whisper_init_from_file_no_state: loading model from 'ggml-large.bin'
   
3. whisper_model_load: loading model
   / B# J" u. g$ Z
4. whisper_model_load: n_vocab       = 51865
   
5. whisper_model_load: n_audio_ctx   = 1500
   
6. whisper_model_load: n_audio_state = 1280
   
7. whisper_model_load: n_audio_head  = 20
   
8. whisper_model_load: n_audio_layer = 32
   7 n1 W9 A, m( \$ M! Y
9. whisper_model_load: n_text_ctx    = 448
   
10. whisper_model_load: n_text_state  = 1280
    5 G: W  ~" E& T4 H. r
11. whisper_model_load: n_text_head   = 20
    + i# ~2 |0 a  x; |+ R9 K" z
12. whisper_model_load: n_text_layer  = 32
    
13. whisper_model_load: n_mels        = 80
    # Y) w0 s9 R* E% n6 z
14. whisper_model_load: ftype         = 1
    
15. whisper_model_load: qntvr         = 0
    
16. whisper_model_load: type          = 5
    
17. whisper_model_load: mem required  = 3557.00 MB (+   71.00 MB per decoder)
    
18. whisper_model_load: adding 1608 extra tokens
    
19. whisper_model_load: model ctx     = 2951.27 MB
    0 J  m; |! F, Z, e; [
20. whisper_model_load: model size    = 2950.66 MB
    
21. whisper_init_state: kv self size  =   70.00 MB
    
22. whisper_init_state: kv cross size =  234.38 MB
    * k7 D  X/ k& \/ _) o
23. New Segment: 00:00:00 ==> 00:00:02.7600000 :  (birds chirping)
    
24. New Segment: 00:00:03.6600000 ==> 00:00:05.9000000 :  (exhaling)
    9 }7 z) r2 A( n, M
25. New Segment: 00:00:05.9000000 ==> 00:00:08.6600000 :  (birds chirping)
    
26. New Segment: 00:00:08.6600000 ==> 00:00:35.1200000 :  (gun firing)
      l7 p5 h7 `2 l3 \
27. New Segment: 00:00:36.1200000 ==> 00:00:38.5400000 :  (gun firing)
    - E) ]: {8 H0 ]" D- }& @' ?) k4 _. H9 e
28. New Segment: 00:00:39.0600000 ==> 00:00:41.4800000 :  (gun firing)
    
29. New Segment: 00:00:41.4800000 ==> 00:00:49.4000000 :  (tires screeching)
    
30. New Segment: 00:00:49.4000000 ==> 00:00:58.5800000 :  (glass shattering)
    * ~8 m1 g. T! l
31. New Segment: 00:00:58.5800000 ==> 00:01:07.7400000 :  (singing in foreign language)
    
32. New Segment: 00:01:07.7400000 ==> 00:01:11.5800000 :  (singing in foreign language)
    
33. New Segment: 00:01:11.5800000 ==> 00:01:17 :  (tires screeching)
    
34. New Segment: 00:01:17 ==> 00:01:24.8400000 :  (singing in foreign language)
    7 ~. m; S; g2 q9 b8 Y
35. New Segment: 00:01:24.8400000 ==> 00:01:28.6400000 :  (panting)
    - V+ z% O0 ~# m9 @
36. New Segment: 00:01:36.7800000 ==> 00:01:39.2000000 :  (gun firing)
    
37. New Segment: 00:01:39.2000000 ==> 00:01:43.4600000 :  - Adrian.
    5 n( `9 x7 d7 j9 @
38. New Segment: 00:01:43.4600000 ==> 00:01:45.6200000 :  - Oh God.
    " R* b; E- |" Z1 l* G
39. New Segment: 00:01:45.6200000 ==> 00:01:48.2000000 :  - What's the matter sweetheart?
    ! x6 d. f, f- q: t8 z9 ?
40. New Segment: 00:01:48.2000000 ==> 00:01:50.4200000 :  Oh.
    + z  r5 ~6 P7 x5 ~, w6 A9 q2 u
41. New Segment: 00:01:50.4200000 ==> 00:01:53.4600000 :  - Oh it's horrible.
    & Q( e* Z' S* A9 w$ g( [
42. New Segment: 00:01:53.4600000 ==> 00:01:55.3000000 :  - Shh.
    
43. New Segment: 00:01:55.3000000 ==> 00:02:02.3400000 :  It was just a bad dream.
    
44. New Segment: 00:02:05.4200000 ==> 00:02:09.8800000 :  - You don't ever have to be afraid of anything.
    
45. New Segment: 00:02:09.8800000 ==> 00:02:12.8000000 :  I'll always be here to protect you.
    
46. New Segment: 00:02:12.9200000 ==> 00:02:15.5000000 :  (gentle music)
    
47. New Segment: 00:02:16.4800000 ==> 00:02:19.0600000 :  (gentle music)
    6 C1 F- [3 W+ P; _5 |6 E9 B
48. New Segment: 00:02:19.0600000 ==> 00:02:21.6400000 :  (gentle music)
    , t  a) ^+ Q" g/ K- f
49. New Segment: 00:02:21.6400000 ==> 00:02:24.2200000 :  (gentle music)
    
50. New Segment: 00:02:24.5400000 ==> 00:02:27.1200000 :  (gentle music)
    
51. New Segment: 00:02:27.1200000 ==> 00:02:29.7000000 :  (gentle music)
    2 X% v3 |. q8 w; A, g& l
52. New Segment: 00:02:29.7000000 ==> 00:02:33.1800000 :  [Music]
    

复制代码

& X3 G  q! S7 W% e# l% l1 c

星之韶华

学习学习一下