【游戏视频汉化 #1】 AI语音识别软件 Whisper编译

shane007

多年之前，就有为不带字幕的游戏视频配上字幕的想法。
7 [, H- @0 ?7 v0 W) v 但是当时条件不成熟，但是目前来看，条件似乎成熟了
8 {% H- I3 Y  L$ \. V: u7 N
3 l6 L" g0 D+ Q# [+ U Whisper是openAI的开源语音识别软件。
% o5 d& v6 ~. g- J) m$ s; z, d 它有一个.net的版本，在这个版本的基础上进行少量修改，就能将游戏视频对应的字幕识别成srt格式。
, |% }) ~6 o6 I 之后，对这个srt文件再进行在线批量翻译之后，进行少量调整之后，汉化工作就完成了。
0 _( ]: r' m$ C! L0 e. I
% S2 Z- B+ Q2 t9 f- O) K# y 地址如下
https://github.com/sandrohanea/whisper.net


& y# F/ ?) J5 j1 U+ {9 F$ \5 A 编译最好使用vs2022编译，否则在.net sdk版本上会出很多问题。

编译好之后，有几个注意点
2 ], U, f- l  m
) C! p& h) T, W$ F$ N( F* J <0>使用的模型文件修改为大模型，ggml-large.bin，用这个模型效果比较好。
    当然，所有时间也会比较多，估计转换一批文件需要几个甚至几十个小时。

<1>Language要设定为"english"。
( r1 L, c/ K" Z& p
9 ~/ V4 L0 V3 ?/ ~' U) I7 _0 W

1. /*    var builder = factory.CreateBuilder()
   
2.         .WithLanguage(opt.Language);*/
   % }( x" b, |8 ~+ M3 S: Z
3.     var builder = factory.CreateBuilder()
   8 Y# Q3 R2 E& p8 w/ C
4.     .WithLanguage("english");

复制代码

. k+ X, [! j$ [. u- K <2>缺省好像只支持Wav格式，而且是要16K采样率的，需要实现转换成这种格式，否则会出错。

<3>缺省只提供了一个例子wav文件的转换，需要改为批量形式。
  ], J7 D& J& r. d6 i! B   （遍历某个目录中的所有文件）

! {% [: n+ W9 c, J3 b: N( ^6 E <4>输出的文件，需要稍加整理，以符合srt格式

* w( w8 Q6 v% P4 ^5 b9 W   以下是一个Wav文件的控制台输出（幽魂开场动画）

1. 
   
2. whisper_init_from_file_no_state: loading model from 'ggml-large.bin'
   & [8 e' ?# i  r7 _  m' d# L
3. whisper_model_load: loading model
   
4. whisper_model_load: n_vocab       = 51865
   & h( ?8 Q/ d) R+ U- |8 C
5. whisper_model_load: n_audio_ctx   = 1500
   5 e8 ^# K" V% r& l
6. whisper_model_load: n_audio_state = 1280
   
7. whisper_model_load: n_audio_head  = 20
   
8. whisper_model_load: n_audio_layer = 32
   , \+ `: M: l) J+ }9 p; w$ x
9. whisper_model_load: n_text_ctx    = 448
   
10. whisper_model_load: n_text_state  = 1280
    2 j+ ^8 B, A6 O0 \: c5 N
11. whisper_model_load: n_text_head   = 20
    
12. whisper_model_load: n_text_layer  = 32
    
13. whisper_model_load: n_mels        = 80
    % V2 @- v# k) l8 f! `
14. whisper_model_load: ftype         = 1
    / g' r8 k5 V0 D( H7 C) C3 A
15. whisper_model_load: qntvr         = 0
    6 z2 {4 F7 c3 v5 A% l0 \
16. whisper_model_load: type          = 5
    
17. whisper_model_load: mem required  = 3557.00 MB (+   71.00 MB per decoder)
    
18. whisper_model_load: adding 1608 extra tokens
    ' g6 I! b0 |: x$ W- y  n
19. whisper_model_load: model ctx     = 2951.27 MB
    
20. whisper_model_load: model size    = 2950.66 MB
    
21. whisper_init_state: kv self size  =   70.00 MB
    
22. whisper_init_state: kv cross size =  234.38 MB
    
23. New Segment: 00:00:00 ==> 00:00:02.7600000 :  (birds chirping)
    
24. New Segment: 00:00:03.6600000 ==> 00:00:05.9000000 :  (exhaling)
    ! R: c  W4 a5 B% h8 l9 J( g
25. New Segment: 00:00:05.9000000 ==> 00:00:08.6600000 :  (birds chirping)
    
26. New Segment: 00:00:08.6600000 ==> 00:00:35.1200000 :  (gun firing)
    * g4 D- P7 j9 X; a) k' Y
27. New Segment: 00:00:36.1200000 ==> 00:00:38.5400000 :  (gun firing)
    2 W$ [' D3 p* u( v7 A6 D2 c
28. New Segment: 00:00:39.0600000 ==> 00:00:41.4800000 :  (gun firing)
    
29. New Segment: 00:00:41.4800000 ==> 00:00:49.4000000 :  (tires screeching)
      c3 Y, w. \$ A) j6 n% H" m% t# ^
30. New Segment: 00:00:49.4000000 ==> 00:00:58.5800000 :  (glass shattering)
    & G- T% c$ x' H9 {5 ~
31. New Segment: 00:00:58.5800000 ==> 00:01:07.7400000 :  (singing in foreign language)
    7 N% I2 d4 W  g! h( I
32. New Segment: 00:01:07.7400000 ==> 00:01:11.5800000 :  (singing in foreign language)
    
33. New Segment: 00:01:11.5800000 ==> 00:01:17 :  (tires screeching)
    
34. New Segment: 00:01:17 ==> 00:01:24.8400000 :  (singing in foreign language)
    $ H# X0 w; E7 r0 {  ]
35. New Segment: 00:01:24.8400000 ==> 00:01:28.6400000 :  (panting)
    
36. New Segment: 00:01:36.7800000 ==> 00:01:39.2000000 :  (gun firing)
    
37. New Segment: 00:01:39.2000000 ==> 00:01:43.4600000 :  - Adrian.
    7 f, u* g  r2 f
38. New Segment: 00:01:43.4600000 ==> 00:01:45.6200000 :  - Oh God.
    
39. New Segment: 00:01:45.6200000 ==> 00:01:48.2000000 :  - What's the matter sweetheart?
    6 j$ S& n7 J/ f/ M# r4 B9 b
40. New Segment: 00:01:48.2000000 ==> 00:01:50.4200000 :  Oh.
    ' H6 \  S( A9 A5 F# P8 [
41. New Segment: 00:01:50.4200000 ==> 00:01:53.4600000 :  - Oh it's horrible.
    ! |6 M, W4 l  z$ t% U. g8 x7 N
42. New Segment: 00:01:53.4600000 ==> 00:01:55.3000000 :  - Shh.
    2 v( m: J* g2 j# _. e
43. New Segment: 00:01:55.3000000 ==> 00:02:02.3400000 :  It was just a bad dream.
    $ m; s$ A- b6 N  @4 S" k# j
44. New Segment: 00:02:05.4200000 ==> 00:02:09.8800000 :  - You don't ever have to be afraid of anything.
    1 d  I  c1 B' i7 k0 y* F8 b: R
45. New Segment: 00:02:09.8800000 ==> 00:02:12.8000000 :  I'll always be here to protect you.
    # c4 V# h- B& p, j! u% w3 S' _
46. New Segment: 00:02:12.9200000 ==> 00:02:15.5000000 :  (gentle music)
    
47. New Segment: 00:02:16.4800000 ==> 00:02:19.0600000 :  (gentle music)
    
48. New Segment: 00:02:19.0600000 ==> 00:02:21.6400000 :  (gentle music)
    ; v- {- s8 @6 G$ B
49. New Segment: 00:02:21.6400000 ==> 00:02:24.2200000 :  (gentle music)
    
50. New Segment: 00:02:24.5400000 ==> 00:02:27.1200000 :  (gentle music)
    
51. New Segment: 00:02:27.1200000 ==> 00:02:29.7000000 :  (gentle music)
    9 J" O# l, j; `- s8 A
52. New Segment: 00:02:29.7000000 ==> 00:02:33.1800000 :  [Music]
    " f+ u9 ?& E9 ^' U0 ~

复制代码

1 J" q0 Q- z$ m9 x