【游戏视频汉化 #1】 AI语音识别软件 Whisper编译

shane007

多年之前，就有为不带字幕的游戏视频配上字幕的想法。
但是当时条件不成熟，但是目前来看，条件似乎成熟了
! I  d  }- o' K+ d/ A- i$ k8 [
Whisper是openAI的开源语音识别软件。
" T# f% |) i! B4 |% M 它有一个.net的版本，在这个版本的基础上进行少量修改，就能将游戏视频对应的字幕识别成srt格式。
! X1 `' U. h) C* m4 `! w 之后，对这个srt文件再进行在线批量翻译之后，进行少量调整之后，汉化工作就完成了。
5 N  y3 C( X" m# h
5 k; [. `7 a  Z! o- ^9 y, V 地址如下
https://github.com/sandrohanea/whisper.net
( J3 o* [) ^2 F+ `8 W2 i$ a# k
  w4 M5 S$ ~6 b) c+ B7 t
- K4 Q: i3 e1 j 编译最好使用vs2022编译，否则在.net sdk版本上会出很多问题。

( I" s9 [! ]' g# X* x0 b 编译好之后，有几个注意点
- s' E- l" [! m6 T
, ^$ _6 ?6 ]) e% j/ p" _ <0>使用的模型文件修改为大模型，ggml-large.bin，用这个模型效果比较好。
    当然，所有时间也会比较多，估计转换一批文件需要几个甚至几十个小时。

<1>Language要设定为"english"。

6 d( E. }" Y/ C% K+ ?( ~

1. /*    var builder = factory.CreateBuilder()
   2 @4 o- @. d  |3 v# }/ n* a, }* ^
2.         .WithLanguage(opt.Language);*/
   
3.     var builder = factory.CreateBuilder()
   
4.     .WithLanguage("english");

复制代码

# Y2 S& z1 M* H2 a, s5 d <2>缺省好像只支持Wav格式，而且是要16K采样率的，需要实现转换成这种格式，否则会出错。
- Q0 K! m4 u* B. A/ w$ |
<3>缺省只提供了一个例子wav文件的转换，需要改为批量形式。
7 {5 q6 A% e) A6 \   （遍历某个目录中的所有文件）
- Q3 _" \% T/ T/ m9 d% ~, C
<4>输出的文件，需要稍加整理，以符合srt格式
  d, [% S/ H( N! Y0 Y
   以下是一个Wav文件的控制台输出（幽魂开场动画）
# A4 G8 X' Z6 E3 y- Q: V

1. 
   
2. whisper_init_from_file_no_state: loading model from 'ggml-large.bin'
   / X* k  e) p  w. P$ f1 ~
3. whisper_model_load: loading model
   9 g- r1 `. c8 {: j
4. whisper_model_load: n_vocab       = 51865
   
5. whisper_model_load: n_audio_ctx   = 1500
   8 a! e; h5 k6 W
6. whisper_model_load: n_audio_state = 1280
   7 b0 D0 v( b: j, N& @3 L
7. whisper_model_load: n_audio_head  = 20
   
8. whisper_model_load: n_audio_layer = 32
   , X! d1 G9 s- E% B- r
9. whisper_model_load: n_text_ctx    = 448
   - |$ [( H# l' @0 N
10. whisper_model_load: n_text_state  = 1280
    
11. whisper_model_load: n_text_head   = 20
    ! g& r! r* w6 ^; N- k
12. whisper_model_load: n_text_layer  = 32
    ' ~2 U% n- B: l
13. whisper_model_load: n_mels        = 80
    $ M+ e' c  c4 f" o. T
14. whisper_model_load: ftype         = 1
      A3 y- j- k: i* J
15. whisper_model_load: qntvr         = 0
    + v8 P9 S" R! g0 P' _" U
16. whisper_model_load: type          = 5
    
17. whisper_model_load: mem required  = 3557.00 MB (+   71.00 MB per decoder)
    
18. whisper_model_load: adding 1608 extra tokens
    
19. whisper_model_load: model ctx     = 2951.27 MB
    ( F/ C/ m8 C1 U) d. v" Z
20. whisper_model_load: model size    = 2950.66 MB
    ) V4 ^) ~8 Q5 f8 K% D* J. m
21. whisper_init_state: kv self size  =   70.00 MB
    - ]" C3 @3 m* p9 c
22. whisper_init_state: kv cross size =  234.38 MB
    9 a9 v. R/ u6 @' f4 I2 g
23. New Segment: 00:00:00 ==> 00:00:02.7600000 :  (birds chirping)
    2 y0 I+ `! J: ^0 [+ Q
24. New Segment: 00:00:03.6600000 ==> 00:00:05.9000000 :  (exhaling)
    
25. New Segment: 00:00:05.9000000 ==> 00:00:08.6600000 :  (birds chirping)
      h/ k$ K' t( b
26. New Segment: 00:00:08.6600000 ==> 00:00:35.1200000 :  (gun firing)
    
27. New Segment: 00:00:36.1200000 ==> 00:00:38.5400000 :  (gun firing)
    9 Q* E) M/ e" L" T( J7 b7 r. C8 j
28. New Segment: 00:00:39.0600000 ==> 00:00:41.4800000 :  (gun firing)
    0 U! p" Z% v+ d
29. New Segment: 00:00:41.4800000 ==> 00:00:49.4000000 :  (tires screeching)
    - ^8 a: B; @* h! n3 u3 p4 {, m* @
30. New Segment: 00:00:49.4000000 ==> 00:00:58.5800000 :  (glass shattering)
    ) c% w. O0 b/ z- r7 c, q
31. New Segment: 00:00:58.5800000 ==> 00:01:07.7400000 :  (singing in foreign language)
    
32. New Segment: 00:01:07.7400000 ==> 00:01:11.5800000 :  (singing in foreign language)
    ' u; c2 d: P8 M( h/ u, x4 x
33. New Segment: 00:01:11.5800000 ==> 00:01:17 :  (tires screeching)
    
34. New Segment: 00:01:17 ==> 00:01:24.8400000 :  (singing in foreign language)
    
35. New Segment: 00:01:24.8400000 ==> 00:01:28.6400000 :  (panting)
    % D1 j# P/ d  b' K/ x
36. New Segment: 00:01:36.7800000 ==> 00:01:39.2000000 :  (gun firing)
    
37. New Segment: 00:01:39.2000000 ==> 00:01:43.4600000 :  - Adrian.
    
38. New Segment: 00:01:43.4600000 ==> 00:01:45.6200000 :  - Oh God.
    ) r  \/ b. ?0 ?5 ?
39. New Segment: 00:01:45.6200000 ==> 00:01:48.2000000 :  - What's the matter sweetheart?
    
40. New Segment: 00:01:48.2000000 ==> 00:01:50.4200000 :  Oh.
    & @6 F: ?7 Z5 V; x. v; D
41. New Segment: 00:01:50.4200000 ==> 00:01:53.4600000 :  - Oh it's horrible.
    : [9 b7 Z$ {; h$ }+ x
42. New Segment: 00:01:53.4600000 ==> 00:01:55.3000000 :  - Shh.
    8 W* U9 N3 S9 T5 l% P
43. New Segment: 00:01:55.3000000 ==> 00:02:02.3400000 :  It was just a bad dream.
    9 O' u% n) T0 k* i
44. New Segment: 00:02:05.4200000 ==> 00:02:09.8800000 :  - You don't ever have to be afraid of anything.
    
45. New Segment: 00:02:09.8800000 ==> 00:02:12.8000000 :  I'll always be here to protect you.
    
46. New Segment: 00:02:12.9200000 ==> 00:02:15.5000000 :  (gentle music)
    
47. New Segment: 00:02:16.4800000 ==> 00:02:19.0600000 :  (gentle music)
    
48. New Segment: 00:02:19.0600000 ==> 00:02:21.6400000 :  (gentle music)
    6 Z; w9 ^# H9 c( Z( Q8 m
49. New Segment: 00:02:21.6400000 ==> 00:02:24.2200000 :  (gentle music)
    0 d  @. ?5 Y; y6 }9 e
50. New Segment: 00:02:24.5400000 ==> 00:02:27.1200000 :  (gentle music)
    4 h  V# s: N" ~; M
51. New Segment: 00:02:27.1200000 ==> 00:02:29.7000000 :  (gentle music)
    
52. New Segment: 00:02:29.7000000 ==> 00:02:33.1800000 :  [Music]
    2 P6 z+ _# u8 s0 C+ v! {) i9 x

复制代码

0 n! Q" q9 E! V# y4 {, R