Transcribe MP4 Video with Hello Transcribe (and add captions/subtitles)
Introduction
You can now transcribe MP4 video files with Hello Transcribe 2.7 (get it on the App Store).
This works just like audio files — you can open an MP4 file with the Document picker in the app, share an MP4 file with Hello Transcribe on your iPhone/iPad, or drag & drop a file on macOS.
N.B. Hello Transcribe can also translate any video to English, so you can use it to add English subtitles to a video with a non-English audio language.
Here’s a MP4 I’ve transcribed:
And the results below. If you click or tap the “subtitles” button you can export the SRT (or VTT) file.
SRT is a standard subtitle file format which you can use to add closed captions to a hosting platform like YouTube or Vimeo, or embed captions in the video using ffmpeg. See below.
Upload SRT subtitles to your hosting platform
The SRT file can be uploaded to YouTube or Vimeo to add subtitles or closed captions. Here is the same video with the uploaded SRT file (turn on subtitles/closed captions):
You can do the same for German, French etc.
Note that YouTube has it’s own free auto-generated closed captions, whereas on Vimeo you need to pay. Hello Transcribe does a slightly better job of transcribing “Bhagavad Gita”:
Embed subtitles in a video using ffmpeg
If you’re on macOS you can embed subtitles in a video using ffmpeg. I recommend you use HomeBrew to install it.
Step 1 - Check your ffmpeg has libass support
Run "ffmpeg -version" on the command line and confirm that "--enable-libass" is output:
$ ffmpeg -version
ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
built with Apple clang version 14.0.3 (clang-1403.0.22.14.1)
configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/6.0 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox --enable-neon
Step 2 - Use Hello Transcribe
Use Hello Transcribe to transcribe the video and save the subtitles in SRT format as described above.
Step 3 - Use ffmpeg to create a new video file
Assuming the video has filename "oppenheimer.mp4" and the SRT file is "oppenheimer.srt", the command below will create a video "oppenheimer_with_subtitles.mp4" with embedded subtitles:
$ ffmpeg -i oppenheimer.mp4 -vf subtitles=oppenheimer.srt oppenheimer_with_subtitles.mp4
And here is the result:
Conclusion
I had this feature on the backlog but it turned out to be super simple so I added it to the app quickly before reworking the Whisper models.
Building subtitles into the app itself (on all platforms) will take some more effort but it’s coming. Also support for more formats like .MOV.
If you find an MP4 that doesn’t work let me know!