The FFmpeg wiki estimates that you should be able to produce video files of similar video quality as H.264 that are about half the size.
ffmpeg -c:v libx265 -preset fast -crf 28 -tag:v hvc1 -c:a eac3 -b:a 224k -i <source> <target>
Or for GPU-based encoding, which is much faster but produces a larger file size:
ffmpeg -vcodec hevc_videotoolbox -b:v 6000k -tag:v hvc1 -c:a eac3 -b:a 224k -i <source> <target>
libx265 encodes via the CPU while hevc_videotoolbox uses the GPU.
libx265 produces a much smaller file size, but hevc_videotoolbox runs significantly faster (5 to 10 times the speed often).
For hevc_videotoolbox a target bitrate is specified with -b:v 6000k. Trying to get away without doing that, regardless of the selected video profile (main or main10), unless a higher bitrate was forced, resulted in the the output video quality being garbage; totally unwatchable, with visual artifacts and blurriness everywhere. Specifying a target video bitrate works around the problem, at the cost of producing an even larger final video.
The right compromise is to use libx265 for any videos that you want to keep around, but for videos that you’d prefer encoded quickly and will probably delete (say you want them for a single trip), hevc_videotoolbox is perfectly fine. Output videos are still small enough, and the significantly faster encoding speeds mean that FFmpeg finishes far more quickly.
-c:a eac3 tells FFmpeg to re-encode video using Dolby Digital Plus. You might wonder why I would ever choose that over AAC, and the answer is simple: Apple.
Dolby Digital Plus is strictly worse than AAC, so you should prefer the latter in most situations. However, if you’re encoding video for QuickTime (or any Apple target), you have to consider that thanks to the company’s opaque quirks, their devices support AAC in stereo only. They support multi-channel audio, but only if it uses a “blessed” multi-channel codec like Dolby Digital Plus 1.
You might want to change this parameter if (1) your input sources are just stereo anyway, or (2) you never expect to watch the output video on anything but a stereo device (i.e., headphones, TV minus sound system). It’s trivial to have FFmpeg downmix to stereo AAC instead:
-c:a aac -ac 2
The libx265 preset setting accepts the wide array of adjectives ultrafast, superfast, veryfaster, faster, fast, medium, slow, slower, veryslow, and placebo. Specifying a faster setting means that encoding speed will be preferred over file size. Both ends of the spectrum have extreme diminishing returns.
There is generallyt very little file size difference between fast, medium, and slow, but some encoding speed difference, so I just default to -preset fast for everything.
The argument -tag:v hvc1 tags the video with hvc1, which is purely for QuickTime’s benefit. It allows this Very Stupid Player to recognize the fact that it will be able to play the resulting file.
Why QuickTime won’t open the encoded files: It isn’t the video Apple didn’t like, it is the 5.1 AAC.