Hardware vs Software Video Encoding: When Each One Wins
Hardware vs Software Video Encoding: When Each One Wins
The traditional advice is simple and increasingly misleading: hardware for speed, software for quality.
It was clearly true a decade ago, when early fixed-function encoders produced visibly worse output at any bitrate. It is much less true now. Recent generations of NVENC, Quick Sync and Apple's media engine produce results that are hard to distinguish from a mid-preset software encode at reasonable bitrates — and they do it 10 to 50 times faster while barely touching the CPU.
The useful version of the rule is more specific: the quality gap scales inversely with bitrate. Starve the encoder and software wins clearly. Give it room and the difference shrinks toward invisibility.
This guide covers what each approach actually is, where the gap is real, and how to choose per job.
What They Actually Are
Software encoding runs entirely on the CPU. An encoder like x264, x265 or SVT-AV1 is a program, so it can be arbitrarily sophisticated — evaluate every prediction mode, run multi-pass rate control, apply psychovisual optimisations tuned to how human vision works, spend as long as you let it. The preset control exists precisely to trade time for thoroughness.
Hardware encoding runs on a dedicated block of silicon whose job is video encoding and nothing else:
- NVIDIA NVENC — a separate engine on the GPU die, independent of the CUDA cores, so encoding does not compete with rendering.
- Intel Quick Sync — part of the integrated graphics, present on most Intel CPUs.
- AMD AMF / VCN — AMD's equivalent.
- Apple VideoToolbox — the framework exposing the media engine on Apple Silicon, with dedicated H.264, HEVC and ProRes blocks. Pro, Max and Ultra chips include multiple engines.
The critical structural point: hardware encoders implement a fixed set of decisions in circuitry. They cannot be given more time to think, because the pipeline is physical. When you pick a "quality" preset in a hardware encoder, you are choosing between a small number of pre-baked configurations, not asking it to search harder.
This is also why hardware encoders improve in generational jumps rather than continuously. NVENC on an RTX 40-series card is materially better than on a GTX 10-series card, because it is different silicon. x264 improved through a decade of software refinement on the same hardware.
The Quality Gap, Honestly
Rough guidance at 1080p H.264:
| Bitrate | Practical difference |
|---|---|
| Very low (1–3 Mbps) | Software clearly better — visible blocking and smearing from hardware |
| Typical streaming (4–8 Mbps) | Close. Recent hardware ≈ x264 medium. Needs freeze-frame comparison |
| Generous (10–20 Mbps) | Effectively indistinguishable in normal viewing |
| Very high / near-lossless | Indistinguishable |
The mechanism behind that table: at low bitrates, the encoder must make hard choices about what to discard, and better analysis produces better choices. At high bitrates there is enough budget to represent the frame well regardless, so sophisticated analysis has less to contribute.
Two additional weaknesses worth knowing about, because they show up in specific content:
Rate control on complex scenes. Software encoders with multi-pass or well-tuned CRF distribute bits across a video more intelligently. Hardware encoders can spike or starve on abrupt complexity changes — the classic case being a sudden scene change or a burst of motion, where a hardware encode momentarily softens in a way a good software encode does not.
Psychovisual tuning. x264 in particular includes optimisations tuned to human perception rather than mathematical error, preserving film grain and fine texture that pure error-minimisation would smooth away. Hardware encoders generally do not implement equivalents, which is why hardware encodes of grainy film can look slightly "plasticky" at moderate bitrates.
Neither is a reason to avoid hardware encoding. Both are reasons to give it a somewhat higher bitrate than you would give x264 — which is very often a good trade, because you get the file in a fiftieth of the time.
The Speed Difference Is Not Subtle
This is where hardware wins so decisively that it reframes the whole comparison.
Rough throughput for a 1080p H.264 encode:
| Encoder | Typical speed |
|---|---|
x264 veryslow | Well below real time |
x264 medium | ~1–3× real time |
x264 veryfast | ~5–10× real time |
| NVENC / Quick Sync / VideoToolbox | ~10–50× real time |
A one-hour 1080p video encoded with x264 at slow can take several hours. The same file through a hardware encoder often finishes in a few minutes. At 4K the gap widens further, because software encoding cost scales with pixel count while hardware encoders have fixed-throughput pipelines.
There is a second dimension that matters as much on laptops: power. Saturating every CPU core for an hour heats the machine, spins the fans, throttles other work and drains the battery. A hardware encode runs on a small, purpose-built block at a fraction of the power. On an Apple Silicon MacBook the practical difference between the two approaches is not just minutes versus hours — it is whether the machine remains usable while it works.
Choosing Per Job
Live streaming and screen recording → hardware
Not really a debate. You need real-time encoding while the machine simultaneously runs the game or app being captured. Software encoding competes directly with that workload for the same cores, which is exactly how you get dropped frames and stuttering gameplay.
Hardware encoding moves the load onto silicon that is otherwise idle. Use NVENC, Quick Sync or VideoToolbox and give it a slightly higher bitrate than you would give x264. See OBS recording settings for concrete configuration.
The single exception is a two-machine setup where a dedicated capture PC does nothing but encode. There, software encoding costs you nothing.
Archival masters and distribution encodes → software
If you are producing the version that will be stored for years, or the master a platform will re-encode from, spend the time. x265 at a slow preset produces the smallest file at a given quality, and storage saved is saved permanently. The hours are paid once.
Which software encoder is a separate question — see x264 vs x265.
Low-bitrate delivery → software
Anywhere bandwidth is genuinely scarce, better analysis translates directly into better-looking video. This is the case where the quality gap is most visible.
Batch transcoding a library → usually hardware
Converting hundreds of files is the case where the speed difference compounds. A library that takes a week in x265 finishes in an afternoon on hardware. Give up a few percent of efficiency and get the job done.
Editing proxies and intermediates → hardware
Proxies exist to be fast and are deleted afterwards. Quality-per-bit is irrelevant. See how to create proxy files.
Apple Silicon Specifics
Worth calling out, because Mac users are in an unusually good position here.
Apple Silicon includes a dedicated media engine handling H.264, HEVC and ProRes encode and decode. Pro, Max and Ultra chips ship additional engines, which is why higher-tier chips transcode dramatically faster despite similar CPU performance on other tasks.
Two things follow:
VideoToolbox HEVC is genuinely good. Apple's HEVC encoder is well regarded relative to other hardware encoders, and for typical delivery bitrates it is a reasonable default rather than a compromise.
ProRes encoding is hardware-accelerated on Pro/Max/Ultra chips, which is a large deal for editors — it makes ProRes practical as a working format on a laptop.
The practical Mac guidance: use VideoToolbox for anything that is not an archival master. The efficiency you give up is small; the time and battery you save is not. Background on the formats is in HEVC vs H.264 and what video encoding is.
Quick Reference
| Hardware | Software | |
|---|---|---|
| Speed | 10–50× real time | 0.2–10× real time |
| CPU load | Minimal | Very high |
| Power draw | Low | High |
| Quality at low bitrate | Weaker | Better |
| Quality at generous bitrate | Comparable | Marginally better |
| Tunability | Limited presets | Extensive |
| Improves via | New hardware generation | Software updates |
| Best for | Streaming, batch, proxies, laptops | Masters, low bitrate, archival |
You Usually Do Not Have to Choose Manually
Most of the time the right answer is contextual — hardware for the routine job, software when quality per bit genuinely matters — and constantly making that call by hand is more overhead than it is worth.
Compresto handles it on macOS. It uses Apple's media engine where that is the right trade and falls back to software encoding where it is not, so a folder of 4K video compresses in minutes without you reasoning about encoders at all. You choose a target size or quality level; it picks the path. Everything runs locally — files are never uploaded — and it handles images and PDFs alongside video.
If you want the controls, HandBrake and FFmpeg expose every encoder discussed here. If you want the result, this is the shorter path.
Frequently Asked Questions
Is hardware encoding good enough for YouTube?
Yes. YouTube re-encodes everything you upload, so your file is an intermediate rather than the final product. Upload a high-bitrate hardware encode and the result is indistinguishable from a slow software encode that took twenty times longer. See best video format for YouTube.
Does hardware encoding reduce file size as much as software?
At the same visual quality, hardware encodes are typically somewhat larger — commonly in the region of 10–30% depending on encoder generation, content and bitrate. At the same bitrate, files are the same size and the difference shows as quality instead.
Can I use both?
Yes, and it is a sensible workflow. Use hardware for drafts, proxies and anything time-sensitive, then run a software encode for the final master. Most tools let you switch encoders per job.
Which hardware encoder is best?
Broadly, recent NVIDIA NVENC and Apple's media engine lead, with Intel Quick Sync competitive on recent generations. The gap between encoder generations is usually larger than the gap between vendors, so a current AMD GPU will beat an old NVIDIA one.
Does hardware encoding support HEVC and AV1?
HEVC is supported by all current hardware encoders. AV1 encoding is present on recent NVIDIA RTX 40-series and newer, Intel Arc, and recent AMD GPUs, but is not yet universal. Apple Silicon currently offers AV1 hardware decode on recent chips rather than encode. See AV1 vs H.265.
Why does my hardware encode look blocky?
Almost always the bitrate is too low. Hardware encoders need more headroom than x264 to reach the same quality. Raise the bitrate 20–40% and the problem usually disappears.