# How Does AI Noise Cancellation Work? A Practical Breakdown

URL: https://anyvoice.app/journal/how-does-ai-noise-cancellation-work
Type: blog
Locale: en
Published: 2026-07-20
Updated: 2026-07-20

---

> A frame-by-frame breakdown of how AI noise cancellation actually works, from gain masks to RNNoise, DeepFilterNet, and Krisp latency tradeoffs.

How does AI noise cancellation work? A neural network runs on small audio frames, usually 10 to 40 milliseconds each, and for every frame it predicts which parts of the signal are voice and which are noise, then passes through only the voice. That's the whole trick, at least at the conceptual level. The engineering that makes it happen fast enough for a live call, without turning your voice into a robot, is where the real differences between tools show up. This piece walks through the signal chain frame by frame, compares the three architectures you'll actually run into (RNNoise, DeepFilterNet, Krisp), and flags where the technology still falls apart.

## What's actually happening between your mic and the listener's ear

Your microphone captures one continuous waveform: your voice, the fridge hum, the neighbor's leaf blower, the click of your mechanical keyboard, all mixed into a single signal. A noise suppression model doesn't try to identify and subtract each of those sources individually. Instead, it works frame by frame, estimating a gain mask, essentially a volume knob per frequency band, that it applies before the audio leaves your device.

Traditional noise gates use a fixed threshold: below X dB, mute. AI-based suppression replaces that fixed rule with a model trained on thousands of hours of paired clean and noisy audio. It learns what voice looks like across pitch, accent, and recording condition, and applies that learned pattern in real time instead of a static cutoff.

The practical result: a fixed threshold either lets noise through during quiet speech or clips the tail end of your words. A trained model adapts the gain per band, per frame, which is why it holds up better on non-steady noise, someone talking in the background, a dog barking, a door slamming, than a classic gate ever could.

![Close-up of a condenser microphone with pop filter, illustrating the raw signal a noise cancellation model has to work from](https://fdzlnqpwsaniezitwiuw.supabase.co/storage/v1/object/public/cms-media/anyvoice/2026-07/52a1e1-detail-microphone.webp)

## The frame-by-frame loop: how the model tells voice from noise

Here's the loop, stripped to its parts: capture a frame, extract features (usually a spectrogram, since frequency content separates voice from most noise better than raw amplitude), run the model, get a gain value per frequency band, apply it, output the frame. Repeat 25 to 100 times a second depending on frame size.

Two architecture families dominate this space right now. Recurrent networks, like the GRU (Gated Recurrent Unit) core in RNNoise, process audio sequentially and carry a memory of recent frames, which helps with temporal patterns like a voice trailing off. Convolutional approaches extract spatial features from the spectrogram directly, which is closer to how CNNs handle images, and tend to generalize better across noise types they weren't explicitly trained on.

*The session note: the frame size is the actual lever most people ignore. A shorter frame (10ms) means lower latency but less context for the model to work with. A longer frame (20-40ms) gives the model more to reason about, which usually means cleaner output, at the direct cost of a delay you'll hear as a lag on a live call.*

That tradeoff, frame size against latency against quality, is the entire design space every noise cancellation tool operates in. Nobody escapes it. What differs is where each product chooses to sit.

## RNNoise vs DeepFilterNet vs Krisp: same problem, three different tradeoffs

RNNoise is the open-source baseline most people cite. It pairs classic signal processing with a compact GRU network, predicting gains across 22 frequency bands from 10-millisecond frames. The model file is a few hundred kilobytes, it runs comfortably on a single CPU core, and it compiles to WebAssembly for browser use. It's excellent on steady noise, a fan, HVAC hum, a hard drive whirring, but its single-gain-per-band design shows its age on harder cases: overlapping speech in the background, sudden clatter, heavy reverberation.

DeepFilterNet takes a two-stage approach: a first pass applies gains on perceptually-spaced bands, then a second stage runs a short multi-frame filter on the frequencies below roughly 5 kHz to reconstruct voice detail that a simple gain mask would otherwise lose. It measurably outperforms band-gain-only methods and is the quality leader among open-source options, but it needs more compute to get there: a real-time factor of 0.19 on a laptop CPU thread and 0.42 on a Raspberry Pi 4, against RNNoise's much lighter footprint. Added latency lands around 40ms.

Krisp runs a proprietary architecture on-device, also processing in 10-millisecond frames, at roughly 25ms of added latency for the full model (15ms for its lighter voice-isolation variant). The internal design isn't published, but the product tradeoff is clear: cross-platform consistency across Windows, macOS, Linux, Android, iOS, and browsers, with zero server round-trip, which matters for privacy as much as for latency. For context on why these numbers matter: ITU-T G.114 recommends keeping total one-way mouth-to-ear delay under 150ms for a conversation to feel natural, so even the slowest of these three options ([latency comparison in this deep dive](https://www.forasoft.com/learn/ai-for-video-engineering/articles-ai/real-time-noise-suppression-krisp-rnnoise-deepfilternet)) leaves plenty of headroom before a call starts feeling laggy.

## Why bidirectional filtering changes the calculus

Most built-in noise suppression, the kind baked into your laptop's OS or your video call app, only cleans your outgoing mic signal. It does nothing about the noise arriving from the other participants on the call. [Krisp's own documentation on the technique](https://krisp.ai/blog/active-noise-cancellation-technology-vs-ai-based-noise-cancellation-algorithms/) describes filtering both directions: your mic before it leaves your machine, and the incoming audio before it hits your speakers.

That distinction matters more than most explainers give it credit for. If you're recording a remote interview, or running a podcast session with a guest calling in from an airport lounge, one-directional suppression only solves half the problem. You'll clean your own signal and still have to deal with their background noise in post, or worse, live, with no clean way to separate it after the fact. Bidirectional processing catches the incoming noise before it's baked into the recording.

![Person on a video call wearing headphones in a busy cafe, the kind of noisy environment AI noise cancellation is built to handle](https://fdzlnqpwsaniezitwiuw.supabase.co/storage/v1/object/public/cms-media/anyvoice/2026-07/e90eed-ambiance-call-cafe.webp)

## Where AI noise cancellation still gets it wrong

Skip aggressive settings if your source material is music, ambient field recordings, or anything with sustained non-voice content you actually want to keep. These models are trained to isolate speech; anything else gets treated as noise and suppressed, including instrumentation, room tone you might want for editing, or a co-host's laugh in the far background that a producer would normally keep.

Overlapping speech is still the hardest case. When two people talk at once, even DeepFilterNet's two-stage approach struggles to cleanly separate them, because the model is picking gain values per frame, not identifying individual speakers. If your workflow involves multiple mics picking up crosstalk, source separation at the mic level (physical distance, directional pickup patterns) still beats anything a post-processing model can fix.

And there's a real ceiling on aggressiveness. A suppressor that removes 90 percent of the noise while keeping the voice sounding like a person beats one that removes 99 percent and leaves you sounding like you're talking through a tin can. Push any of these models past their comfortable operating range and you start hearing artifacts: a warbly, underwater quality on sibilants, or breath sounds getting chopped mid-word. If your recording sounds worse after suppression than before, you've overshot; dial it back rather than layering a second pass on top.

## AI noise cancellation vs traditional ANC: different tools, not competitors

Active Noise Cancellation, the kind in your headphones, is a different problem entirely, and worth separating clearly because the two get conflated constantly. ANC generates an inverted sound wave to physically cancel incoming ambient noise before it reaches your ear, a purely acoustic, hardware-dependent process that dates back to the 1970s and works best on steady, low-frequency sound like engine rumble. It reacts near-instantly because there's no model inference in the loop, just phase-inverted waveform generation.

AI noise cancellation solves a different problem: cleaning a voice signal for transmission or recording, working on higher-frequency, non-steady sources like speech and chatter that ANC was never designed to touch. One is about what reaches your ears. The other is about what reaches everyone else's. A good remote setup often uses both: ANC headphones to block your side, and AI suppression on your mic feed to clean what you're sending out.

![Overhead flat-lay of home studio recording gear: interface, cables, and notes, part of the chain noise suppression models process](https://fdzlnqpwsaniezitwiuw.supabase.co/storage/v1/object/public/cms-media/anyvoice/2026-07/d62d61-flatlay-studio-gear.webp)

## Before your next call: what's actually worth checking

Run a quick test before you rely on any of this for something that matters: record a sample with suppression on and one with it off, then listen back on headphones, not laptop speakers. Check for two things specifically: whether sibilants (s, sh, f sounds) hold up without a warble, and whether the tail end of your sentences gets clipped when you trail off quietly. Those two failure modes show up before anything else.

If you're on a budget or CPU-constrained device, RNNoise's WASM build is a legitimate free option for steady background noise. If you need the cleanest possible open-source result and have the compute headroom, DeepFilterNet's two-stage filtering is worth the extra latency. If you want something that works the same way across every platform you touch, with no server round-trip and no model to configure, that's the case for a commercial layer like Krisp running underneath whatever app you're already using.

None of these fix a bad mic position or a room with hard, reflective surfaces. Suppression is a repair step, not a substitute for treating the source. Get the signal right at capture and the model has less work to do, which is where clean output actually starts.

## FAQ

### Does AI noise cancellation work on music, or just voice?

Just voice, effectively. These models are trained to isolate speech, so anything else (instruments, ambient room tone, sustained non-voice sound) gets treated as noise and suppressed. Don't run it on music tracks or field recordings you want to keep intact.

### Why does my voice sound underwater when I max out the suppression?

You've pushed the model past its comfortable operating range. Aggressive suppression distorts sibilants and can chop breath sounds mid-word. A suppressor that removes 90 percent of the noise while keeping your voice sounding like a person beats one that removes 99 percent and leaves you sounding robotic. Dial it back rather than stacking a second pass.

### What's the actual latency added by AI noise suppression?

It varies by architecture: RNNoise adds around 10ms, Krisp around 25ms (15ms for its lite variant), and DeepFilterNet around 40ms due to its two-stage filtering. For reference, ITU-T G.114 recommends staying under 150ms total one-way delay for a call to feel natural, so all three leave headroom.

### Is RNNoise still worth using in 2026, or should I switch to DeepFilterNet?

RNNoise is still the right call for tight compute budgets, browser deployment, or steady background noise like a fan or HVAC hum. DeepFilterNet's two-stage filtering measurably outperforms it on harder cases (overlapping speech, sudden clatter) but needs more CPU headroom to run in real time.

### Does Krisp process audio in the cloud, or on my device?

On-device. There's no server round-trip for the noise cancellation itself, which is why it holds a consistent ~25ms latency across platforms and doesn't raise the privacy concerns that a cloud pass would.

### Can AI noise cancellation fix overlapping speech from two people talking at once?

Not cleanly. The model predicts a gain value per frame, not per speaker, so it can't separate two overlapping voices the way it separates voice from non-voice noise. If crosstalk is a recurring problem, fix it at the mic level with distance or directional pickup rather than expecting post-processing to solve it.

### What's the difference between AI noise cancellation and the ANC in my headphones?

ANC is acoustic: it generates an inverted sound wave to physically cancel ambient noise before it reaches your ear, and it's built for steady, low-frequency sound like engine rumble. AI noise cancellation is a software model cleaning a voice signal for transmission or recording. They solve different problems and often get used together.