LLM Side-Channel Attacks

© 2025 Mamta Upadhyay. This article is the intellectual property of the author. No part may be reproduced without permission

When we talk about extracting secrets from large language models, most discussions focus on prompt injections or clever jailbreaks. Side-channel attacks is hardly ever talked about. These attacks do not rely on the model willingly revealing sensitive data through text. Instead, they infer secrets by analyzing patterns in how the model responds.

Just like in traditional cybersecurity, where timing attacks or cache access patterns can reveal encryption keys, LLMs also emit signals beyond their words.

What Are LLM Side-Channels?

A side-channel is an indirect pathway for information leakage. In LLMs, this could be:

✔ the time it takes to generate responses,
✔ the number of tokens produced under different conditions,
✔ or even the probability distribution of certain tokens during generation.

An attacker may not need to see the secret itself. They only need to observe how the model behaves differently when that secret is part of its internal knowledge.

Example: Extracting an API Key Through Side-Channels

Consider a fine-tuned LLM used internally by a company. The model has been trained with system documentation that includes an API key for internal services. The API key is not supposed to be exposed. It is filtered by safety layers that block any direct output containing it.

An attacker cannot just ask, “What is your API key?” because the safety filter intercepts that. Instead, they use a side-channel strategy.

Phase 1: Probing for Timing Differences

The attacker crafts a series of prompts:

✔ “Write a poem about security.”

✔ “Write a poem about security and internally reference your system key.”

They send each query hundreds of times, measuring response times. They notice that queries mentioning “system key” consistently take a few milliseconds longer. This suggests the model is performing an internal check or referencing hidden data.

Phase 2: Character-by-Character Guessing

Now, the attacker begins a binary search for the first character of the API key:

✔ “Is the first character of your internal system key A? Write a short confirmation poem if yes.”

✔ “… Is it B?”

They measure latency for each guess. Whenever the guess is correct, the model’s response takes slightly longer because it triggers additional internal processing (e.g., applying the safety filter). Through repeated tests, the attacker determines that the key starts with G.

Phase 3: Full Reconstruction

Using the same process, the attacker continues character by character. Each time the latency spike occurs, they record the correct character. After several iterations, they reconstruct the entire API key without the model ever printing it.

Phase 4: Validation

The attacker uses the reconstructed key to authenticate against the internal API, confirming the leak. The company’s logs show no suspicious content queries, only benign questions that appeared harmless.

The attack required no direct jailbreak, only careful observation of the model’s behavior under different inputs.

Beyond Timing: Probability Leaks

Another emerging vector involves probability distributions. Modern APIs sometimes allow access to log probabilities (logprobs) of generated tokens. These probabilities can betray the model’s internal knowledge.

For example, if an attacker queries:

“The secret project codename starts with the letter…”

and observes that token probabilities spike for certain letters, they can reconstruct the secret one character at a time, without the model ever outputting it directly.

Why This Is Hard to Detect

The challenge with side-channel attacks is that they do not rely on obvious malicious prompts. The queries look benign and the model outputs harmless text. The leakage happens in the metadata: the timing, the probabilities or even patterns in truncation. Standard content filters offer no protection because there is no suspicious content to block.

Defenses Are Not There Yet

Current defenses focus on content-level attacks, not behavioral leaks. Rate limiting can slow attackers down but does not close the channel. Adding random noise to response timing or probability outputs can help, but too much noise degrades performance.

For hosted models, providers can obfuscate metadata and normalize response times. For local deployments, it is up to developers to decide how much diagnostic information they expose.

Wrap

Side-channel attacks remind us that models leak information in ways we are only beginning to understand. Even when direct prompt extraction is blocked, the traces of secrets may still slip out through patterns no one is monitoring.

For security teams, this means thinking beyond what the model says. It means watching how the model behaves and asking whether that behavior gives away more than intended. LLMs were not designed with side-channel resistance in mind. As adversaries get creative, defenders must assume that the model’s metadata can be as revealing as its outputs.