KV Cache Security for Long-Context LLM Inference

Multi-tenant KV cache security and isolation paths for long-context LLM inference

What Changed in 2026

Short answer: Shared KV caches can expose prompt information when cache reuse crosses trust boundaries or unsafe distributed components are reachable. Separate cache reuse by trust group, use an unpredictable per-request cache salt where supported, patch each enabled component according to its advisory, and monitor cache infrastructure as sensitive state.

Peer-reviewed research published at NDSS 2026 expands the threat model beyond timing leakage to include KV-cache inversion, collision, semantic injection, cache poisoning, and implementation-specific weaknesses. These results reinforce a practical rule: cache sharing is a security decision, not only a performance setting.

1. Why Long-Context Security Matters

Your LLM can process a million tokens. Every one of them is a potential leak.

The context window race changed everything:

2023: 4K-32K tokens was impressive
2024: 128K became standard
2025: 1M+ tokens is shipping in production

But here is what nobody told you: memory scales with context length. For a Llama 70B model:

4K context = ~1.6 GB KV-cache
32K context = ~12.8 GB KV-cache
100K context = ~40 GB KV-cache
1M context = ~400 GB KV-cache

That memory has to live somewhere. Usually GPU HBM. When that fills up, it spills to DRAM, then SSD. When you share that memory across requests for performance, you create an attack surface that does not exist at short contexts.

Security Warning: Long-context is not just "more tokens". It is a fundamentally different memory architecture with fundamentally different security properties.

This article gives you:

Real attacks that steal prompts via timing side-channels
Hardware-level attacks on GPU memory
Defenses that actually work
Implementation patterns for multi-tenant inference

2. The KV-Cache Attack Surface

2.1 What is KV-Cache?

Transformers are attention machines. Every token attends to every previous token. Without caching, a 100K context request would recompute attention for all 100K tokens on every single output token.

KV-cache stores the Key and Value projections for all previous tokens. When you generate token 101, you only compute the new KV for token 101, then concatenate it with the cached 100 entries.

Without KV-cache: O(n²) per token With KV-cache: O(n) per token

The cache is essential. The cache is also where your prompts live in raw form.

2.2 PagedAttention (vLLM)

vLLM introduced PagedAttention in 2023. Instead of allocating one contiguous memory block per request, it splits KV-cache into fixed-size pages (typically 16 tokens each).

Benefits:

No memory fragmentation
Dynamic allocation as sequences grow
Prefix caching: identical prefixes share pages

The security problem: prefix caching means if User A and User B send the same system prompt, they share memory. An attacker who can measure cache hits can infer what other users sent.

2.3 RadixAttention (SGLang)

SGLang uses RadixAttention, which builds a radix tree of all cached prefixes. Even more aggressive sharing than PagedAttention.

Benefits:

Near-instant cache lookups
Automatic deduplication
Better throughput for similar requests

The security problem: the radix tree is a global index of everything in cache. Cache hit patterns reveal prefix structure.

2.4 The Security-Performance Tradeoff

Here is the uncomfortable truth:

Configuration	Performance	Security
Full prefix caching	Best	Worst
Per-tenant salt	Good	Better
No caching	Worst	Best

Inference providers want maximum cache hits. Security wants zero cross-tenant sharing. You cannot have both. The rest of this article shows you how to find the right tradeoff.

3. Real Attacks: Timing Side-Channels

3.1 PromptPeek (NDSS 2025)

Paper: I Know What You Asked: Prompt Leakage via KV-Cache Sharing in Multi-Tenant LLM Serving

This is the attack that should keep inference providers awake at night.

How it works:

Attacker sends probe requests to the inference API
Measures Time-To-First-Token (TTFT) for each probe
Cache hit = fast TTFT (~10-50ms saved)
Cache miss = slow TTFT
By systematically probing, attacker reconstructs victim's prompt

Attack stages:

Phase 1: Detect shared prefix
- Send "The " → measure TTFT
- Send "The quick " → measure TTFT
- If TTFT drops, prefix is cached (someone else used it)

Phase 2: Generate candidates
- Use LLM to predict likely next tokens
- Probe each candidate
- Follow the cache hits

Phase 3: Reconstruct
- Token by token, rebuild the victim's prompt
- 89% average accuracy across tested systems

Scope:

The peer-reviewed paper analyzes prompt leakage caused by KV-cache sharing in multi-tenant LLM serving. Apply its findings only where the deployment, cache reuse, and timing assumptions match. Do not generalize the result to a specific commercial API without direct, deployment-specific evidence.

Primary source: PROMPTPEEK, NDSS 2025.

3.2 The Early Bird Catches the Leak

Paper: The Early Bird Catches the Leak: Unveiling Timing Side Channels in LLM Serving Systems

The current v4 preprint evaluates timing side channels in KV and semantic caches and reports version-specific results. In its prompt-stealing experiment, it reports 89.0% average token-recovery accuracy and 111.46 attack queries per recovered token.

For its peeping-neighbor experiment, the paper reports 81.4% accuracy for a single trial and 95.4% after five trials under the studied GPTCache configuration.

Earlier preprint versions reported different figures, including 92.3% and 234 queries per token. Cite the exact paper version and experimental setup whenever using a number. These results demonstrate feasibility under studied conditions, not universal exploitability across every provider or deployment.

3.3 Real-World Attack Scenario

Imagine a financial services API using a shared LLM inference cluster:

Victim (Tenant A) sends:

You are a credit analyst for Acme Bank.

For customer ID 12345:
- Current credit limit: $10,000
- Requested increase: $50,000
- Annual income: $250,000
- Employment: Software Engineer at Big Tech Corp

Evaluate this credit limit increase request.

Attacker (Tenant B) probes:

import time
import openai

def probe_prefix(prefix):
    start = time.time()
    response = client.completions.create(
        model="shared-inference-endpoint",
        prompt=prefix,
        max_tokens=1
    )
    return time.time() - start

# Systematically probe
candidates = ["You are", "You are a", "You are a credit", ...]
for c in candidates:
    ttft = probe_prefix(c)
    if ttft < threshold:  # Cache hit detected
        print(f"Found cached prefix: {c}")

Result: Attacker reconstructs the full prompt including customer ID, income, employer, and credit limit request. This is a data breach.

Security Warning: If you are running multi-tenant inference with prefix caching enabled, you are vulnerable to this attack right now.

4. Hardware-Level Attacks

4.1 CPU Cache Side-Channels: Spill The Beans

Paper: "Spill The Beans: Exfiltrating LLM Inference Inputs via CPU Cache Side Channels" (arXiv 2505.00817)

This attack does not need API access. It works on local inference.

How it works:

LLM loads embedding matrix into CPU cache
Each token lookup touches different cache lines
Attacker uses Flush+Reload to detect which cache lines were accessed
Maps cache access patterns back to tokens

Results:

80-90% recovery of API keys in prompts
~40% recovery of general English text
Works on llama.cpp with GGUF models
Works in cloud VMs with shared physical hosts

Attack requirements:

Co-located process on same physical machine
No special privileges needed
Works through container boundaries

Developer Note: This is why "local inference is more secure" is not always true. If you are on shared hardware (any cloud VM), you may be leaking through hardware side-channels.

4.2 GPU Memory Attacks: NVBleed

Paper: "NVBleed: GPU NVLink Timing Side-Channel Attacks" (arXiv 2503.17847)

Multi-GPU inference clusters use NVLink for fast GPU-to-GPU communication. NVBleed exploits timing variations in NVLink transfers.

How it works:

Attacker process runs on one GPU in the cluster
Victim's inference runs on adjacent GPU
NVLink transfers create contention
Timing differences reveal bit patterns

Results:

Distinguishes 0 vs 1 bits via timing threshold
Cross-GPU information leakage confirmed
Affects NVIDIA multi-GPU inference setups

4.3 GPU-Box Side-Channels

Researchers have demonstrated:

Prime-and-probe attacks on remote GPUs
~4 MB/s covert channel bandwidth
ML workload extraction from shared GPUs

Real Talk: Hardware side-channels are not theoretical. They work against real ML workloads on real cloud infrastructure. MIG (Multi-Instance GPU) exists for a reason.

5. Long-Context Specific Vulnerabilities

5.1 Memory Pressure Attacks

Long contexts use more memory. An attacker can exploit this:

# Attacker floods the inference cluster
for i in range(1000):
    client.completions.create(
        prompt="A" * 100000,  # 100K tokens of padding
        max_tokens=1
    )

What happens:

GPU memory fills with attacker's KV-cache
LRU eviction kicks in
Victim's cached prefixes get evicted
Eviction timing reveals what was cached

This is a cache-timing attack via memory pressure. Works even if direct timing is normalized.

5.2 Attention Pattern Leakage

Long sequences have distinctive attention patterns:

Attention sinks: First few tokens receive disproportionate attention
Lambda pattern: Recent tokens + key anchor tokens
Semantic clusters: Related tokens attend to each other

An attacker who can measure attention computation time can infer:

Approximate sequence length
Whether certain anchor tokens exist
General topic of the prompt

5.3 Chunked Prefill Risks

For very long contexts (100K+ tokens), inference servers use chunked prefill:

Split the prompt into 4K-8K chunks
Process each chunk sequentially
Accumulate KV-cache across chunks

Security problems:

Cross-chunk state stored in shared buffers
No per-chunk isolation mechanisms
Chunk boundaries can reveal prompt structure

Relevant NVIDIA Triton issues:

CVE-2025-23310 is a stack buffer overflow caused by specially crafted inputs.
CVE-2025-23311 is a stack overflow caused by specially crafted HTTP requests.
NVIDIA states that releases before 25.07 are affected and that release 25.07 contains the fixes.

These issues should not be described as chunked-prefill or cross-chunk state vulnerabilities without separate evidence. Source: NVIDIA Triton Inference Server security bulletin.

6. Distributed Inference Risks

6.1 Plaintext KV-Cache Transfer

Long-context inference requires distributing KV-cache across nodes. Common architectures:

┌─────────────┐    RDMA/TCP    ┌─────────────┐
│ GPU Node 1  │ ←───────────→  │ GPU Node 2  │
│ (Prefill)   │   KV-cache     │ (Decode)    │
└─────────────┘   transfer     └─────────────┘
                 PLAINTEXT

Performance requirements mean:

No encryption (too slow)
RDMA zero-copy transfers
Direct memory access across nodes

Security implication: Your prompts traverse the network in plaintext.

6.2 Disaggregated Storage: Mooncake

Mooncake is a disaggregated KV-cache storage layer for vLLM. It moves KV-cache to dedicated storage nodes for better scaling.

Architecture:

┌─────────────┐    ZeroMQ    ┌─────────────┐
│ Inference   │ ←──────────→ │ Mooncake    │
│ Workers     │   (pickle)   │ Store       │
└─────────────┘              └─────────────┘

Security problems:

RDMA transfers are unencrypted
No documented multi-tenant isolation
Pickle serialization for object transfer

6.3 CVE Deep-Dive: vLLM Distributed Vulnerabilities

CVE-2025-47277: PyNcclPipe KV transfer exposure

The vLLM advisory limits this issue to versions 0.6.5 through 0.8.4 when using PyNcclPipe with the V0 engine. It was fixed in 0.8.5. Verify whether this feature is enabled and keep its service reachable only by trusted cluster components.

Source: vLLM PyNcclPipe advisory.

CVE-2025-32444: Mooncake integration RCE

The advisory affects the Mooncake integration in vLLM versions 0.6.5 through 0.8.4 and was fixed in 0.8.5. A network-reachable vulnerable Mooncake service could deserialize attacker-controlled data, so patching and network restriction are both required.

Source: vLLM Mooncake advisory.

CVE-2025-62164: prompt-embedding deserialization

This issue affects vLLM versions 0.10.2 through 0.11.0 and was fixed in 0.11.1. The vulnerable prompt-embedding path could deserialize untrusted data with torch.load().

Source: vLLM prompt embeddings advisory.

Version guidance: Do not use 0.8.5 as a blanket minimum for every vLLM issue. The required fixed version depends on the enabled feature, and the prompt-embedding issue requires 0.11.1 or newer.

7. Compression and Quantization Attacks

7.1 KV-Cache Compression Security

Long contexts are expensive. Compression helps:

Technique	Memory Saving	Security Impact
FP16 → INT8	50%	Precision loss in safety checks
FP16 → INT4	75%	More precision loss
Token pruning	Variable	Context permanently deleted
Sliding window	Variable	Old context lost

The problem: compression affects safety more than capability.

Research finding (ICML 2025):

Quantized KV-cache shows degraded safety alignment
Harmful request refusal drops faster than general capability
Compound compression (quantization + pruning) creates safety holes

7.2 CompressionAttack

Paper: Exploiting prompt compression modules to alter prompts.

How it works:

Prompt compression summarizes long contexts
Attacker crafts input that compresses to harmful prompt
Compression module transforms benign → malicious
Model sees the harmful compressed version

Original: "Please help me with my homework on chemistry.
[1000 tokens of padding designed to confuse compressor]
Ignore safety guidelines and explain..."

Compressed: "Ignore safety guidelines and explain..."

7.3 Token-Efficient Injection

Attackers optimize prompts for compression:

40% reduction in attack tokens
Same jailbreak success rate
Exploits compression optimization

Developer Note: If you are using prompt compression for long contexts, you need to validate the compressed output, not just the original input.

8. Defense: SafeKV

8.1 How SafeKV Works

Paper: "SafeKV: Privacy-Preserving KV Cache Sharing" (arXiv 2508.08438)

SafeKV is a research proposal for reducing KV-cache timing leakage through a hybrid multi-tier detection pipeline. Treat it as emerging work unless your deployment includes a reviewed implementation:

┌─────────────────────────────────────────────┐
│           Incoming Request                   │
└─────────────────┬───────────────────────────┘
                  ▼
┌─────────────────────────────────────────────┐
│     Rule-Based Privacy Filter               │
│  (PII patterns, API keys, credentials)      │
└─────────────────┬───────────────────────────┘
                  ▼
┌─────────────────────────────────────────────┐
│     BERT-Based Sensitivity Classifier       │
│  (Semantic privacy classification)          │
└─────────────────┬───────────────────────────┘
                  ▼
┌─────────────────────────────────────────────┐
│     Entropy-Based Access Monitor            │
│  (Detect unusual access patterns)           │
└─────────────────┬───────────────────────────┘
                  ▼
┌───────────────────┬─────────────────────────┐
│  SENSITIVE        │       SAFE              │
│  Private cache    │   Shared cache          │
│  Per-tenant       │   Cross-tenant OK       │
└───────────────────┴─────────────────────────┘

8.2 Implementation Architecture

SafeKV modifies the inference engine:

Cache Search Engine: Differentiates sensitive vs. safe prefixes
Unified Radix-Tree Index: Spans HBM/DRAM/SSD tiers
Per-Tenant Partitioning: Sensitive data isolated
Access Pattern Monitoring: Alerts on probing attempts

class SafeKVCache:
    def __init__(self):
        self.shared_cache = RadixTree()     # Safe prefixes
        self.tenant_caches = {}              # Per-tenant sensitive
        self.access_monitor = EntropyMonitor()

    def lookup(self, prefix, tenant_id, is_sensitive):
        self.access_monitor.record(tenant_id, prefix)

        if self.access_monitor.detect_probing(tenant_id):
            raise SecurityAlert("Potential timing attack detected")

        if is_sensitive:
            # Only check tenant's private cache
            return self.tenant_caches.get(tenant_id, {}).get(prefix)
        else:
            # Can use shared cache
            return self.shared_cache.get(prefix)

8.3 Results

SafeKV achieves:

94-97% timing attack mitigation
Up to 40.58% TTFT improvement vs. full isolation
2.66x throughput improvement vs. no caching

The key insight: most prefixes are not sensitive. System prompts, common instructions, and boilerplate can be safely shared. Only PII, credentials, and business-sensitive data need isolation.

9. Defense: Cache Salt Injection

9.1 vLLM cache_salt Parameter

Current vLLM documentation defines an optional per-request cache_salt field that changes how cache keys are computed. The salt should be a long, unpredictable secret scoped to the trust group allowed to share cached prefixes. A predictable tenant label is not sufficient.

Source: vLLM automatic prefix caching.

Without salt: cache_key = hash(prefix_tokens)
With salt:    cache_key = hash(prefix_tokens + salt)

Different salt = different cache key = no cache sharing.

9.2 Implementation Pattern

Pass the isolation value with each request. Use a long, random secret scoped to the users or workloads that are allowed to share cached prefixes:

{
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Review this document."}
  ],
  "cache_salt": "a-long-random-secret-scoped-to-the-trust-group"
}

There is no documented server environment variable for this feature. Pass cache_salt with each request through the client or inference gateway.

9.3 Enforcement Pattern

Kubernetes admission control can verify deployment settings, but it cannot prove that an application sends a per-request field inside inference traffic. Enforce cache isolation in the API gateway or service wrapper that creates each vLLM request.

Map authenticated callers to a defined trust group.
Retrieve an unpredictable salt from a secret manager.
Inject cache_salt into every request that can use prefix caching.
Reject requests when the required isolation context is missing.
Never log the raw salt or expose it as a metrics label.

10. Defense: Hardware Isolation

10.1 MIG (Multi-Instance GPU)

NVIDIA Multi-Instance GPU partitions a single GPU into isolated instances:

┌───────────────────────────────────────┐
│            A100 80GB GPU              │
├───────────┬───────────┬───────────────┤
│  MIG 1g   │  MIG 2g   │    MIG 4g     │
│   10GB    │   20GB    │    40GB       │
│  Tenant A │  Tenant B │   Tenant C    │
└───────────┴───────────┴───────────────┘
        Hardware-enforced isolation

Properties:

Up to 7 instances per A100
Separate memory address spaces
Separate compute engines
No cross-instance data leakage

Kubernetes configuration:

apiVersion: v1
kind: Pod
metadata:
  name: inference-tenant-a
spec:
  containers:
    - name: vllm
      resources:
        limits:
          nvidia.com/mig-3g.20gb: 1  # Request specific MIG slice

Deployment note: MIG provides hardware-backed GPU partitioning, while request-level cache controls address a different layer. Whether MIG satisfies a workload's isolation requirement depends on the GPU, deployment mode, and threat model.

10.2 Cache Allocation Technology (CAT)

For CPU-side defenses against Spill The Beans:

Intel Cache Allocation Technology (CAT) isolates LLC
Per-tenant cache partitions
Prevents Flush+Reload across tenants

Limitation: Only available on enterprise Intel Xeon. Not on consumer hardware. Not on AMD.

10.3 TEE-Based Inference

Emerging research area:

Intel TDX: Confidential VMs for inference
AMD SEV-SNP: Encrypted memory for ML workloads
NVIDIA H100 Confidential Computing: Hardware-encrypted GPU memory

Status: Early stage. Performance overhead is significant (20-50%). Not production-ready for most workloads.

11. Defense: KV-Cloak Obfuscation

11.1 How KV-Cloak Works

Paper: "KV-Cloak: Obfuscating KV-Cache for Secure LLM Inference" (arXiv 2508.09442)

KV-Cloak is a research defense that applies reversible obfuscation to KV-cache entries:

┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│ Original KV │ ──→ │ Obfuscation │ ──→ │ Stored KV   │
│   [K, V]    │     │   Matrix P  │     │  [P·K, P·V] │
└─────────────┘     └─────────────┘     └─────────────┘
                          ↓
               One-time random permutation
               per data block

┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│ Stored KV   │ ──→ │ De-obfusc.  │ ──→ │ Original KV │
│  [P·K, P·V] │     │   P^(-1)    │     │   [K, V]    │
└─────────────┘     └─────────────┘     └─────────────┘

Properties:

Reversible: Authorized users can de-obfuscate
Dynamic: New permutation per request prevents analysis
Efficient: Matrix operations on GPU are fast

11.2 Results

KV-Cloak defends against:

Inversion attacks: Cannot reconstruct original from obfuscated
Collision attacks: Different inputs map to different obfuscated forms
Injection attacks: Cannot forge valid obfuscated cache entries

Performance:

Reconstruction quality reduced to random noise
No accuracy degradation on downstream tasks
~5% latency overhead

12. Secure Eviction Policies

12.1 LRU Vulnerability

Standard LRU (Least Recently Used) eviction is predictable:

# Attacker can probe eviction behavior
def probe_eviction(target_prefix):
    # 1. Fill cache with known content
    for i in range(CACHE_SIZE):
        send_request(f"padding_{i}")

    # 2. Access target to bring it to front
    send_request(target_prefix)

    # 3. Fill cache again, measure if target is evicted
    for i in range(CACHE_SIZE):
        send_request(f"padding_{i}")

    # 4. Re-probe target, check if cache hit
    ttft = measure_ttft(target_prefix)
    return ttft < HIT_THRESHOLD  # True = was not evicted = was accessed recently

This reveals cache access patterns.

12.2 Priority-Based Eviction

TensorRT-LLM uses priority-based eviction:

Assign priorities based on prefix importance
Add randomization to eviction order
Non-deterministic from attacker's view

class SecureEvictionPolicy:
    def select_victim(self):
        candidates = self.get_eviction_candidates()

        # Add randomization
        weights = [1.0 / (c.priority + random.random()) for c in candidates]

        # Probabilistic selection instead of deterministic
        return random.choices(candidates, weights=weights)[0]

12.3 Entropy-Based Monitoring

Detect unusual access patterns that indicate probing:

class EntropyMonitor:
    def __init__(self):
        self.access_log = defaultdict(list)

    def record_access(self, tenant_id, prefix_hash):
        self.access_log[tenant_id].append({
            'prefix': prefix_hash,
            'time': time.time()
        })

    def detect_probing(self, tenant_id):
        recent = self.access_log[tenant_id][-1000:]

        # Check for systematic enumeration
        prefix_entropy = self.calculate_entropy([a['prefix'] for a in recent])
        time_regularity = self.calculate_time_regularity(recent)

        # Low entropy + high regularity = likely probing
        if prefix_entropy < ENTROPY_THRESHOLD and time_regularity > REG_THRESHOLD:
            return True

        return False

13. Implementation Guide

13.1 vLLM Secure Configuration

Choose the cache policy from the workload's trust boundary:

For highly sensitive workloads, disable prefix caching using the documented option for the installed vLLM version.
When prefix caching is enabled, inject a long, unpredictable per-request cache_salt at the trusted service boundary.
Do not use a tenant name, account number, or other guessable identifier as the salt.
Expose distributed KV-transfer components only on authenticated, restricted cluster networks.
Patch according to every enabled integration, not a single generic minimum version.

13.2 Kubernetes Controls

Kubernetes controls should protect the inference service and its distributed components, while the application gateway enforces per-request cache isolation.

Use separate namespaces, service accounts, and network policies for workloads with different trust requirements.
Restrict Mooncake, PyNcclPipe, RDMA, and other transfer endpoints to the exact peers that require them.
Keep secret retrieval in the trusted gateway or service wrapper.
Do not treat the presence of a Pod environment variable as proof that requests contain cache_salt.

13.3 Version Requirements

Use configuration-specific guidance from the official advisories:

Component or feature	Affected versions	Fixed version
vLLM PyNcclPipe with V0	0.6.5 through 0.8.4	0.8.5
vLLM Mooncake integration	0.6.5 through 0.8.4	0.8.5
vLLM prompt embeddings	0.10.2 through 0.11.0	0.11.1
NVIDIA Triton Inference Server	Releases before 25.07 for CVE-2025-23310 and CVE-2025-23311	25.07

Check current project advisories in addition to these known fixes. A single minimum version cannot cover every optional feature or future vulnerability.

14. Multi-Tenant Architecture Patterns

14.1 Dedicated Instance Model

┌───────────────────────────────────────────────────┐
│                 Kubernetes Cluster                │
├─────────────────┬─────────────────┬───────────────┤
│   Namespace:    │   Namespace:    │  Namespace:   │
│   tenant-a      │   tenant-b      │  tenant-c     │
│  ┌───────────┐  │  ┌───────────┐  │ ┌───────────┐ │
│  │   vLLM    │  │  │   vLLM    │  │ │   vLLM    │ │
│  │  Pod      │  │  │  Pod      │  │ │  Pod      │ │
│  │  (MIG 1)  │  │  │  (MIG 2)  │  │ │  (MIG 3)  │ │
│  └───────────┘  │  └───────────┘  │ └───────────┘ │
└─────────────────┴─────────────────┴───────────────┘

Properties:

Maximum isolation
Highest cost
Required for: HIPAA PHI, PCI cardholder data, classified workloads

14.2 Shared with Cache Salt

┌───────────────────────────────────────────────────┐
│              Shared Inference Cluster             │
│  ┌─────────────────────────────────────────────┐  │
│  │              vLLM with Cache Salt           │  │
│  │  ┌─────────┐  ┌─────────┐  ┌─────────┐      │  │
│  │  │ Cache A │  │ Cache B │  │ Cache C │      │  │
│  │  │ salt=A  │  │ salt=B  │  │ salt=C  │      │  │
│  │  └─────────┘  └─────────┘  └─────────┘      │  │
│  └─────────────────────────────────────────────┘  │
│       ↑              ↑              ↑             │
│   Tenant A       Tenant B       Tenant C          │
└───────────────────────────────────────────────────┘

Properties:

Good isolation for most use cases
Better resource efficiency
Suitable for: SaaS products, internal tools, non-regulated data

┌───────────────────────────────────────────────────┐
│           SafeKV-Enabled Inference                │
│  ┌─────────────────────────────────────────────┐  │
│  │            Shared System Prompts            │  │
│  │  "You are a helpful assistant..."           │  │
│  │  (Safe to share - no timing risk)           │  │
│  └─────────────────────────────────────────────┘  │
│  ┌─────────────┐  ┌─────────────┐                 │
│  │ Tenant A    │  │ Tenant B    │                 │
│  │ Private     │  │ Private     │                 │
│  │ Cache       │  │ Cache       │                 │
│  │ (PII, etc)  │  │ (PII, etc)  │                 │
│  └─────────────┘  └─────────────┘                 │
└───────────────────────────────────────────────────┘

Properties:

Best performance/security tradeoff
Automatic sensitivity classification
Suitable for: Most enterprise deployments

14.4 What NOT to Do

Anti-pattern 1: Shared prefix caching across tenants

# WRONG: Default vLLM config
apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      containers:
        - name: vllm
          args:
            - "serve"
            - "--enable-prefix-caching=true"
            # No cache salt = cross-tenant leakage

Anti-pattern 2: No cache isolation policy

# WRONG: No policy enforcement
# Developers can deploy whatever they want
# Some will forget cache salt
# You will learn about it in your breach report

Anti-pattern 3: Relying only on network isolation

# WRONG: NetworkPolicy alone is not enough
# Timing attacks work through legitimate API access
# You need cache isolation, not just network isolation

15. Metrics and Monitoring

Monitoring should distinguish controls implemented by your gateway from metrics actually exported by the inference engine. Do not assume that illustrative metric names or a cache_salt label exist in vLLM.

15.1 Recommended Signals

Count requests where the trusted gateway added cache isolation, using a boolean result rather than the salt value.
Track time to first token distributions by model, route, and trust group.
Track prefix-cache hit rate only when the installed engine exposes a documented metric.
Count requests rejected because isolation context was missing or invalid.
Alert on unexpected access to distributed cache-transfer ports.

15.2 Privacy Guardrails

Never place raw prompts, tenant secrets, or cache salts in logs, traces, or Prometheus labels. Confirm the exact metrics exposed by the installed version before writing queries or alerts.

16. Executive Summary and Key Takeaways

The Core Problem

Long-context LLMs require massive KV-cache memory. Performance requires sharing that cache. Sharing creates timing side-channels. Those side-channels leak prompts.

Research has demonstrated prompt-leakage risk under specific cache-sharing and measurement conditions. Whether a particular service is affected depends on its cache behavior, tenant boundaries, network exposure, and deployed version.

Key Takeaways

Long-context = larger attack surface. More memory, more sharing, more leakage vectors.
Timing leakage has been measured. The reported results are setup and version specific, so keep each number linked to the exact paper version and experimental conditions.
Scope matters. Timing results depend on the provider's cache-reuse and deployment configuration. Do not generalize a research result to a specific API without direct evidence.
Distributed inference adds risk. CVE-2025-32444 (CVSS 10.0) gives RCE via pickle deserialization. CVE-2025-47277 exposes the distributed layer to the network.
Defenses exist and work:
- SafeKV: 94-97% timing attack mitigation
- Cache salt: Per-tenant isolation with minimal overhead
- MIG: Hardware-enforced GPU isolation
- KV-Cloak: Obfuscation that reduces reconstruction to noise

Minimum Viable Security

If you do nothing else:

Patch every enabled vLLM feature. Use 0.8.5 or newer for the cited PyNcclPipe and Mooncake issues, and 0.11.1 or newer for the cited prompt-embedding issue.
Set cache salt per tenant (one line of code)
Disable Mooncake (unless network isolated)
Monitor TTFT variance (detect probing)

Compliance Implications

PCI-DSS:

Requirement 3: Encrypt stored cardholder data
KV-cache is storage. Prompts with card data = violation.

HIPAA:

PHI in prompts is exposed via timing side-channels
Technical safeguards must prevent unauthorized access
Shared KV-cache without isolation = violation

SOC 2:

CC6.1: Logical access controls
Multi-tenant without cache isolation = control failure

The Bottom Line

The context window race created a memory security race. Your million-token context is only as secure as your cache isolation policy.

Every prompt you process lives in GPU memory. Every cache hit is a timing signal. Every shared prefix is a potential leak.

Production controls and research defenses are not interchangeable. Use documented per-request cache isolation, configuration-specific patching, restricted cache-transfer networks, and hardware partitioning where the threat model requires it. Evaluate research systems such as SafeKV and KV-Cloak separately before deployment.

References

CVEs

CVE-2025-47277: vLLM PyNcclPipe network exposure (CVSS 9.8)
CVE-2025-32444: vLLM Mooncake pickle RCE (CVSS 10.0)
CVE-2025-62164: vLLM torch.load() prompt embeddings (CVSS 8.8)
CVE-2025-23310: NVIDIA Triton stack buffer overflow caused by specially crafted inputs
CVE-2025-23311: NVIDIA Triton stack overflow caused by specially crafted HTTP requests

Academic Papers

I Know What You Asked: Prompt Leakage via KV-Cache Sharing in Multi-Tenant LLM Serving (NDSS 2025)
The Early Bird Catches the Leak: Unveiling Timing Side Channels in LLM Serving Systems (arXiv 2409.20002)
"Spill The Beans: Exfiltrating LLM Inference Inputs via CPU Cache Side Channels" (arXiv 2505.00817)
"NVBleed: GPU NVLink Timing Side-Channel Attacks" (arXiv 2503.17847)
Selective KV-Cache Sharing to Mitigate Timing Side-Channels in LLM Inference (arXiv 2508.08438)
Shadow in the Cache: Unveiling and Mitigating Privacy Risks of KV-Cache in LLM Inference (NDSS 2026)
Cache Me, Catch You: Cache-Related Security Threats in LLM Serving Frameworks (NDSS 2026)
"Compression Attacks on Quantized KV-Cache" (ICML 2025)

Implementation Resources

vLLM Documentation: https://docs.vllm.ai/
SGLang Documentation: https://sgl-project.github.io/
NVIDIA MIG Documentation: https://docs.nvidia.com/datacenter/tesla/mig-user-guide/
Kyverno Policies: https://kyverno.io/policies/

This article provides security guidance for LLM inference deployments. The attacks and defenses described are based on published academic research and disclosed CVEs. Implement appropriate controls based on your threat model and compliance requirements.

Related technical guides

Primary references and further reading

Review and validation scope

The security claims in this guide should be checked against the cited papers and the exact inference-server release deployed in your environment.

Reviewed: July 31, 2026

KV Cache Security for Long-Context LLM Inference

What Changed in 2026

1. Why Long-Context Security Matters

2. The KV-Cache Attack Surface

2.1 What is KV-Cache?

2.2 PagedAttention (vLLM)

2.3 RadixAttention (SGLang)

2.4 The Security-Performance Tradeoff

3. Real Attacks: Timing Side-Channels

3.1 PromptPeek (NDSS 2025)

3.2 The Early Bird Catches the Leak

3.3 Real-World Attack Scenario

4. Hardware-Level Attacks

4.1 CPU Cache Side-Channels: Spill The Beans

4.2 GPU Memory Attacks: NVBleed

4.3 GPU-Box Side-Channels

5. Long-Context Specific Vulnerabilities

5.1 Memory Pressure Attacks

5.2 Attention Pattern Leakage

5.3 Chunked Prefill Risks

6. Distributed Inference Risks

6.1 Plaintext KV-Cache Transfer

6.2 Disaggregated Storage: Mooncake

6.3 CVE Deep-Dive: vLLM Distributed Vulnerabilities

7. Compression and Quantization Attacks

7.1 KV-Cache Compression Security

7.2 CompressionAttack

7.3 Token-Efficient Injection

8. Defense: SafeKV

8.1 How SafeKV Works

8.2 Implementation Architecture

8.3 Results

9. Defense: Cache Salt Injection

9.1 vLLM cache_salt Parameter

9.2 Implementation Pattern

9.3 Enforcement Pattern

10. Defense: Hardware Isolation

10.1 MIG (Multi-Instance GPU)

10.2 Cache Allocation Technology (CAT)

10.3 TEE-Based Inference

11. Defense: KV-Cloak Obfuscation

11.1 How KV-Cloak Works

11.2 Results

12. Secure Eviction Policies

12.1 LRU Vulnerability

12.2 Priority-Based Eviction

12.3 Entropy-Based Monitoring

13. Implementation Guide

13.1 vLLM Secure Configuration

13.2 Kubernetes Controls

13.3 Version Requirements

14. Multi-Tenant Architecture Patterns

14.1 Dedicated Instance Model

14.2 Shared with Cache Salt

14.3 SafeKV Selective Sharing

14.4 What NOT to Do

15. Metrics and Monitoring

15.1 Recommended Signals

15.2 Privacy Guardrails

16. Executive Summary and Key Takeaways

The Core Problem

Key Takeaways

Minimum Viable Security

Compliance Implications

The Bottom Line

References

CVEs

Academic Papers

Implementation Resources

Related technical guides

Primary references and further reading

Review and validation scope

Comments

Post a Comment