AES-GCM and why a nonce must never repeat

What a nonce is

AES-GCM encrypts a message and protects it from changes in one step. Each encryption uses a secret key, a nonce and the message. A nonce is a "number used once". It can be public, but it must be new for every message that you encrypt with the same key.

This post uses .NET code to show what breaks when a nonce repeats. Then it shows two safe ways to make nonces.

What the nonce does in GCM

AES is a block cipher. With a key, it turns one 16-byte block into another. NIST defines GCM (Galois/Counter Mode) in Special Publication 800-38D (opens in a new tab) (SP 800-38D). GCM has two parts. Counter mode hides the message. GHASH uses a secret hash key, called H, to turn the ciphertext and the lengths into one 16-byte value. GCM then hides that value with a tag mask, and the result is the authentication tag. The tag is a short check value that lets the receiver detect any change.

The usual GCM nonce is 12 bytes (96 bits), the length that NIST recommends. GCM puts a 4-byte counter after it to make a 16-byte counter block. AES encrypts each counter block with the key, and the output is the keystream. GCM combines the keystream with the message using XOR (exclusive or), which gives 1 where two bits differ. The result is the ciphertext.

The counter starts at 1, and that first block is kept for the tag. The message uses counters 2, 3, 4 and so on, as Figure 1 shows. RFC 7714, section 16.1.1 (opens in a new tab), lists these blocks byte by byte in a test vector, which is a published example with exact inputs and the exact outputs they must give. A short program that I do not show here checks .NET AesGcm against it, and the ciphertext and the tag match.

Only the key and the nonce go into the keystream. The message does not. So one key and one nonce always give the same keystream.

Same nonce, same keystream

The program below reuses a nonce on purpose. It encrypts two messages with one key and one nonce. Then it combines the two ciphertexts with XOR.

// Shows what leaks when two messages use the same key and the same nonce.
// Run with: dotnet run nonce-reuse.cs
using System.Security.Cryptography;
using System.Text;

byte[] key = RandomNumberGenerator.GetBytes(32);   // a new AES-256 key
byte[] nonce = RandomNumberGenerator.GetBytes(12); // one 96-bit nonce...
using var aes = new AesGcm(key, tagSizeInBytes: 16);

byte[] p1 = Encoding.ASCII.GetBytes("Send 0100 to Ann");
byte[] p2 = Encoding.ASCII.GetBytes("Send 0250 to Bob");
byte[] c1 = new byte[16], c2 = new byte[16], tag = new byte[16];

aes.Encrypt(nonce, p1, c1, tag);
aes.Encrypt(nonce, p2, c2, tag); // ...used twice: this is the bug

byte[] cx = Xor(c1, c2);
byte[] px = Xor(p1, p2);
if (!cx.AsSpan().SequenceEqual(px))
    throw new InvalidOperationException("c1 XOR c2 should equal p1 XOR p2");

Console.WriteLine($"c1 XOR c2: {Convert.ToHexString(cx)}");
Console.WriteLine($"p1 XOR p2: {Convert.ToHexString(px)}");

// Anyone who knows p1 can now read p2 without the key.
Console.WriteLine($"p2 is:     {Encoding.ASCII.GetString(Xor(cx, p1))}");

static byte[] Xor(byte[] a, byte[] b)
{
    var result = new byte[a.Length];
    for (int i = 0; i < a.Length; i++) result[i] = (byte)(a[i] ^ b[i]);
    return result;
}

c1 XOR c2: 0000000000000305000000000003010C
p1 XOR p2: 0000000000000305000000000003010C
p2 is:     Send 0250 to Bob

The key and the nonce are random, yet the first two lines are the same on every run. Both ciphertexts hold the same keystream, and XOR of a value with itself is zero. So the keystream drops out, and the XOR of the two messages is left.

Each zero byte shows a place where the two messages match. If an attacker knows or guesses one message, they can read the other one, as the last line shows. RFC 5116, section 5.1.1 (opens in a new tab), lists this leak of the messages as the first result of nonce reuse in GCM. XOR with a repeated key causes the same trouble elsewhere. In Why a WebSocket client masks its frames, four known bytes of text give away a whole 4-byte masking key.

The tag breaks too

The second result is worse, because it affects every message under that key. In 2006, Antoine Joux sent NIST comments on the GCM draft. He described a "forbidden attack" with a repeated nonce. It recovers H, the secret hash key of GHASH. With H, an attacker can change any message they have seen under that key and compute a new valid tag for it.

Take a one-block message, which is a message of exactly 16 bytes, with no associated data. Associated data is extra data that the tag protects but does not encrypt, such as a header. The tag of that one-block message has this form:

tag = C*H^2 + L*H + mask

Here, + is XOR and * is multiplication in the GCM field, a set of 128-bit values with its own rules. C is the ciphertext block, and L is a block that holds the lengths. The value mask is AES of the counter block with counter 1.

Two messages of the same length with the same key and nonce share L and mask. So XOR of the two tags removes both:

tag1 + tag2 = (C1 + C2) * H^2

The attacker knows every value here except H. They solve for H squared, then take the square root to get H. Next, they compute the mask from one tag. Now they can make a valid tag for any one-block message sent with that nonce.

A short program, not shown here, runs this attack against .NET AesGcm. The attacker side sees the nonce, both ciphertexts, both tags and the first message. It never sees the key.

field multiply matches RFC 7714: True
attacker found the hash key H:   True
receiver accepted and decrypted: Send 9999 to Eve
attack worked in 1000 of 1000 new runs
with two different nonces:       (rejected)

Line one checks the field math against RFC 7714. Line two compares the found H with the real one, AES of a zero block under the key. In line three, Decrypt checks the forged tag with the real key and accepts a message that the key owner never wrote. Then the program repeats the attack 1,000 times with new random keys and nonces. To check the test itself, the program then runs the same steps on two messages with different nonces. There the receiver rejects the forged message.

Longer messages give an equation of higher degree, which can have more than one possible value for H. Usually there are only a few, and Joux shows how a second pair of messages with a repeated nonce narrows them down. SP 800-38D gives the same warning in Appendix A.

What the standards ask for

Section 8 of SP 800-38D sets the rule. IV (initialization vector) is the NIST name for the nonce.

The probability that the authenticated encryption function ever will be invoked with the same IV and the same key on two (or more) distinct sets of input data shall be no greater than 2^-32.

NIST SP 800-38D, section 8

The standard allows two ways to build IVs, and a key must use only one of them.

In .NET, AesGcm.NonceByteSizes allows only 12 bytes, the 96-bit form. For the tag, NIST allows 128, 120, 112, 104 or 96 bits, and 64 or 32 bits for special cases. In 2024, NIST announced its plan to remove tags under 96 bits in a revision. Create AesGcm with the constructor that takes tagSizeInBytes. On Windows and Linux it accepts 12 to 16 bytes, and on macOS only 16. RFC 5116 uses 16 for AES-GCM, so use 16 everywhere. The constructors without a tag size have been obsolete since .NET 8.

Two safe ways to make nonces

A counter per key

A counter is the simplest safe nonce. For 96-bit nonces, NIST suggests a 32-bit fixed field and then a 64-bit counter. The fixed field names the sender, so two senders that share a key need different fixed fields.

// A counter nonce for one sender and one key (NIST SP 800-38D, section 8.2.1):
// a 4-byte fixed field for the sender, then an 8-byte counter.
// Run with: dotnet run counter-nonce.cs
using System.Buffers.Binary;
using System.Security.Cryptography;

byte[] key = RandomNumberGenerator.GetBytes(32);
using var sender = new CounterNonceSender(key, fixedField: 7);

for (int i = 0; i < 3; i++)
{
    EncryptedMessage message = sender.Encrypt("hello"u8);
    Console.WriteLine($"nonce: {Convert.ToHexString(message.Nonce)}");
}

record EncryptedMessage(byte[] Nonce, byte[] Ciphertext, byte[] Tag);

sealed class CounterNonceSender(byte[] key, uint fixedField) : IDisposable
{
    private readonly AesGcm _aes = new(key, tagSizeInBytes: 16);
    private readonly Lock _gate = new();
    private ulong _counter; // never goes back while this key is in use

    public EncryptedMessage Encrypt(ReadOnlySpan<byte> plaintext)
    {
        lock (_gate)
        {
            if (_counter == ulong.MaxValue)
                throw new InvalidOperationException("Counter used up. Use a new key.");

            var nonce = new byte[12];
            BinaryPrimitives.WriteUInt32BigEndian(nonce, fixedField);
            BinaryPrimitives.WriteUInt64BigEndian(nonce.AsSpan(4), _counter++);

            var ciphertext = new byte[plaintext.Length];
            var tag = new byte[16];
            _aes.Encrypt(nonce, plaintext, ciphertext, tag);
            return new EncryptedMessage(nonce, ciphertext, tag);
        }
    }

    public void Dispose() => _aes.Dispose();
}

nonce: 000000070000000000000000
nonce: 000000070000000000000001
nonce: 000000070000000000000002

Without the lock, two threads could read the same counter value and build the same nonce. If the key survives a restart, the counter must survive it too. RFC 5116, section 3.1 (opens in a new tab), suggests saving a counter value ahead of the ones in use. It also notes that a new key often helps when nonces are hard to coordinate.

TLS 1.3, a version of the protocol behind HTTPS, works this way. TLS sends data in records. Each direction keeps a 64-bit record counter, which the RFC calls the sequence number. It starts at zero for every new key. The nonce is that counter XOR a fixed value from the key setup (RFC 9846, section 5.3 (opens in a new tab)). The key usually comes from a key exchange, as in Key exchange in plain words, with X25519 or A first look at ML-KEM.

A counter keeps the nonces unique, but a key should still be changed after a set amount of data. TLS 1.3 does this before 2^24.5 full-size records (RFC 9846, section 5.5 (opens in a new tab)).

Random nonces with a budget

A counter is hard to share when many servers use one key and do not talk to each other. Then you can pick each nonce at random. NIST allows this when at least 96 bits come from an approved random bit generator. In .NET, use RandomNumberGenerator. Microsoft Learn names it as the class for cryptographically secure random numbers, which are numbers that are safe to use for secrets such as passwords and keys.

Random values can collide. With q random 96-bit nonces, the chance that any two are equal is less than q² / 2⁹⁷. Section 8.3 limits a key to 2³² messages with random nonces. NIST says that this limit and 96 random bits are enough to meet the rule. The sample prints this bound and counts the messages for each key.

// Random 96-bit nonces with a message budget per key.
// NIST SP 800-38D, section 8.3: at most 2^32 messages per key with random nonces.
// Run with: dotnet run random-nonce.cs
using System.Security.Cryptography;

// Chance that any two of q random 96-bit nonces match: at most q(q-1)/2 * 2^-96.
foreach (int log2q in new[] { 30, 32 })
{
    double q = Math.Pow(2, log2q);
    string chance = $"2^{Math.Log2(q * (q - 1) / 2) - 96:F1}";
    Console.WriteLine($"2^{log2q} messages: chance of a repeat is at most {chance}");
}

using var sender = new RandomNonceSender(RandomNumberGenerator.GetBytes(32));
EncryptedMessage message = sender.Encrypt("hello"u8);
Console.WriteLine($"nonce: {Convert.ToHexString(message.Nonce)} (new every run)");

record EncryptedMessage(byte[] Nonce, byte[] Ciphertext, byte[] Tag);

sealed class RandomNonceSender(byte[] key) : IDisposable
{
    // My own margin: a quarter of the NIST limit.
    public const long MaxMessages = 1L << 30;

    private readonly AesGcm _aes = new(key, tagSizeInBytes: 16);
    private readonly Lock _gate = new();
    private long _used;

    public EncryptedMessage Encrypt(ReadOnlySpan<byte> plaintext)
    {
        lock (_gate)
        {
            if (_used == MaxMessages)
                throw new InvalidOperationException("Budget used up. Use a new key.");
            _used++;

            byte[] nonce = RandomNumberGenerator.GetBytes(12);
            var ciphertext = new byte[plaintext.Length];
            var tag = new byte[16];
            _aes.Encrypt(nonce, plaintext, ciphertext, tag);
            return new EncryptedMessage(nonce, ciphertext, tag);
        }
    }

    public void Dispose() => _aes.Dispose();
}

2^30 messages: chance of a repeat is at most 2^-37.0
2^32 messages: chance of a repeat is at most 2^-33.0
nonce: 445D744A7F1C36AD77B51CD7 (new every run)

At 2³² messages the bound is 2^-33, inside the rule of 2^-32. The sample stops at 2³⁰ messages, which is my own margin, and there the bound is 2^-37. When the budget runs out, get a new key. The last line of the output changes on every run.

The sample counts the messages of one object in one process, and the count starts at 0 again after a restart. The limit of 2³² messages covers every sender and every restart that uses the key. If several servers share one key, each server's budget must be the global budget divided by the number of servers, or each server needs its own key. If the key outlives the process, save the count too.

When you cannot keep a counter

Some systems cannot promise unique nonces, for example many writers that share one key but no state. For them, RFC 8452 (opens in a new tab) describes AES-GCM-SIV, which is "nonce misuse resistant". If a nonce repeats, the only thing that leaks is whether two messages were equal. The cost is speed: encryption needs two passes over the data.

RFC 8452 still recommends random nonces, so treat its protection as a safety net. At the time of writing, the .NET API reference on Microsoft Learn has no AES-GCM-SIV class. With AesGcm, use one of the two patterns above.

Before you ship

1. Create AesGcm with tagSizeInBytes: 16.
2. Choose one nonce method for each key: a counter or random values. Do not mix them.
3. Guard a counter with a lock. Never let it go back while the key is in use, even after a restart.
4. Take random nonces from RandomNumberGenerator, count the messages, and change the key well before 2³² messages. If several servers share a key, split the budget between them.
5. Never write a fixed nonce into your code.
6. Change the key after a set amount of data, even with a counter, as TLS 1.3 does.
7. Test your code against a published test vector, such as RFC 7714, section 16.1.1.

Read more

• NIST SP 800-38D (opens in a new tab), Galois/Counter Mode (GCM) and GMAC, mainly sections 8 and 8.3 and Appendix A
• NIST to revise SP 800-38D (opens in a new tab), the planned change to tag lengths
• Authentication Failures in NIST version of GCM (opens in a new tab), comments by Antoine Joux on the forbidden attack
• RFC 5116 (opens in a new tab), an interface and algorithms for authenticated encryption, sections 3.1 and 5.1.1
• RFC 7714 (opens in a new tab), with an AES-GCM test vector in section 16.1.1 that lists every step
• RFC 8452 (opens in a new tab), AES-GCM-SIV, nonce misuse resistant encryption
• RFC 9846 (opens in a new tab), TLS 1.3, with the per-record nonce in section 5.3
• AesGcm class (opens in a new tab) on Microsoft Learn, with the constructors and the nonce and tag sizes
• Key exchange in plain words, with X25519, one way to get the key
• A first look at ML-KEM, the post-quantum key exchange, another way to get the key