When it comes to protecting an API from duplicate requests, the first thing that comes to mind is using idempotency. The client passes a unique key (X-Request-Id, nonce, or Idempotency-Key), and the server checks whether a request with that key has already been made. It sounds logical, but in practice, this doesn't prevent race conditions.
Let's figure out why.
What is idempotency and how does it typically work
Idempotency is a property of an operation such that calling it again with the same parameters doesn't change the result. That is, no matter how many times the request is executed, the effect is the same as if it had been executed only once.
This is typically implemented as follows:
- The client generates a unique request identifier, for example, abc-123.
- Sends it to the server in a header.
- Before executing the operation, the server checks the database (or cache) to see if the request with that key has already been made. - If the key is found, the server returns the saved result of the first execution.
- If the key is not found, it performs the operation, storing the key and the result.
At first glance, the problem seems solved: even if the client sends the same request twice, reprocessing won't occur.
Case 1: Two different requests with different keys
Idempotency works well against sending the same request twice—for example, when a user clicks a button twice or a network interruption causes the client to retry the request with the same key.
But what happens when two different requests (with different keys) attempt to perform the same logical operation simultaneously? For example, two different users (or two different threads of the same application) simultaneously want to debit the same balance.
- Request 1 with key abc-111 reads the balance: 100 rubles.
- Request 2 with key xyz-222 reads the balance: 100 rubles (because the first one hasn't yet had time to debit it).
- Request 1 debits 100 rubles. The balance becomes 0.
- Request 2 also debits 100 rubles because it saw 100 when checking the balance. The balance becomes -100.
Idempotency doesn't help here because the requests have different keys. The server checked honestly: key xyz-222 wasn't encountered, meaning the operation can be performed. The problem isn't the resending, but that both requests read outdated data before either had time to modify it. This is a classic race condition.
Case 2: Even the same UUID might not save the situation (if there's no atomicity)
One could speculate: what if the same key is used for two simultaneous requests? For example, a client accidentally generated the same identifier, or an attacker intentionally sent duplicates with the same key.
It would seem that idempotency should work: the server would see the key, understand that it has already executed it, and not repeat it. But in practice, if the implementation is not atomic, a race condition can occur even here.
Let's look at what a typical key check looks like in code:
- Check if there is a record with the key abc-123 in the processed_requests table.
- If not, perform the operation (for example, write off).
- Save the key abc-123 to the processed_requests table.
If two requests with the same key arrive simultaneously, the following sequence is possible:
- Thread A checks the key abc-123 - there is no record.
- Thread B checks the key abc-123 - there is also no record (because A hasn't created it yet).
- Thread A writes off and saves the key.
Thread B also performs a write (because it was checked earlier) and also attempts to save the key (it may receive a uniqueness error, but the operation has already been performed).
As a result, the operation was executed twice, even though the key is the same. The problem is that checking for the key's presence and inserting the key are not a single (atomic) operation. There's a window between them, through which the second thread can squeeze.
This is why many experienced developers say: idempotency alone doesn't protect against race conditions. It solves the problem of repeated requests in normal situations (when there's a delay between requests), but it doesn't prevent simultaneous concurrent requests.
True race protection requires other mechanisms that guarantee that two concurrent requests cannot simultaneously read stale data or perform an operation based on outdated information.
Concurrency must always be considered and the system must be designed so that critical operations are atomic.
At RK-CMS, we understand this well and use a combination of mechanisms to ensure data integrity even under high load.
