The Webhook Reliability Gap
You built a Stripe webhook with Cursor, deployed it, and within a week you have duplicate charges, missed webhooks, and a security researcher who spoofed a webhook. What happened? Your AI assistant built a webhook that works in testing but is not production-ready, because it lacks retries, signature verification, and idempotency. This is the webhook reliability gap, and it is one of the most common failures in AI-generated apps that handle webhooks. Here are the 6 reasons AI assistants build unreliable webhooks, and the production checklist to fix them.
The direct answer is that webhooks are HTTP callbacks from third-party services (e.g., Stripe, GitHub, Slack), and they are notoriously difficult to handle correctly. AI assistants generate webhook handlers that work in testing (where webhooks arrive once, in order, and are not spoofed) but fail in production (where webhooks can arrive multiple times, out of order, and can be spoofed). The 6 reasons are: no signature verification, no idempotency, no retry handling, slow responses, no error handling, and no logging. Each one has a known cause and a known fix. For more on production reliability, see our article on why AI apps break on the first real user.
Reason 1: No Signature Verification
The most dangerous reason AI assistants build unreliable webhooks is the lack of signature verification. Webhook providers (e.g., Stripe, GitHub) sign their webhooks with a secret, so you can verify the webhook is from the provider and not from an attacker. AI assistants rarely implement signature verification, which means an attacker can send a fake webhook and trigger actions (e.g., mark an invoice as paid without actually paying). The fix is to verify the webhook signature using the provider's SDK (e.g., stripe.webhooks.constructEvent for Stripe). For more on security, see our article on the secrets management gap.
Reason 2: No Idempotency
The second reason is no idempotency. Webhook providers retry webhooks if they do not receive a 200 response within a timeout, which means the same webhook can arrive multiple times. AI assistants rarely implement idempotency, which means duplicate webhooks cause duplicate actions (e.g., double charges, duplicate records). The fix is to make webhook handlers idempotent: check if the webhook has already been processed (by storing the webhook ID in a database), and if so, return 200 without reprocessing.
Reason 3: No Retry Handling
The third reason is no retry handling. Webhook providers retry webhooks with exponential backoff, which means a failed webhook will be retried multiple times. AI assistants rarely handle retries, which means a temporary failure (e.g., database timeout) causes the webhook to be retried, but the retry also fails, and the webhook is eventually abandoned. The fix is to handle errors gracefully: return 200 for webhooks that cannot be processed (e.g., malformed payload) and return 500 for temporary failures (e.g., database timeout), so the provider retries.
Reason 4: Slow Responses
The fourth reason is slow responses. Webhook providers have a timeout (e.g., 30 seconds for Stripe), and if your handler does not respond within the timeout, the provider retries. AI assistants often build handlers that do synchronous work (e.g., send an email, generate a report), which can take longer than the timeout. The fix is to respond immediately (return 200) and process the webhook asynchronously (e.g., via a job queue). For more on job queues, see our article on running BullMQ background workers.
Reason 5: No Error Handling
The fifth reason is no error handling. If a webhook handler throws an unhandled error, the server returns a 500, which causes the provider to retry. If the error is permanent (e.g., malformed payload), the retry also fails, and the webhook is retried indefinitely. The fix is to handle errors: catch all errors, log them, and return 200 for permanent errors (to stop retries) or 500 for temporary errors (to trigger retries).
Reason 6: No Logging
The sixth reason is no logging. Without logging, you cannot debug webhook issues (e.g., "did the webhook arrive?", "what was the payload?", "why did it fail?"). The fix is to log every webhook: the timestamp, the provider, the event type, the payload (redacting sensitive data), and the result (success or failure). For more on logging, see our article on the logging gap.
Step-by-Step: Building a Production-Ready Stripe Webhook
Here is how to build a production-ready Stripe webhook in Express.
// server.js
const express = require('express');
const Stripe = require('stripe');
const app = express();
const stripe = Stripe(process.env.STRIPE_SECRET_KEY);
const webhookSecret = process.env.STRIPE_WEBHOOK_SECRET;
// Use raw body for webhook signature verification
app.post('/webhooks/stripe', express.raw({ type: 'application/json' }), async (req, res) => {
const sig = req.headers['stripe-signature'];
let event;
// Fix 1: Verify the signature
try {
event = stripe.webhooks.constructEvent(req.body, sig, webhookSecret);
} catch (err) {
console.error('Webhook signature verification failed:', err.message);
return res.status(400).send(`Webhook Error: ${err.message}`);
}
// Fix 2: Check idempotency
const processed = await WebhookLog.findOne({ where: { eventId: event.id } });
if (processed) {
console.log(`Webhook ${event.id} already processed, skipping`);
return res.json({ received: true, duplicate: true });
}
// Log the webhook (Fix 6)
await WebhookLog.create({
eventId: event.id,
type: event.type,
payload: JSON.stringify(event.data.object),
});
// Fix 4: Respond immediately, process asynchronously
res.json({ received: true });
// Process the event asynchronously (Fix 5: error handling)
try {
switch (event.type) {
case 'checkout.session.completed':
await jobQueue.add('process-payment', event.data.object);
break;
case 'invoice.paid':
await jobQueue.add('process-invoice', event.data.object);
break;
default:
console.log(`Unhandled event type: ${event.type}`);
}
} catch (err) {
console.error('Webhook processing error:', err);
// Don't return 500 here, because we already returned 200
// The job queue will handle the retry
}
});
app.listen(process.env.PORT || 3000);Common Pitfalls and Troubleshooting
The first pitfall is using JSON body parser for webhooks. Stripe's signature verification requires the raw body, but express.json() parses it into a JavaScript object, which changes the body. The fix is to use express.raw({ type: 'application/json' }) for the webhook route. The second pitfall is not handling duplicate webhooks. Stripe retries webhooks, which means the same event can arrive multiple times. The fix is to check if the event has already been processed (by storing the event ID in a database). The third pitfall is slow webhook handlers. If the handler takes too long, Stripe retries, which means the same event is processed multiple times. The fix is to respond immediately and process asynchronously. The fourth pitfall is not logging webhooks. Without logs, you cannot debug webhook issues. The fix is to log every webhook (event ID, type, payload, result). The fifth pitfall is not testing webhooks. Webhooks are difficult to test locally, because they require a public URL. The fix is to use a tool like Stripe CLI or ngrok to forward webhooks to your local machine.
Conclusion: Build Reliable Webhooks or Don't Build Them at All
The webhook reliability gap is not a sign that your AI assistant did a bad job. It is a sign that webhooks are notoriously difficult to handle correctly, and AI assistants do not consider the production constraints. By applying the 6 fixes above (signature verification, idempotency, retry handling, fast responses, error handling, logging), you can build production-ready webhooks that handle real-world conditions. Stop building unreliable webhooks and start building them right.
Ready to ship reliable webhooks? Drag your project to Deployxa Drop for an instant live preview, or install the CLI with npm i -g @deployxa/cli and deploy from your terminal. For more on AI coding patterns, see our articles on the secrets management gap and the CDN configuration gap. Learn about the font loading trap and the image optimization gap in our companion articles. Explore our free developer tools to speed up your workflow.