New top story on Hacker News: Tell HN: I DDoSed myself using AWS CloudFront and Lambda Edge and got $4.5k bill

Tell HN: I DDoSed myself using AWS CloudFront and Lambda Edge and got $4.5k bill
38 by huksley | 17 comments on Hacker News.
I am using awesome NextJS and serverless-nextjs and deploy my app to CloudFront and Lambda@Edge. I made a mistake and accidentally created a serverless function that called itself. In a recursive loop, with a 30s timeout. I thought I fixed it and deployed the code to the dev environment. I have had an AWS Billing alert (Budgets) set up to prompt me when my monthly budget goes over $300 (my usual bill is $200/month). Imagine the terror when I woke up the next day to see the AWS Billing alert email saying I already owed $1,484! I removed a function and deployed it again in 30 minutes, but it was too late. It has already run for 24 hours, using over 70 million Gb-Second! Only after that I've learned that AWS Billing alerts do not work this way for CloudFront. You get delayed information on charges because they collect them from all regions. On the following day, the bill settled at a shocking $4600. This is more than we have ever spent on AWS all time. CloudFront includes the AWS Shield Standard feature, but somehow, it was not activated for this case (Lambda@Edge calling itself via CloudFront). Now, I understand that I should have created CloudWatch alarms, which would alert me when the number of requests exceeds the limit. The problem is, that they need to be set up per region, and I got CloudFront charges from all points of presence. I am a big proponent of the serverless approach. It makes it easy to scale and develop things (e.g., you get PR review version branches for free, both frontend and backend code like Vercel does). But now, I am unsure because such unexpected charges can ruin a side-project or emerging startup. Now I am waiting on a response from AWS Support on these charges; maybe they can help me waive part of that. What is your experience with it? Would you recommend to use to build a new product if you are bootstrapped, 3-person startup?