At Koan, we use DynamoDB for all of our application’s database needs. It’s fully managed, helping us maintain reliable query latency regardless of traffic volume and reduce our operational workload. But one of the challenges we’ve had to deal with has been around capacity planning. Our traffic pattern is generally very spiky: we see our highest usage on Mondays and Fridays, with the middle of the week and weekends being low points.
This usage pattern complicates capacity planning. We’re generally either over-provisioned, and thus wasting money, or under-provisioned, and thus risking our queries being throttled as auto-scaling kicks in.
In November of last year, Amazon announced a new pricing structure for DynamoDB called On-Demand Mode. With it, your capacity costs are strictly a function of how many capacity units you consume, rather than how many you would have provisioned in Provisioned Capacity Mode. Amazon claims that DynamoDB will continue handling your traffic load without any provisioning on your part at all, removing the pain around capacity planning.
This pricing structure seemed perfect for us: high performance when we need it, and no wasted money from over-provisioning. So did it actually live up to the claims?
In January, we dipped our toe into this new world by enabling on-demand pricing on our most used tables. We saw an immediate dip in our costs, so at the end of February we enabled it for the rest of our tables. When all was said and done, our average daily cost was down about 90%! From our testing, we were unable to detect any hits to performance either. Overall, a huge win.
Know your traffic patterns
Our service has very spiky traffic, so on-demand mode is ideal for us. If your service has a level traffic load, on-demand mode will probably end up costing you more money, not less. As an extreme example, suppose we have a table with 1000 Read Capacity Units (RCUs) and 1000 Write Capacity Units (WCUs). Per Amazon’s pricing, this would cost us $561.60 per month in total. With that capacity, if our traffic were perfectly even, and our capacity were fully utilized, we’d be able to perform 5,184,000,000 reads per month and 2,592,000,000 writes per month. If we were using on-demand pricing, we’d be paying $4,536 per month. That’s over 9x the price! This is a very contrived example, but the overall lesson is clear: on-demand mode is only worth it if your traffic is very uneven.
Over-provision like there’s no tomorrow before switching to on-demand mode
Under the hood, DynamoDB maintains capacity equal to double your previous peak traffic when in on-demand mode. When you exceed this amount, it can take up to 30 minutes to scale up to meet your new peak. But there’s a subtle detail in the documentation around how it provisions capacity when you first switch over to on-demand mode: the capacity it initially provisions will be whatever your capacity was immediately before you switched. This means that if you scale up to 10,000 RCUs and WCUs in every index just before flipping the switch, your capacity in on-demand mode will stay at 10,000, even if your peak traffic is typically much lower than that. As best we can tell, there’s no drawback to this trick.
Switch back to provisioned mode for scheduled operations
On-demand mode is great for handling spiky user traffic, but it’s much more expensive per-query than a fully utilized provisioned capacity setup. When performing operations such as large backfills or data migrations, it may be cheaper to switch to provisioned mode with just the right estimated capacity that you’ll need for your operation. DynamoDB allows you to switch back and forth once per day, so a well-timed switchover could save you a lot of money.