I think soft caps on AWS were not just insufficient, they also plain didn’t work: many of the horror stories included the first notification when the bill was already way past the configured limit. I guess this became enough of a regulatory risk to introduce hard caps and eat the difference when their billing system cannot stay close to realtime.

But also, assuming LLMs can do standard well-known things, there is a solution with hard budget cap by default and inavoidably. Install stuff on VPSes with purely monthly pricing, do not give agent access to order more (so no hoster management UI access, just SSH access to VPS). There are clauses in ToS about things that get your VPS shut down, and in some cases your account terminated (like on AWS, of course), but what you pay is known in advance.

score 1 · canonical

I expect that most businesses and individuals would prefer errors to a surprise $10,000+ bill.

These businesses have no say in the matter, they are serfs. Why on earth would a business that stands to collect on these massive mistakes put up guardrails against them? That’s leaving money on the table.

If somebody tripped and dropped $10k on your patio you’d probably give it back, you’re also not a product designer for a cloud host, and this is why. Think like a ravenous greedy ghoul with an unending thirst for money and no integrity or decency, and you’ll start to understand how these “experiences” are designed.

score 1 · canonical