ARM64 can reduce your infra cost - with a catch

Money doesn't grow on trees

Cloud spending can go off the rails, and no one is really happy about this. The finance team is grilling the ops team why the operating cost is very high, and then ops ask the devs to curb the spending, but they don't want to run into a situation where "I reduced the specs for our instance types, but turns out it causes a service disruption because our spike workloads need more memory." While understandable, this puts every team in a stalemate because no one is willing to make the change.

If autoscaling is enabled, it can mitigate the too-little-memory-which-means-a-service-disruption, but then many workloads are run via systemd or docker on a single machine. You could try setting up observability to see how much resources your services require during idle and peak load, but that's assuming someone set up observability, and that people are actually using it!!!

Switching compute type is easier

Since cloud providers offer arm64 compute, you can utilize this to reduce your cloud spend. On AWS RDS instance types there's arm64 compute, which should be around 5-10% cheaper hourly. Doesn't sound like much, but compounding it into a monthly bill in addition to reserved instances adds up to a lot. This does not require you to change the code, only to migrate your RDS to a new instance type.

For applications, if you are a Go or Rust shop, things are looking good because these runtimes produce a binary, which tends to have less issues on arm64. Python and Node probably run into some issues on arm64 if they depend on amd64 system libraries from dependencies - so YMMV.

On GCP you can use arm64 with Compute Engine. On Azure you can also provision arm64 compute for AKS as well. Check out cloud provider docs for more services supporting arm64 compute.

Added benefit of using Go and Rust is less compute footprint compared to Python or Node, which means more throughput for the same compute size. Even the baseline latency is significantly less, see benchmark here.

Do not rewrite unless you absolutely have to

If you are contemplating rewriting everything into Go or Rust, hold that thought. A rewrite means you need to train your employees to learn a new language, get familiar with its ecosystem, and rewrite a service. Sure, you can use LLMs, but this doesn't mean LLMs can catch edge cases, or port them from original implementation. Also for Go and Rust, there are less available candidates compared to other popular runtimes. Although for heavily utilized small services or endpoints, it might be worth it to port to Go or Rust.

But make sure that if the original employees or teams that rewrote the services are gone, current team members can continue supporting these services. Otherwise you'll have to migrate everything back because no one understands how to maintain the rewritten services.

But if you got to the point where you successfully built a new team for Go or Rust, you'll have to do something with the existing team members who haven't picked up these languages. There's probably some legacy services they have to maintain so there's work for them. But then maybe it's time for an org restructure.

Technological changes do not end at the technical layer, they always affect peopleware. There are real stories out there about orgs that can't scale their team because they can't find enough Rust developers. Do not rush the decision, ponder it. Try the least effort paths first, then consider more destructive paths if you need to reduce your cloud spend further.