What is platform engineering?

Back in 2017-2018, everyone wanted to be a data scientist. Then reality hits, that they need a data engineer for a successful machine learning project. Things didn’t end there, since they also need a machine learning engineer to create production-ready code.

Some people think you only need an MLE and suddenly your ML project would become a reality, sadly the reality begs to differ, because you also need to find someone to deploy and scale it, enter DevOps engineer (who understands ML, this is very important).

So what’s next? What if you are working on a dozen ML projects, and you find that you have to create everything from scratch, every time. Would things be better if you can somehow only need to customize the relevant bits, and reuse templates for the rest?

I’m happy to tell you that there is a role for this, and it is platform engineering. This applies to other kinds of software development as well, I just use data works as an example.

In a nutshell, a platform engineer is someone who understands how to implement end-to-end projects, and is familiar with how teams/roles collaborate, so they can offer shortest path (read: abstractions) to delivery.

This is very important, because you have to speak everyone’s language, so you can “sell” on why your solutions benefit them, and how they can use it to achieve faster time-to-market.

Failing this, you can end up with abstractions no one wants to work with, because it doesn’t actually solve their problems.

As for abstractions, think of writing functions for reusable parts in your codebase. Just that this time it’s at a project level. You could say it’s a larger scale of abstractions. Definitely more moving parts and details to consider, which makes this role very challenging!

Although, if you look up platform engineering in 2024, many people would tell you it’s about Internal Developer Platform (IDP). It is part of a toolbox, but not necessarily what you need for platform engineering. At a certain scale, IDP would be beneficial, but it would be a very long time before you reach that point.

Jokingly, I say platform engineering is a glue job. We understand and can function in many different roles, but we don’t like it as much as guiding people how to collaborate and work together!