Cost optimization with kubernetes

Correction 2023-07-02: fix homelab specs and corresponding AWS EC2 instance class (it’s actually 32GB RAM, not 64GB) Congratulations, you managed to successfully deployed a few services on kubernetes! But this is not the end 👀. Unfortunately money doesn’t grow on trees, and if you can’t justify your infra expenses, finance department won’t be happy. If you’re using Terraform, you can use Infracost to create a cost report. Pretty nifty. But what about kubernetes? Given cost reporting is a basic feature, kubernetes is no exception. ...

April 1, 2023 · 2 min · Karn Wong

Bare metal works, until it doesn't. Hello, cloud.

Background Ever wonder how websites (and everything in between) work? Chances are you can create a project running on your local machine. It works as you expected, but to let other people access it, you have to “deploy” it. For many years, to support a lot of request volumes you need to run your applications in a data center. These days this setup is known as on-premise. ...

March 24, 2023 · 4 min · Karn Wong

Google Analytics v4 ingestion via BigQuery

Background You want to track who access your site, Google Analytics can do that. To see the data, you can use Google Analytics dashboard, the default settings is good enough for most use cases. But what if you have a lot of tracking data, and you want to streamline a way to analyze it? You could use Data Studio for this, so it’s cool for the moment. But what if you want to use Google Analytics data in conjunction with other data? Say, product SKUs? There are multiple ways to achieve that, but one approach that provides a lot of flexibility and allow for long-term maintenance is to write a custom pipeline yourself. ...

March 19, 2023 · 7 min · Karn Wong

SecretOps with teller

Raise your hands if you normally have to send .env files to your team members so they can start a project in dev environment. While there is nothing wrong with this approach, it could introduce a lot of security risks, namely sharing secrets via plaintext protocol. Sure, you can share them on Slack, Discord, etc. But unless it’s encrypted with your own keys, it could be leaked if the said communication platform were to be breached. ...

March 19, 2023 · 2 min · Karn Wong

Data transformation - python vs sql showdown

For most people, using SQL to transform data is a no-brainer, seeing it’s a very versatile language, and doesn’t have quite a steep learning curve compared to python. There are some cases where SQL is more suitable for a task, but the reverse can also happen as well. For instance, given a string conversion problem: if a string occurs only one time, replace it with # if a string occurs multiple times, replace it with & > one ### > three ###&& > Heartbreak hotel &&&&&#&&&##&#&&# A solution in python would be: ...

March 18, 2023 · 1 min · Karn Wong