For our production workload, we deploy the workloads on Kubernetes, in which an ingress resource is created per each deployment. Resources in ingress are GCP Load Balancer and SSL Certificate. As for DNS, we use Cloudflare since it enables CDN without extra configurations on our part. A few months after the deployment went live initially, we were informed that the website couldn’t be accessed. Turns out GCP couldn’t renew the SSL Certificate (error FAILED_NOT_VISIBLE.) Looking at GCP docs, turns out if the DNS couldn’t be resolved to the Load Balancer IP, it couldn’t provision/renew a certificate. ...

For a machine learning project, usually it is divided into two main categories: research and production. For research ML project, the model would be created and used locally on a researcher’s machine. For a production ML project, a deployment would be involved. Usual pattern is to create a service to load a model, accept input, then return a prediction. Production ML is also divided into two main patterns: batch or real-time. For batch inference, a job would be triggered on an interval to pre-calculate predictions, then store somewhere. As for real-time inference, it is more tricky, since this involves web application architecture (at least the data and application tier). ...

When we face with engineering problems, it’s too easy to fall into the trap thinking it should be solved with a technical solution. Seasoned engineers think differently, because they realize that most of the time, it’s “people” or “workflow” problems. Let me provide a few examples. Management wants analysts to use Jupyter notebook to reduce time required to create a routine report Background: Most analysts are comfortable using Microsoft Excel to work with data, some can also use SQL, but it’s rare for analysts to be familiar with Python. Jupyter notebook is an interactive development interface for data works, since users can execute a chunk of code at a time, and render data without requiring re-running the full code. Problem: every month a two-person analyst team would spend two days stitching up multiple CSV files (can be up to 60) via VLookup for a monthly report. This is because analysts have to look up information for each record, in which they use a template query and manually execute 50 queries with changed parameters. ...

Every business generates data, some very little, some do generate ginormous amount of data. If you are familiar with the basic web application architecture, there are data, application and web tier. But it doesn’t end there, because the data generated has to be analyzed for reports. A lot of organizations have analysts working on production database directly. This works fine and well, until the data they are working with is very large to the point that a single query can take half a day to process! ...

Background Think of websites you visit each day. Most likely they are hosted on a cloud provider such as AWS, GCP, Azure. The good news is it’s very easy to create a simple deployment with a virtual machine, but for scalable and high-availability workloads, usual recommendations is to use a container-based runtime such as AWS ECS/EKS, GCP Cloud Run/GKE. These services also require more configurations than a simple VM deployment. ...