Calling C from Go, Python and Rust benchmark

There’s a consensus that generally, c is very fast, and python is very slow. But if we are talking about go and rust, you would find that rust is slightly faster than go. So from fastest to slowest: c, rust, go, python. But what if you have go, python and rust calling c function? There would be more overhead, but how much? Calling C from Go, Python and Rust Stats ...

October 12, 2024 · 2 min · Karn Wong

Hello World API Performance Benchmark (Go, Node, Python, Rust)

Correction 2024-09-21: after using multi-stage build, Node image size dropped from 1.1GB to 130MB. The first programming language I achieved proficiency was Python, so for the longest time I’ve been using it to do most stuff. Last year I picked up Go, and I had a blast with it. This month I picked up Rust for data/ml works, and so far I was very impressed. It got me thinking - it’s been said multiple times is Python is slower than compiled languages, and Node is very easy to use but uses a lot of memory. Since I can code in multiple languages, why not do a simple API benchmark? So here’s the results. ...

September 20, 2024 · 3 min · Karn Wong

Streamlit load test performance

Streamlit is well-loved by many people, especially among data folks due to the fact that it does not require prior web programming knowledge to get started. Popular use cases for streamlit can be anything from a quick machine learning application poc or internal dashboards. But what if you want to create a production deployment? Would streamlit still be a viable option? Experiment setup For a production deployment, let’s assume there are 500 concurrent users. For a single-page streamlit with a chat box taking 3 seconds to return a single-paragraph message: ...

September 7, 2024 · 2 min · Karn Wong

Slim down python docker image size with poetry and pip

Python package management is not straightforward, seeing default package manager (pip) does not behave like node’s npm, in a sense that it doesn’t track dependencies versions. This is why you should use poetry to manage python packages, since it creates a lock file, so you can be sure that on every re-install, the versions would be the same. However, this poses a challenge when you want to create a docker image with poetry, because you need to do an extra pip install poetry (unless you bake this into your base python image). Additionally, turns out using poetry to install packages results in larger docker image size. ...

April 7, 2024 · 2 min · Karn Wong

Python venv management

Updated 2023-09-09: I now revert back to plain requirements.txt, since pipenv is very poor at resolving large dependencies list. Poetry still remains funky so that’s off the table as well. Updated 2024-01-20: I’ve been using poetry in production for a few months now and so far it works great! Looks like they ironed out a lot of rough edges. Although I would still recommend people to use pre-commit to generate plain requirements.txt, this is so that in ci or dockerfile you don’t have to install poetry and set in-project venv every time. ...

July 2, 2021 · 4 min · Karn Wong