ONNX multi-platform benchmark
Jan 7, 2026
Intro
Previously I worked on ONNX inference benchmark, but that was for REST api. This time, I'm seeing how onnx performs across platforms.
ONNX is a portable format, which means you can run it on many ecosystems and platforms. Which means you can run it on edge devices like IoT and mobile as well. This means you can embed onnx model into your apps or binaries and perform an inference locally without talking to a REST endpoint, which is great for when your target devices might not always be online, or the connection might be spotty.
Imagine a use case where you have IoT collecting sensor data, but you can't store all the data because they take up a lot of space, and some of them are noise. Using rules to filter out the data is possible but it still doesn't cover more cases. But you can train a small classification model and deploy it on the said IoT devices to filter what will be forwarded to a database.
In this benchmark, I'm evaluating onnx inference across pc, mobile (android) and IoT (raspberry pi 4). I did try on iOS but I couldn't get the dependencies working, feel free to work on it yourself and let me know how it goes. I did try flutter on iOS which works fine, but it's older than my android device, so the performance isn't really comparable.
Experiment Design
- Model is random forest regressor (scikit-learn to onnx)
- Tested against various model size.
- stdout strings are the same across all implementations.
- There are no optimizations.
- Inference accelerators are not used.
Steps
- Initialize the model and input data.
- Run inference on warmup for 3 iterations.
- Run inference for 20 iterations.
- Measure average inference time.
Platform / Device Matrix
| Python | Rust | Web | Kotlin | Flutter | |
|---|---|---|---|---|---|
| PC (Thinkpad X13 Gen 6) | ✅ | ✅ | ✅ | ||
| Android (Samsung Galaxy S25) | ✅ | ✅ | ✅ | ||
| IoT (Raspberry Pi 4 8GB) | ✅ |
Model Size
| Rows | ONNX Model Size (MB) |
|---|---|
| 7,000 | 27 |
| 20,000 | 74 |
| 60,000 | 227 |
| 100,000 | 377 |
SBOM
Click to expand
NAME VERSION TYPE
@babel/code-frame 7.27.1 npm
@babel/helper-string-parser 7.27.1 npm
@babel/helper-validator-identifier 7.28.5 npm
@babel/parser 7.28.5 npm
@babel/types 7.28.5 npm
@bufbuild/protobuf 2.10.1 npm
@esbuild/aix-ppc64 0.25.12 npm
@esbuild/android-arm 0.25.12 npm
@esbuild/android-arm64 0.25.12 npm
@esbuild/android-x64 0.25.12 npm
@esbuild/darwin-arm64 0.25.12 npm
@esbuild/darwin-x64 0.25.12 npm
@esbuild/freebsd-arm64 0.25.12 npm
@esbuild/freebsd-x64 0.25.12 npm
@esbuild/linux-arm 0.25.12 npm
@esbuild/linux-arm64 0.25.12 npm
@esbuild/linux-ia32 0.25.12 npm
@esbuild/linux-loong64 0.25.12 npm
@esbuild/linux-mips64el 0.25.12 npm
@esbuild/linux-ppc64 0.25.12 npm
@esbuild/linux-riscv64 0.25.12 npm
@esbuild/linux-s390x 0.25.12 npm
@esbuild/linux-x64 0.25.12 npm
@esbuild/netbsd-arm64 0.25.12 npm
@esbuild/netbsd-x64 0.25.12 npm
@esbuild/openbsd-arm64 0.25.12 npm
@esbuild/openbsd-x64 0.25.12 npm
@esbuild/openharmony-arm64 0.25.12 npm
@esbuild/sunos-x64 0.25.12 npm
@esbuild/win32-arm64 0.25.12 npm
@esbuild/win32-ia32 0.25.12 npm
@esbuild/win32-x64 0.25.12 npm
@eslint-community/eslint-utils 4.9.0 npm
@eslint-community/regexpp 4.12.2 npm
@eslint/config-array 0.21.1 npm
@eslint/config-helpers 0.4.2 npm
@eslint/core 0.17.0 npm
@eslint/eslintrc 3.3.1 npm
@eslint/js 9.39.1 npm
@eslint/object-schema 2.1.7 npm
@eslint/plugin-kit 0.4.1 npm
@humanfs/core 0.19.1 npm
@humanfs/node 0.16.7 npm
@humanwhocodes/module-importer 1.0.1 npm
@humanwhocodes/retry 0.4.3 npm
@inquirer/external-editor 1.0.3 npm
@inquirer/figures 1.0.15 npm
@isaacs/cliui 8.0.2 npm
@jridgewell/gen-mapping 0.3.13 npm
@jridgewell/resolve-uri 3.1.2 npm
@jridgewell/source-map 0.3.11 npm
@jridgewell/sourcemap-codec 1.5.5 npm
@jridgewell/trace-mapping 0.3.31 npm
@nodelib/fs.scandir 2.1.5 npm
@nodelib/fs.stat 2.0.5 npm
@nodelib/fs.walk 1.2.8 npm
@parcel/watcher 2.5.1 npm
@parcel/watcher-android-arm64 2.5.1 npm
@parcel/watcher-darwin-arm64 2.5.1 npm
@parcel/watcher-darwin-x64 2.5.1 npm
@parcel/watcher-freebsd-x64 2.5.1 npm
@parcel/watcher-linux-arm-glibc 2.5.1 npm
@parcel/watcher-linux-arm-musl 2.5.1 npm
@parcel/watcher-linux-arm64-glibc 2.5.1 npm
@parcel/watcher-linux-arm64-musl 2.5.1 npm
@parcel/watcher-linux-x64-glibc 2.5.1 npm
@parcel/watcher-linux-x64-musl 2.5.1 npm
@parcel/watcher-win32-arm64 2.5.1 npm
@parcel/watcher-win32-ia32 2.5.1 npm
@parcel/watcher-win32-x64 2.5.1 npm
@pkgjs/parseargs 0.11.0 npm
@pkgr/core 0.2.9 npm
@protobufjs/aspromise 1.1.2 npm
@protobufjs/base64 1.1.2 npm
@protobufjs/codegen 2.0.4 npm
@protobufjs/eventemitter 1.1.0 npm
@protobufjs/fetch 1.1.0 npm
@protobufjs/float 1.0.2 npm
@protobufjs/inquire 1.1.0 npm
@protobufjs/path 1.1.2 npm
@protobufjs/pool 1.1.0 npm
@protobufjs/utf8 1.1.0 npm
@quasar/app-vite 2.4.0 npm
@quasar/extras 1.17.0 npm
@quasar/render-ssr-error 1.0.3 npm
@quasar/ssl-certificate 1.0.0 npm
@quasar/vite-plugin 1.10.0 npm
@rolldown/pluginutils 1.0.0-beta.29 npm
@rollup/rollup-android-arm-eabi 4.53.2 npm
@rollup/rollup-android-arm64 4.53.2 npm
@rollup/rollup-darwin-arm64 4.53.2 npm
@rollup/rollup-darwin-x64 4.53.2 npm
@rollup/rollup-freebsd-arm64 4.53.2 npm
@rollup/rollup-freebsd-x64 4.53.2 npm
@rollup/rollup-linux-arm-gnueabihf 4.53.2 npm
@rollup/rollup-linux-arm-musleabihf 4.53.2 npm
@rollup/rollup-linux-arm64-gnu 4.53.2 npm
@rollup/rollup-linux-arm64-musl 4.53.2 npm
@rollup/rollup-linux-loong64-gnu 4.53.2 npm
@rollup/rollup-linux-ppc64-gnu 4.53.2 npm
@rollup/rollup-linux-riscv64-gnu 4.53.2 npm
@rollup/rollup-linux-riscv64-musl 4.53.2 npm
@rollup/rollup-linux-s390x-gnu 4.53.2 npm
@rollup/rollup-linux-x64-gnu 4.53.2 npm
@rollup/rollup-linux-x64-musl 4.53.2 npm
@rollup/rollup-openharmony-arm64 4.53.2 npm
@rollup/rollup-win32-arm64-msvc 4.53.2 npm
@rollup/rollup-win32-ia32-msvc 4.53.2 npm
@rollup/rollup-win32-x64-gnu 4.53.2 npm
@rollup/rollup-win32-x64-msvc 4.53.2 npm
@types/body-parser 1.19.6 npm
@types/chrome 0.0.262 npm
@types/compression 1.8.1 npm
@types/connect 3.4.38 npm
@types/cordova 11.0.3 npm
@types/estree 1.0.8 npm
@types/express 4.17.25 npm
@types/express 5.0.5 npm
@types/express-serve-static-core 4.19.7 npm
@types/express-serve-static-core 5.1.0 npm
@types/filesystem 0.0.36 npm
@types/filewriter 0.0.33 npm
@types/har-format 1.2.16 npm
@types/http-errors 2.0.5 npm
@types/json-schema 7.0.15 npm
@types/mime 1.3.5 npm
@types/node 20.19.25 npm
@types/node 24.10.1 npm
@types/node-forge 1.3.14 npm
@types/qs 6.14.0 npm
@types/range-parser 1.2.7 npm
@types/send 0.17.6 npm
@types/send 1.2.1 npm
@types/serve-static 1.15.10 npm
@typescript-eslint/eslint-plugin 8.47.0 npm
@typescript-eslint/parser 8.47.0 npm
@typescript-eslint/project-service 8.47.0 npm
@typescript-eslint/scope-manager 8.47.0 npm
@typescript-eslint/tsconfig-utils 8.47.0 npm
@typescript-eslint/type-utils 8.47.0 npm
@typescript-eslint/types 8.47.0 npm
@typescript-eslint/typescript-estree 8.47.0 npm
@typescript-eslint/utils 8.47.0 npm
@typescript-eslint/visitor-keys 8.47.0 npm
@vitejs/plugin-vue 6.0.1 npm
@volar/language-core 2.4.23 npm
@volar/source-map 2.4.23 npm
@volar/typescript 2.4.23 npm
@vue/compiler-core 3.5.24 npm
@vue/compiler-dom 3.5.24 npm
@vue/compiler-sfc 3.5.24 npm
@vue/compiler-ssr 3.5.24 npm
@vue/devtools-api 6.6.4 npm
@vue/eslint-config-prettier 10.2.0 npm
@vue/eslint-config-typescript 14.6.0 npm
@vue/language-core 3.1.4 npm
@vue/reactivity 3.5.24 npm
@vue/runtime-core 3.5.24 npm
@vue/runtime-dom 3.5.24 npm
@vue/server-renderer 3.5.24 npm
@vue/shared 3.5.24 npm
Flutter 1.0.0 pod
abort-controller 3.0.0 npm
accepts 1.3.8 npm
acorn 8.15.0 npm
acorn-jsx 5.3.2 npm
adler2 2.0.1 rust-crate
ajv 6.12.6 npm
alien-signals 3.1.0 npm
ansi-escapes 4.3.2 npm
ansi-regex 5.0.1 npm
ansi-regex 6.2.2 npm
ansi-styles 4.3.0 npm
ansi-styles 6.2.3 npm
anyio 4.12.0 python
anymatch 3.1.3 npm
archiver 7.0.1 npm
archiver-utils 5.0.2 npm
argparse 2.0.1 npm
array-flatten 1.1.1 npm
async 2.13.0 dart-pub
async 3.2.6 npm
autocfg 1.5.0 rust-crate
autoprefixer 10.4.22 npm
b4a 1.7.3 npm
balanced-match 1.0.2 npm
bare-events 2.8.2 npm
base64 0.22.1 rust-crate
base64-js 1.5.1 npm
base64ct 1.8.0 rust-crate
baseline-browser-mapping 2.8.29 npm
binary-extensions 2.3.0 npm
bitflags 2.10.0 rust-crate
bl 4.1.0 npm
block-buffer 0.10.4 rust-crate
body-parser 1.20.3 npm
boolbase 1.0.0 npm
boolean_selector 2.1.2 dart-pub
brace-expansion 1.1.12 npm
brace-expansion 2.0.2 npm
braces 3.0.3 npm
browserslist 4.28.0 npm
buffer 5.7.1 npm
buffer 6.0.3 npm
buffer-builder 0.2.0 npm
buffer-crc32 1.0.0 npm
buffer-from 1.1.2 npm
bundle-name 4.1.0 npm
byteorder 1.5.0 rust-crate
bytes 1.11.0 rust-crate
bytes 3.1.2 npm
call-bind-apply-helpers 1.0.2 npm
call-bound 1.0.4 npm
callsites 3.1.0 npm
camel-case 4.1.2 npm
caniuse-lite 1.0.30001755 npm
cc 1.2.48 rust-crate
cfg-if 1.0.4 rust-crate
chalk 4.1.2 npm
characters 1.4.0 dart-pub
chardet 2.1.1 npm
chokidar 3.6.0 npm
chokidar 4.0.3 npm
ci-info 4.3.1 npm
clean-css 5.3.3 npm
cli-cursor 3.1.0 npm
cli-spinners 2.9.2 npm
cli-width 4.1.0 npm
click 8.3.1 python
cliui 8.0.1 npm
clock 1.1.2 dart-pub
clone 1.0.4 npm
clone-deep 4.0.1 npm
collection 1.19.1 dart-pub
color-convert 2.0.1 npm
color-name 1.1.4 npm
colorama 0.4.6 python (+1 duplicate)
coloredlogs 15.0.1 python (+1 duplicate)
colorjs.io 0.5.2 npm
commander 10.0.1 npm
commander 2.20.3 npm
compress-commons 6.0.2 npm
compressible 2.0.18 npm
compression 1.8.1 npm
concat-map 0.0.1 npm
confbox 0.1.8 npm
content-disposition 0.5.4 npm
content-type 1.0.5 npm
contourpy 1.3.3 python
cookie 0.7.1 npm
cookie-signature 1.0.6 npm
core-foundation 0.9.4 rust-crate
core-foundation-sys 0.8.7 rust-crate
core-util-is 1.0.3 npm
cpufeatures 0.2.17 rust-crate
crc-32 1.2.2 npm
crc32-stream 6.0.0 npm
crc32fast 1.5.0 rust-crate
cross-spawn 7.0.6 npm
crypto-common 0.1.7 rust-crate
cssesc 3.0.0 npm
csstype 3.2.3 npm
cupertino_icons 1.0.8 dart-pub
cycler 0.12.1 python
debug 2.6.9 npm
debug 4.4.3 npm
deep-is 0.1.4 npm
default-browser 5.4.0 npm
default-browser-id 5.0.1 npm
defaults 1.0.4 npm
define-lazy-prop 2.0.0 npm
define-lazy-prop 3.0.0 npm
depd 2.0.0 npm
der 0.7.10 rust-crate
destroy 1.2.0 npm
detect-libc 1.0.3 npm
digest 0.10.7 rust-crate
docutils 0.22.3 python
dot-case 3.0.4 npm
dot-prop 9.0.0 npm
dotenv 16.6.1 npm
dotenv-expand 11.0.7 npm
dunder-proto 1.0.1 npm
eastasianwidth 0.2.0 npm
ee-first 1.1.1 npm
electron-to-chromium 1.5.255 npm
elementtree 0.1.7 npm
emoji-regex 8.0.0 npm
emoji-regex 9.2.2 npm
encodeurl 1.0.2 npm
encodeurl 2.0.0 npm
entities 4.5.0 npm
errno 0.3.14 rust-crate
es-define-property 1.0.1 npm
es-errors 1.3.0 npm
es-object-atoms 1.1.1 npm
esbuild 0.25.12 npm
escalade 3.2.0 npm
escape-html 1.0.3 npm
escape-string-regexp 4.0.0 npm
eslint 9.39.1 npm
eslint-config-prettier 10.1.8 npm
eslint-plugin-prettier 5.5.4 npm
eslint-plugin-vue 10.5.1 npm
eslint-scope 8.4.0 npm
eslint-visitor-keys 3.4.3 npm
eslint-visitor-keys 4.2.1 npm
espree 10.4.0 npm
esquery 1.6.0 npm
esrecurse 4.3.0 npm
estraverse 5.3.0 npm
estree-walker 2.0.2 npm
esutils 2.0.3 npm
etag 1.8.1 npm
event-target-shim 5.0.1 npm
events 3.3.0 npm
events-universal 1.0.1 npm
express 4.21.2 npm
fake_async 1.3.3 dart-pub
fast-deep-equal 3.1.3 npm
fast-diff 1.3.0 npm
fast-fifo 1.3.2 npm
fast-glob 3.3.3 npm
fast-json-stable-stringify 2.1.0 npm
fast-levenshtein 2.0.6 npm
fastq 1.19.1 npm
fastrand 2.3.0 rust-crate
fdir 6.5.0 npm
ffi 2.1.4 dart-pub
file-entry-cache 8.0.0 npm
filetime 0.2.26 rust-crate
fill-range 7.1.1 npm
finalhandler 1.3.1 npm
find-msvc-tools 0.1.5 rust-crate
find-up 5.0.0 npm
flat 5.0.2 npm
flat-cache 4.0.1 npm
flatbuffers 25.9.23 npm
flatbuffers 25.9.23 python (+1 duplicate)
flate2 1.1.5 rust-crate
flatted 3.3.3 npm
flutter 3.35.0 dart-pub
flutter_lints 5.0.0 dart-pub
flutter_onnxruntime 0.0.1 pod
flutter_onnxruntime 1.6.0 dart-pub
flutter_test 3.35.0 dart-pub
flutter_web_plugins 3.35.0 dart-pub
fonttools 4.61.0 python
foreground-child 3.3.1 npm
foreign-types 0.3.2 rust-crate
foreign-types-shared 0.1.1 rust-crate
forwarded 0.2.0 npm
fraction.js 5.3.4 npm
fresh 0.5.2 npm
fs-extra 11.3.2 npm
fsevents 2.3.3 npm
function-bind 1.1.2 npm
generic-array 0.14.7 rust-crate
get-caller-file 2.0.5 npm
get-intrinsic 1.3.0 npm
get-proto 1.0.1 npm
getrandom 0.3.4 rust-crate
glob 10.5.0 npm
glob-parent 5.1.2 npm
glob-parent 6.0.2 npm
globals 14.0.0 npm
globals 16.5.0 npm
gopd 1.2.0 npm
graceful-fs 4.2.11 npm
gradle-wrapper UNKNOWN java-archive
graphemer 1.4.0 npm
guid-typescript 1.0.9 npm
h11 0.16.0 python
has-flag 4.0.0 npm
has-symbols 1.1.0 npm
hasown 2.0.2 npm
html-minifier-terser 7.2.0 npm
http 1.4.0 rust-crate
http-errors 2.0.0 npm
httparse 1.10.1 rust-crate
humanfriendly 10.0 python (+1 duplicate)
iconv-lite 0.4.24 npm
iconv-lite 0.7.0 npm
idna 3.11 python
ieee754 1.2.1 npm
ignore 5.3.2 npm
ignore 7.0.5 npm
immutable 5.1.4 npm
import-fresh 3.3.1 npm
imurmurhash 0.1.4 npm
inherits 2.0.4 npm
inquirer 9.3.8 npm
ipaddr.js 1.9.1 npm
is-binary-path 2.1.0 npm
is-docker 2.2.1 npm
is-docker 3.0.0 npm
is-extglob 2.1.1 npm
is-fullwidth-code-point 3.0.0 npm
is-glob 4.0.3 npm
is-inside-container 1.0.0 npm
is-interactive 1.0.0 npm
is-number 7.0.0 npm
is-plain-object 2.0.4 npm
is-stream 2.0.1 npm
is-unicode-supported 0.1.0 npm
is-wsl 2.2.0 npm
is-wsl 3.1.0 npm
isarray 1.0.0 npm
isbinaryfile 5.0.7 npm
isexe 2.0.0 npm
isobject 3.0.1 npm
itoa 1.0.15 rust-crate
itsdangerous 2.2.0 python
jackspeak 3.4.3 npm
jedi 0.19.2 python
joblib 1.5.2 python (+1 duplicate)
js-tokens 4.0.0 npm
js-yaml 4.1.1 npm
json-buffer 3.0.1 npm
json-schema-traverse 0.4.1 npm
json-stable-stringify-without-jsonify 1.0.1 npm
jsonfile 6.2.0 npm
keyv 4.5.4 npm
kind-of 6.0.3 npm
kiwisolver 1.4.9 python
kolorist 1.8.0 npm
lazystream 1.0.1 npm
leak_tracker 11.0.2 dart-pub
leak_tracker_flutter_testing 3.0.10 dart-pub
leak_tracker_testing 3.0.2 dart-pub
levn 0.4.1 npm
libc 0.2.177 rust-crate
libredox 0.1.10 rust-crate
lints 5.1.1 dart-pub
linux-raw-sys 0.11.0 rust-crate
locate-path 6.0.0 npm
lodash 4.17.21 npm
lodash.merge 4.6.2 npm
log 0.4.28 rust-crate
log-symbols 4.1.0 npm
loguru 0.7.3 python (+1 duplicate)
long 5.3.2 npm
loro 1.10.0 python
lower-case 2.0.2 npm
lru-cache 10.4.3 npm
magic-string 0.30.21 npm
marimo 0.18.1 python
markdown 3.10 python
matcher 0.12.17 dart-pub
material_color_utilities 0.11.1 dart-pub
math-intrinsics 1.1.0 npm
matplotlib 3.10.7 python
matrixmultiply 0.3.10 rust-crate
media-typer 0.3.0 npm
merge-descriptors 1.0.3 npm
merge2 1.4.1 npm
meta 1.17.0 dart-pub
methods 1.1.2 npm
micromatch 4.0.8 npm
mime 1.6.0 npm
mime-db 1.52.0 npm
mime-db 1.54.0 npm
mime-types 2.1.35 npm
mimic-fn 2.1.0 npm
minimatch 3.1.2 npm
minimatch 5.1.6 npm
minimatch 9.0.5 npm
minimist 1.2.8 npm
minipass 7.1.2 npm
miniz_oxide 0.8.9 rust-crate
ml-dtypes 0.5.4 python (+1 duplicate)
mlly 1.8.0 npm
mpmath 1.3.0 python (+1 duplicate)
ms 2.0.0 npm
ms 2.1.3 npm
msgspec 0.20.0 python
muggle-string 0.4.1 npm
mute-stream 1.0.0 npm
nanoid 3.3.11 npm
narwhals 2.12.0 python
native-tls 0.2.14 rust-crate
natural-compare 1.4.0 npm
ndarray 0.16.1 rust-crate
negotiator 0.6.3 npm
negotiator 0.6.4 npm
no-case 3.0.4 npm
node-addon-api 7.1.1 npm
node-forge 1.3.1 npm
node-releases 2.0.27 npm
normalize-path 3.0.0 npm
normalize-range 0.1.2 npm
npm-run-path 6.0.0 npm
nth-check 2.1.1 npm
num-complex 0.4.6 rust-crate
num-integer 0.1.46 rust-crate
num-traits 0.2.19 rust-crate
numpy 2.3.5 python (+1 duplicate)
object-inspect 1.13.4 npm
on-finished 2.4.1 npm
on-headers 1.1.0 npm
once_cell 1.21.3 rust-crate
onetime 5.1.2 npm
onnx 1.19.1 python (+1 duplicate)
onnx-create 0.1.0 python
onnx-inference 0.1.0 python
onnx-inference 0.1.0 rust-crate
onnxinference 1.0.0+1 dart-pub
onnxruntime 1.23.2 python (+1 duplicate)
onnxruntime-c 1.22.0 pod
onnxruntime-common 1.23.2 npm
onnxruntime-objc 1.22.0 pod
onnxruntime-objc/Core 1.22.0 pod
onnxruntime-web 1.23.2 npm
open 10.2.0 npm
open 8.4.2 npm
openssl 0.10.75 rust-crate
openssl-macros 0.1.1 rust-crate
openssl-probe 0.1.6 rust-crate
openssl-src 300.5.4+3.5.4 rust-crate
openssl-sys 0.9.111 rust-crate
optionator 0.9.4 npm
ora 5.4.1 npm
ort 2.0.0-rc.10 rust-crate
ort-sys 2.0.0-rc.10 rust-crate
p-limit 3.1.0 npm
p-locate 5.0.0 npm
package-json-from-dist 1.0.1 npm
packaging 25.0 python (+1 duplicate)
pandas 2.3.3 python (+1 duplicate)
param-case 3.0.4 npm
parent-module 1.0.1 npm
parseurl 1.3.3 npm
parso 0.8.5 python
pascal-case 3.1.2 npm
path 1.9.1 dart-pub
path-browserify 1.0.1 npm
path-exists 4.0.0 npm
path-key 3.1.1 npm
path-key 4.0.0 npm
path-scurry 1.11.1 npm
path-to-regexp 0.1.12 npm
path_provider 2.1.5 dart-pub
path_provider_android 2.2.22 dart-pub
path_provider_foundation 0.0.1 pod
path_provider_foundation 2.5.1 dart-pub
path_provider_linux 2.2.1 dart-pub
path_provider_platform_interface 2.1.2 dart-pub
path_provider_windows 2.3.0 dart-pub
pathe 2.0.3 npm
pem-rfc7468 0.7.0 rust-crate
percent-encoding 2.3.2 rust-crate
picocolors 1.1.1 npm
picomatch 2.3.1 npm
picomatch 4.0.3 npm
pillow 12.0.0 python
pin-project-lite 0.2.16 rust-crate
pkg-config 0.3.32 rust-crate
pkg-types 1.3.1 npm
platform 1.3.6 npm
platform 3.1.6 dart-pub
plugin_platform_interface 2.1.8 dart-pub
polars 1.35.2 python
polars-runtime-32 1.35.2 python
portable-atomic 1.11.1 rust-crate
portable-atomic-util 0.2.4 rust-crate
postcss 8.5.6 npm
postcss-selector-parser 6.1.2 npm
postcss-value-parser 4.2.0 npm
prelude-ls 1.2.1 npm
prettier 3.6.2 npm
prettier-linter-helpers 1.0.0 npm
proc-macro2 1.0.103 rust-crate
process 0.11.10 npm
process-nextick-args 2.0.1 npm
protobuf 6.33.1 python (+1 duplicate)
protobufjs 7.5.4 npm
proxy-addr 2.0.7 npm
psutil 7.1.3 python
punycode 2.3.1 npm
py4j 0.10.9.9 python
pyarrow 22.0.0 python
pygments 2.19.2 python
pymdown-extensions 10.17.2 python
pyparsing 3.2.5 python
pyreadline3 3.5.4 python (+1 duplicate)
pyspark 4.0.1 python
python-dateutil 2.9.0.post0 python (+1 duplicate)
pytz 2025.2 python (+1 duplicate)
pyyaml 6.0.3 python
qs 6.13.0 npm
quasar 2.18.6 npm
queue-microtask 1.2.3 npm
quote 1.0.42 rust-crate
r-efi 5.3.0 rust-crate
randombytes 2.1.0 npm
range-parser 1.2.1 npm
raw-body 2.5.2 npm
rawpointer 0.2.1 rust-crate
readable-stream 2.3.8 npm
readable-stream 3.6.2 npm
readable-stream 4.7.0 npm
readdir-glob 1.1.3 npm
readdirp 3.6.0 npm
readdirp 4.1.2 npm
redox_syscall 0.5.18 rust-crate
relateurl 0.2.7 npm
require-directory 2.1.1 npm
resolve-from 4.0.0 npm
restore-cursor 3.1.0 npm
reusify 1.1.0 npm
rollup 4.53.2 npm
rollup-plugin-visualizer 5.14.0 npm
run-applescript 7.1.0 npm
run-async 3.0.0 npm
run-parallel 1.2.0 npm
rustix 1.1.2 rust-crate
rustls-pki-types 1.13.1 rust-crate
rxjs 7.8.2 npm
safe-buffer 5.1.2 npm
safe-buffer 5.2.1 npm
safer-buffer 2.1.2 npm
sass 1.93.3 npm
sass-embedded 1.93.3 npm
sass-embedded-all-unknown 1.93.3 npm
sass-embedded-android-arm 1.93.3 npm
sass-embedded-android-arm64 1.93.3 npm
sass-embedded-android-riscv64 1.93.3 npm
sass-embedded-android-x64 1.93.3 npm
sass-embedded-darwin-arm64 1.93.3 npm
sass-embedded-darwin-x64 1.93.3 npm
sass-embedded-linux-arm 1.93.3 npm
sass-embedded-linux-arm64 1.93.3 npm
sass-embedded-linux-musl-arm 1.93.3 npm
sass-embedded-linux-musl-arm64 1.93.3 npm
sass-embedded-linux-musl-riscv64 1.93.3 npm
sass-embedded-linux-musl-x64 1.93.3 npm
sass-embedded-linux-riscv64 1.93.3 npm
sass-embedded-linux-x64 1.93.3 npm
sass-embedded-unknown-all 1.93.3 npm
sass-embedded-win32-arm64 1.93.3 npm
sass-embedded-win32-x64 1.93.3 npm
sax 1.1.4 npm
schannel 0.1.28 rust-crate
scikit-learn 1.7.2 python (+1 duplicate)
scipy 1.16.3 python (+1 duplicate)
seaborn 0.13.2 python
security-framework 2.11.1 rust-crate
security-framework-sys 2.15.0 rust-crate
selfsigned 2.4.1 npm
semver 7.7.3 npm
send 0.19.0 npm
serialize-javascript 6.0.2 npm
serve-static 1.16.2 npm
setprototypeof 1.2.0 npm
sha2 0.10.9 rust-crate
shallow-clone 3.0.1 npm
shebang-command 2.0.0 npm
shebang-regex 3.0.0 npm
shlex 1.3.0 rust-crate
side-channel 1.1.0 npm
side-channel-list 1.0.0 npm
side-channel-map 1.0.1 npm
side-channel-weakmap 1.0.2 npm
signal-exit 3.0.7 npm
signal-exit 4.1.0 npm
simd-adler32 0.3.7 rust-crate
six 1.17.0 python (+1 duplicate)
skl2onnx 1.19.1 python (+1 duplicate)
sky_engine 3.35.0 dart-pub
smallvec 2.0.0-alpha.10 rust-crate
socks 0.3.4 rust-crate
source-map 0.6.1 npm
source-map 0.7.6 npm
source-map-js 1.2.1 npm
source-map-support 0.5.21 npm
source_span 1.10.1 dart-pub
stack-trace 1.0.0-pre2 npm
stack_trace 1.12.1 dart-pub
starlette 0.50.0 python
statuses 2.0.1 npm
stream_channel 2.1.4 dart-pub
streamx 2.23.0 npm
string-width 4.2.3 npm
string-width 5.1.2 npm
string_decoder 1.1.1 npm
string_decoder 1.3.0 npm
string_scanner 1.4.1 dart-pub
strip-ansi 6.0.1 npm
strip-ansi 7.1.2 npm
strip-json-comments 3.1.1 npm
supports-color 7.2.0 npm
supports-color 8.1.1 npm
sympy 1.14.0 python (+1 duplicate)
syn 2.0.111 rust-crate
sync-child-process 1.0.2 npm
sync-message-port 1.1.3 npm
synckit 0.11.11 npm
tar 0.4.44 rust-crate
tar-stream 3.1.7 npm
tempfile 3.23.0 rust-crate
term_glyph 1.2.2 dart-pub
terser 5.44.1 npm
test_api 0.7.7 dart-pub
text-decoder 1.2.3 npm
threadpoolctl 3.6.0 python (+1 duplicate)
tiny-invariant 1.3.3 npm
tinyglobby 0.2.15 npm
to-regex-range 5.0.1 npm
toidentifier 1.0.1 npm
tomlkit 0.13.3 python
tracing 0.1.43 rust-crate
tracing-core 0.1.35 rust-crate
ts-api-utils 2.1.0 npm
ts-essentials 9.4.2 npm
tslib 2.8.1 npm
type-check 0.4.0 npm
type-fest 0.21.3 npm
type-fest 4.41.0 npm
type-is 1.6.18 npm
typenum 1.19.0 rust-crate
typescript 5.9.3 npm
typescript-eslint 8.47.0 npm
typing-extensions 4.15.0 python (+1 duplicate)
tzdata 2025.2 python (+1 duplicate)
ufo 1.6.1 npm
undici-types 6.21.0 npm
undici-types 7.16.0 npm
unicode-ident 1.0.22 rust-crate
unicorn-magic 0.3.0 npm
universalify 2.0.1 npm
unpipe 1.0.0 npm
update-browserslist-db 1.1.4 npm
ureq 3.1.4 rust-crate
ureq-proto 0.5.3 rust-crate
uri-js 4.4.1 npm
utf-8 0.7.6 rust-crate
util-deprecate 1.0.2 npm
utils-merge 1.0.1 npm
uvicorn 0.38.0 python
varint 6.0.0 npm
vary 1.1.2 npm
vcpkg 0.2.15 rust-crate
vector_math 2.2.0 dart-pub
version_check 0.9.5 rust-crate
vite 7.2.2 npm
vite-plugin-checker 0.11.0 npm
vm_service 15.0.2 dart-pub
vscode-uri 3.1.0 npm
vue 3.5.24 npm
vue-eslint-parser 10.2.0 npm
vue-router 4.6.3 npm
vue-tsc 3.1.4 npm
wasip2 1.0.1+wasi-0.2.4 rust-crate
wcwidth 1.0.1 npm
web 1.1.1 dart-pub
webpack-merge 6.0.1 npm
webpki-root-certs 1.0.4 rust-crate
websockets 15.0.1 python
which 2.0.2 npm
wildcard 2.0.1 npm
win32-setctime 1.2.0 python (+1 duplicate)
winapi 0.3.9 rust-crate
winapi-i686-pc-windows-gnu 0.4.0 rust-crate
winapi-x86_64-pc-windows-gnu 0.4.0 rust-crate
windows-link 0.2.1 rust-crate
windows-sys 0.60.2 rust-crate
windows-sys 0.61.2 rust-crate
windows-targets 0.53.5 rust-crate
windows_aarch64_gnullvm 0.53.1 rust-crate
windows_aarch64_msvc 0.53.1 rust-crate
windows_i686_gnu 0.53.1 rust-crate
windows_i686_gnullvm 0.53.1 rust-crate
windows_i686_msvc 0.53.1 rust-crate
windows_x86_64_gnu 0.53.1 rust-crate
windows_x86_64_gnullvm 0.53.1 rust-crate
windows_x86_64_msvc 0.53.1 rust-crate
wit-bindgen 0.46.0 rust-crate
word-wrap 1.2.5 npm
wrap-ansi 6.2.0 npm
wrap-ansi 7.0.0 npm
wrap-ansi 8.1.0 npm
wsl-utils 0.1.0 npm
xattr 1.6.1 rust-crate
xdg_directories 1.1.0 dart-pub
xml-name-validator 4.0.0 npm
y18n 5.0.8 npm
yargs 17.7.2 npm
yargs-parser 21.1.1 npm
yocto-queue 0.1.0 npm
yoctocolors-cjs 2.1.3 npm
zeroize 1.8.2 rust-crate
zip-stream 6.0.1 npmFor Kotlin:Click to expand
[versions]
agp = "8.13.1"
kotlin = "2.0.21"
coreKtx = "1.10.1"
junit = "4.13.2"
junitVersion = "1.1.5"
espressoCore = "3.5.1"
appcompat = "1.6.1"
material = "1.10.0"
activity = "1.8.0"
constraintlayout = "2.1.4"
onnxruntimeAndroid = "1.23.2"
Results
Note: each platform has its own elbow point (it means when something no longer works, or works but takes a significant performance hit).
We can see that on web, onnx takes very long to perform an inference. Additionally, without further optimization, the onnx model size has to be very small (around 30 MB). On web via pc, it takes even longer than android (kotlin). Even onnx on IoT is still faster than web.
On mobile front, flutter takes significantly longer than kotlin. This makes sense because kotlin is "native" but flutter is not.
Zooming in with python, rust, kotlin (all native runtimes), you can see that even with larger model size, the inference time does not increase significantly (it does use more RAM though). It's also very interesting that python on pc has very similar performance to kotlin on android (python is notorious for being slow). On pc, rust is around 2x faster than python, this is to be expected.
Elbow Points
| 74 MB | 227 MB | 377 MB | |
|---|---|---|---|
| Web | 🫠 | ||
| Kotlin | 🫠 | ||
| Flutter | 🫠 |
- At 74 MB, crashes on
web- takes 7s to initialize. - At 227 MB, crashes on
android(kotlin). APK takes around 4 minutes to compile. - At 377 MB, crashes on
android(flutter).
What Does All This Mean?
Mostly there's nothing to worry about if you are on pc. If python is too slow, you can use rust. For web, it's best for small models because it takes longest to perform an inference. On mobile, you would benefit from using inference accelerators available on ios and android. For IoT go for binary-based solution, and rust is perfect for this.
Note that on web, ios and android, you would really benefit from using accelerators and further inference optimization.