ModelPort：用 Python 打包模型，在 Flutter 应用一行 Dart 跑端侧推理

Run any AI model in your Flutter app. Prepare a model once with Python. Run it on-device in Flutter with one line of Dart.

Status: early development. Nothing is published yet. Follow along or star the repo to get notified when v0.1.0 ships.

Why

Today, putting your own PyTorch or Hugging Face model into a Flutter app means doing all of this by hand:

converting the model to a mobile format (ONNX, ExecuTorch .pte, GGUF)
guessing input names, shapes, and dtypes
re-implementing image preprocessing in Dart, where a single wrong number gives silently wrong results
wiring labels, tokenizers, and chat templates
writing download, resume, checksum, and cache logic for large model files
learning a different API for every inference engine
hoping the phone gives the same output as Python

Engine packages such as flutter_onnxruntime, executorch_flutter, and llm_llamacpp run models well. ModelPort does everything around them.

How it works

  Python (your computer)                    Flutter (user's phone)
 ┌──────────────────────────┐              ┌──────────────────────────────┐
 │ modelport CLI            │   bundle     │ modelport (Dart)             │
 │ export → verify → pack   │ ───────────► │ download · cache · preprocess│
 │ writes modelport.json    │  HF Hub /    │ run · postprocess            │
 └──────────────────────────┘  asset / URL └──────────────┬───────────────┘
                                                          ▼
                                     ONNX Runtime · ExecuTorch · llama.cpp

modelport CLI (Python) converts a model, checks that the converted model matches the original, and writes a modelport.json manifest describing inputs, outputs, preprocessing, labels, files, and checksums.
modelport (Dart) reads the manifest, downloads and verifies the files, prepares inputs, runs the model through an adapter, and returns typed results.
Adapters (modelport_onnx, modelport_executorch, modelport_llamacpp) connect existing engines to one API. Add only the engines you need.

Planned usage

pip install "modelport[onnx,hf]"
modelport export torchvision:mobilenet_v3_small --target onnx
modelport verify dist/mobilenet_v3_small
modelport publish dist/mobilenet_v3_small --hf modelport-dev/mobilenet_v3_small

final classifier = await ImageClassifier.load('hf://modelport-dev/mobilenet_v3_small');
final results = await classifier.classify(imageBytes);
print(results.first); // golden retriever (0.93)

final llm = await TextGenerator.load('hf://modelport-dev/qwen2.5-0.5b-instruct');
await for (final piece in llm.chat([ChatMessage.user('What is Flutter?')])) {
  stdout.write(piece);
}

Repository layout

Path	What lives there
`spec/`	The `modelport.json` manifest schema and examples
`python/`	The `modelport` CLI (PyPI)
`dart/packages/`	Dart and Flutter packages (pub.dev)
`apps/demo/`	Demo Flutter app
`zoo/`	Curated, tested model manifests
`docs/`	Documentation site

Roadmap

The full plan is in PLAN.md (written in Bangla). In short:

Manifest spec v0.1
Python CLI with ONNX export and verification
Dart core with download, cache, and preprocessing
ONNX, ExecuTorch, and llama.cpp adapters
Image classification, object detection, and text generation task APIs
Demo app, model zoo, and docs
v0.1.0 release

Contributing

Contributions are welcome. Read CONTRIBUTING.md to get started.

License

Apache-2.0