Reproduce or contribute a model
The Model Browser lists models that run on SiMa.ai platforms, each with its measured performance and accuracy. This repository holds what produces a model: its pinned source, its compile recipe, and the scripts that measure it. You can run those scripts yourself and reproduce every step on your own machine.
Start here
Read Getting started first. It shows the path every model takes, from the official model to the measured results, and walks through it once with real commands. A contributor follows the same path, so read it before the contribution guides below. Concepts defines the terms both use, such as build, build profile, stage, and fingerprint.
Contribute a model
A contribution adds or updates one model folder. Its model.yaml records the
exact source, components, build profiles, and datasets, and its scripts compile
and measure each build. These guides explain how to define a model, qualify it
locally, and submit it for review, and what CI does after the merge.
Where each step runs
| Phase | What runs | What it shows |
|---|---|---|
| Local development | Validation, verified downloads, the compile of each build, MPK checks, and the performance and accuracy stages on a DevKit. | You have reproducible results for every build you submit. |
| Pull request | Target Branch Checks: lint, the repository tests and model tests, the model-registry package build, and the model browser checks. | The committed files are valid on the merge result. Pull requests do not run recipe or surgery code to compile or measure a model. |
| After the merge | CI compiles and benchmarks each changed build and publishes the results. | The published model and its measurements come from the reviewed revision. |
The pinned Model Compiler and a compatible Modalix DevKit are submission prerequisites, not optional CI substitutes. Do not open a model pull request until every submitted build has compiled, produced valid MPKs, and completed its declared performance and accuracy stages on a DevKit. CI repeats those stages after the merge. It does not replace local qualification.
Workflow
A contribution moves through six steps. Each produces what the next one needs. Select a step to open its guide.
- Establish contractPin source + datasetsDefine YAML, recipe, testsSuccessComplete, reviewablemodel contractDefine the model contract →
- Qualify locallyValidate + fetch inputsCompile + verify variantsSuccessOne valid MPKper tested variantQualify locally →
- Benchmark MPKRun the exact MPKon a Modalix DevKitSuccessMachine-readableperformance resultBenchmarking →
- Prepare GitCommit inputs onlyDocument local evidenceSuccessFocused, reviewablepull requestSubmit for review →
- PR validatesValidate catalog + matrixKeep contributor code off VulcanSuccessSafe, deterministicreview signalPR checks →
- Trusted automationCompile + measure accuracyPublish + benchmark stagingSuccessTrusted evidenceand provenanceAfter merge →
Before you begin
You need a checkout of this repository, sima-cli, and the repository-pinned
Model Compiler from the NEAT SDK or a standalone installation. The DevKit
stages need a compatible Modalix DevKit. Models that use calibration,
fidelity, or accuracy data also need private Artifactory access.
Start with Define the model contract. The local qualification guide owns the tool installation and complete test sequence. Follow it before you submit anything.
Guides
Follow these guides in order:
- Define the model contract: the model directory, scripts, components, sources, build profiles, packages, and stage entries.
- Qualify locally: install the contributor tool, then compile and measure every submitted build.
- Submit for review: commit only reviewed inputs and report the stages you ran.
- Promote a model: make a tested build of a build-on-demand model available for download.
Use these references when the model needs them:
- Concepts: build, build profile, stage, benchmark, fingerprint, model component, model package, and MPK.
- Model README: the README and hero rules, with the README template.
- Datasets for local model development: Artifactory access, registered dataset resolution, and adding a dataset or variant.
- Accuracy and fidelity: compile-time numerical fidelity and DevKit task accuracy.
- DevKit performance: the performance script, its workload, and how results are published.
model.yamlreference: the meaning of each field.- Expensive and on-demand compile runners: decide whether a model qualifies for an exception, prepare approval evidence, configure the request, and verify a build.
- Model taxonomy: sanctioned category IDs.
The JSON Schemas under
schemas/ and the
taxonomy files under
taxonomies/
are authoritative. These guides explain how to work with them.