Cadence is the workbench where we fine-tune and evaluate the models behind Oasis Speech. People and coding agents work in it through the same API, every change is versioned, and no model reaches production without passing a gate. The public repository, github.com/usunrise88/cadence, is a fork of the system we run in production, under the MIT license.
This note describes the repository. It is not a measurement note.
| In short | What it means |
|---|---|
| Production fork | The public repository is a fork of the system we train Oasis Speech in |
| MIT | The license of the public repository |
| One contract | One OpenAPI file generates the server, the web client, the agent tools and the command line |
| Train, evaluate, gate | A model is registered only after it passes its gate |
Why we built it
A speech recognizer improves by fine-tuning on the kind of audio it will hear. Doing that once takes a script. Doing it every month, for many languages, with a team and with coding agents helping, takes a process, and the process needs a place to live. That place records where the data came from, which recipe produced which model, what the evaluation set was and who approved the run that cost a day of GPU time.
Cadence is that place. We train and compare Oasis Speech in it. It covers the whole life of a fine-tune: data, training, evaluation, export, deployment and the production loop that feeds new data back.
Two ideas
Agent-native
A coding agent works in Cadence through the same API as the interface. Its edits arrive as drafts attributed to the agent’s session. Actions that spend money or change a shared asset wait for a person’s approval. Every panel in the interface updates live as the agent works, so a person watching sees a mix of datasets change while the agent edits it.
API-first
There is one OpenAPI contract. The Go server, the web client, the agent tools (MCP) and the command line are generated from it, so an action has one name everywhere: mixes.edit is the operation, the tool and the interface command.
Cadence versions everything. Recipes live in git. Datasets, models and golden evaluation sets are immutable versions in a registry. Every change has an actor, an idempotency key and an audit row.
What it does
Training
- Workers. GPU workers register what they can run, lease jobs over HTTP, send heartbeats, stream logs and metrics, and publish checkpoints while they train.
- Pipelines. Pipelines are YAML in the project’s git repository, pinned to step versions and validated before they run. Cadence reuses results by input hash, so an unchanged step does not run twice.
- Runs. A run is a training stage over a pipeline, with calibrated time estimates, top-k checkpoints, live metrics, and pause and resume.
- Budgets. Each project and each agent session has a GPU budget. An action over budget waits for approval, in the Approvals panel or from Telegram. The budget fails closed and counts queued work, so a queue of cheap jobs cannot add up past the limit.
- Operations. A content store with a disk gauge and eviction, nightly backups with a weekly restore test, notifications and a daily digest.
Evaluation
- Golden sets. Cadence freezes golden sets from datasets marked evaluation-only, with an approval, and ties each one to a scoring normaliser. It checks every training input for the audio of every golden set, so an evaluation clip cannot leak into training.
- Evals. An eval runs a checkpoint and its baseline through the real streaming decoder at each latency profile. It scores error rates, substitutions, deletions and insertions, streaming behavior and entity metrics. Cells are cached across projects, and differences carry paired bootstrap intervals.
- Gates. A gate turns the whole matrix into one verdict. A model that passes its gate can be registered. One that fails stays out.
- Panels. The report with charts, a diff between runs, an audio view (waveform, spectrogram, words), live manual transcription tests with up to three models side by side (microphone, file or a span of a file; Cadence stores nothing from them), experiments and sweeps, language packs with boost lists, and lineage.
What is next
Planned next: mounts, ingest, freezing and annotation. After them come export, shadow and canary deployment, triage and schedules, which close the loop from production calls back to training data. The roadmap is in the repository.
Where it fits
Cadence is the training side of the stack. Oasis Speech is the recognition service that runs on the models Cadence produces, Polygon grades the agents that use them, and the load test measures the servers that run them.
- Oasis Speech 2
- How we grade a conversation
- Server Unit under load
- Cadence is public, the announcement