Show HN: BYOSM, Turn one sentence into a small specialist model

Tour · Goal and bet · Methodology · Results · Details

Build Your Own Specialist Model.

One sentence in, a small model for that one job, scored against LLMs, Jev and fine-tuned models on public benchmarks, on any CPU.

Run it »  ·  Results »  ·  Models on Hugging Face »

Build Evaluate Observe Ship

✅ Claude Code or Codex ✅ Same scorer for all rivals ✅ Every job live ✅ int8 ONNX on any CPU

✅ Ollama, LM Studio, API keys ✅ No leaks between splits ✅ LLM decisions as a tree ✅ HTTP server and Docker

✅ Hugging Face datasets ✅ Overfitting check ✅ Tokens per task and model ✅ Publish to Hugging Face

✅ LLM data, judge-checked ✅ Health score ✅ Test sets, Excel export ✅ Teams that act as you type

✅ Experiments and autopilot ✅ Hugging Face rivals ✅ Paged tables, live counts ✅ SQLite or Postgres

What's new, and what isn't

The recipe is not new: fine-tune a small pretrained encoder on labelled examples. BYOSM automates everything around it from one sentence. It finds a public benchmark and training data on Hugging Face, or has an LLM write the data and a second LLM check the labels. It trains, proposes and runs experiments, and scores the result against the models you would otherwise use, with the same code and a paired bootstrap:

Close relatives: SetFit (few-shot fine-tuning of sentence encoders) and AutoTrain (no-code fine-tuning on your data). BYOSM starts from a sentence instead of a dataset, and its output is a comparison you can trust as much as the model.

git clone https://github.com/bsye/BYOSM && cd BYOSM python3.11 -m venv .venv && .venv/bin/pip install -e '.[dev,silero]' .venv/bin/byosm app # opens http://127.0.0.1:8765 with its token

State lives in one database: SQLite by default, Postgres with DATABASE_URL (pip install -e '.[postgres]').

Step What you do

1. Describe the task One sentence, and the LLM you have

2. Watch it build Follow each job live

3. Check its health See what fails and what to try next

4. Compare with rivals Every model on the same splits

5. Test it on new inputs Your test sets and Hugging Face rivals

6. Act while the user types A team fires before the sentence ends

7. Choose the LLMs What each task uses and costs

8. Ship it Python, HTTP, Docker or Hugging Face

9. Describe the task

One sentence and the LLM you have; the autopilot does the rest.

Every job live: its stages, the LLM calls behind each, progress and the log.

A health score with every check behind it, and the experiments the LLM proposes to fix what fails.

Test and calib scores with 95% intervals for every experiment and rival, scored by the same code.

Hugging Face models for the same task, ready to add as rivals, and test sets with every model's score.

A team of specialists picks the action and its arguments while the sentence is still being typed.

Tokens and calls per task and model, and the model and effort each task uses.

One ONNX bundle to call from Python, the command line, HTTP or a Docker image.

Models on Hugging Face: bsye · byosm-doctype · byosm-intent · byosm-vad

A small model "beating GPT" is easy to fake. Each of these guards runs in code on every specialist; none is a promise in prose.

Rules and benchmarks: Methodology. Every number: Results.

Tour · Goal and bet · Methodology · Results · Details