
Show HN: Free2Genius – A growth system where a model decides who to contact
A growth system where a model decides who to contact and a locally-hosted agent decides
以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。
A growth system where a model decides who to contact and a locally-hosted agent decides what to say — and where both are built so they can say "not you" and "not worth it".
All data here is synthetic, generated by f2g/data/generate.py. No Albert data, customer data or production model was used. Pricing is an illustrative placeholder. Every number in this README is reproducible from a clean checkout with make all.
A narrated tour of the running system: the value-fit gate that suppresses 7,530 candidates the product would not pay for, evidence chips that trace every figure back to the tool call behind it, the scope policy that declines investment questions before any model loads, a real open-weights model answering through Ollama, and always-valid intervals on the experiment.
It also shows the system getting something wrong. The local model grounds every number correctly and then draws the opposite conclusion from them — kept in the cut, because no guardrail here checks reasoning, and that is the argument for human review before send.
The video is rendered from this repository rather than screen-recorded: voice, browser and edit all come from python scripts/demo/record.py. See scripts/demo/.
Most free users never see why a paid tier is worth paying for, and generic upsell copy does not help because it is generic. The obvious build is a propensity model — predict who will convert, contact the top decile — and it is the wrong build twice over.
It optimises the wrong quantity. On this population, ranking by predicted conversion spends 96.7% of a 10% contact budget on users who were going to convert anyway, then books their conversions as programme impact.
It cannot decline. Some users respond negatively to contact. Here they are the fee-stressed, high-support-contact segment, at −2.2 pp uplift. A propensity model has no way to express "do not contact this person"; it only knows "less likely".
Decision Owner Signal
Who is movable? Uplift model (X-learner) Treatment effect estimated from a randomised pilot
Who should be moved? Targeting policy Effect, subject to budget, do-no-harm, fairness and value fit
What do we say? Concierge agent The user's own fee history, narrated under hard guardrails
The idea that ties them together
The agent's grounded savings estimate is a hard constraint inside the targeting decision, not a check beside it. Before contacting anyone, the system computes from that user's own 90-day fee history what Genius would actually save them, and suppresses them if it does not clear the subscription price — no matter how movable the model says they are.
The consequence is structural rather than aspirational: the system cannot hit its conversion target by pushing users who will not benefit, because the honesty check sits inside the decision. On the live cohort the gate is the binding constraint, suppressing 7,530 of 12,000 candidates and raising the mean estimated saving of contacted users from $34.75 to $70.21.
All reproducible. Commands in Running it.
Uplift ranking vs propensity, at the 20% operating budget +32.9% more incremental conversions
X-learner Qini 0.782 vs propensity 0.673, against an oracle ceiling of 1.282
Correlation with the true effect Spearman 0.633
Bottom-decile realised uplift −2.39 pp — the model locates users contact would harm
Doubly-robust policy value covers the oracle for all five candidate policies
The oracle column exists because the population is simulated, and it is the strongest evidence here: we can check the estimator against a known answer, which no real dataset permits.
Runs on Qwen2.5-1.5B / 3B, quantized, in-process — no API key, no network, zero marginal cost per generation.
Naive fixed-horizon test, dashboard read 200 times under no true effect 49% false positives at a nominal 5%
Always-valid confidence sequence, same conditions 3%
Three decisions worth arguing about
Local open-weights inference, by default. (ADR-001) The agent reads transactions, advances and fee events. Running a 1.5–3B model in-process means no financial attribute crosses a network boundary, and — the part that matters more — the evaluation suite costs nothing to run, so the zero-grounding-violation gate can cover every case on every commit. A metered API would have forced sampling, and a sampled safety gate is not a gate.
Reliability is the runtime's job, not the model's. (ADR-003) Small models emit malformed tool calls constantly. Rather than accept that, tool calling is enforced by a GBNF grammar generated from the tool schemas, so malformed JSON is unrepresentable rather than handled. Enum parameters become literal alternatives — which means a hallucinated product feature cannot be emitted at all. Then schema validation, then a bounded repair loop that raises rather than returning a best guess.
Grounding is a mechanism, not an instruction. (ADR-007) Every number any tool returns is recorded in a per-session ledger. Every number in the draft message must be in it. An unaccounted figure blocks the message and the turn degrades to a templated writer that is grounded by construction. This is why a 1.5B model is safe to deploy here: it did fabricate seven dollar figures in testing, and none of them could reach a user.
Prerequisites
Python 3.11+ required — everything except the browser console (tested on 3.11 and 3.13)
Node 18+ only for the browser console
Check Python is present before anything else. On Windows use the launcher, which is what the official installer puts on PATH:
py --version # Windows python3 --version # macOS / Linux
If that fails, install Python 3.11 or newer:
This project lives on the branch claude/sleepy-clarke-2h5iey, which is currently the only branch on the remote. After cloning, make sure you are on it and up to date:
git fetch origin && git checkout claude/sleepy-clarke-2h5iey && git pull
./run.sh setup # macOS / Linux run.bat setup # Windows ./run.sh all # run.bat all ./run.sh api # :8000, OpenAPI at /docs ./run.sh web # :5173 <- open this ./run.sh test # 205 tests
The wrappers find a working interpreter for you. If you already know your Python command, python run.py <task> (or py run.py <task> on Windows) is equivalent. Run with no arguments to list every task. A Makefile with the same targets exists for anyone who prefers make.
To run the agent on a real local model rather than the deterministic writer, pick one of two paths and then choose it from the provider dropdown next to Regenerate (nudge card) or Send (chat) in the console's Concierge tab.
Already running Ollama, LM Studio or vLLM — nothing to download, it reuses the models you have. Defaults assume Ollama:
ollama list # pick any instruct model you already have export F2G_OPENAI_MODEL=qwen2.5:7b # default; set F2G_OPENAI_BASE_URL for non-Ollama python run.py api
Then select Ollama / local server in the dropdown.
In-process llama.cpp — no server, but it downloads its own weights:
.venv/bin/pip install llama-cpp-python # .venv\Scripts\pip on Windows python run.py models # ~2.9GB of open weights
Either way generation takes 60-110 seconds — the latency gap recorded as unmet in spec 005. An explicitly chosen provider fails with a 503 naming what it needs, rather than quietly falling back.
The system is fully functional without those weights: the deterministic provider is a production runtime, not a test stub (ADR-006), and the default provider falls back to it with a warning when no weights are present.
Start with docs/learn/ — a seven-part guide that teaches the system from zero: the business problem, the architecture, a low-level walk through each layer, the nine build stages and what broke at each, and a demo script with the questions you will be asked. About four hours end to end; the ten-minute orientation is at the top.
Spec-driven, using GitHub Spec Kit. Nine features across five domains, each with a specification, plan, task list, and high- and low-level design documents. The register is at specs/README.md; the governing principles are in .specify/memory/constitution.md.
Several specifications were corrected by their own implementation — those corrections are recorded in place rather than quietly edited out, because they are the most honest part of the record. Two examples:
f2g/ ├── data/ synthetic population, account ledger, pricing catalog ├── ml/ feature pipeline, estimators, evaluation, policy, offline policy evaluation ├── llm/ provider abstraction, GBNF grammar, validation, repair loop ├── agent/ tools, scope policy, guardrails, the concierge loop ├── evals/ golden cases, LLM judge, the CI gate ├── experiment/ assignment, always-valid analysis ├── api/ FastAPI service, event store, telemetry └── governance/ model card generator web/ React + TypeScript console specs/ nine feature specifications, plans and task lists docs/ architecture (HLD/LLD), ADRs, governance, demo script tests/ 205 tests
Limitations
Stated here rather than left to be found:
正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。