返回资讯中心
外部精选
软件工程
#开发者实时流

Basal-Omni: An open-weight 31B model for AI decision-making (nr 1 on JevBench)

Use this model Instructions to use Remek/basal-omni-1.0-ultra with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Hacker News 最新lukasm2026/10/18 · 2026/10/1830 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

Like 6

Text Generation

Transformers

Safetensors

Polish

English

gemma4

image-text-to-text

typed-decisions

decision-making

classification

calibration

polish

experimental

conversational

License: apache-2.0

Model card Files Files and versions xet

Community

Deploy

Copy to bucket new

Use this model Instructions to use Remek/basal-omni-1.0-ultra with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

Transformers How to use Remek/basal-omni-1.0-ultra with Transformers:

Use a pipeline as a high-level helper

from transformers import pipeline

pipe = pipeline("text-generation", model="Remek/basal-omni-1.0-ultra") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)

pip install -U transformers accelerate

Load model directly

from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Remek/basal-omni-1.0-ultra") model = AutoModelForMultimodalLM.from_pretrained("Remek/basal-omni-1.0-ultra", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks

Google Colab

Kaggle

Local Apps Settings

vLLM How to use Remek/basal-omni-1.0-ultra with vLLM:

Install vLLM from pip:

pip install vllm

Start the vLLM server:

vllm serve "Remek/basal-omni-1.0-ultra"

Call the server using curl (OpenAI-compatible API):

curl -X POST "http://localhost:8000/v1/chat/completions"
-H "Content-Type: application/json"
--data '{ "model": "Remek/basal-omni-1.0-ultra", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'

Use Docker

docker model run hf.co/Remek/basal-omni-1.0-ultra

SGLang How to use Remek/basal-omni-1.0-ultra with SGLang:

Install SGLang from pip:

pip install sglang

Start the SGLang server:

python3 -m sglang.launch_server
--model-path "Remek/basal-omni-1.0-ultra"
--host 0.0.0.0
--port 30000

Call the server using curl (OpenAI-compatible API):

curl -X POST "http://localhost:30000/v1/chat/completions"
-H "Content-Type: application/json"
--data '{ "model": "Remek/basal-omni-1.0-ultra", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'

Use Docker images

docker run --gpus all
--shm-size 32g
-p 30000:30000
-v ~/.cache/huggingface:/root/.cache/huggingface
--env "HF_TOKEN=<secret>"
--ipc=host
lmsysorg/sglang:latest
python3 -m sglang.launch_server
--model-path "Remek/basal-omni-1.0-ultra"
--host 0.0.0.0
--port 30000

Call the server using curl (OpenAI-compatible API):

curl -X POST "http://localhost:30000/v1/chat/completions"
-H "Content-Type: application/json"
--data '{ "model": "Remek/basal-omni-1.0-ultra", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'

Docker Model Runner How to use Remek/basal-omni-1.0-ultra with Docker Model Runner:

docker model run hf.co/Remek/basal-omni-1.0-ultra

Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

basal-omni-1.0-ultra (experimental) Results

Quick start

Images

Speed

Files

How it works

Training

Limitations

Citation

License and attribution

basal-omni-1.0-ultra (experimental)

basal-omni-1.0-ultra (31B) is the largest model of basal-omni, basal's line on Google Gemma 4. It is an experimental release: it is trained on text decisions only, and it also answers the same questions about images, read by Gemma 4's own image encoder without media training (zero-shot; media training comes in the next releases). The 31B base has no audio encoder, so this tier takes no audio. It answers the same typed questions as basal-1.5, over the same API.

What it is. Inspired by System 1 (fast, intuitive) decision models such as Jev. Instead of writing an answer, the model reads a state (a message, a document, retrieved passages, an agent trace, a web page as JSON). It then answers a typed question about it (choice, yes/no noul, score, and in the basal engine multi, act and tool; a tool question over OpenAI or MCP tool definitions picks the tool, not its arguments). The answer is a calibrated probability for each allowed answer, in one forward pass, without generating text. The answer never falls outside the options you give, and the probability says how sure the model is.

Results

Served with the basal engine: basal-omni-1.0-ultra on one H200 and basal-omni-1.0-max on one H100, both with --mode eager, bf16, one option order; basal-1.5-max as released (--mode fast, both option orders). Same items and the same clients for all three. Measured with basal 2.0; basal 2.1 gives the same top answer on 23,278 of 23,278 served-check rows.

Quick start

  1. Install into a fresh uv environment:

uv venv --python 3.12 --allow-existing ~/basal-env && source ~/basal-env/bin/activate uv pip install torch==2.11.0 torchvision==0.26.0 --index-url https://download.pytorch.org/whl/cu128 uv pip install "basal[media] @ https://github.com/rkinas/basal/archive/refs/tags/v2.1.0.tar.gz"

  1. Start the server. It prints basal: model ... ready:

basal-serve --model Remek/basal-omni-1.0-ultra --mode eager --media on --port 8000

This release runs in the engine's eager mode only (PyTorch, bf16, about 62 GB of GPU memory for the weights; on one 80 GB GPU, states up to about 10,000 tokens, see Limitations). The fast mode (shared prefix, CUDA graphs), the FP8 checkpoints and the MLX / GGUF ports of basal-1.5 come with a later release of this line.

  1. Ask questions, exactly as with basal-1.5:

curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{ "state": "Klient: od wczoraj nie mogę zalogować się do bankowości internetowej, system pokazuje błąd hasła.", "questions": { "dept": {"type": "choice", "instructions": "Do którego działu skierować zgłoszenie?", "criteria": {"cards": "Reklamacje kart", "online": "Wsparcie bankowości elektronicznej", "loans": "Kredyty"}}, "urgent": {"type": "noul", "instructions": "Czy klient nie może korzystać z usługi?"}}}'

Full API (multi, act, tool, option keys, max_error): API.

Images

With --media on, a request may carry images with its state. They are placed where the state text goes, before it, in the order given; the state text may be empty.

IMG=$(base64 < paragon.jpg | tr -d '\n') curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{ "state": "", "lang": "pl", "media": [{"type": "image", "data": "'"$IMG"'"}], "questions": { "card": {"type": "noul", "instructions": "Czy zapłacono kartą?"}, "vat": {"type": "choice", "instructions": "Jaka jest najwyższa stawka VAT na paragonie?", "criteria": {"23": "23%", "8": "8%", "5": "5%", "0": "0% lub zwolniony"}}}}'

"media": a list of {"type": "image", "data": "<base64>"} (a data: URL works too), any format Pillow reads. Audio is refused with HTTP 422 (this model does not take audio (it takes: image)): the 31B base has no audio encoder.

"detail": "default" (280 image tokens) or "high" (1,120; small print, slower).

"lang": "pl" or "en", the prompt language when the state text does not show it (a state that is only media).

Limits: 8 items and 20 MB each per request, 50 MP per image.

Every question of a request sees all of its images. For the steadiest answers, send each question with only the images it is about.

Answers carry "calibration": "text": the temperatures and the certified thresholds were fitted on text; validate the thresholds on your own media before automating with them.

On omni-probe-pl, our small Polish probe (each item asked from the image and from its text):

Speed

One question per request, one option order, on one H200 (--mode eager, bf16, --media on):

Files

model-*.safetensors, tokenizer, chat_template.jinja, processor_config.json: the model. Gemma 4 architecture, 31B, 262k vocabulary.

Gemma's image encoder is included and unchanged (this release is trained on text); the engine reads images through it with --media on (processor_config.json). The 31B base has no audio encoder.

CALIBRATION.json: per-type temperatures and certified confidence thresholds for 1% / 5% error, fitted on the served engine (--mode eager, bf16, one option order).

CALIBRATION.o2.json: the same for --orders 2 (both option orders averaged).

basal.json: prompt format 2 and the readout protocol.

How it works

The model receives a fixed chat prompt with the state, the question and coded options (A–Z, then two-character codes, up to 255). The assistant turn is prefilled with {"answer": ", and the decision is the softmax over the next-token logits of the option codes only: one forward pass, no text generation. The server applies the calibrated temperature of the question type from CALIBRATION.json.

Use the confidence. CALIBRATION.json stores thresholds for a target error of 1% or 5% among accepted decisions. They were chosen on calibration data before testing; on the test data they accept 56.8% / 94.2% of decisions at 0.47% / 3.8% observed error (at the 1% target score questions have no certified threshold and are deferred). Accept decisions above the threshold automatically and route the rest to a person.

Training

Base and method: fine-tuned from google/gemma-4-31B-it with LoRA (rank 32) on the language-model layers, merged into the weights. The image encoder was not trained.

Data: about 640,000 typed decisions in Polish and English: rules and deadlines, real documents, long documents, retrieval decisions, contracts, routing, ordinal scores, judging answers, agent and tool decisions, and "cannot be determined" answers.

Distillation: basal-1.5 provides soft targets on the decisions it answers correctly.

Schedule: 10,000 of the 12,741 planned steps of 64 decisions (640,000 of the basal-1.7 mix's 815,386, without its 1,931 items longer than 6,144 tokens) at peak learning rate 3e-5 with cosine decay; stopped at step 10,000 because the dev panel had stopped improving (0.908, 0.909 and 0.909 at steps 8,000, 9,000 and 10,000), then the LoRA was merged into the weights in float64 and rounded once to bf16.

Limitations

Experimental. It is the first model of a new base; validate it on your own documents before relying on it.

Images are zero-shot. Gemma 4's encoder reads them, but this model was trained on text decisions only; image answers are weaker than text where the image itself is hard (phone photos), see "Images". Their calibration is the text calibration. No audio: the 31B base has no audio encoder.

Size and speed: 31B parameters, about 62 GB in bf16, about 2× slower than basal-omni-1.0-max (see Speed). The longest state it answered with one option order: about 10,000 tokens with the memory capped at an 80 GB H100's (8,000 with --orders 2), and about 32,000 tokens on one 141 GB H200. A longer state fails with an out-of-memory error (HTTP 422), and the server keeps serving.

Exact arithmetic and thresholds are a weak spot of one-pass decisions: compute them in code.

Option order: one order is asked by default; --orders 2 reduces sensitivity to option order at twice the cost.

tool with no fitting tool: ultra usually answers "no suitable tool" (4 of 6 out-of-scope requests), b

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →