返回资讯中心
外部精选
企业动态
#开源#模型发布

Hugging Face 介绍决策模型及其生态支持

来源标题:Decision Models in Hugging Face

Hugging Face 发布文章介绍决策模型,说明并非所有问题都需要用 LLM 生成文本。文章对比了决策模型与传统分类、零样本分类的区别,并介绍其接口、状态与问题原语,以及在 Transformers、llama.cpp 和 Hub 中的使用与训练方式。

Hugging Face Blog2026/11/10 · 8 天前30 分钟阅读

以下正文同步自 Hugging Face Blog,版权归原站所有,已转换为易读排版。

Back to Articles

a]:hidden">

Decision Models in Hugging Face

Published November 10, 2026

Update on GitHub

Upvote -

Aritra Roy Gosthipaty ariG23498 Follow

Lysandre lysandre Follow

Pedro Cuenca pcuenq Follow

Tom Aarsen tomaarsen Follow

:last-child]:mb-0"> What are decision models?

Why not use LLMs for everything?

How are decision models different from classification models? Traditional classification

Zero-shot classification

Where decision models differ

The decision model interface State

Questions and their primitives

Decision models in the Hugging Face ecosystem 1. Serving with Transformers

  1. Serving with llama.cpp

  2. Discovering models on the Hugging Face Hub

  3. Training decision models using Hugging Face tooling

Final thoughts

Large Language Models (LLMs) are exceptionally good at generating text. We use them to answer questions, write code, reason about problems, and so much more. But not every problem requires generating text.

Consider a customer support system receiving the following message:

"I was charged twice for my subscription. I need a refund today."

The system needs to determine which team should handle the request, how urgent it is, and whether the customer is requesting a refund.

We could ask an LLM to return a JSON object containing these answers.

llama cli -hf ggml-org/gemma-4-e4b-it-GGUF:Q4_0

We have the following customer support ticket, "I was charged twice for my subscription. I need a refund today.", which team should handle the request, how urgent it is, and whether the customer is requesting a refund. Give the answers in JSON.

[Start thinking] ... [end thinking]

{
 "team_to_handle": "Billing",
 "urgency": "High",
 "request_refund": true
}

But what if the model could directly evaluate the available options and return a probability distribution over them, without generating any text?

from transformers import pipeline

decide = pipeline("decision", model="autotrust/GEV-26B-Decide")
decision = decide(
 {
 "state": "I was charged twice for my order and want a refund.",
 "questions": {
 "department": {
 "type": "choice",
 "instructions": "Which team should handle this?",
 "criteria": {
 "billing": "Payments and refunds",
 "technical": "Bugs and technical issues",
 "sales": "Pricing and subscriptions"
 }
 },
 }
 }
)

# {
# 'department': {
# 'type': 'choice',
# 'choice': 'billing',
# 'confidence': 0.9974000453948975,
# 'probabilities': {
# 'billing': 0.9982666969299316,
# 'technical': 0.0010246571619063616,
# 'sales': 0.0007087149424478412
# }
# }
# }

This is the key idea behind decision models.

What are decision models?
 

A decision model takes some input context (state), evaluates a question with a set of predefined possible answers (primitives), and returns a structured decision along with probabilities. Instead of asking a model to generate an answer token by token, we ask it to assign probabilities to the answers that our application already understands.

TypeSafe AI introduced Jev, describing it as a System One model. The name draws inspiration from the distinction between fast, intuitive thinking (System 1) and slower, deliberative thinking (System 2).

The community has converged to the term decision model, because it describes the model's computational role rather than invoking a particular theory of cognition. In the rest of the post we will refer to the models as "decision models" instead of "system one models". (A poll on twitter deciding what we should be calling such models)

For a decision model, the fundamental interface is:

State + Questions + Allowed Answers → Decisions + Probabilities

 
 
 
 
 Why not use LLMs for everything?
 

We could simply prompt an LLM to return a JSON object containing our decisions. In fact, this is how many applications already use language models for classification, routing, and other structured tasks.

However, there are a few problems with this approach.

How are decision models different from classification models?
 

Before understanding the interface of a decision model, it is useful to revisit two closely related approaches namely traditional classification and zero-shot classification.

Traditional classification
 

Suppose we train a classifier that routes customer-support requests to three departments:

A conventional classification model learns a mapping from an input to a predefined set of labels. During training, the model learns to predict these labels from examples. A classification head typically produces one logit per class, and softmax turns those logits into a probability distribution.

The limitation here is that the output classes are baked into the model's classification head.

If we introduce a fourth department, such as Legal, the existing classifier does not automatically acquire a corresponding output class. We may need to modify the head, collect training data, and retrain or adapt the model.

Zero-shot classification
 

Zero-shot classification addresses an important part of this limitation. Instead of requiring the classifier to be trained specifically on every output category, we supply the candidate labels at inference time.

from transformers import pipeline

classifier = pipeline(
 "zero-shot-classification",
 model="facebook/bart-large-mnli",
)

classifier(
 "My payment was deducted twice.",
 candidate_labels=["billing", "technical support", "sales"],
)

# {'sequence': 'My payment was deducted twice.',
# 'labels': ['billing', 'sales', 'technical support'],
# 'scores': [0.8466812372207642, 0.0962134450674057, 0.05710534751415253]}

Unlike a conventional fixed-label classifier, the categories can change from one request to another. Let’s add another department, such as “Legal”, and see the model outputs.

from transformers import pipeline

classifier = pipeline(
 "zero-shot-classification",
 model="facebook/bart-large-mnli",
)

classifier(
 "My payment was deducted twice.",
 candidate_labels=["billing", "technical support", "sales", "legal"],
)

# {'sequence': 'My payment was deducted twice.',
# 'labels': ['billing', 'sales', 'technical support', 'legal'],
# 'scores': [0.8185835480690002, 0.09302053600549698, 0.05521026626229286, 0.03318570926785469]}

One common implementation uses a model trained for natural language inference (NLI). Each candidate label is converted into a hypothesis, and the model measures how strongly the input entails that hypothesis.

This already sounds quite similar to a decision model. And in many ways, it is.

Where decision models differ
 

Decision models extend this idea beyond simply assigning an input to a category. A decision model can accept several named questions about the same state, with different answer types. The types can be choosing between alternatives (choice), scoring an ordered scale (score), or estimating the probability that a statement is true (noul).

More importantly, its interface is designed around the probability distribution over possible decisions.

The advantage of decision models is not that they invented runtime-defined labels or probabilistic classification. It is that they organize these capabilities into a consistent interface, potentially backed by architectures and training procedures optimized specifically for decisions.

The decision model interface
 

A decision model request has the following two components:

State
 

The state is the information the model should evaluate. It may be a string, a structured JSON object, or, other modalities for supported omni-modal models.

{
 "customer_message": "I was charged twice. Please refund me today.",
 "subscription": "pro",
 "previous_tickets": 2
}

 
 
 
 
 Questions and their primitives
 

Questions describe what we want to know about the state. In the decision models interface, three fundamental primitives represent different forms of decisions:

Choice: A probability distribution over options
 

Choice is the most intuitive primitive and the closest to traditional classification.

Which department should handle this customer request?

The available options are Billing, Technical support, and Sales.

from transformers import pipeline

decide = pipeline("decision", model="autotrust/GEV-26B-Decide")
decision = decide(
 {
 "state": "I was charged twice for my order and want a refund.",
 "questions": {
 "department": {
 "type": "choice",
 "instructions": "Which team should handle this?",
 "criteria": {
 "billing": "Payments and refunds",
 "technical": "Bugs and technical issues",
 "sales": "Pricing and subscriptions"
 }
 },
 }
 }
)

# {
# 'department': {
# 'type': 'choice',
# 'choice': 'billing',
# 'confidence': 0.9974000453948975,
# 'probabilities': {
# 'billing': 0.9982666969299316,
# 'technical': 0.0010246571619063616,
# 'sales': 0.0007087149424478412
# }
# }
# }

The model assigns a logit to each option. For logits ($z_1,\ldots,z_K$), (with K classes) we can obtain the probabilities using softmax:

pi=exp⁡(zi/T)∑j=1Kexp⁡(zj)
p_i=\frac{\exp(z_i/T)}{\sum_{j=1}^{K}\exp(z_j)}
pi​=∑j=1K​exp(zj​)exp(zi​/T)​

∑i=1Kpi=1.
\sum_{i=1}^{K}p_i=1.
i=1∑K​pi​=1.

The model chooses the option with the largest probability:

y^=arg⁡max⁡ipi
\hat y=\arg\max_i p_i
y^​=argimax​pi​

In this case, the choice is Billing, with the probability of 0.998, but the remaining probabilities are useful too. They tell us how the model distributes its belief among the alternatives.

Score: The expectation of an ordered distribution
 

Choice works when there are several distinct alternatives. But what if we want to measure something on an ordered scale?

from transformers import pipeline

decide = pipeline("decision", model="autotrust/GEV-26B-Decide")
decision = decide(
 {
 "state": "I was charged twice for my order and want a refund.",
 "questions": {
 "urgency": {
 "type": "score",
 "instructions": "How urgent is the request?",
 "criteria": ["low", "medium", "high"]
 },
 }
 }
)

# {
# 'urgency': {
# 'type': 'score',
# 'score': 1.7053890228271484,
# 'legend': {
# '0': 'low',
# '1': 'medium',
# '2': 'high'
# },
# 'confidence': 0.5580834252759814,
# 'probabilities': {
# '0': 0.014908230863511562,
# '1': 0.26479458808898926,
# '2': 0.7202972173690796
# }
# }
# }

The model again produces a probability distribution:

Rather than returning only the most likely level, the Score primitive computes the expected value of the level index.

For a random variable (X) representing the level, the expectation is:

E[X]=∑i=0K−1ipi
\mathbb{E}[X]=\sum_{i=0}^{K-1}ip_i
E[X]=i=0∑K−1​ipi​

E[X]=0(0.10)+1(0.26)+2(0.72)=1.7
\mathbb{E}[X] = 0(0.10)+1(0.26)+2(0.72)=1.7
E[X]=0(0.10)+1(0.26)+2(0.72)=1.7

This does not mean that the model selected a fictional level halfway between medium and high. It means that the expectation of its probability distribution is 1.7.

Noul: A Bernoulli decision
 

The third primitive is Noul, which represents a binary question.

from transformers import pipeline

decide = pipeline("decision", model="autotrust/GEV-26B-Decide")
decision = decide(
 {
 "state": "I was charged twice for my order and want a refund.",
 "questions": {
 "refund": {
 "type": "noul",
 "instructions": "Is the customer requesting a refund?"
 }
 }
 }
)

# {
# 'refund': {
# 'type': 'noul',
# 'noul': 0.9947046637535095
# }
# }

Rather than defining a list of choices, we ask the model to return the probability that the answer is true. We can understand this using the Bernoulli distribution (noul comes from Bernoulli).

Y∼Bernoulli⁡(p)
Y\sim\operatorname{Bernoulli}(p)
Y∼Bernoulli(p)

P(Y=1)=p,P(Y=0)=1−p.
P(Y=1)=p,\qquad P(Y=0)=1-p.
P(Y=1)=p,P(Y=0)=1−p.

{"type": "noul", "noul": 0.92}

it assigns a probability of 0.92 to Yes and 0.08 to No.

The model is not randomly sampling a binary answer here. It returns the probability parameter itself, leaving the application to decide how to act on it.

Confidence: How certain is the model?
 

So far, we have obtained probability distributions from the model. But the decision model

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →