返回资讯中心
外部精选
软件工程
#开发者实时流

Classifying images with a 2B decision model instead of a vision LLM

Classifying Images with Strands Decider 2B Decision Model

Hacker News 最新burningman19492026/11/10 · 7 天前30 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

...

Classifying Images with Strands Decider 2B Decision Model

AWS Builder Center

Classifying Images with Strands Decider 2B Decision Model

Classifying Images with Strands Decider 2B Decision Model

Classify images with Strands Decider, a 2B multimodal decision model. A security camera example covers setup, code, and accuracy and cost vs. Bedrock LLMs.

Follow

AWS Employee

Published Nov 10, 2026

Last modified Nov 10, 2026

60

Classifying images without generating text

Many applications that need visual understanding send an image to a model and only need a short, fixed answer back. A loan or claims intake system checks whether the signature block on an uploaded form is filled in. An insurance app sorts claim photos by what is damaged: roof, window, siding, or nothing. A camera on a factory line grades each part's defect as none, cosmetic, or functional. A security monitoring system asks whether the fire exit in a camera frame is blocked. In each case the answer is yes or no, one option from a list, or a level on a scale, and software acts on it: it routes the claim, rejects the part, or alerts an operator.

Today there are two common ways to answer these questions:

A vision LLM. It works with no training, but it writes text. You pay for output tokens, parse the answer, and handle the cases where the format is wrong.

A trained image classifier. It is fast and cheap to run, but each new label needs its own labeled data and training.

A decision model sits between the two. Like an LLM, it takes a question in plain language and needs no training for a new question. Like a classifier, it returns a fixed answer with a probability, and it never generates text. That gives you an LLM's flexibility at a lower cost per answer: a 2B model needs far less compute than a large vision LLM and generates no output tokens, whether it runs on your own hardware or on a pay-per-token service.

This article introduces Strands Decider, an open-source decision model that reads images, and tests it on one example: frames from security cameras. On a hand-labeled set of camera stills it scored 85.2% with no fine-tuning, against 88.0% for the best vision LLMs on Amazon Bedrock.

What a decision model is

 Strands Decider  is an open-source model from the Strands Agents project. It picks from a fixed set of answers and returns a probability for each one. It supports three question types:

Type

Returns

Example

 noul 

Probability of "yes"

"Is the signature block filled in?"

 choice 

One option from a named set, with a probability for each

"What is damaged? roof, window, siding, none"

 score 

A level on an ordered scale

"How severe is the defect? none, cosmetic, functional"

The model is Qwen3.5-2B with its text output layer replaced by a small scoring head (about 1M parameters). All questions about one input are answered in a single forward pass.

The launch blog covers text only. The repo's  docs/vision.md  documents image input: load the model with vision enabled and it keeps Qwen3.5's own vision encoder (about 331M parameters). The decider was never trained on images, and it answers image questions with no retraining. On public image benchmarks the current release scores 0.776 on NaturalBench and 0.872 on POPE-adversarial.

Where it fits

A decision model fits any application where the input is an image and the output is a fixed answer. The table marks which examples I measured in the security camera test below and which are untested ideas.

Use case

Example question

Type

Tested?

Parking lot occupancy

"Is the lot more than half full?"

 noul 

Yes: 89%

Service vehicles

"Is a delivery or service vehicle at the dock?"

 noul 

Yes: 95%

Retail shelves and aisles

"Is there clutter blocking the aisle?"

 noul 

Yes: 100%

Workplace safety

"Is anyone at height without a harness?"

 noul 

Partly: broad safety questions scored about 77%

Document intake

"Is the signature block filled in?" (repo README)

 noul 

No

Screens for UI agents

"Did the payment succeed?" (repo  docs/vision.md )

 noul ,  choice 

No

Insurance or field-service photos

"What is damaged? roof, window, siding, none"

 choice 

No

Manufacturing inspection

"How severe is the defect? none, cosmetic, functional"

 score 

No

Image moderation

"Does the image break the content policy?"

 noul 

No

The tested categories are small samples, so treat them as signs, not measurements. The pattern was consistent: concrete, visible questions scored well, and questions that need judgment scored worst.

Example: security camera motion frames

A security camera with motion detection sends a frame each time something moves. Most of those frames show nothing of interest: a shadow, a tree, a car driving past. An operator still needs an answer to a fixed question for each frame, such as "Is the fire exit blocked?" or "Is anyone on the roof?" Sending every frame to a vision LLM means paying for a full LLM call on each one, including the empty ones.

On the labeled camera stills, Strands Decider 2B scored 85.2% on its own at a tuned cutoff, in about 1 second per frame on a laptop GPU. Gemma 4 31B and GPT-6 Luna on Amazon Bedrock scored 88.0%. Used as a filter in front of an LLM, the decider removed 7-8% of LLM calls with no accuracy loss, or about a third of calls for a 2-3 point loss, and every error it added was a missed event. On cost, a self-hosted decider costs less per frame than Gemma 4 on Bedrock only if one GPU instance handles about 3 or more frames per second, around the clock.

The rest of this article builds the pipeline for this example, then compares the results with the LLMs.

Architecture





Each camera has one or more fixed questions. When motion triggers a frame, the frame and its questions go to the decider in one request. The decider returns one probability per question, and your code compares each one to a cutoff.

The server has no authentication, so if other hosts need to call it, keep it in a private subnet behind your own access controls.

Install

python3.12 -m venv .venv

source .venv/bin/activate

pip install "strands-decider[vision]==0.3.0"

The first call downloads the model from Hugging Face. On my machine, the download and first load took about 3 minutes. Later loads take seconds.

Test it on one frame:

strands-decider ask StrandsAgents/strands-decider-2B-qwen3.5-v1-2610 \

--image frame.jpg --state "Security camera image." \

--noul "Is the fire exit blocked?"

Versions I used: checkpoint  StrandsAgents/strands-decider-2B-qwen3.5-v1-2610  (released 10/9/26), Python 3.12, transformers 5.19.0, torch 2.14.1. Vision needs transformers 5.18 or later.

Hardware requirements

Strands Decider is not currently available on Amazon Bedrock (as of November 2026), so for now you host it yourself and pay for the instance by the hour, whether frames arrive or not.



GPU

CPU only

Model weights

about 4.7 GB (bf16)

about 9.4 GB (fp32)

Memory

8 GB GPU memory

16 GB RAM (32 GB safer)

Speed

0.5-0.9 s per frame (measured on Apple M3 Pro)

several seconds per frame (not measured)

The memory figures are my estimates from the parameter count. The repo does not list them.

Instance (us-east-1, on demand)

Hardware

$/hour

$/month, always on

Use it for

m7i.2xlarge

8 vCPU, 32 GB RAM

$0.403

about $294

Testing only

g4dn.xlarge

NVIDIA T4, 16 GB

$0.526

about $384

Lowest-cost GPU that fits. No native bf16, so expect slower runs.

g6.xlarge

NVIDIA L4, 24 GB

$0.805

about $588

Recommended minimum for production. The repo's vision benchmarks ran on an L4.

The repo also has deploy guides for Amazon SageMaker (a real-time endpoint,  ml.g6.xlarge  by default) and AWS Lambda. The repo's Lambda deployment runs a text-only build of the model without the vision encoder, so it refuses requests with images and does not work for this use case as published.

Calling the API

Option 1: HTTP server

strands-decider serve StrandsAgents/strands-decider-2B-qwen3.5-v1-2610 --vision --port 8000

Request ( POST /v1/systemone ):

{

"state": "Security camera image.",

"images": ["<base64 JPEG>"],

"questions": {

"exit_blocked": {"type": "noul", "instructions": "Is the fire exit blocked?"},

"person_on_roof": {"type": "noul", "instructions": "Is anyone on the roof?"}

}

}

Response:

{

"answers": {

"exit_blocked": {"type": "noul", "noul": 0.1153},

"person_on_roof": {"type": "noul", "noul": 0.0412}

},

"usage": {"input_tokens": 0, "output_tokens": 0}

}

The  state  field is context text and can be empty. The  images  field takes up to 4 base64 images in PNG, JPEG, WebP, or GIF format. Images, text, and questions share a 4,096-token window. The server shrinks each image so its longer side is at most 448 px, and that setting is fixed in the server.

Option 2: Python, with a larger image size

In my tests, 768 px images were more accurate than 448 px at every cutoff (85.2% against 76.1% at the 0.25 cutoff). To set the size, use the Python API:

import base64

from strands_decider.infer import EngineConfig

from strands_decider.schema import NoulQuestion

from strands_decider.vision import load_vision_engine



engine = load_vision_engine(

"StrandsAgents/strands-decider-2B-qwen3.5-v1-2610",

EngineConfig(device="cuda"), # "mps" on a Mac, "cpu" otherwise

image_long_side=768,

)



with open("frame.jpg", "rb") as f:

img = base64.b64encode(f.read()).decode("ascii")



resp = engine.ask_images(

"Security camera image.",

{"q": NoulQuestion(instructions="Is the fire exit blocked?")},

[img],

)

p_yes = resp.answers["q"].noul

Speed on the M3 Pro GPU, one frame at a time: median 0.46 s at 448 px and 0.89 s at 768 px.

Parsing results

A  noul  answer is a probability, not a yes or no. You choose the cutoff that turns it into a decision.

CUTOFF = 0.25 # tune this on your own labeled frames



def decide(resp, image_present: bool) -> dict:

if not image_present:

raise ValueError("no image in request; the decider will still answer")

return {

name: {"yes": ans.noul >= CUTOFF, "p_yes": ans.noul}

for name, ans in resp.answers.items()

}

Three rules from my tests:

Do not use 0.5 as the cutoff. At 0.5 the decider leaned hard toward "no" and missed 43% of actual events. On my data the useful range was 0.2 to 0.3.

Tune the cutoff on labeled frames from your own cameras, then check it on a separate set. A cutoff tuned on the test set looks better than it will on new frames.

Check that every request has an image. With the image removed, the decider's accuracy falls to chance, but it still answers with high confidence (the repo reports 0.71 to 0.83 on average). Its confidence does not tell you the image is missing.

Results: the decider against vision LLMs on Amazon Bedrock

How I tested

The test set was a hand-labeled set of security camera stills from working cameras and public sources, including still frames I pulled from videos in the  CCTV Action Recognition Dataset  on Kaggle, 57% "yes" and 43% "no", across dozens of different yes/no questions. Each LLM got the same frame and question and returned  {"result": true|false, "description": "..."} . I counted answers I could not parse as wrong, so every model is scored over every frame. The decider ran through the Python API above, with  "Security camera image."  as context, at 448 px and 768 px.

Accuracy

Model

Where it runs

Accuracy

Gemma 4 31B

Amazon Bedrock

88.0%

GPT-6 Luna, full detail

Amazon Bedrock

88.0%

GPT-6 Luna,  detail=low 

Amazon Bedrock

86.1%

GPT-6 Luna,  detail=low , no reasoning

Amazon Bedrock

85.2%

Strands Decider 2B, 768 px, cutoff 0.25

Self-hosted

85.2%

Claude Haiku 5.5

Amazon Bedrock

78.0% (8% of answers unparsed)

Amazon Nova 2 Lite, no reasoning

Amazon Bedrock

77.0%

Strands Decider 2B, 768 px, cutoff 0.5 (default)

Self-hosted

74.6%

I picked the 0.25 cutoff by looking at these same answers and have not yet tested it on a separate set. The peak is na

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →