WILD!!!, Fovea-4B model Outperforms ImaJev-4B

Like 1

Image-Text-to-Text

Safetensors

English

qwen3_5

decision-model

typed-decisions

vision-language

calibrated-probabilities

jev

conversational

License: apache-2.0

Model card Files Files and versions xet

Community

Copy to bucket new

Fovea-4B Serving

Evaluation

Training

Limits

Acknowledgements

Fovea-4B

Fovea-4B answers typed questions about an image and a record in one forward pass, with a probability for every option. It reads photos, documents, receipts, charts and screenshots, together with a JSON state, and answers yes/no (noul), multiple-choice (choice) and ordinal (score) questions over TypeSafe's System One API (POST /v1/systemone).

Serving

git clone https://github.com/Sivasuthan9/fovea && cd fovea python3.12 -m venv .venv && . .venv/bin/activate pip install torch==2.13.0 torchvision==0.28.0 --index-url https://download.pytorch.org/whl/cu130 pip install -r requirements.txt python scripts/serve.py --model Sivasuthan/fovea-4b --port 8000

The server loads the model, records CUDA graphs and warms up the kernels before it reports ready; then POST http://127.0.0.1:8000/v1/systemone with a System One request, for example

{"state": {"photo": "img-1"}, "assets": {"img-1": {"type": "image", "format": "jpg", "data_base64": "..."}}, "questions": {"decision": {"type": "choice", "instructions": "What is the total on the receipt in photo?", "criteria": {"12.40": null, "14.20": null, "21.40": null}}}}

returns {"model": "fovea-4b", "answers": {"decision": {"type": "choice", "choice": "14.20", "probabilities": {...}, "confidence": ...}}, "usage": {...}}. Images may also be sent as data:image/...;base64, URIs inside the state or as a top-level images list. Every answer covers exactly the requested labels and sums to 1; a question that cannot be scored gets a uniform answer rather than an error.

Evaluation

Fovea-4B against the two strongest open 4B models on the ImageJevBench v0.3 leaderboard by Intelligence, Imajev-4B (#1) and H2O-Lightning-4B image (#2), and the larger Jev-Vision. All four answer the same 2,360 held-out multiple-choice questions, each served as its authors release it. Scores are Intelligence, 100 × (accuracy − chance) / (1 − chance); the best value in each column is in bold.

Paired on the same questions, Fovea-4B leads Imajev-4B by +1.1 points (95% interval +0.3 to +2.0), H2O-Lightning-4B by +0.7 (−0.7 to +2.0) and Jev-Vision by +4.9 (+3.0 to +6.7) overall. Fovea-4B answers 80.6% of the questions correctly.

How each model was served. Fovea-4B: images up to 2 MP, its calibrated temperatures. Imajev-4B: native resolution, temperature 1.305. H2O-Lightning-4B image: images up to 1 MP, temperature 0.65. Jev-Vision: its own decision function, images up to 1.29 MP, temperature 0.5.

Evaluation data. Test or validation splits of public benchmarks, none used for training: DocVQA (400 document questions), ChartQA (300) and SEED-Bench-2-Plus charts (100), AI2D (300 diagrams) and SEED-Bench-2-Plus maps (100), CV-Bench (300: depth, distance, spatial relation, counting), BLINK (560: spatial reasoning, counting and nine multi-image tasks) and MMStar (300 general photos). Overall weights the categories by the ImageJevBench core-track mix; ECE is the ten-bin expected calibration error of the top answer. This is our own held-out evaluation, not a leaderboard result: the sets are easier than ImageJevBench's sealed items and contain only multiple-choice questions, so the absolute numbers do not predict a leaderboard score.

Training

Fovea-4B was trained on 57,441 typed image decisions with a cross-entropy loss on the decision readout (teacher distributions where available), options reshuffled every time a question is seen.

Each source question becomes a choice question, and a share of them also a noul claim check (does the image confirm a claimed answer?) or, for exact counts, a score question on an ordinal scale. A LoRA of rank 32 was trained on every language-model projection (attention, MLP and the Gated DeltaNet projections; the vision encoder frozen), together with the readout, in two runs (peak learning rate 5e-5 and 2e-5, 8 questions per step, cosine decay to 10%). The released weights are the exact weight-space average of the best checkpoint of each run, merged into the model. 300 source questions were held out and checked every 500 steps. No benchmark test items were used for training.

Limits

One forward pass, no reasoning: multi-step arithmetic and long chains of inference are weak.

English only.

Very small text in large images is read only as well as the image resolution allows (2,097,152 pixels per image).

Acknowledgements

The base model is Qwen3.5-4B (Apache-2.0). See NOTICE.

","pad_token":"<|endoftext|>","unk_token":null},"chat_template_jinja":"{%- set image_count = namespace(value=0) %}\n{%- set video_count = namespace(value=0) %}\n{%- macro render_content(content, do_vision_count, is_system_content=false) %}\n {%- if content is string %}\n {{- content }}\n {%- elif content is iterable and content is not mapping %}\n {%- for item in content %}\n {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}\n {%- if is_system_content %}\n {{- raise_exception('System message cannot contain images.') }}\n {%- endif %}\n {%- if do_vision_count %}\n {%- set image_count.value = image_count.value + 1 %}\n {%- endif %}\n {%- if add_vision_id %}\n {{- 'Picture ' ~ image_count.value ~ ': ' }}\n {%- endif %}\n {{- '<|vision_start|><|image_pad|><|vision_end|>' }}\n {%- elif 'video' in item or item.type == 'video' %}\n {%- if is_system_content %}\n {{- raise_exception('System message cannot contain videos.') }}\n {%- endif %}\n {%- if do_vision_count %}\n {%- set video_count.value = video_count.value + 1 %}\n {%- endif %}\n {%- if add_vision_id %}\n {{- 'Video ' ~ video_count.value ~ ': ' }}\n {%- endif %}\n {{- '<|vision_start|><|video_pad|><|vision_end|>' }}\n {%- elif 'text' in item %}\n {{- item.text }}\n {%- else %}\n {{- raise_exception('Unexpected item type in content.') }}\n {%- endif %}\n {%- endfor %}\n {%- elif content is none or content is undefined %}\n {{- '' }}\n {%- else %}\n {{- raise_exception('Unexpected content type.') }}\n {%- endif %}\n{%- endmacro %}\n{%- if not messages %}\n {{- raise_exception('No messages provided.') }}\n{%- endif %}\n{%- if tools and tools is iterable and tools is not mapping %}\n {{- '<|im_start|>system\n' }}\n {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}\n {%- for tool in tools %}\n {{- "\n" }}\n {{- tool | tojson }}\n {%- endfor %}\n {{- "\n</tools>" }}\n {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}\n {%- if messages[0].role == 'system' %}\n {%- set content = render_content(messages[0].content, false, true)|trim %}\n {%- if content %}\n {{- '\n\n' + content }}\n {%- endif %}\n {%- endif %}\n {{- '<|im_end|>\n' }}\n{%- else %}\n {%- if messages[0].role == 'system' %}\n {%- set content = render_content(messages[0].content, false, true)|trim %}\n {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}\n {%- endif %}\n{%- endif %}\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for message in messages[::-1] %}\n {%- set index = (messages|length - 1) - loop.index0 %}\n {%- if ns.multi_step_tool and message.role == "user" %}\n {%- set content = render_content(message.content, false)|trim %}\n {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}\n {%- set ns.multi_step_tool = false %}\n {%- set ns.last_query_index = index %}\n {%- endif %}\n {%- endif %}\n{%- endfor %}\n{%- if ns.multi_step_tool %}\n {{- raise_exception('No user query found in messages.') }}\n{%- endif %}\n{%- for message in messages %}\n {%- set content = render_content(message.content, true)|trim %}\n {%- if message.role == "system" %}\n {%- if not loop.first %}\n {{- raise_exception('System message must be at the beginning.') }}\n {%- endif %}\n {%- elif message.role == "user" %}\n {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}\n {%- elif message.role == "assistant" %}\n {%- set reasoning_content = '' %}\n {%- if message.reasoning_content is string %}\n {%- set reasoning_content = message.reasoning_content %}\n {%- else %}\n {%- if '</think>' in content %}\n {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}\n {%- set content = content.split('</think>')[-1].lstrip('\n') %}\n {%- endif %}\n {%- endif %}\n {%- set reasoning_content = reasoning_content|trim %}\n {%- if loop.index0 > ns.last_query_index %}\n {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}\n {%- else %}\n {{- '<|im_start|>' + message.role + '\n' + content }}\n {%- endif %}\n {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}\n {%- for tool_call in message.tool_calls %}\n {%- if tool_call.function is defined %}\n {%- set tool_call = tool_call.function %}\n {%- endif %}\n {%- if loop.first %}\n {%- if content|trim %}\n {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}\n {%- else %}\n {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}\n {%- endif %}\n {%- else %}\n {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}\n {%- endif %}\n {%- if tool_call.arguments is defined %}\n {%- for args_name, args_value in tool_call.arguments|items %}\n {{- '<parameter=' + args_name + '>\n' }}\n {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}\n {{- args_value }}\n {{- '\n</parameter>\n' }}\n {%- endfor %}\n {%- endif %}\n {{- '</function>\n</tool_call>' }}\n {%- endfor %}\n {%- endif %}\n {{- '<|im_end|>\n' }}\n {%- elif message.role == "tool" %}\n {%- if loop.previtem and loop.previtem.role != "tool" %}\n {{- '<|im_start|>user' }}\n {%- endif %}\n {{- '\n<tool_response>\n' }}\n {{- content }}\n {{- '\n</tool_response>' }}\n {%- if not loop.last and loop.nextitem.role != "tool" %}\n {{- '<|im_end|>\n' }}\n {%- elif loop.last %}\n {{- '<|im_end|>\n' }}\n {%- endif %}\n {%- else %}\n {{- raise_exception('Unexpected message role.') }}\n {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n {{- '<|im_start|>assistant\n' }}\n {%- if enable_thinking is defined and enable_thinking is false %}\n {{- '<think>\n\n</think>\n\n' }}\n {%- else %}\n {{- '<think>\n' }}\n {%- endif %}\n{%- endif %}"},"createdAt":"2026-10-24T05:40:35.000Z","discussionsDisabled":false,"discussionsSorting":"recently-created","downloads":21,"downloadsAllTime":21,"id":"Sivasuthan/fovea-4b","isLikedByUser":false,"availableInferenceProviders":[],"showHuggingChatEntry":false,"inference":"","lastModified":"2026-10-24T09:48:45.000Z","likes":1,"pipeline_tag":"image-text-to-text","librariesOther":[],"trackDownloads":true,"model-index":null,"private":false,"repoType":"model","gated":false,"tags":["safet