返回资讯中心
外部精选
软件工程
#开发者实时流

Octen-Team 发布 4B 模型 Octen-WebGR,记忆 2 亿条 URL 并开源权重

来源标题:Octen-WebGR, a 4B model that memorized 200M URLs (open weights, live demo)

Octen-Team 发布 4B 参数模型 Octen-WebGR-4B,记忆了 2 亿条 URL,权重以 cc-by-nc-4.0 许可开放并提供在线演示。该模型基于 qwen3,面向生成式检索、语义 ID 与网页导航搜索场景,可通过 Transformers 调用。

Hacker News 最新kunka_octen2026/11/13 · 5 天前30 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

Like 65

Follow

Octen-Team 106

Text Generation

Transformers

Safetensors

qwen3

generative-retrieval

semantic-id

navigational-search

web-search

conversational

text-generation-inference

arxiv: 2610.35256

License: cc-by-nc-4.0

Model card Files Files and versions xet

Community 1

Deploy

Copy to bucket

Use this model Instructions to use Octen/Octen-WebGR-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

Transformers How to use Octen/Octen-WebGR-4B with Transformers:

Use a pipeline as a high-level helper

from transformers import pipeline

pipe = pipeline("text-generation", model="Octen/Octen-WebGR-4B") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)

pip install -U transformers accelerate

Load model directly

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Octen/Octen-WebGR-4B") model = AutoModelForCausalLM.from_pretrained("Octen/Octen-WebGR-4B", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks

Google Colab

Kaggle

Local Apps Settings

vLLM How to use Octen/Octen-WebGR-4B with vLLM:

Install vLLM from pip:

pip install vllm

Start the vLLM server:

vllm serve "Octen/Octen-WebGR-4B"

Call the server using curl (OpenAI-compatible API):

curl -X POST "http://localhost:8000/v1/chat/completions"
-H "Content-Type: application/json"
--data '{ "model": "Octen/Octen-WebGR-4B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'

Use Docker

docker model run hf.co/Octen/Octen-WebGR-4B

SGLang How to use Octen/Octen-WebGR-4B with SGLang:

Install SGLang from pip:

pip install sglang

Start the SGLang server:

python3 -m sglang.launch_server
--model-path "Octen/Octen-WebGR-4B"
--host 0.0.0.0
--port 30000

Call the server using curl (OpenAI-compatible API):

curl -X POST "http://localhost:30000/v1/chat/completions"
-H "Content-Type: application/json"
--data '{ "model": "Octen/Octen-WebGR-4B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'

Use Docker images

docker run --gpus all
--shm-size 32g
-p 30000:30000
-v ~/.cache/huggingface:/root/.cache/huggingface
--env "HF_TOKEN=<secret>"
--ipc=host
lmsysorg/sglang:latest
python3 -m sglang.launch_server
--model-path "Octen/Octen-WebGR-4B"
--host 0.0.0.0
--port 30000

Call the server using curl (OpenAI-compatible API):

curl -X POST "http://localhost:30000/v1/chat/completions"
-H "Content-Type: application/json"
--data '{ "model": "Octen/Octen-WebGR-4B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'

Docker Model Runner How to use Octen/Octen-WebGR-4B with Docker Model Runner:

docker model run hf.co/Octen/Octen-WebGR-4B

Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Octen-WebGR-4B Model

Results

How it works

Usage

Files

Screening of the table

Intended use and limitations

License

Citation

Contact

Hosted API: Pulse Search. Don't want to run the GPU yourself? Octen-WebGR powers Pulse Search, a hosted API with ~10 ms server-side p50 latency. Commercial use is allowed through the API, and lite mode starts at $0.1 per 1,000 calls.

Octen-WebGR-4B is a generative retriever for navigational web search, developed by Octen. Given a query, it generates a two-token semantic identifier (SID) that is constrained to a table of 195.9 million real URLs, so the web index lives in the model's parameters: retrieval needs one GPU and about 10 GB of memory, instead of hundreds of gigabytes of vectors.

It is described in the technical report Octen-WebGR: Scaling Generative Retrieval to 200 Million Web Pages (Xu et al., 2026). A smaller model for a 31-million-URL table is available as Octen-WebGR-0.6B.

Model

Base model Qwen3-4B-Base, full-parameter fine-tuning

Layers / hidden size 36 / 2,560

Parameters, total (non-embedding) 4.36B (3.63B)

Identifier tokens 2 × 65,536 (vocabulary ids 151,936 to 283,007)

URLs in the training universe 201,231,792

URLs in the released table 195,921,090 (after screening, see below)

Training data Query–URL click pairs up to August 2026 (proprietary, not released)

Weights bf16, 8.7 GB

Results

Navigational queries from September 2026, the month after the training data, weighted by clicks. A system scores a hit when the clicked target page is among its top k URLs; targets outside a system's candidate set count as misses.

GR is strongest on new ways of asking for familiar pages: on unseen queries for pages with many training queries it ranks the target first 30.9% of the time, against 17.2% for dense retrieval (21.4% with the click prior). Dense retrieval remains better on rarely visited pages, and a click table cannot answer new queries at all. All numbers use the full 201M universe; restricting decoding to the screened table lowers hit@1 from 59.6 to 58.7, because 1.2% of the traffic targets a removed URL. See the report for the full evaluation.

How it works

Normalise the query: collapse whitespace, strip, lowercase. Keep at most the first 32 tokens.

Prompt: Query: {q}\nURL: without special tokens.

Generate two SID tokens. Level t (0 or 1) uses vocabulary id 151936 + t * 65536 + code.

Constrained beam search: at each level, only prefixes present in sid_table/ are allowed and the softmax is taken over the valid children only; 20 beams are kept. The score is the sum of the two log-probabilities.

Look (sid0, sid1) up in sid_table/ to get the URL.

Do not use model.generate(). The tokenizer has no entries for the SID tokens, and unconstrained generation returns identifiers that map to no URL. Use the inference code in this repository.

Usage

pip install -r requirements.txt # torch 2.8.0, transformers 4.57.1; flash-attn optional python infer_gpu.py --model Octen/Octen-WebGR-4B --query "youtube" "boeing internship" python infer_gpu.py --model Octen/Octen-WebGR-4B --check # compare against smoke_test.json

from huggingface_hub import snapshot_download import sys

root = snapshot_download("Octen/Octen-WebGR-4B") sys.path.insert(0, root) from infer_gpu import GR

gr = GR(root) # loads the model and the 195.9M-row table for url, logp, row in gr.search(["hugging face"], K=10)[0]: print(url, logp)

infer_gpu.py is a reference implementation: it recomputes the prompt for every beam at the second level and pads each batch to its longest prompt. On an idle H200 it answers a single query in 56 ms (median) and needs 10.1 ms per query in batches of 32. infer_cpu.py implements the same protocol on CPUs and reuses the prompt's key–value cache.

Memory: about 9.7 GiB of GPU memory (bf16 weights and SID keys). Peak host RAM is about 23 GiB with infer_cpu.py, which keeps the weights and the table in memory.

Files

Path Content License

*.safetensors, config.json, generation_config.json Model weights (bf16) CC BY-NC 4.0

tokenizer*.json, vocab.json, merges.txt Qwen3 tokenizer, unchanged Apache 2.0

sid_table/part-*.parquet One row per URL: sid0, sid1 (int32), url (string, without scheme), sorted by sid0 * 65536 + sid1, unique CC BY-NC 4.0

infer_gpu.py, infer_cpu.py Inference code Apache 2.0

smoke_test.json 31 hand-picked queries with expected top-20 URLs, for --check –

Screening of the table

Before release we removed every URL whose host, or a parent domain of it, is on the adult, malware, phishing, cryptojacking or stalkerware lists of the UT1 blacklists; every URL listed by URLhaus; and every URL whose text contains an e-mail address or a credential (5,310,702 URLs, 2.6%, mostly adult sites). Removing a row from the table removes the URL from every possible output, because decoding is constrained to the table. The tables were not checked against lists of child sexual abuse material, nor screened for personal pages; both are left to the takedown process.

URL removal requests: research@octen.ai. Removals are published as versioned changes to sid_table/.

Intended use and limitations

Intended for research on generative retrieval and for navigational search over the URLs in the released table.

The model can only return URLs in the table. Pages that appeared after August 2026 are not covered, and adding a page requires training on it.

It is weaker than dense retrieval on rarely visited pages, and it is not meant for informational queries.

Its knowledge reflects clicks up to August 2026 and the biases of the rankings under which those clicks were made.

It was trained on aggregated click data without user identifiers. It must not be used to infer anything about individual users.

License

The model weights and the SID-to-URL table are released under CC BY-NC 4.0 (non-commercial use). The inference code is released under the Apache License 2.0. The model is derived from Qwen3-4B-Base, Copyright 2024 Alibaba Cloud, licensed under the Apache License 2.0 (LICENSE-QWEN3); see NOTICE. Unlike the Octen-Embedding models, the WebGR weights are not licensed for commercial use; contact Octen for commercial licensing, or use Pulse Search for commercial access through the hosted API.

Citation

@misc{octen2026webgr, title = {Octen-WebGR: Scaling Generative Retrieval to 200 Million Web Pages}, author = {Xu, Guangwei and Zhang, Wenkai and Liu, Yawen and Zhang, Dongxu and Xu, Xijie and Zou, Kuan}, year = {2026}, eprint = {2610.35256}, archivePrefix = {arXiv}, primaryClass = {cs.IR}, url = {https://arxiv.org/abs/2610.35256} }

Contact

For API access, deployment solutions, and technical documentation, visit octen.ai.

Join our Discord community for questions, feedback, and the latest updates.

","unk_token":null,"pad_token":"<|endoftext|>","chat_template":"{%- if tools %}\n {{- '<|im_start|>system\n' }}\n {%- if messages[0].role == 'system' %}\n {{- messages[0].content + '\n\n' }}\n {%- endif %}\n {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}\n {%- for tool in tools %}\n {{- "\n" }}\n {{- tool | tojson }}\n {%- endfor %}\n {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}\n{%- else %}\n {%- if messages[0].role == 'system' %}\n {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}\n {%- endif %}\n{%- endif %}\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for message in messages[::-1] %}\n {%- set index = (messages|length - 1) - loop.index0 %}\n {%- if ns.multi_step_tool and message.role == "user" and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}\n {%- set ns.multi_step_tool = false %}\n {%- set ns.last_query_index = index %}\n {%- endif %}\n{%- endfor %}\n{%- for message in messages %}\n {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}\n {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}\n {%- elif message.role == "assistant" %}\n {%- set content = message.content %}\n {%- set reasoning_content = '' %}\n {%- if message.reasoning_content is defined and message.reasoning_content is not none %}\n {%-

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →