返回资讯中心
外部精选
软件工程
#开发者实时流

Maxwell发布V1.0-Flash混合专家模型,总参数35B、激活3B

来源标题:Maxwell-V1-Flash 35B-A3B

Maxwell推出V1.0-Flash混合专家(MoE)语言模型,总参数35B、每token激活3B,上下文窗口最高256K tokens。该模型在约26T多语言token上预训练,结合块稀疏注意力与细粒度专家路由,经两阶段监督微调、强化学习及多教师在线策略蒸馏。官方称其在推理、智能体搜索和通用能力上表现强劲。

Hacker News 最新nateb20222026/11/17 · 今天16 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

We present Maxwell-V1.0-Flash, a high-performance Mixture-of-Experts (MoE) language model with 35B total parameters and 3B active parameters per token, supporting a context window of up to 256K tokens. Maxwell combines learned block-sparse attention, fine-grained expert routing, and manifold-constrained hyper-connections for efficient long-context inference. It is pre-trained on approximately 26T multilingual tokens with the Muon optimizer and multi-token prediction, followed by two-stage supervised fine-tuning, specialized reinforcement learning, and multi-teacher on-policy distillation. Together, these choices deliver strong reasoning, agentic search, and broad general capabilities.

Maxwell-V1.0-Flash uses a 45-layer decoder-only Transformer backbone. Its feed-forward stack begins with 3 dense layers followed by 42 MoE layers, while block-sparse attention and manifold-constrained hyper-connections support selective context access and information flow across residual streams.

Maxwell Block Sparse Attention (MBSA) combines fixed prefix and local coverage with learned remote block selection. Each KV group attends to up to 32 blocks, capping support at 4,096 tokens while retaining full-history KV. MBSA delivers a 6.7x kernel speedup over dense GQA at 256K under matched shapes.

Four residual streams use learned input-dependent aggregation and constrained mixing with 20 Sinkhorn-Knopp iterations. This expands information flow across the network without adding extra attention or feed-forward evaluations.

The first 3 decoder layers use dense feed-forward networks, followed by 42 MoE layers. Each MoE layer contains 256 routed experts, activates 12 per token, and includes 2 always-active shared experts, with a dynamic selection bias for auxiliary-loss-free load balancing.

Maxwell-V1.0-Flash is pre-trained on approximately 26T multilingual tokens using the Muon optimizer and an auxiliary multi-token prediction objective. Context length is progressively extended from 4K to 256K, with block-sparse attention introduced during mid-training after indexer warm-up.

Two-stage supervised fine-tuning establishes general, reasoning, and agentic capabilities. Independent reinforcement-learning branches then specialize these capabilities, and multi-teacher on-policy distillation consolidates them into a single model.

Model Name Description Hugging Face

Maxwell-V1-Flash 35B-total / 3B-active MoE model 🤗 Hugging Face

The model repository above contains the official Maxwell-V1.0-Flash release used by the examples in this repository.

Base model. Maxwell-V1.0-Flash (Base) is compared with two Qwen base models of comparable scale under a unified internal evaluation framework. All scores are percentages; the best score in each row is shown in bold. GSM8K is evaluated using pass@8.

Base model, long context. Scores are on a 0–100 scale. All results use block-sparse attention in every layer.

Post-trained model. Maxwell-V1.0-Flash is compared with four models of comparable scale: Qwen3.6-35B-A3B, Gemma-4-26B-A4B-it, Nemotron-3.5-Lightning, and Muse-Glimmer-30B. All scores are percentages; the highest score for each metric is shown in bold. Multilingual results cover only market languages. Model names wrap across lines to keep the complete comparison compact.

  • Maxwell with context management enabled.

Post-training evaluations mostly use temperature 0.6, top-p 0.95, top-k 20, and a 32K-token context. AA-LCR, mathematical benchmarks, and HLE use 128K; scientific benchmarks use 80K; LiveCodeBench v6 uses 64K; multilingual evaluation uses a maximum generation length of 16K tokens. SWE-Bench Verified is evaluated with the Claude Code harness (version 2.1.212) in a network-isolated environment, with a 256K-token context window, at most 240 interaction turns, and a four-hour wall-clock limit per task. Terminal-Bench 2.1 is evaluated in the Harbor framework with the Terminus-2 harness, using a two-hour timeout per task, a 256K-token context window, 32K max tokens, temperature 1.0, top-p 0.95, top-k 20, and interleaved thinking.

Requires NVIDIA driver >= 580 (CUDA 13.0). On older drivers, add -e VLLM_ENABLE_CUDA_COMPATIBILITY=1 to docker run.

Download the weights to a local directory first with the Hugging Face CLI (install it with pip install -U huggingface_hub if hf is not available):

export MODEL_DIR=$PWD/Maxwell-V1-Flash

hf download maxwell-model/Maxwell-V1-Flash --local-dir "$MODEL_DIR"

Serve on B300

A single GPU is enough:

docker run --rm --init
--name vllm-maxwell
--gpus all
--shm-size 8g
--network host
-v "$MODEL_DIR":/models/Maxwell-V1-Flash:ro
--entrypoint vllm
maxwellmodel/vllm-openai:v0.30.0
serve /models/Maxwell-V1-Flash
--trust-remote-code
--block-size 128
--served-model-name Maxwell-V1.0-Flash
--port 8000
--tensor-parallel-size 1
--reasoning-parser qwen3
--enable-auto-tool-choice
--tool-call-parser qwen3_coder

Serve on H100

Requires at least two GPUs. Use the same command with --tensor-parallel-size 2.

You can also try Maxwell-V1.0-Flash at https://maxwell-llm.ai/playground.

Maxwell-V1.0-Flash uses the bundled chat template for system, user, assistant, and tool messages, with thinking enabled by default. For multi-turn conversations and tool calls, pass the complete assistant message back into messages, including reasoning_content and tool_calls when present. OpenAI-compatible tool definitions are accepted through the tools field.

import openai

def chat_completion( client: openai.OpenAI, model_name: str, ) -> str: messages = [ { "role": "user", "content": ( "Why is it harder to keep a room cool when the door is open " "on a hot day? Explain in 3-4 simple sentences." ), }, ]

response = client.chat.completions.create( model=model_name, messages=messages, stream=False, max_tokens=4096, temperature=0.6, top_p=0.95, extra_body={ "top_k": 20, "chat_template_kwargs": { "enable_thinking": True, }, }, )

return response.choices[0].message.content

For full guides and examples, see the Maxwell-V1.0-Flash Quickstart.

Maxwell-V1.0-Flash is released under the Apache License 2.0. See LICENSE for details.

If you have any questions or suggestions, feel free to reach out to our R&D and product teams via email: contact@maxwell-llm.ai

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →