
NCM: Next Concept Modeling, Parallel Text Generation
Fast, Parallel Coarse-to-Fine Text Generation
以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。
Fast, Parallel Coarse-to-Fine Text Generation
Next-Concept Modeling (NCM) is a two-stage framework for parallel text generation. A multiscale residual VQ tokenizer represents a sequence as a hierarchy of discrete codes, and a transformer predicts that hierarchy from coarse to fine. Positions within a level are generated in parallel, so generation requires one forward evaluation per level.
Coarse-to-fine generation
NCM generates coarse to fine: each level predicts a longer sequence in parallel, and decoding every level shows a sample being refined from a rough sketch to the full output in a few steps.
NCM beats or matches strong diffusion baselines on different tasks & datasets while generating 9-31× faster.
Weights and tokenized datasets are hosted in the NCM Hugging Face collection. Each released model consists of a frozen tokenizer and a next-concept generator.
git clone https://github.com/HazyResearch/ncm.git cd ncm pip install -r requirements.txt
Python 3.10 or newer and a CUDA-capable PyTorch installation are recommended.
Download the checkpoints and evaluation inputs:
hf download hazyresearch/ncm-checkpoints --local-dir artifacts hf download hazyresearch/ncm-datasets --repo-type dataset --local-dir data
Then run:
python evaluate.py --config config/eval/<task>.yaml --checkpoint-dir artifacts/<task> --data-dir data/<task>
<task> is one of xsum, wmt14, sudoku-easy, sudoku-medium or sudoku-hard
python evaluate.py --config config/eval/<dataset>.yaml --checkpoint-dir artifacts/<dataset>
<dataset> is tinystories, owt or lm1b-<size> with <size> one of small, medium or large
This runs one generation shard per visible GPU, merges them, and prints ROUGE-1/2/L and generative perplexity for XSum; cased and lowercased sacreBLEU, chrF and length ratio for WMT14; solved, cell and valid-grid rates for Sudoku; and generative perplexity under GPT-2 large with unigram entropy, over 5 seeds of 256 unconditional samples, for TinyStories, OpenWebText and LM1B, the latter under the FLM scoring protocol. Add --cfg <scale> for classifier-free guidance on XSum and WMT14; the guided WMT14 row uses --cfg 1.0.
evaluate.py: evaluation entry point
scripts/: per-GPU generation shard and the merge that scores it
config/eval/: one config per released checkpoint, identical to the config.yaml shipped with it
models/, tokenization/: generator and tokenizer implementations
Citation
@misc{kumbong2026nextconcept, title={Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation}, author={Hermann Kumbong and Ishan S. Khare and Iddah Mlauzi and Andrew Woen and Charles Ding and E. Kelly Buchanan and Simran Arora and Azalia Mirhoseini and Christopher Ré}, year={2026}, eprint={2610.14162}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2610.14162}, }
Released under the MIT license.
正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。