
Training Natural Language Verifiers with Lean
LeanPRM: Lean-Grounded Process Supervision for Natural Language Proof Verification
以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。
Papers arxiv:2610.17697
Copy markdown
LeanPRM: Lean-Grounded Process Supervision for Natural Language Proof Verification
Published on Oct 13
· Submitted by
Salman Rahman on Oct 16
Upvote 3
Authors: Rodney Lafuente-Mercado ,
Salman Rahman ,
Jingyan Shen ,
James Shiffer ,
Saadia Gabriel ,
Pavel Izmailov
Abstract
LeanPRM uses the Lean proof assistant to automatically generate step-level correctness labels for natural-language mathematical proofs, enabling scalable training of learned verifiers without human annotation.
Generated by thinkingmachines/Inkling-Small
AI systems now write long and difficult mathematical proofs, and checking their step-by-step validity is becoming a bottleneck. A learned verifier could check proofs cheaply, but training one requires a correctness label on every step, and such labels are hard to obtain for open-ended proofs. Existing methods take them from human annotators, which is costly, or derive them from final answers, which proofs do not have, and formalizing each proof in Lean requires expert effort. We introduce LeanPRM, a framework that uses the Lean proof assistant to create step-level labels for natural-language proofs. A model writes a natural-language proof together with a paired formal proof, without feedback from Lean. Lean then labels every step, and a second model discards failures caused only by the formal syntax, so that the negative labels mark errors in the mathematics. LeanPRM contains 10,634 proofs with 152K labeled steps and no human annotation. A Qwen3-4B verifier trained on LeanPRM raises exact step-label accuracy on a held-out set of 200 proofs (LeanPRM-Bench) from 48.2 to 74.7, as opposed to 60.3 when the same base model is trained on PRM800K. It also selects better proofs in best-of-N selection on IMO-level problems, and outperforms the PRM800K-trained verifier on three of four external verification benchmarks. Training on LeanPRM improves verifiers at every size from 0.6B to 8B parameters on LeanPRM-Bench, Hard2Verify and IMO-GradingBench. Given the labels need no human annotation, LeanPRM can grow with the supply of formal statements, offering a path to step-level feedback on AI-generated proofs that are hard for humans to check.
View arXiv page View PDF Project page Add to collection
Community
salmannyu
Paper submitter 18 minutes ago
We introduce LeanPRM, a framework that uses the Lean proof assistant to automatically label each step of natural-language proofs, for training verifiers that check AI-generated mathematical reasoning without human annotation.
Upvote 3
Get this paper in your agent:
Don't have the latest CLI? curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。