返回资讯中心
外部精选
软件工程
#开发者实时流

开发者呼吁为LLM打造更好的语言服务器工具

来源标题:Better Language Server Tools for LLMs

语言服务器能为代码提供语义和诊断信息,支持符号重命名、调用树遍历等重构与发现工具,这些能力对LLM编码代理同样有价值。作者指出LSP工具具有局部性优势,单符号悬停或单文件错误查询比完整编译更快获取信息。文章认为编码代理在获得丰富信息、自动护栏和高效调试工具时表现更好。

Hacker News 最新killcoder2026/11/11 · 6 天前24 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

Back to Blog

Better Language Server Tools for LLMs

They should have good DevEx because it improves my DevEx

Language servers expose semantic and diagnostic information about code for low-latency iteration during development. They provide refactor tools that are ‘language aware’: symbol renaming, ‘move to file’, ‘organise imports’. They also provide discovery tools to walk call trees and find usage sites of functions.

function build_tools( for: “humans” | “llms” | “llms_for_humans”) { /* … */ }build_tools()humansllmsllms_for_humans

These are useful tools that aren’t otherwise available - while a compilation will perform a type-check, it won’t provide inlay hints or the ability to rename a function across a codebase.

There’s also a specificity to LSP tooling, you can request hover information for a single symbol, or get the errors in a single file. A compile will provide global information unrelated to your local focus. This locality allows much faster time-to-information compared to full or even incremental compilation. It is a sharper tool.

Coding agents perform better with access to rich information, automatic guard rails, and tools to debug problems efficiently. They can work faster when a tool returns information in milliseconds instead of seconds. They are cheaper when tools emit fewer tokens, especially when they’re the more relevant tokens.

Coding agents should have access to these LSP systems!

Language servers have APIs designed for cursor based (mouse or keyboard) positioning within files, because humans tend to use mice and keyboards. LLMs don’t have cursors, and are pretty poor at line and character based positioning.

As always, everything is a data representation problem and there’s a clear mismatch here.

What’s the primary modality of the LLM? Text! Lets design tools that match their modality.

LSP integrations tend to use line and character indexes to localise text in their tool calls:

{ "operation": "hover", "filePath": "packages/dataflow/test/operators/multiply-schemas.test.ts", "line": 4, "character": 6 }

While the modality is text, the input stream into the LLM is a sequence of tokens.

3504 1134 19772

Tokens in the o200k tokeniser can be up to 128 characters long. For the model to accurately keep track of character and line positioning, it needs to accumulate that information over the entire file, which it likely read in chunks, in between reasoning and edits via tool calls. There’s a substantial amount working against the model here.

Guess the line and character index of the second e in needle in this block below. As a hint, this is the whole file read.

';--token-additional-padding:1.5"> 200006 44580 38155 316 28 173781 ';--token-additional-padding:2.5"> 200005 12606 815 ';--token-additional-padding:2.5"> 200008 2706 2123 416 405 220 3887 2659 706 976 29082 6446 48704 1072 290 392 needle 1 306 290 8312 15145 134541 4066 943 ';--token-additional-padding:0.5"> 200007

Or what about this one?

16198 176161 256 needle

Answers below:

If you guessed line 1, column 47 for the first, and line 2, column 4 for the second, you were right! Well done!

It’s pretty hard! Sure, these models are Turing complete and can learn anything if they believe in themselves, but is there a better way?

OpenAI’s Codex uses a text based format for its apply_patch tool. It supports concise file moves, additions, updates and deletions. It has a Lark grammar to enforce structural correctness of the calls.

Codex has been extensively post-trained to understand and output this format. An apply_patch call looks like this:

*** Begin Patch *** Add File: hello.txt +Hello world *** Update File: src/app.rs *** Move to: src/main.rs @@ fn greet() {

  • println!("Hi");
  • println!("Hello world"); } @@ fn farewell() {
  • println!("Goodbye");
  • println!("Fare Thee Well"); } *** Delete File: obsolete.txt *** End Patch

We can take advantage of their hard work and design our own ‘language server’ query format that’s idiomatic to the modality of LLMs while aligning with OpenAI’s post-training.

Lets begin with the ability to make queries via hover and inlay :

Here’s a small TypeScript file with some types:

Know that it is within my power to have fancy hovering in the browser - but this has been taken away such that we may continue to empathise with the machine.

const vS = schemaNumber({ unit: volts }) const aS = schemaNumber({ unit: amperes })

type V = typeof vS['infer'] type A = typeof aS['infer']

const eq = math({ v: vS, a: aS }, v * a) const res = eq.evaluate({ v: 230, a: 5 })

const wrong: string = 42

This request will fetch the hover information for the V, A types and the res declaration. It will also fetch inlay hints and JSDoc content for the math function call.

*** Begin Language Server Request *** File: packages/dataflow/test/operators/multiply-schemas.test.ts @@ type V = typeof vS['infer'] ^ V @@ type A = typeof aS['infer'] ^ A ? const eq = math({ v: vS, a: aS }, v * a) ^ math[docs] @@ const res = eq.evaluate({ v: 230, a: 5 }) ^ res *** End Language Server Request

The server responds with:

*** Begin Language Server Response *** File: packages/dataflow/test/operators/multiply-schemas.test.ts @@ type V = typeof vS['infer'] ^ V /*hover:V```typescript type V = number & UnitOfQuantity<"volts", "Voltage">

@@ type A = typeof aS['infer']
^ A
/*<hover:A>```typescript
type A = number & UnitOfQuantity<"amperes", "Electric Current">
```</hover:A>*/
? const eq = math(/*<inlay>inputs:</inlay>*/{ v: vS, a: aS },/*<inlay>equation:</inlay>*/`v * a`)
^ math[docs]
/*<docs:math>
The `math` function takes an object of input variable schemas and a math string. A handle is returned with properties:
- `schema`: Containing the calculated unit.
- `evaluate`: Takes an object of the variable's values, returns the answer.
</docs:math>*/
@@ const res = eq.evaluate({ v: 230, a: 5 })
^ res
/*<hover:res>```typescript
const res: number & UnitOfQuantity<"watts", "Power">
```</hover:res>*/
@@ const wrong: string = 42
/*<diagnostics:error:wrong:ts>
Type 'number' is not assignable to type 'string'.ts(2322)
</diagnostics:error:wrong:ts>*/
*** End Language Server Response

While this looks gross to a human, this is great for the LLM!

The xml-like tags unambiguously bind each hint to its source. The markdown format is well understood by LLMs, and all additional information is colocated with the textual source. The LLM is able to request the minimal information it needs, and we can inject errors and other diagnostic information automatically. The sorted context @@ lines concept works both ways, it makes it unambiguous for the parser to locate lines, and makes it unambiguous to the LLM when we inject diagnostics (like the const wrong: string = 42 line).

This ability to push faults into the model’s context is currently difficult in the first-party harnesses.

This is the spiritual successor to my copy paste adventures in apply_patch. I still think apply_patch should support copy paste, since not all of text is code. However, code should be moved and renamed with a language server.

*** Begin Language Server Request
*** File: packages/dataflow/test/operators/multiply-schemas.test.ts
[@ const vS = schemaNumber({ unit: volts })
@@ const eq = math({ v: vS, a: aS }, `v * a`)
! eq RENAME_TO powerEquation
@@ const res = eq.evaluate({ v: 230, a: 5 })
! res RENAME_TO power
@) const wrong: string = 42
! MOVE_TO_FILE packages/dataflow/test/fixtures/electrical-power.ts
*** End Language Server Request

And returns this:

*** Begin Language Server Response
*** File: packages/dataflow/test/operators/multiply-schemas.test.ts
@@ const wrong: string = 42
/*<diagnostics:error:wrong:ts>
Type 'number' is not assignable to type 'string'.ts(2322)
</diagnostics:error:wrong:ts>*/
*** File: packages/dataflow/test/fixtures/electrical-power.ts
@@ const vS = schemaNumber({ unit: volts })
@@ const powerEquation = math({ v: vS, a: aS }, `v * a`)
@@ const power = powerEquation.evaluate({ v: 230, a: 5 })
*** End Language Server Response

Success and Failure

It’s almost comical to do the inverse of the experiment and ask you to ‘type out needle ’ but the triviality is kind of the point.

16198 176161 256 needle 
 
 
 
 

Answer below:

Of course it’s in the last place you look, who keeps looking after they’ve found it?

We should make it easy for the models to do the right thing.

I’ve been using this language server integration for a while now, it’s working great! It’s substantially more token efficient to do language-server enabled renames and it’s much faster to not have to wait for a compile just to do a type check.

It doesn’t work every time. GPT5.5 can do some silly things sometimes. It is funny when the model gets into a loop of not being able to uniquely identify some lines for hover information, after a few tries it will apply_patch a random comment to ‘provide a unique anchor’ (and have no trouble anchoring the comment). So while the format training is helpful, it’s not perfect. Ultimately the models need to be post-trained to use tools effectively…

Humans interact with their IDE via text selection, range edits, multiple cursors, tab based indentation, selection/file/project local find and replace, and many other tools. We should provide agents with the same capabilities, tuned for their modality.

*** End Language Server Response

If you would like further reading on this topic, I recommend:

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →