
A Schema Named Hash Sent Our Ruby SDK into Infinite Recursion
Schemas named after standard-library types broke our generated SDKs in TypeScript, Rust, Kotlin, Python and Go, and exposed a naming problem that reserved-keyword lists couldn't solve.
以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。
Schemas named after standard-library types broke our generated SDKs in TypeScript, Rust, Kotlin, Python and Go, and exposed a naming problem that reserved-keyword lists couldn't solve.
We generate SDKs from OpenAPI documents in ten languages. When an API defines a schema, the generator produces a model with the same name. A schema named User becomes a User class. A schema named Invoice becomes an Invoice class.
Those names belong to the API's author, who picks them without knowing which languages they'll end up in. For years we treated that as a solved problem. Every language publishes its reserved keywords, so we kept a table for each one and renamed anything that appeared in it. A property named class or a schema named void got escaped, and every other name passed through untouched.
Then a test schema named Hash sent our Ruby SDK into infinite recursion. Hash isn't a Ruby keyword, so our table let it through.
It was one of 150 standard-library names we had put into a single test document. Across our ten target languages, TypeScript, Rust, Kotlin, Python and Go failed their type checks or compiles straight away. Ruby loaded fine and failed at runtime instead. This post covers what those collisions looked like and the three different fixes they needed.
It started with Ruby running out of stack while converting an ordinary hash into a plain Ruby value.
{ "a" => 1 }
There was no circular reference, no deeply nested object, and nothing particularly interesting about the data.
The problem was a class named Hash. When the specification defines a schema named Hash, our Ruby generator produces:
module Sdk class Hash end end
Ruby has a built-in Hash class, of course. But defining another class with that name inside a module is perfectly legal.
The consequences showed up elsewhere in the generated SDK.
Our runtime had a helper responsible for converting values into plain Ruby data structures. Here's a reduced version:
module Sdk class Hash; end
def self.deep_to_h(val) case val when Hash val.transform_values { |v| deep_to_h(v) } when NilClass, TrueClass, FalseClass, Integer, Float, String val else deep_to_h(val.to_h) end end end
Sdk.deep_to_h({ "a" => 1 })
On Ruby 3.4.9, this raised:
SystemStackError: stack level too deep
The interesting part is the when Hash branch.
Inside module Sdk, Ruby resolves the unqualified constant Hash to Sdk::Hash. That's the generated model, rather than Ruby's built-in ::Hash.
1 } misses the branch, falls through to deep_to_h(val.to_h), and calls itself with the same hash until the call stack ends in SystemStackError. Right: "when ::Hash" resolves to Ruby's built-in Hash, the same input matches, reaches transform_values and returns normally." loading="lazy" decoding="async">
Our input is an instance of ::Hash, so it doesn't match.
deep_to_h(val.to_h)
For an ordinary Ruby hash, to_h returns the same hash. The helper calls itself with the same value, misses the same branch, and continues until Ruby runs out of stack.
Every line of the generated source is valid Ruby.
The failure came from one name resolving to a different class than the runtime expected.
This was one of the more interesting failures we found while testing schema names against the standard libraries of our ten target languages.
It also exposed a problem with an approach we'd been relying on for years.
Every SDK generator needs a list of reserved keywords.
An API might define a property named class, an operation named return, or a schema named void.
Those names can conflict with the syntax of the target language, so a generator needs rules for escaping or renaming them.
Fortunately, programming languages document their reserved words. You can build a reasonably complete keyword table from the language specification and verify it against the compiler.
Hash isn't a Ruby keyword. It's the name of a core class.
The same is true of Record in TypeScript, Iterator in Rust, list in Python, and TimeInterval in Swift.
These are ordinary identifiers that developers can legally reuse.
That's useful when you're writing code by hand. You can deliberately introduce a name into a particular scope, and the language gives you rules for resolving it.
An SDK generator combines names chosen by the API author with names used by its own runtime, templates, and dependencies. Neither side necessarily knows what the other will call things.
An API author might choose Record, Event, File, Task, Duration, Option, Result, Date, Value, or Error.
They're reasonable names for API resources.
Several also have established meanings in common programming languages.
A generator has to figure out which of those meanings wins in every scope it produces.
And unlike a keyword table, there isn't a fixed list you can copy from the language documentation.
The collisions depend on the source code your generator emits.
Before the Ruby stack overflow, we had already encountered several naming collisions in real API specifications.
In Java, that collided with java.lang.Deprecated.
In Swift, TimeInterval is a Foundation type, and our runtime uses it when working with retry delays.
Each time, we investigated the failure and added a targeted fix.
That worked for the immediate problem, but we were effectively waiting for API authors to discover our naming constraints for us.
We already maintained a deliberately difficult OpenAPI fixture for regression testing. Whenever we found an unusual specification shape that broke an SDK, we added it to the fixture.
The generated SDKs then went through the real formatters, linters, type checkers, and compilers for their respective languages.
On October 3, we expanded that fixture with a different idea.
We added a schema that referenced 150 other schemas, each named after a standard-library, framework, or runtime type in at least one of our target languages.
The names weren't random strings. They were deliberately selected to overlap with things our generated code might already depend on.
These were the first errors reported, rather than an exhaustive count of collisions.
In several languages, the compiler couldn't get far enough to report everything else.
Ruby was an interesting exception. The generated source could load successfully, leaving the failure to appear when the affected runtime helper was called.
That distinction mattered because it changed what we needed from our tests. Compiling the SDK would catch many naming mistakes, but it wouldn't catch every case where a valid name changed runtime behavior.
We also tested a related problem involving property names.
In Python, a property named copy, json, or dict on a Pydantic model can produce a warning and then replace the corresponding method on model instances.
A property named model_config is more disruptive: it fails during class creation because the name has a special meaning to Pydantic.
At that point, the naming problem had become broader than the original Ruby example.
We needed a consistent way to decide which names the API author could keep, which names the generator had to change, and where those changes belonged.
The Ruby fix was straightforward once we understood the lookup rules.
Our runtime meant Ruby's built-in Hash, so we made that explicit:
return hash unless hash.is_a?(::Hash)
The leading :: tells Ruby to resolve the constant from the top-level namespace.
A generated model named Sdk::Hash can now exist without changing what the runtime means by ::Hash.
This turned out to be the least disruptive kind of fix.
The API author keeps the schema name Hash. The generated model keeps its expected name. We only change the internal reference that was ambiguous.
The same principle applies in other languages, although the syntax differs.
In Rust, a generated model can interfere with names imported from the prelude or used in trait implementations.
Where appropriate, we made those references explicit:
impl ::std::convert::From<T>
TypeScript had a particularly neat example.
Suppose an API defines a schema named Record, and the generator produces:
export interface Record { id: string; }
The same file also needs a map type.
export type Labels = Record<string, string>;
But Record now refers to our generated interface, which doesn't accept type parameters.
TS2315: Type 'Record' is not generic
We verified the alternative with TypeScript 6.0.2:
export interface Record { id: string; }
// Error: resolves to the generated interface export type Labels = Record<string, string>;
// Resolves to the built-in utility type export type Fixed = globalThis.Record<string, string>;
Using globalThis.Record allows the generator to retain the API's model name while referring to the built-in utility type where it needs to.
These changes have an advantage that's easy to overlook: they don't affect the SDK's public API.
Users can still work with the models whose names came from their specifications.
Only the generator's internal references change.
We started there whenever the language made it possible.
Qualification solved a good number of collisions, but some names were too deeply embedded in the generated code to handle that way.
Our Rust SDK, for example, imports models with:
use crate::types::*;
That brings generated model names into scopes where other code expects standard-library types and traits.
A model named Option, Some, Vec, or Iterator can change the meaning of signatures and implementations throughout the crate.
Kotlin has a related problem with types available through its default imports, including Map, List, and Result.
For these cases, we reserve names that the generated templates depend on and rename conflicting models.
The important detail is how we decide which names to reserve.
We don't need every type that has ever existed in a language's standard library.
We need the names that our emitted code uses in scopes where a generated model could interfere.
That makes the reservation list specific to our generator.
If a template begins using another unqualified standard-library name, the list has to change with it.
This is more maintenance than a fixed keyword table, but it also gives us a much clearer rule.
The names we reserve are derived from the code we generate.
The most instructive mistake came from Python.
REST APIs frequently have operations named list.
client.pets.list()
That's a natural interface for retrieving a collection.
In our Python SDK, however, the generated implementation also contains module-level functions.
The module may use Python's built-in list for annotations:
items: list[str]
It may also use the built-in in runtime checks:
isinstance(value, list)
Now define a function with the same name in that module:
def list(...): ...
The module-level list name refers to the generated function.
The original built-in is no longer what those other references resolve to.
TypeError: isinstance() arg 2 must be a type, a tuple of types, or a union
We fixed it by adding a suffix to operations whose names collided with built-ins used by the module.
def list_(...): ...
That solved the internal collision.
Unfortunately, we'd used the same naming logic for the public client method.
So an API operation called list now appeared as:
client.pets.list_()
Every user of an SDK containing a list operation would see the renamed method.
The suffix was technically valid. It just belonged in the wrong place.
This was the moment that changed how we approached naming.
A Python module-level function and an attribute on a client object have different lookup rules.
The module-level list conflicts with references to Python's built-in. An instance attribute named list doesn't create that same collision.
We had been forcing both names through one naming function because they represented the same OpenAPI operation.
Once we separated them, the fix became simple.
The implementation function ke
正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。