Latai – run GGUF compliant/local models upto 35 MB on edge

import latai

with latai.Runtime.open() as rt: rt.install("contract-terms.gguf") answer = rt.run(latai.text_request("contract-terms", clause_text)) print(answer["output"], answer["latai"]["timing"]["total_ms"], "ms")

Add the SDK, install a pack, make one run call. How the engine itself reaches the SDK differs by language, and the table below says which is which.

Where it came from

We did not set out to build a runtime. We built ej, and at 11 MB it was the smallest, best performing decision model of its size that could run on an edge device. Then the same question started arriving in private Discord servers and Slack channels, in different clothes every time: fine, but how do I actually run this thing?

That is not a documentation problem. People were asking for the layer underneath: somewhere to put a small model so it runs on the device, the same way, every time, whatever the language and whatever the hardware.

latai is named for the spool a kite line winds onto, the thing in your hand that lets the kite go somewhere.

Every SDK is a thin binding over the same C interface, so the request and the error codes are identical across all six.

The desktop and server SDKs run today. The two mobile ones are published and still being proven on real hardware, so read their READMEs before you plan a ship date around them.

The SDK source is Apache-2.0 or MIT, your choice. The engine inside ships as a binary, free to use and to redistribute unchanged inside your own software.

latai is the engine. Models give it its skills, and each one is a signed pack rather than a library you wire in by hand. Swap the pack, keep the call.

maap is the deliberate exception. Estimation at 4 billion parameters is work for the cloud, and it stays there.

Cloud is not the only other option. Apple and Google each ship an on-device model inside the platform itself, and you can always build on a runtime yourself.

Measured on Linux x86_64 with our own harness, one machine, single run, not independently re-run.

Start with the SDK for your stack. Add it, install a pack, make one run call.

No server, no upload, no cost per answer.