
5分钟读懂Decision AI:开源模型可在本地运行决策系统
来源标题:Decision AI Explained in 5 Minutes
Decision AI 可通过 Ollama 和 llama.cpp 的开源模型在本地构建决策系统,无需依赖闭源 API。Ollama 支持 Nimble、Tev1,llama.cpp 支持 Laya、Kev、Julia-1、OpenJev,可通过 /v1/systemone 接口运行。示例用 Laya 模型将客服消息自动分类到对应部门。
以下正文同步自 KDnuggets,版权归原站所有,已转换为易读排版。
The interesting part is that we now have several ways to build decision-making applications without depending on a closed-source API. We can choose smaller models for speed, fine-tune them for our specific tasks, or run everything locally to reduce API costs.
Things are moving quickly. Both Ollama and llama.cpp now support decision models through the TypeSafe-compatible /v1/systemone endpoint.
Ollama supports models such as Nimble and Tev1, while llama.cpp supports Laya, Kev, Julia-1, and OpenJev.
Let's try Laya using the latest llama.cpp. Start the local server:
llama update llama serve -hf ggml-org/Laya-GGUF
Once the model is running, open another terminal and send a customer support request:
curl http://localhost:8080/v1/systemone
-H "Content-Type: application/json"
-d '{
"state": "I was charged twice for my order.",
"questions": {
"department": {
"type": "choice",
"instructions": "Which team should handle this?",
"criteria": {
"billing": "Payments and refunds",
"shipping": "Delivery problems",
"technical": "Bugs and account issues"
}
}
}
}'
The model will evaluate the message and return the selected department along with a probability for each option.
That's it. We can now experiment with a decision model locally without using a paid API.
I think the most interesting part of Decision AI is how it can work alongside existing LLMs to make AI applications faster and cheaper.
Imagine an AI coding agent using a small decision model to select the right tool, decide whether to search the web, or determine if a task needs a more powerful LLM. Instead of using an expensive reasoning model for every little decision, we can reserve it for tasks that actually need it.
Another exciting area is real-time applications. Decision models are already being tested in games, where they can choose the next move in milliseconds. We could see similar applications in robotics, fraud detection, and autonomous AI agents.
With Ollama and llama.cpp now supporting these models, we can also run them locally, reducing API costs and keeping sensitive data on our own hardware.
The biggest challenge is reliability. If a model says it is 95% confident, we need to know whether we can trust that number. Better training, calibration, and human review will be important as these models move into real-world applications.
正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。