pip install -U transformers accelerate torch import json from transformers import AutoModelForCausalLM, AutoTokenizer model_name = “katanemo/Arch-Router-1.5B” model = AutoModelForCausalLM.from_pretrained( model_name, device_map=“auto”, torch_dtype=“auto” ) tokenizer = AutoTokenizer.from_pretrained(model_name)

Prompt 模板来自模型卡片；模型在使用这段确切措辞时效果最佳

TASK_INSTRUCTION = “”“ You are a helpful assistant designed to find the best suited route. You are provided with route description within <routes></routes> XML tags: <routes> {routes} </routes> <conversation> {conversation} </conversation> FORMAT_PROMPT = ”“” Your task is to decide which route is best suit with user intent on the conversation in <conversation></conversation> XML tags. Follow the instruction:

1. If the latest intent from user is irrelevant or user intent is full filled, response with other route {“route”: “other”}.
2. You must analyze the route descriptions and find the best match route for user latest intent.
3. You only response the name of the route that best matches the user's request, use the exact name in the <routes></routes>. Based on your analysis, provide your response in the following JSON formats if you decide to match any route: {“route”: “route_name”}

你的路由，用通俗英文编写

route_config = [ {“name”: “billing