I keep seeing the same pattern. Someone needs to classify text (route support tickets, tag documents, detect intent) so they prompt an LLM. It works in the demo. A few months later the complaints are about latency, cost, and malformed JSON breaking things downstream

There's a simpler approach that gets better results on classification: freeze a sentence embedding model and train a logistic regression head on top. That's it. No fine-tuning, no GPU, no prompt engineering

I'll show both approaches on the same task so you can see the difference

Banking77 is a customer intent dataset with 77 classes. Things like "lost_or_stolen_card", "card_payment_not_recognised", "contactless_not_working". Many labels are semantically close, which is what makes it a real test

This is the kind of problem that shows up everywhere: support ticket routing, email classification, voice assistant intent detection. Lots of categories, subtle differences between them

Here's how you'd classify text with an LLM. Using OpenAI as the example since that's what most people reach for:

LLM classification with OpenAI

import openai import json

client = openai.OpenAI()

labels = [ "activate_my_card", "age_limit", "apple_pay_or_google_pay", "atm_support", "automatic_top_up", "balance_not_updated_after_bank_transfer", "balance_not_updated_after_cheque_or_cash_deposit", "beneficiary_not_allowed", "cancel_transfer", "card_about_to_expire", "card_acceptance",

... 66 more labels ...

"virtual_card_not_working", "visa_or_mastercard", "why_verify_identity", "wrong_amount_of_cash_received", "wrong_exchange_rate_for_cash_withdrawal" ]

def classify_with_llm(text): response = client.chat.completions.create( model="gpt-4o-mini", temperature=0, messages=[{ "role": "user", "content": f"""Classify this customer message into exactly one of these categories: {json.dumps(labels)}

Message: {text}

Reply with ONLY the category name, nothing else.""" }] ) return response.choices[0].message.content.strip()

result = classify_with_llm("I was charged twice for the same transaction")

→ "card_payment_not_recognised" (maybe, if the JSON doesn't break)

Problems with this: