Where Jev Fits in a Large-Scale Data Pipeline
DevelopersWhere Jev Fits in a Large-Scale Data Pipeline
以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。
Back to Blog
DevelopersWhere Jev Fits in a Large-Scale Data Pipeline
By Evan Fenster·Oct 5, 2026
We're building a universal product catalog, which means we process data for tens of millions of physical products every day. In simplest terms, we take raw unstructured data from products and transform it to fit our internal product taxonomy. A multi-stage pipeline performs data extraction, classification, and named-entity resolution to get structured product info and establish relationships across brands, variants, and canonical product offers across different retailers.
Whilst much of the public discourse today focuses on the frontier models, we care much more about the cheapest models that we can run at enormous scale.
The majority of our work is high quantity, low intelligence tasks. That makes Jev an interesting candidate. Jev is TypeSafe AI's decision model: instead of generating free-form text, it answers structured questions, such as choosing an option from a supplied list, and returns a confidence estimate. We tested it against our existing pipeline, starting with product categorization.
Category taxonomies take the form of a tree with subcategories getting more specific the further down you go. This often leads to hierarchical classification as the standard approach: find the root node(s) that match best, then which of their children match best, and repeat recursively until none of the children are a valid match or you've reached leaf nodes.
Take “Nike Sportswear Club Fleece.” We may find node “Clothing & Accessories” matches best at the root level, then its child “Clothing,” then its child “Outerwear,” and finally “Coats & Jackets.”
TypeSafe's cookbook pattern walked through doing this top-down walk with beam search, a similar pattern as described above but additionally maintaining K best paths as you traverse. We replicated this as our first attempt at integrating Jev, and while it looked promising initially, it broke down in some crucial cases early on.
Consider the case below, where our product is a “Youth Bicycle.” In both Shopify and Google's taxonomy, this sits at Sporting Goods > Outdoor Recreation > Cycling > Bicycles. It takes 3 layers before you start getting any semblance of a bicycle with “Cycling,” whereas other root nodes like “Vehicles & Parts” score much higher. Across the entire tree, the path we need falls out of our beam quickly, and we incorrectly match our “Youth Bicycle” to a “Motor Vehicle.”
Why don't we do a Jev call per node, since it's so cheap? Then, we certainly would've found our match with a direct comparison to Bicycles at a score of 0.94. The answer is that Jev may not be as cheap as you think. With thousands of nodes in the full category taxonomy, this would have increased our costs ~2,000× compared to our current production setup. Scoring all 6,000 categories reached 98.0% accuracy, at $1.08 per product.
After a few more experiments, we did find a solution with Jev that worked incredibly well. Our existing pipeline uses an embedding space of all categories, over which we query with product info. Recall@1 was low (sub-40%), but Recall@30 was high (above 99%). So, we return the top 30 results and use Gemma 4 31B to select the best option out of the results, resulting in 81.9% accuracy.
We swapped Jev in as a reranker on that last step. Once we had our top 30 results, we performed a Choice call over the returned categories. Here, we were able to achieve 97.2% accuracy with similar costs at $0.0005 per product.
TLDR: Jev improved our product categorization accuracy from 81.9% to 97.2% at roughly the same cost when we used it to choose from 30 prefiltered categories. Searching the full taxonomy with Jev was too expensive.
For those coming from frontier models like Astra and Fable, using Jev probably feels close to free. But, if Jev can solve tasks that you've been using frontier models on, you've likely been overpaying for inference for a while, and models like GLM-5.3-Flash or DeepSeek V4.1 Flash could have performed similarly for pennies on the dollar. For those benchmarking Jev against cheaper open source models, it's closer to a 50% discount than many multiples of savings. At Channel3, we use batched inference that makes the costs nearly identical to Jev.
Field notes from the rapidly changing world of AI commerce
正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。