返回资讯中心
外部精选
软件工程
#开发者实时流

Show HN: Typed data pipeline – in TypeScript and DuckDB

croft is one tool that pulls data from APIs and files, transforms it, keeps it fresh on a schedule, and serves it to your app. Pipelines are TypeScript, with SQL where you want it, and everything lives in one DuckDB fil…

Hacker News 最新uptownhr2026/10/18 · 2026/10/187 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

croft is one tool that pulls data from APIs and files, transforms it, keeps it fresh on a schedule, and serves it to your app. Pipelines are TypeScript, with SQL where you want it, and everything lives in one DuckDB file. No Python, no servers, no glue between tools.

A croft project is a folder of TypeScript and SQL files. Each file becomes one table, and croft runs them in order.

From Stripe, GitHub or any API, or from CSV exports. Incremental, typed, and safe to rerun.

// assets/stripe_charges.ts export default ingest({ key: "id", schedule: "every hour", async *rows({ http }) { … }, });

Transform

Shape it

A SQL query or a TypeScript function per table, with checks that run before anything is saved.

-- assets/revenue_by_month.sql -- check: revenue >= 0 SELECT month, sum(amount) AS revenue FROM stripe_charges GROUP BY month

Schedule

Keep it fresh

A schedule in plain words. Everything downstream updates after it, in order.

$ croft schedule on $ croft status ASSET NEXT stripe_charges every hour revenue_by_month after inputs

Serve

Use it in your app

Query the tables from any Bun or Node app, with no database server to run.

import { query } from "@zabaca/croft/read";

const rows = await query( "from revenue_by_month order by month", );

Typed across SQL and TypeScript

Every table gets a TypeScript type, generated from its real columns. Change a SQL query, and croft checks the TypeScript that reads it, before anything runs.

// .croft/types/revenue_by_region.d.ts, written by croft export type RevenueByRegionRow = { region: string; // VARCHAR, the key orders: number | bigint | null; revenue: number | null; // DOUBLE };

After renaming revenue to total in the SQL:

$ croft validate --types error UNKNOWN_INPUT_COLUMN assets/region_labels.ts:9:65 revenue_by_region has no column "revenue" fix: revenue_by_region has these columns: region, orders, total, _loaded_at

What it replaces

A typical data stack is several tools, in Python and YAML, glued together. croft is one, in TypeScript.

croft does these jobs; it does not read those tools' projects. It is built for one machine and one DuckDB file. If your team already runs a warehouse with dbt, keep it.

Tell Claude Code what you want. It writes the pipeline, and croft checks every step.

Some data can't be fetched again. croft never throws it away on its own.

Each update is saved whole or not at all, and a failed check keeps the old data.

Anything replaced or deleted goes to the trash first, and comes back with one command.

One file on your computer. No account, no server, no cloud to pay for.

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →