返回资讯中心
外部精选
软件工程
#开发者实时流

Dbt-lineage – see what a dbt change breaks, without a warehouse

Every dbt PR gets one comment: what changed, which downstream columns and dashboards it breaks, and what to rebuild. Live demo →

Hacker News 最新sipemu2026/11/5 · 今天30 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

Every dbt PR gets one comment: what changed, which downstream columns and dashboards it breaks, and what to rebuild. Live demo →

See what a dbt change breaks — downstream models, columns, dashboards and contracts — before it ships. dbt-lineage parses your project directly (no warehouse, no Python, no dbt compile) and puts lineage and impact wherever you work.

Where What you get Get it

Pull requests (GitHub) One sticky comment per PR: lineage diff, column impact, affected dashboards, breaking changes, rebuild selector; inline lint annotations uses: sipemu/dbt-lineage-viewer@v0 · live demo

GitLab · Bitbucket · Azure DevOps The same report as an MR/PR comment, plus Code Quality / Code Insights / build warnings docs/CI.md

VS Code, Cursor, VSCodium Show impact of my changes panel, hover and go-to-definition on ref()/source(), lint diagnostics; binary bundled VS Code Marketplace · Open VSX

Terminal Lineage, impact, diff, plan, lint, coverage, interactive TUI pip install dbt-lineage-cli · brew install sipemu/tap/dbt-lineage · cargo binstall dbt-lineage

Browser Static lineage site with column lineage for GitHub/GitLab Pages dbt-lineage site · demo site

BI tools Dashboards from Metabase, Tableau, Looker, Power BI, Superset and anofox-visualization as dbt exposures dbt-lineage exposures <tool>

AI agents MCP server with lineage, impact and column-lineage tools dbt-lineage mcp

Neovim, Helix Language server (dbt-lineage lsp) docs/EDITORS.md

Features

Source modes

Direct SQL parsing — extracts ref() and source() via a Jinja-aware lexer, no dbt compile needed

Manifest support — optionally read manifest.json for column metadata, materializations, and full graph fidelity

Macro-aware ref() — SQL-parse mode follows simple {% macro smart_ref(name) %}{{ ref(name) }}{% endmacro %} wrappers automatically

Parallel parsing + on-disk cache — rayon-parallelized parsing and a content-hash cache at .dbt-lineage/cache.bin; warm runs are near-instant on large monorepos. Bypass with --no-cache. See docs/CACHE.md

Analysis subcommands

summary — one-shot project overview (counts, tags, top fan-out, orphans)

impact <model> — downstream impact with severity scoring (Critical / High / Medium / Low)

diff --base <ref> — compares lineage between git refs, showing added / removed / modified nodes and edges

plan --base <ref> — emits a dbt selector for the minimal rebuild set; pipes into dbt run -s "$(…)"

coverage — tests-per-model + per-column map; text, JSON, or SARIF (PR annotations)

lint — flags unused sources, undefined refs, dead-end models, missing descriptions, models with only unresolved upstream, circular refs

site — static lineage site (search, per-node pages with lineage diagram, column-level lineage, SQL) for GitHub/GitLab Pages

docs — generates per-model Markdown with description, upstream / downstream, columns, embedded Mermaid lineage

perf — joins run_results.json with the DAG: slowest models, critical paths

check-manifest — flags drift between manifest.json and current SQL files using dbt's recorded SHA-256 (mtime fallback); exits non-zero for CI gating

Column-level lineage

AST-backed analyzer via sqlparser (0.6.0+) handles aliases, JOINs, aggregations, and CTEs with proper transformation classification (Direct, Aliased, Derived)

Regex-based analyzer remains as a fallback for dialect-specific syntax sqlparser can't yet parse

MCP server (AI-agent integration)

dbt-lineage mcp — stdio Model Context Protocol server. See docs/MCP.md for setup and cookbook.

10 tools: summary, search_models, lineage, impact, get_model_details, read_model_sql, column_upstream, column_downstream, lineage_bundle, propose_test

Resources: model SQL, focused lineage diagrams, source/exposure metadata, always-fresh summary

Prompts: review-impact, propose-tests, explain-column, find-bottleneck, document-model, onboard-project

Output

6 formats — ASCII, Graphviz DOT, JSON, Mermaid, self-contained SVG, interactive HTML (pan / zoom / search)

Mermaid niceties — --show-columns inlines column names in node labels; --group-by directory emits Mermaid subgraphs

Graph collapse — --collapse / --collapse=focal drops intermediate nodes and labels transitive paths with (via N) edges

Structured errors — --error-format json writes {level, what, why, hint} to stderr for deterministic agent / CI consumers

Editor integration

dbt-lineage lsp — language server: hover and go-to-definition on ref() / source(), lineage code lens per model, lint diagnostics. VS Code extension on the Marketplace and Open VSX (code --install-extension sipemu.dbt-lineage-viewer); Neovim and Helix setup in docs/EDITORS.md

Interactive TUI

Selector expressions — -s tag:finance,path:marts filters by tag, path, or model name (comma-separated)

Node-type filters — --include-tests, --include-seeds, --include-snapshots, --include-exposures

Node-type support — models, sources, seeds, snapshots, tests, exposures

Installation

From crates.io

cargo install dbt-lineage

pip / uv (no Rust needed)

pip install dbt-lineage-cli # installs the dbt-lineage command uvx --from dbt-lineage-cli dbt-lineage --help

Homebrew (macOS, Linux)

brew install sipemu/tap/dbt-lineage

With cargo-binstall (no compile)

cargo binstall dbt-lineage

Pre-built binaries

Every tagged release ships pre-built binaries for Linux (x86_64 + aarch64), macOS (Intel + Apple Silicon), and Windows (x86_64) as downloadable assets on the Releases page. Download the archive for your platform, extract, and put the binary on your PATH.

Live demo: sipemu/dbt-lineage-demo#1 shows the PR comment (lineage diff with SQL body changes, column impact, Mermaid impact diagram, rebuild selector, lint); sipemu.github.io/dbt-lineage-demo is the same project as a dbt-lineage site.

on: pull_request

permissions: contents: read pull-requests: write # sticky PR comment security-events: write # only needed with sarif:

jobs: lineage: runs-on: ubuntu-latest steps:

  • uses: actions/checkout@v5 with: fetch-depth: 0 # diff / plan need the base branch history
  • uses: sipemu/dbt-lineage-viewer@v0 with: base: ${{ github.base_ref }} # e.g. main; resolved to origin/main if needed project-dir: dbt/ comment: true # post a Markdown summary as a PR comment sarif: lint,coverage # also surface findings as PR annotations

@v0 tracks the latest 0.x release; pin an exact tag (e.g. @v0.9.1) if you prefer. The action installs the matching dbt-lineage binary from this repo's release artifacts, runs summary, diff, plan, and optionally lint/coverage SARIF, then posts a single sticky comment on the PR with the rendered report. See action.yml for the full input/output reference.

docker run --rm -v "$PWD:/workspace" ghcr.io/sipemu/dbt-lineage:latest summary

Published to ghcr.io/sipemu/dbt-lineage on every release (latest, v0, 0.x, 0.x.y; amd64 + arm64), with git included for diff / plan / ci report. In GitLab CI, override the entrypoint:

dbt-lineage: image: { name: ghcr.io/sipemu/dbt-lineage:v0, entrypoint: [""] } script: dbt-lineage ci report --out report.md

As a pre-commit hook

.pre-commit-config.yaml in your dbt repo

repos:

  • repo: https://github.com/sipemu/dbt-lineage-viewer rev: v0.10.0 hooks:
  • id: dbt-lineage-lint
  • id: dbt-lineage-check-manifest # needs a compiled target/manifest.json args: [-p, path/to/dbt] # only if the project isn't at the repo root

From source

git clone https://github.com/sipemu/dbt-lineage-viewer.git cd dbt-lineage-viewer cargo install --path .

The binary is installed to ~/.cargo/bin/dbt-lineage.

Static output

Full lineage of current directory's dbt project

dbt-lineage

Focus on a specific model

dbt-lineage stg_orders

Point at a different project directory

dbt-lineage -p path/to/dbt/project

Show 2 levels upstream, 1 downstream

dbt-lineage stg_orders -u 2 -d 1

Include seeds, tests, snapshots, exposures

dbt-lineage --include-seeds --include-tests --include-snapshots --include-exposures

Selector expressions

dbt-lineage -s tag:finance,path:marts

Use manifest.json instead of parsing SQL

dbt-lineage --manifest target/manifest.json

Output formats

dbt-lineage -o dot > lineage.dot # Graphviz DOT dbt-lineage -o json # JSON graph dbt-lineage -o mermaid # Mermaid diagram dbt-lineage -o svg > lineage.svg # Self-contained SVG dbt-lineage -o html > lineage.html # Interactive HTML (pan/zoom/search)

Interactive TUI

dbt-lineage -i dbt-lineage -i -p path/to/dbt/project dbt-lineage -i stg_orders -u 3 -d 3

Breaking changes to governed models

diff, ci report and the PR comment flag changes that break consumers of governed models (dbt 1.5+ model governance in schema YAML):

ci report --fail-on breaking-changes fails the check. lint adds a private-ref error for refs to access: private models from outside their group, which dbt rejects at parse time.

Compute downstream impact for a model with severity scoring:

dbt-lineage impact orders -p path/to/project # text report dbt-lineage impact orders -o json # JSON for CI dbt-lineage impact orders --manifest target/manifest.json

Severity levels:

Compare lineage between git refs to see what changed:

dbt-lineage diff --base main # compare main to working tree dbt-lineage diff --base main --head feature-branch # compare two branches dbt-lineage diff --base HEAD~1 -o json # JSON for CI integration

Shows added, removed, and modified nodes and edges with a summary of changes.

A self-contained static site — search, one page per model/source/seed/exposure with its lineage diagram, columns with column-level lineage, and highlighted SQL — ready for GitHub or GitLab Pages:

dbt-lineage site -p path/to/project --out site

Light/dark, phone-friendly, no server or CDN. See docs/SITE.md for Pages workflows.

One-shot project overview, designed as the first thing an agent (or human) runs against an unfamiliar dbt project:

dbt-lineage summary -p path/to/project # text dbt-lineage summary --manifest target/manifest.json # manifest mode dbt-lineage summary -o json # structured for piping into jq

Reports counts per node type, tag distribution, top downstream-reach models, and orphan models (no downstream consumers).

Generate dbt exposures from Metabase, anofox-visualization, Tableau, Looker, Superset or Power BI dashboards, so impact analysis knows which dashboards a change reaches:

METABASE_API_KEY=mb_… dbt-lineage exposures metabase --url https://metabase.example.com
--write models/exposures_metabase.yml

From a LookML project, offline (LookML dashboards → explores → views → tables):

dbt-lineage exposures looker --lookml path/to/lookml-project --write models/exposures_looker.yml

From anofox-visualization dashboards (DuckDB .sql files with ::ROLE panels):

dbt-lineage exposures anofox --dashboards dashboards/ --url 'https://viz.example.com/d/{name}'
--write models/exposures_anofox.yml

From Tableau workbooks, offline (.twb / .twbx):

dbt-lineage exposures tableau --workbooks workbooks/ --write models/exposures_tableau.yml

From Superset dashboard exports, offline (ZIPs from superset export-dashboards or the UI; dashboard → charts → datasets → tables, virtual-dataset SQL included):

dbt-lineage exposures superset --exports exports/
--url 'https://superset.example.com/superset/dashboard/{name}/' --write models/exposures_superset.yml

From Power BI, offline: PBIP projects (*.Report + *.SemanticModel, TMDL or model.bim) and .pbit templates; tables come from the partitions' Power Query (M) sources and native SQL:

dbt-lineage exposures powerbi --reports powerbi/ --url 'https://app.powerbi.com/groups/me/reports/{name}'
--write models/exposures_powerbi.yml

Tables the dashboards read are mapped onto models and sources; unmatched tables are listed on stderr. See docs/BI.md.

One Markdown report for PR/MR comments — lineage diff, rebuild selector, manifest freshness, lint, and project summary — plus exit codes for gating:

dbt-lineage ci report # base = PR/MR target branch from CI env dbt-lineage ci report --b

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →