
Dbt-lineage – see what a dbt change breaks, without a warehouse
Every dbt PR gets one comment: what changed, which downstream columns and dashboards it breaks, and what to rebuild. Live demo →
以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。
Every dbt PR gets one comment: what changed, which downstream columns and dashboards it breaks, and what to rebuild. Live demo →
See what a dbt change breaks — downstream models, columns, dashboards and contracts — before it ships. dbt-lineage parses your project directly (no warehouse, no Python, no dbt compile) and puts lineage and impact wherever you work.
Where What you get Get it
Pull requests (GitHub) One sticky comment per PR: lineage diff, column impact, affected dashboards, breaking changes, rebuild selector; inline lint annotations uses: sipemu/dbt-lineage-viewer@v0 · live demo
GitLab · Bitbucket · Azure DevOps The same report as an MR/PR comment, plus Code Quality / Code Insights / build warnings docs/CI.md
VS Code, Cursor, VSCodium Show impact of my changes panel, hover and go-to-definition on ref()/source(), lint diagnostics; binary bundled VS Code Marketplace · Open VSX
Terminal Lineage, impact, diff, plan, lint, coverage, interactive TUI pip install dbt-lineage-cli · brew install sipemu/tap/dbt-lineage · cargo binstall dbt-lineage
Browser Static lineage site with column lineage for GitHub/GitLab Pages dbt-lineage site · demo site
BI tools Dashboards from Metabase, Tableau, Looker, Power BI, Superset and anofox-visualization as dbt exposures dbt-lineage exposures <tool>
AI agents MCP server with lineage, impact and column-lineage tools dbt-lineage mcp
Neovim, Helix Language server (dbt-lineage lsp) docs/EDITORS.md
Features
Source modes
Direct SQL parsing — extracts ref() and source() via a Jinja-aware lexer, no dbt compile needed
Manifest support — optionally read manifest.json for column metadata, materializations, and full graph fidelity
Macro-aware ref() — SQL-parse mode follows simple {% macro smart_ref(name) %}{{ ref(name) }}{% endmacro %} wrappers automatically
Parallel parsing + on-disk cache — rayon-parallelized parsing and a content-hash cache at .dbt-lineage/cache.bin; warm runs are near-instant on large monorepos. Bypass with --no-cache. See docs/CACHE.md
Analysis subcommands
summary — one-shot project overview (counts, tags, top fan-out, orphans)
impact <model> — downstream impact with severity scoring (Critical / High / Medium / Low)
diff --base <ref> — compares lineage between git refs, showing added / removed / modified nodes and edges
plan --base <ref> — emits a dbt selector for the minimal rebuild set; pipes into dbt run -s "$(…)"
coverage — tests-per-model + per-column map; text, JSON, or SARIF (PR annotations)
lint — flags unused sources, undefined refs, dead-end models, missing descriptions, models with only unresolved upstream, circular refs
site — static lineage site (search, per-node pages with lineage diagram, column-level lineage, SQL) for GitHub/GitLab Pages
docs — generates per-model Markdown with description, upstream / downstream, columns, embedded Mermaid lineage
perf — joins run_results.json with the DAG: slowest models, critical paths
check-manifest — flags drift between manifest.json and current SQL files using dbt's recorded SHA-256 (mtime fallback); exits non-zero for CI gating
Column-level lineage
AST-backed analyzer via sqlparser (0.6.0+) handles aliases, JOINs, aggregations, and CTEs with proper transformation classification (Direct, Aliased, Derived)
Regex-based analyzer remains as a fallback for dialect-specific syntax sqlparser can't yet parse
MCP server (AI-agent integration)
dbt-lineage mcp — stdio Model Context Protocol server. See docs/MCP.md for setup and cookbook.
10 tools: summary, search_models, lineage, impact, get_model_details, read_model_sql, column_upstream, column_downstream, lineage_bundle, propose_test
Resources: model SQL, focused lineage diagrams, source/exposure metadata, always-fresh summary
Prompts: review-impact, propose-tests, explain-column, find-bottleneck, document-model, onboard-project
Output
6 formats — ASCII, Graphviz DOT, JSON, Mermaid, self-contained SVG, interactive HTML (pan / zoom / search)
Mermaid niceties — --show-columns inlines column names in node labels; --group-by directory emits Mermaid subgraphs
Graph collapse — --collapse / --collapse=focal drops intermediate nodes and labels transitive paths with (via N) edges
Structured errors — --error-format json writes {level, what, why, hint} to stderr for deterministic agent / CI consumers
Editor integration
dbt-lineage lsp — language server: hover and go-to-definition on ref() / source(), lineage code lens per model, lint diagnostics. VS Code extension on the Marketplace and Open VSX (code --install-extension sipemu.dbt-lineage-viewer); Neovim and Helix setup in docs/EDITORS.md
Interactive TUI
Selector expressions — -s tag:finance,path:marts filters by tag, path, or model name (comma-separated)
Node-type filters — --include-tests, --include-seeds, --include-snapshots, --include-exposures
Node-type support — models, sources, seeds, snapshots, tests, exposures
Installation
From crates.io
cargo install dbt-lineage
pip / uv (no Rust needed)
pip install dbt-lineage-cli # installs the dbt-lineage command
uvx --from dbt-lineage-cli dbt-lineage --help
Homebrew (macOS, Linux)
brew install sipemu/tap/dbt-lineage
With cargo-binstall (no compile)
cargo binstall dbt-lineage
Pre-built binaries
Every tagged release ships pre-built binaries for Linux (x86_64 + aarch64), macOS (Intel + Apple Silicon), and Windows (x86_64) as downloadable assets on the Releases page. Download the archive for your platform, extract, and put the binary on your PATH.
Live demo: sipemu/dbt-lineage-demo#1 shows the PR comment (lineage diff with SQL body changes, column impact, Mermaid impact diagram, rebuild selector, lint); sipemu.github.io/dbt-lineage-demo is the same project as a dbt-lineage site.
on: pull_request
permissions:
contents: read
pull-requests: write # sticky PR comment
security-events: write # only needed with sarif:
jobs: lineage: runs-on: ubuntu-latest steps:
- uses: actions/checkout@v5
with:
fetch-depth: 0 #
diff/planneed the base branch history - uses: sipemu/dbt-lineage-viewer@v0 with: base: ${{ github.base_ref }} # e.g. main; resolved to origin/main if needed project-dir: dbt/ comment: true # post a Markdown summary as a PR comment sarif: lint,coverage # also surface findings as PR annotations
@v0 tracks the latest 0.x release; pin an exact tag (e.g. @v0.9.1) if you prefer. The action installs the matching dbt-lineage binary from this repo's release artifacts, runs summary, diff, plan, and optionally lint/coverage SARIF, then posts a single sticky comment on the PR with the rendered report. See action.yml for the full input/output reference.
docker run --rm -v "$PWD:/workspace" ghcr.io/sipemu/dbt-lineage:latest summary
Published to ghcr.io/sipemu/dbt-lineage on every release (latest, v0, 0.x, 0.x.y; amd64 + arm64), with git included for diff / plan / ci report. In GitLab CI, override the entrypoint:
dbt-lineage: image: { name: ghcr.io/sipemu/dbt-lineage:v0, entrypoint: [""] } script: dbt-lineage ci report --out report.md
As a pre-commit hook
.pre-commit-config.yaml in your dbt repo
repos:
- repo: https://github.com/sipemu/dbt-lineage-viewer rev: v0.10.0 hooks:
- id: dbt-lineage-lint
- id: dbt-lineage-check-manifest # needs a compiled target/manifest.json args: [-p, path/to/dbt] # only if the project isn't at the repo root
From source
git clone https://github.com/sipemu/dbt-lineage-viewer.git cd dbt-lineage-viewer cargo install --path .
The binary is installed to ~/.cargo/bin/dbt-lineage.
Static output
Full lineage of current directory's dbt project
dbt-lineage
Focus on a specific model
dbt-lineage stg_orders
Point at a different project directory
dbt-lineage -p path/to/dbt/project
Show 2 levels upstream, 1 downstream
dbt-lineage stg_orders -u 2 -d 1
Include seeds, tests, snapshots, exposures
dbt-lineage --include-seeds --include-tests --include-snapshots --include-exposures
Selector expressions
dbt-lineage -s tag:finance,path:marts
Use manifest.json instead of parsing SQL
dbt-lineage --manifest target/manifest.json
Output formats
dbt-lineage -o dot > lineage.dot # Graphviz DOT dbt-lineage -o json # JSON graph dbt-lineage -o mermaid # Mermaid diagram dbt-lineage -o svg > lineage.svg # Self-contained SVG dbt-lineage -o html > lineage.html # Interactive HTML (pan/zoom/search)
Interactive TUI
dbt-lineage -i dbt-lineage -i -p path/to/dbt/project dbt-lineage -i stg_orders -u 3 -d 3
Breaking changes to governed models
diff, ci report and the PR comment flag changes that break consumers of governed models (dbt 1.5+ model governance in schema YAML):
ci report --fail-on breaking-changes fails the check. lint adds a private-ref error for refs to access: private models from outside their group, which dbt rejects at parse time.
Compute downstream impact for a model with severity scoring:
dbt-lineage impact orders -p path/to/project # text report dbt-lineage impact orders -o json # JSON for CI dbt-lineage impact orders --manifest target/manifest.json
Severity levels:
Compare lineage between git refs to see what changed:
dbt-lineage diff --base main # compare main to working tree dbt-lineage diff --base main --head feature-branch # compare two branches dbt-lineage diff --base HEAD~1 -o json # JSON for CI integration
Shows added, removed, and modified nodes and edges with a summary of changes.
A self-contained static site — search, one page per model/source/seed/exposure with its lineage diagram, columns with column-level lineage, and highlighted SQL — ready for GitHub or GitLab Pages:
dbt-lineage site -p path/to/project --out site
Light/dark, phone-friendly, no server or CDN. See docs/SITE.md for Pages workflows.
One-shot project overview, designed as the first thing an agent (or human) runs against an unfamiliar dbt project:
dbt-lineage summary -p path/to/project # text dbt-lineage summary --manifest target/manifest.json # manifest mode dbt-lineage summary -o json # structured for piping into jq
Reports counts per node type, tag distribution, top downstream-reach models, and orphan models (no downstream consumers).
Generate dbt exposures from Metabase, anofox-visualization, Tableau, Looker, Superset or Power BI dashboards, so impact analysis knows which dashboards a change reaches:
METABASE_API_KEY=mb_… dbt-lineage exposures metabase --url https://metabase.example.com
--write models/exposures_metabase.yml
From a LookML project, offline (LookML dashboards → explores → views → tables):
dbt-lineage exposures looker --lookml path/to/lookml-project --write models/exposures_looker.yml
From anofox-visualization dashboards (DuckDB .sql files with ::ROLE panels):
dbt-lineage exposures anofox --dashboards dashboards/ --url 'https://viz.example.com/d/{name}'
--write models/exposures_anofox.yml
From Tableau workbooks, offline (.twb / .twbx):
dbt-lineage exposures tableau --workbooks workbooks/ --write models/exposures_tableau.yml
From Superset dashboard exports, offline (ZIPs from superset export-dashboards or the UI; dashboard → charts → datasets → tables, virtual-dataset SQL included):
dbt-lineage exposures superset --exports exports/
--url 'https://superset.example.com/superset/dashboard/{name}/' --write models/exposures_superset.yml
From Power BI, offline: PBIP projects (*.Report + *.SemanticModel, TMDL or model.bim) and .pbit templates; tables come from the partitions' Power Query (M) sources and native SQL:
dbt-lineage exposures powerbi --reports powerbi/ --url 'https://app.powerbi.com/groups/me/reports/{name}'
--write models/exposures_powerbi.yml
Tables the dashboards read are mapped onto models and sources; unmatched tables are listed on stderr. See docs/BI.md.
One Markdown report for PR/MR comments — lineage diff, rebuild selector, manifest freshness, lint, and project summary — plus exit codes for gating:
dbt-lineage ci report # base = PR/MR target branch from CI env dbt-lineage ci report --b
正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。