返回资讯中心
外部精选
大数据
#数据科学#机器学习

From Messy Documents to Structured Data with Docling

Docling takes documents in whatever inconsistent format they arrive in, and converts them into one unified, structured representation that both people and AI systems can work with reliably, rather than everyone downstre…

KDnuggetsShittu Olumide2026/10/1 · 25 天前29 分钟阅读

以下正文同步自 KDnuggets,版权归原站所有,已转换为易读排版。

Once you have a DoclingDocument, getting it into whatever format your downstream system actually wants is a one-line call, and it's worth knowing your options rather than defaulting to Markdown out of habit.

from docling.document_converter import DocumentConverter

converter = DocumentConverter() doc = converter.convert("quarterly_report.pdf").document

Markdown: the natural choice when the output is headed

into an LLM prompt or a RAG pipeline

markdown_output = doc.export_to_markdown()

JSON: the structured, lossless option -- best when a downstream

system needs to parse specific fields programmatically

json_output = doc.export_to_dict()

HTML: useful when a human is going to browse the result

directly in a browser rather than consume it programmatically

html_output = doc.export_to_html()

The choice here really comes down to who or what reads the output next. Markdown is the right call when feeding into most language models, since it's compact and models are heavily trained on it. JSON is the right call when another piece of code needs to reliably find, say, "the third table on page 4" without re-parsing anything. HTML earns its place when the result needs to actually render for a person, preserving visual structure a plain text or Markdown export would flatten.

This is where the pain points from earlier in the article actually get resolved. Scanned pages, for instance, need OCR before there's any text to extract at all, and Docling handles this as a pipeline option rather than a separate tool you'd have to bolt on:

from docling.document_converter import DocumentConverter, PdfFormatOption from docling.datamodel.pipeline_options import PdfPipelineOptions from docling.datamodel.base_models import InputFormat

Configure the pipeline to run OCR -- needed for scanned pages

where there's no embedded text layer to read directly

pipeline_options = PdfPipelineOptions() pipeline_options.do_ocr = True pipeline_options.do_table_structure = True # explicitly enable

table structure detection

converter = DocumentConverter( format_options={ InputFormat.PDF: PdfFormatOption(pipeline_options=pipeline_options) } )

doc = converter.convert("scanned_invoice.pdf").document print(doc.export_to_markdown())

The do_ocr flag is what triggers text recognition on pages that don't already have a text layer, which is exactly the situation with a scanned document, a fax, or a photographed receipt. do_table_structure is worth calling out on its own, because it's doing more than most people expect: Docling isn't just locating where a table sits on the page, it's reconstructing actual rows, columns, and multi-level headers, and it can correctly interpret cell content that's more complex than a single value — a list embedded inside one cell, for instance — rather than flattening everything into a single blob of text the way a naive extractor would.

This is the first genuinely advanced step in this article, and it's the one most relevant if you're feeding documents into a retrieval-augmented generation (RAG) system. Splitting a document into chunks sounds simple until you've watched a naive character-count splitter cut a sentence in half, or separate a table's header row from the data beneath it — both of which quietly wreck retrieval quality.

Docling's HybridChunker is built specifically to avoid that. It starts from the document's actual structure — the same body tree covered in Step 2 — rather than blindly counting characters, and then applies tokenizer-aware refinements on top: splitting a chunk further only when it's genuinely too large for your target token limit, and merging adjacent undersized chunks back together when they share the same heading, so you don't end up with dozens of tiny, context-poor fragments either.

from docling.document_converter import DocumentConverter from docling.chunking import HybridChunker

converter = DocumentConverter() doc = converter.convert("employee_handbook.pdf").document

HybridChunker defaults to a tokenizer aligned with common

embedding models; merge_peers=True (the default) combines

small adjacent chunks that share the same heading

chunker = HybridChunker()

chunks = list(chunker.chunk(dl_doc=doc))

for chunk in chunks[:3]:

.contextualize() returns the chunk text enriched with

its metadata (like its section heading), which is what

you actually want to feed to an embedding model

print(chunker.contextualize(chunk)) print("---")

chunker.chunk(dl_doc=doc) returns an iterator of chunk objects, each one a genuine piece of the document's structure rather than an arbitrary character slice. The contextualize() call matters more than it looks: a raw chunk of text loses the section heading it lived under, but contextualize() folds that context back in, so a chunk about "termination policy" still carries the fact that it came from a section called "Employee Conduct" — which is exactly the kind of context an embedding model needs to retrieve it correctly later. merge_peers, on by default, is what stops the chunker from producing a flood of tiny, nearly useless fragments out of a document with lots of short paragraphs under the same heading.

Everything up to this point has been about getting a clean, structured representation of a document. This last step is where that structure actually turns into the specific data you need, and it's the part of Docling most tutorials skip past — which is a shame, because it's the feature that most directly matches what this article's title promises.

Docling's DocumentExtractor lets you define a schema — either as a simple dictionary or as a full Pydantic model — and get back validated, typed data instead of a wall of text you'd still need to parse yourself. The official example uses a real Swiss QR-bill — an invoice with a bill number, a total, and other fields — as its running case, and it's worth reproducing here because it demonstrates the idea cleanly.

from docling.datamodel.base_models import InputFormat from docling.document_extractor import DocumentExtractor from pydantic import BaseModel, Field from typing import Optional

DocumentExtractor works across both images and PDFs

extractor = DocumentExtractor(allowed_formats=[InputFormat.IMAGE, InputFormat.PDF])

Define exactly the fields you want back, as a normal Pydantic model.

Field(examples=[...]) helps guide extraction without forcing a value.

class Invoice(BaseModel): bill_no: str = Field(examples=["A123", "5414"]) total: float = Field(default=10, examples=[20]) tax_id: Optional[str] = Field(default=None, examples=["1234567890"])

result = extractor.extract( source="invoice_scan.jpg", template=Invoice, # the Pydantic class itself becomes the extraction template )

extracted_data is already a plain dict matching the Invoice schema

print(result.pages[0].extracted_data)

Running this against a real invoice image returns something like {'bill_no': '3139', 'total': 3949.75, 'tax_id': None} — genuinely typed values, not a string you'd still need to regex apart. The Field(examples=[...]) pattern is worth understanding specifically: it doesn't force a value, it gives the extraction model a hint about the shape and format of what it's looking for, which measurably improves accuracy on fields that could otherwise be ambiguous — a bill number that could be read as a date, for instance.

It's worth taking this one step further, because Docling doesn't limit you to flat fields. Nested Pydantic models work directly:

class Contact(BaseModel): name: Optional[str] = Field(default=None, examples=["Smith"]) address: str = Field(default="123 Main St", examples=["456 Elm St"]) city: str = Field(default="Anytown", examples=["Othertown"])

class ExtendedInvoice(BaseModel): bill_no: str = Field(examples=["A123", "5414"]) total: float = Field(default=10, examples=[20]) sender: Contact = Field(default=Contact()) receiver: Contact = Field(default=Contact())

result = extractor.extract(source="invoice_scan.jpg", template=ExtendedInvoice)

Validate and load the result back into a real Pydantic object --

not just a dict -- so you get type checking and IDE autocomplete

invoice = ExtendedInvoice.model_validate(result.pages[0].extracted_data)

print(f"Invoice #{invoice.bill_no} was sent by {invoice.sender.name} to {invoice.receiver.name}.")

That last block is the whole article's argument in miniature. sender and receiver are their own full Pydantic models nested inside ExtendedInvoice, and model_validate() takes the extracted dictionary and turns it into an actual typed Python object — with real attribute access and validation — not a loose bag of keys you're hoping are spelled consistently. That's the distance covered between the scanned image this section started with and a line like invoice.sender.name you can trust enough to put directly into a database write or an API call.

The single-script examples above are how you learn the tool, but it's worth knowing how this fits into an actual production setup. Docling ships native integrations with LangChain, LlamaIndex, and Haystack, so if you're already building a RAG pipeline in one of those frameworks, Docling slots in as the document-loading step rather than requiring you to glue anything together yourself. For agent-based workflows specifically, there's a Model Context Protocol (MCP) server that lets an AI agent call Docling's conversion and extraction capabilities directly as a tool.

If you'd rather not run the models yourself, there are two paths worth knowing. Docling Serve packages the whole engine behind a REST API you can self-host — useful for a team that wants a shared internal conversion service without every application embedding the library directly. And as of June 15, 2026, IBM began offering Docling as a managed software-as-a-service (SaaS) product through watsonx, for teams that would rather not host any of it themselves at all, built directly on the same open-source engine covered in this article.

Worth being upfront about the current edges of the project, since it's under genuinely active development and the feature set keeps expanding month to month. Per the project's own repository roadmap, metadata extraction (pulling out a document's title, authors, references, and language automatically) and complex chemistry understanding (parsing molecular structures) are both listed as coming soon rather than available today, so don't build around either yet.

One practical resource note worth planning around: Docling's layout and table-structure models run locally, which is genuinely good for privacy but means processing time and memory use scale with document complexity and volume. For anyone converting documents at real scale, or reaching for one of the heavier vision-language model pipelines like GraniteDocling, GPU acceleration makes a meaningful difference and is worth budgeting for rather than treating as an afterthought.

The actual distance covered in this article isn't from "PDF" to "Markdown." It's from a document that only a person could reliably make sense of to one a program can trust enough to act on directly — look up a bill number, pull a sender's address, hand a clean chunk to an embedding model without second-guessing where it came from. That's the real difference between section one's garbled spreadsheet paste and section eleven's validated invoice.sender.name, and it's a distance most teams still cross by hand, one copy-paste at a time, simply because they've never had a reliable way not to.

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →