返回资讯中心
外部精选
软件工程
#开发者实时流

File-PII – Extract text, metadata and PII from PDFs in pure JavaScript

Extract text and metadata from files, and check them for personal data, without native dependencies: one class, three methods, one error type. It is built on unpdf (the serverless build of pdf.js) and runs anywhere Node…

Hacker News 最新luisperis10112026/10/21 · 25 天前29 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

Extract text and metadata from files, and check them for personal data, without native dependencies: one class, three methods, one error type. It is built on unpdf (the serverless build of pdf.js) and runs anywhere Node 22 does.

npm install @axium-lab/file-pii

import { FileExtractor } from '@axium-lab/file-pii';

const extractor = new FileExtractor(input); // Buffer, base64 or data URI; detects the format from the bytes

extractor.kind; // 'pdf'

const { raw } = await extractor.extract_text(); // raw: 'Chapter 1: Introduction This page has a link… Appendix Smaller page size.'

await extractor.extract_text({ pages: [2, 4] }); // { raw: '…', pages: [{ number: 2, text: '…' }, { number: 4, text: '…' }] }, only those pages

const metadata = await extractor.get_metadata(); // { title: 'Full Metadata Sample', pageCount: 3, encrypted: true, permissions: ['print', 'copy', 'fill_forms'], … }

const scan = await extractor.scan_pii({ countries: ['ES', 'GLOBAL'], policy: { kinds: { PAYMENT_CARD: 'block' } } }); // { blocked: true, blocked_by: [{ entity: 'CREDIT_CARD', … }], entities: [{ entity: 'ES_NIF', … }, …], metadata: { kind: 'pdf' } }

await extractor.scan_pii(piiOptions, { pages: true }); // same, plus metadata.pages: [{ number: 1, entities: [...] }, …], one entry per page

Text as one line, and per page. raw is the default: the document flattened, ready for an LLM prompt or a search index. pages is opt-in and keeps page numbers and line breaks for chunking and citing; ask for all of them or only some.

Everything the document declares about itself. Info dictionary, XMP, flags, permissions, bookmarks, attachments, form fields, layers. One open, no rendering.

Personal data, page by page. scan_pii() runs euro-pii over the document and tells you what it found, on which page, and whether your policy refuses the file. The file is never rewritten.

Detection by magic bytes, not by extension. The caller cannot lie about the file type.

One error type. Every failure is a FileExtractorError with a category you can switch on.

Lightweight. One runtime dependency (unpdf, ~2 MB, zero transitive dependencies) and one peer (@axium-lab/euro-pii, zero dependencies). No canvas, no WASM to configure.

ESM and CommonJS, with TypeScript declarations included.

Contents

Installation

Usage

Extracting text: extract_text()

Reading metadata: get_metadata()

Detecting personal data: scan_pii()

Errors

API reference

Limitations

Development

Releasing

Installation

npm install @axium-lab/file-pii

or

pnpm add @axium-lab/file-pii

or

bun add @axium-lab/file-pii

Requires Node.js 22 or later. There is nothing to configure.

@axium-lab/euro-pii is a peer dependency with no version constraint: npm 7+, pnpm and Bun install it for you, and any published version works.

FileExtractor takes the file in any of these shapes (FileInput):

The constructor decodes the input, detects the format from its first bytes and fails fast if the input is empty or malformed, or the format is not supported. Nothing is parsed until you call a method.

All three are asynchronous and can be called in any order, as many times as you want.

import { FileExtractor } from '@axium-lab/file-pii';

const extractor = new FileExtractor(base64);

await extractor.extract_text();

{ "kind": "pdf", "raw": "Chapter 1: Introduction This page has a link, a text field and a checkbox. Chapter 2: Rotated page Section 2.1 lives here. Appendix Smaller page size." }

By default you get the whole document as one line. The options choose what to extract; only the selected pages are read.

await extractor.extract_text({ pages: [2, 3] });

{ "kind": "pdf", "raw": "Chapter 2: Rotated page Section 2.1 lives here. Appendix Smaller page size.", "pages": [ { "number": 2, "text": "Chapter 2: Rotated page\nSection 2.1 lives here." }, { "number": 3, "text": "Appendix\nSmaller page size." } ] }

Field Meaning

kind The detected format, 'pdf'.

raw The document, or the selected pages, as a single line: pages and line breaks joined with spaces. Absent when raw is false.

pages[].number 1-based page number. Pages come in ascending order. Present only when pages is requested.

pages[].text The page text with line breaks preserved. Empty for pages without a text layer.

A page number outside the document or a duplicate throws invalid_input, e.g. Page 99 is out of range (document has 3 pages). So does { raw: false, pages: false }, since there is nothing to extract.

To get a single string that keeps line breaks:

Info dictionary. The classic metadata written by the tool that produced the PDF. Any of them can be missing.

Viewer hints. How the author wants a viewer to display the file. Harmless, rarely useful for extraction.

await extractor.scan_pii( { countries: ['ES', 'GLOBAL'], // Spanish identifiers plus IBAN, cards, email… except: ['DATE_TIME'], // but not dates policy: { default: 'keep', // 'keep' | 'block' kinds: { PAYMENT_CARD: 'block' }, }, }, { pages: [1, 3] }, // optional: only these pages, broken down per page );

{ "blocked": true, "blocked_by": [ { "entity": "CREDIT_CARD", "kind": "PAYMENT_CARD", "country": "GLOBAL", "start": 74, "end": 93, "value": "4111 1111 1111 1111", "score": 1, "confirmedBy": "checksum", "…": "…" } ], "entities": [ { "entity": "ES_NIF", "kind": "TAX_ID", "country": "ES", "start": 24, "end": 33, "value": "12345678Z", "…": "…" }, { "entity": "EMAIL_ADDRESS", "kind": "EMAIL", "country": "GLOBAL", "start": 41, "end": 56, "value": "ana@example.com", "…": "…" }, { "entity": "CREDIT_CARD", "kind": "PAYMENT_CARD", "country": "GLOBAL", "start": 74, "end": 93, "value": "4111 1111 1111 1111", "…": "…" }, { "entity": "IBAN_CODE", "kind": "BANK_ACCOUNT", "country": "GLOBAL", "start": 99, "end": 128, "value": "ES91 2100 0418 4502 0005 1332", "…": "…" } ], "metadata": { "kind": "pdf", "pages": [ { "number": 1, "entities": [{ "entity": "ES_NIF", "start": 24, "end": 33, "…": "…" }, { "entity": "EMAIL_ADDRESS", "start": 41, "end": 56, "…": "…" }] }, { "number": 3, "entities": [{ "entity": "CREDIT_CARD", "start": 17, "end": 36, "…": "…" }, { "entity": "IBAN_CODE", "start": 42, "end": 71, "…": "…" }] } ] } }

The first argument is a euro-pii options object, passed through as it is: countries, kinds, entities and except choose what to look for, exactly as euro-pii documents them. The policy is the same too, with one difference: a file is never rewritten, so there is no mask. Each detection is either kept (reported, the file passes) or blocks (the file is refused). The most specific key wins: entities, then kinds, then default, which is 'keep' when left out. Without a policy, everything is reported and nothing blocks. A 'mask' that arrives from plain JSON behaves as 'keep'.

The second argument is the scope, with the same vocabulary as extract_text:

The result is what euro-pii's anonymize() returns for extract_text({ pages }).raw, without the anonymized text, plus metadata:

Each detection is a euro-pii Detection, untouched: entity, kind, dataClass, identifiability, country, start, end, score, value, pattern, confirmedBy. See its README for what each field means.

Pages are read and scanned one at a time, so a document with hundreds of pages costs one page of memory, not the whole text. Each page is scanned as a single line, exactly as raw is built, so the result is the same whether you look at entities or at metadata.pages. A value split across two pages is not detected.

const scan = await extractor.scan_pii({ countries: ['ES', 'GLOBAL'] }, { pages: true });

if (scan.entities.length === 0) { // nothing personal in the file }

if (scan.blocked) { const pages = (scan.metadata.pages ?? []).filter((p) => p.entities.length > 0).map((p) => p.number); throw new Error(Refused: ${scan.blocked_by.map((d) => d.entity).join(', ')} on pages ${pages.join(', ')}); }

Errors

Every error thrown by the library is a FileExtractorError with a category:

unsupported_format is thrown by the constructor, invalid_input by the constructor and by the methods, the rest by the methods. The original error is kept in cause; for scan_pii option errors that is the EuroPiiError.

import { FileExtractor, isFileExtractorError } from '@axium-lab/file-pii';

try { const extractor = new FileExtractor(base64); await extractor.extract_text(); } catch (error) { if (isFileExtractorError(error)) console.error(error.category, error.message); // password_required PDF extraction failed (PasswordException) }

API reference

class FileExtractor

Member Returns

new FileExtractor(input) input: FileInput. Throws invalid_input or unsupported_format

kind FileKind: 'pdf'

extract_text(options?) Promise<ExtractedText>

get_metadata() Promise<FileMetadata>

scan_pii(options?, scope?) options: ScanPiiOptions, scope: ScanPiiScope. Promise<PiiScan>

Other exports

Export What it is

FileExtractorError, isFileExtractorError The error class and its type guard

FILE_KINDS, SUPPORTED_MIME_TYPES ['pdf'] and ['application/pdf'], to skip other files before decoding them

Types: FileInput, ExtractTextOptions, ExtractedText, ExtractedPage, FileKind, FileMetadata, PdfMetadata, PdfPermission, PdfFormField, OutlineItem, ScanPiiOptions, ScanPiiScope, FilePolicy, FileAction, PiiScan, PiiPage, FileExtractorErrorCategory.

type FileAction = 'keep' | 'block';

type ScanPiiOptions = Selection & { // Selection is euro-pii's: countries, kinds, entities, except policy?: { default?: FileAction; kinds?: Partial<Record<Kind, FileAction>>; entities?: Partial<Record<EntityName, FileAction>>; }; };

type ScanPiiScope = { pages?: boolean | number[] };

type PiiScan = ( | { blocked: false; entities: Detection[] } | { blocked: true; blocked_by: Detection[]; entities: Detection[] } ) & { metadata: { kind: FileKind; pages?: { number: number; entities: Detection[] }[] }; };

Limitations

What file-pii does not do matters as much as what it does:

npm run dev # runs tests/manual.ts in watch mode npm run typecheck npm test # runs tests/inputs.ts and tests/pii.ts npm run build # dist/esm + dist/cjs via tsup, then scripts/check-dist.mjs

tests/manual.ts is a playground, not a test suite: it loads a fixture from tests/fixtures/, calls the API and prints the result. tests/fixtures/full.pdf exercises every metadata field. tests/errors.ts does the same for each error category and runs at the end of manual.ts.

docs/og-image.png is the README header, rendered from docs/og-image.html at 1200×630 with headless Chrome:

"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" --headless=new --disable-gpu --hide-scrollbars
--window-size=1200,630 --virtual-time-budget=5000 --screenshot=docs/og-image.png "file://$PWD/docs/og-image.html"

tests/inputs.ts and tests/pii.ts are assertion scripts: npm test runs both, or run one with bun tests/inputs.ts or npx tsx tests/pii.ts. tests/fixtures/pii.pdf has a NIF and an email on page 1, nothing on page 2, and a card and an IBAN on page 3.

Bump version in package.json.

Move the [Unreleased] entries in CHANGELOG.md to a new ## [X.Y.Z] - YYYY-MM-DD section.

Commit on main and leave the working tree clean.

Check you are logged in to the registry in publishConfig (npm whoami --registry=<url>).

Run ./scripts/release.sh (--dry-run first to see what it would do).

The script checks the tree, tag and registry, runs typecheck, tests and a clean build, then tags vX.Y.Z, pushes, and publishes last: a tag can be deleted, a published version cannot. scripts/check-dist.mjs runs as part of npm run build and fails it if any main, module, types or exports path points at a file tsup did not emit.

MIT © 2026 Axium Lab. Some PII patterns, checksums and context words are ported from Microsoft Presidio, also MIT; its notice is included in LICENSE.

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →