返回资讯中心
外部精选
软件工程
#开发者实时流

Parsing EDIFACT in Python with syntax trees

An EDIFACT parser with a real syntax tree.

Hacker News 最新alexprengere2026/10/22 · 23 天前23 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

An EDIFACT parser with a real syntax tree.

Edison is a Python library for parsing EDIFACT messages. Instead of treating a message as a flat list of segments, it models messages as a syntax tree: segments can define groups of child segments, recursively.

You define the syntax tree once, then parse, validate, or pretty-print messages against that structure.

It works by converting the syntax tree to a state machine of possible transitions between segments.

Define EDIFACT syntax trees directly in Python.

Segments can define groups of child segments, which can be conditional, mandatory, or repeatable.

Attach parsing callbacks to the segments you care about.

Select syntax tree by message version.

Handle escaped EDIFACT separators and EBCDIC control characters.

Properly tokenize segments with specified lengths like BLB

Export transition graphs with pydot.

Installation

Edison requires Python 3.12 or newer.

pip install edison-parser

Quick start

Subclass BaseParser, define one or more schemas, and attach callback method names to the schema nodes you want to process.

from edison import BaseParser

class PIZZAQParser(BaseParser): schemas = { (0, 0): [ ["UNH"], ["CVR"], ["REF"], # one type of REF segment [ "PIZ", [ ["TYP", [], "parse_typ"], ["REF", [], "parse_ref"], # another type of REF segment ], ], ["UNT"], ], }

def init(self): self.summary = {} self.pizza = None

def parse_typ(self, segment: str):

Executed when the parser sees a "TYP" segment. The segment is passed as a string.

self.pizza = segment[4:]

def parse_ref(self, segment: str):

Executed when the parser sees a "REF" segment. The segment is passed as a string.

Only executed on the "second" type of "REF" segment!

refs = segment[4:].split("*") self.summary[self.pizza] = len(refs)

edi = """ UNH+1+PIZZAQ:1:1:1A+00012480825035' CVR+USD' REF+LUIS' PIZ+1' TYP+MARGARITA' REF+1*2' PIZ+2' TYP+PESTO' REF+3' UNT+19+1' """

Create the parser and parse this EDIFACT message.

The summary attribute is populated by the callbacks.

parser = PIZZAQParser() parser.parse(edi) parser.summary # {"MARGARITA": 2, "PESTO": 1}

Validate and pretty-print the message.

PIZZAQParser.validate(edi) # True/False PIZZAQParser.show(edi) # to print

Here is the output of PIZZAQParser.show(edi).

A schema is a dictionary keyed by the minimum supported version:

schemas = { (0, 0): [ ["UNH"], ["BGM"], ["UNT"], ], }

Each segment node accepts these positional fields:

["TAG", children, action, repetition, flag]

Field Default Meaning

TAG required Three-character EDIFACT segment tag.

children [] Nested segment group.

action None Parser method name called with the raw segment.

repetition 99999 1 means single-use; any higher value means repeatable.

flag "C" "C" for conditional, "M" for mandatory.

Main API

API Use

parser.parse(edi, version=None) Walk an EDI message and call matching segment actions.

Parser.validate(edi, version=None) Return True when all segments follow the schema transitions.

Parser.show(edi, version=None, colorize=True) Return a formatted, indented representation of an EDI message.

Parser.build_dot_object(version=None, include_up_links=True) Build a pydot.Dot transition graph.

extract_envelope_headers(edi) Quickly return the UNB and UNH envelope segments.

extract_envelope_footers(edi) Quickly return the UNT and UNZ envelope segments.

escape(data) / unescape(data) Escape or unescape EDIFACT separators: +, :, *, '.

normalize_ebcdic_separators(edi) Convert EBCDIC EDIFACT separators to ASCII safely.

Advanced usage

Versioned syntax trees Define multiple syntax trees when the same message evolves across versions. The parser selects the highest schema version less than or equal to the version argument, while parsing callbacks can stay the same.

from edison import BaseParser

class RouteParser(BaseParser): schemas = { (0, 0): [ ["UNH"], ["ORG", [], "parse_origin"], ["DST", [], "parse_destination"], ["UNT"], ], (2, 0): [ ["UNH"], [ "LEG", [ ["ORG", [], "parse_origin"], ["DST", [], "parse_destination"], ], ], ["UNT"], ], }

def init(self): self.route = {}

def parse_origin(self, segment: str): self.route["origin"] = segment[4:]

def parse_destination(self, segment: str): self.route["destination"] = segment[4:]

legacy = RouteParser() legacy.parse("UNH'ORG+PAR'DST+NCE'UNT'", version=(1, 0))

modern = RouteParser() modern.parse("UNH'LEG+1'ORG+PAR'DST+NCE'UNT'", version=(2, 0))

assert legacy.route == modern.route == { "origin": "PAR", "destination": "NCE", } assert RouteParser.validate("UNH'ORG+PAR'DST+NCE'UNT'", version=(1, 0)) assert RouteParser.validate("UNH'LEG+1'ORG+PAR'DST+NCE'UNT'", version=(2, 0))

When version is omitted, incomplete, or newer than all known schemas, Edison uses the latest schema. If a requested version is older than the first schema, find_transitions() raises ValueError.

repetition controls whether the transition graph can come back to the same node:

Mandatory nodes constrain valid paths through the tree. Once a mandatory node is next in a sequence, the parser cannot skip over it to reach a later sibling, a parent, or a child that would bypass it. A mandatory child does not make its parent mandatory by itself: if the whole group is required, mark the group node as "M" too.

from edison import BaseParser

class OrderParser(BaseParser): schemas = { (0, 0): [ ["UNH"], [ "ITM", [ ["DSC"], ["QTY", [], "parse_quantity", 1, "M"], ["PRI"], ], None, 99999, "M", ], ["UNT"], ], }

def init(self): self.quantities = []

def parse_quantity(self, segment: str): self.quantities.append(segment[4:])

assert OrderParser.validate("UNH'ITM'DSC'QTY+2'PRI'ITM'QTY+1'UNT'") assert not OrderParser.validate("UNH'UNT'") # ITM is mandatory. assert not OrderParser.validate("UNH'ITM'PRI'UNT'") # QTY is mandatory. assert not OrderParser.validate("UNH'ITM'QTY+2'QTY+3'UNT'") # QTY repeats once.

Shared headers, footers, and parser hooks Use header and footer when several schemas share envelope segments. They are automatically prepended and appended to every schema at class compilation time.

from edison import BaseParser

class EnvelopeParser(BaseParser): header = [["UNH", [], "on_segment"]] footer = [["UNT", [], "on_segment"]] schemas = {(0, 0): [["BGM", [], "on_segment"]]}

def init(self): self.segments = []

def set_up(self): self.segments.clear()

def on_segment(self, segment: str): self.segments.append(segment)

def tear_down(self): self.count = len(self.segments)

set_up() runs before parsing starts, and tear_down() runs after parsing finishes.

Internally, Edison builds a directed graph of transitions between segments, when the parser class is created.

Each node is a specific segment, and each edge is a valid transition to the next segment.

With this, parsing is just a walk through the graph, and validation is checking if the path exists.

You can visualize the transition graph with build_dot_object(), which returns a pydot.Dot object.

pydot is installed with Edison. To write PNG files, make sure Graphviz is available on your system.

from edison import BaseParser

class Parser(BaseParser): schemas = { (0, 0): [ ["UNH"], [ "BGM", [ ["DTM"], ], ], ["UNT"], ], }

dot = Parser.build_dot_object() dot.write_png("transitions.png")

Here is the resulting graph for the above schema.

You can pass the same version selector used by parsing, and hide upward links for a simpler tree-like graph:

dot = Parser.build_dot_object(version=(13, 2), include_up_links=False) dot.write_png("transitions-v13.png")

Because build_dot_object() returns the underlying pydot.Dot object, you can customize it before writing.

Edison understands escaped EDIFACT separators and can normalize EBCDIC separator characters often found in binary payloads.

from edison import normalize_ebcdic_separators, escape, unescape

assert escape("REC+") == r"REC+" assert unescape(r"REC+") == "REC+" assert normalize_ebcdic_separators("UNB\x1dSENDER\x1fREC") == "UNB+SENDER:REC"

The segment tokenizer also handles escaped quotes, so content such as IFT+1+AIRLINE'SERVER' stays in a single EDIFACT segment.

For the common case of reading EDIFACT envelope boundaries, use extract_envelope_headers() to lazily return the first UNB/UNH segments or extract_envelope_footers() to lazily return the last UNT/UNZ segments:

from edison import extract_envelope_footers, extract_envelope_headers

edi = "UNB+SENDER+RECEIVER'UNH+1+PIZZAQ:1:1:1A'BGM+1'UNT+3+1'UNZ+1+1'" headers = extract_envelope_headers(edi) footers = extract_envelope_footers(edi)

assert headers.unb == "UNB+SENDER+RECEIVER" assert headers.unh == "UNH+1+PIZZAQ:1:1:1A" assert footers.unt == "UNT+3+1" assert footers.unz == "UNZ+1+1"

Development

Local commands This repository uses uv, tox, pytest, ruff, black, ty, and pre-commit hooks.

uv tool install tox --with tox-uv tox -e py

Build distributions with:

uv build --sdist --wheel -o dist/

Run the full pre-commit suite with:

uv tool install prek==0.4.10 prek run --all-files

CI runs tests through tox on CPython 3.12 and 3.14 across Linux, Windows, and macOS, then builds source and wheel distributions.

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →