返回资讯中心
外部精选
软件工程
#开发者实时流

Show HN: A Mojo Script Language Container for the Exasol Database

Mojo is a relatively new language, and its Python affinity makes it an ideal candidate for in-database execution to process data where it is stored. Fast and reliable.

Hacker News 最新ROSEDOLA2026/10/19 · 18 天前30 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

A Script Language Container (SLC) for Exasol written entirely in Mojo — no Rust, no C++ host, no .so plugin loading. A single Mojo binary (mojoudfclient) speaks Exasol's ZMQ + protobuf UDF protocol directly and runs the user function in-process:

Exasol DB <== ZMQ REQ/REP + protobuf ==> mojoudfclient (one Mojo binary: protocol loop + your UDF, baked in) └ only external dependency: libzmq

Everything that a language container normally needs — the ZMQ transport, the protobuf wire format, the handshake/run state machine, and the UDF itself — is Mojo (protobuf is hand-rolled; libzmq is the sole C dependency, since Exasol's protocol is localzmq). There is no Rust anywhere: no exaudfclient, no C-ABI vtable, no fingerprint, no dlopen of a compiled .so.

Status — experimental. The binary compiles (Linux ARM64/x86_64) and passes the full ZMQ/protobuf offline self-test (test/fake_exasol.py: OK: doubling verified). A complete live UDF invocation is still being finalized — see Current limitations and the diagnostic build. Design details are in DESIGN.md.

A Script Language Container (SLC) is a self-contained runtime that you install into Exasol so the database can execute user-defined functions (UDFs) in a language it doesn't ship natively, letting you push custom logic to where the data lives instead of pulling rows out to an external service. Running your code inside the database this way eliminates network round-trips and serialization overhead, so heavy per-row or per-group computation happens in-place, in parallel across Exasol's nodes. Mojo is a compelling language for that hot path because it is a compiled, statically-typed language that produces native machine code — no interpreter loop, no per-call bytecode dispatch — making it dramatically faster than a scripting language for CPU-bound work. At the same time Mojo is a member of the Python ecosystem: it interoperates with CPython via Python.import_module, so you can reuse existing Python packages and gradually move only the performance-critical parts to native Mojo. The result is a UDF container that gives you Python's familiarity and libraries where you want them, and compiled, close-to-the-metal speed where you need it — inside the database engine itself.

Try SQL first; drop to a UDF only when SQL can't express the work efficiently. Exasol's SQL engine is heavily optimized and massively parallel, so set-based work — filters, joins, aggregations, window functions — belongs in SQL, where a hand-written UDF would usually be slower and harder to maintain. A UDF earns its place when the logic is a poor fit for relational algebra: complex per-row or per-group procedures, iterative or stateful algorithms, custom parsing, or bespoke numeric/string routines you'd otherwise pull out of the database into an external service. The rule of thumb: if SQL becomes contorted, needs many passes, or simply can't do it, that's when a UDF is worth it.

Once you do need a UDF, Mojo is the fast path. It compiles to native machine code, so a CPU-bound hot path runs without an interpreter loop — often far faster than a scripting-language UDF for the same inner computation. Because Mojo is part of the Python ecosystem and calls CPython via Python.import_module, you don't have to rewrite everything at once: keep the bulk of your logic in Python and move only the performance-critical inner loop to native Mojo, incrementally. When you want a fully compiled, dependency-light function, you can also write the UDF from scratch in pure Mojo. Reach for this container when you've identified a genuine hot path inside the database and want compiled speed there while keeping Python's familiarity and libraries everywhere else.

Write your UDF in Mojo — src/udf.mojo

Compile & self-test — no Exasol needed

Build the SLC package — one docker/podman build

Deploy: Personal (exasol slc custom install) · Enterprise (BucketFS via curl) · Nano (Docker execution environment)

Activate the language container

Create the script and run it

  1. Write your UDF in Mojo

The entire user-facing UDF lives in src/udf.mojo. The shipped example doubles a BIGINT:

out[i] = 2 * in[i], with NULL in → NULL out (SCALAR)

fn run_double_mojo_native(values: List[Int64], nulls: List[Bool]) -> (List[Int64], List[Bool]): var out = ListInt64 var out_nulls = ListBool for i in range(len(values)): if nulls[i]: out.append(0); out_nulls.append(True) else: out.append(values[i] * 2); out_nulls.append(False) return (out^, out_nulls^)

Dispatch on the SQL script name (already UPPER-cased by Exasol).

fn run_udf(name: String, values: List[Int64], nulls: List[Bool]) -> (List[Int64], List[Bool]): if name == "SUM_POSITIVE": return run_sum_positive(values, nulls) # SET reduce return run_double_mojo_native(values, nulls) # DOUBLE_MOJO_NATIVE (default)

fn is_known(name: String) -> Bool: return name == "DOUBLE_MOJO_NATIVE" or name == "SUM_POSITIVE"

The scalar UDF is named DOUBLE_MOJO_NATIVE — the _NATIVE suffix marks it as baked into mojoudfclient (as opposed to the dynamically-loaded .so UDF DOUBLE_EXT, see Dynamic UDFs), and it avoids DOUBLE, a reserved Exasol keyword.

The binary is built inside the container image (step 3), but you can also compile directly with a local Mojo toolchain:

mojo build src/main.mojo -o mojoudfclient

No linker flags: src/zmq.mojo dlopens libzmq.so.5 at runtime, so mojo build never touches the linker.

The fastest correctness check is the self-test target, which drives the real binary in a chroot through the entire ZMQ/protobuf conversation against a bundled fake Exasol — for every UDF/wire combination:

docker build -f Dockerfile --target selftest --progress=plain .

(Use podman build --arch arm64 … on Apple Silicon.) Each case prints a per-message trace (CLIENT → META → RUN → NEXT → EMIT → DONE → FINISHED) and an OK: <script> verified line; the build fails if any case diverges — invaluable for debugging the wire code before touching a database.

One build produces a hermetic rootfs tarball: the mojoudfclient binary, its dynamic loader, its full ldd shared-object closure (libzmq + the Mojo runtime libs), and the empty mount points Nano's read-only nschroot sandbox requires (/tmp, /var/tmp, /buckets, /dev, /proc, /sys, /run/secrets).

x86_64 host / Enterprise target

docker build -f Dockerfile --target artifact
--output type=local,dest=./out .

→ out/mojo-slc.tar.gz

On Apple Silicon for ARM64 Nano, build with Podman and copy the tarball out of the staging image, then unpack it into a rootfs directory to mount:

podman build --arch arm64 -f Dockerfile --target staging
-t mojo-slc:staging .

cid="$(podman create mojo-slc:staging)" podman cp "$cid:/mojo-slc.tar.gz" ./mojo-slc.tar.gz podman rm "$cid"

mkdir -p mojo-rootfs && tar -xzf mojo-slc.tar.gz -C mojo-rootfs file mojo-rootfs/exaudf/mojoudfclient # ARM64 Linux ELF on an ARM64 host

Build the image for the same architecture as the database container. Never mount an ARM64 client into an x86_64 Exasol, or vice versa.

build_info/language_definitions.json in the rootfs declares the MOJO alias and the executable path /exaudf/mojoudfclient.

Exasol Personal Edition installs a custom SLC directly with the exasol CLI — no BucketFS upload or bind mounts. The container is given as --source, which takes either a local tarball or an https URL, together with the --alias used in CREATE <alias> SCALAR SCRIPT and the --language it provides:

exasol slc custom install
--source ./out/mojo-slc.tar.gz
--alias MOJO
--language mojo
--auto-approve

Like the official commands, a custom install (or update) restarts the database to mount the container, so it accepts --auto-approve to skip the confirmation and --no-restart to record the container and activate it on the next start instead. If the container cannot be made available, the database still starts and the command reports that the container is recorded but not active.

exasol slc list now shows custom containers alongside the official ones with a status column, and --json marks them with a custom type and an available field.

Enterprise fetches the SLC from BucketFS over HTTP. Upload the tarball with a single curl PUT — BucketFS auto-extracts *.tar.gz, so slc/mojoslc.tar.gz becomes browsable at slc/mojoslc/:

HOST=my-exasol-host # BucketFS service host

-u w → curl prompts for the write password (keeps it out of ps/history);

you upload the executable UDF binary here, so DON'T disable TLS in prod.

curl -X PUT -T out/mojo-slc.tar.gz -u w
"https://${HOST}:2581/default/slc/mojoslc.tar.gz"

w is the BucketFS write user; 2581 is the HTTPS BucketFS port (2580 for HTTP). Validate TLS — this channel carries the container binary, so a MITM could swap it. For a self-signed BucketFS cert use --cacert <ca.pem>; only add -k (skip verification) if you accept that risk.

default is the bucket in the bfsdefault service. Verify the executable landed: curl -s -u r -o /dev/null -w '%{http_code}\n'
"https://${HOST}:2581/default/slc/mojoslc/exaudf/mojoudfclient"

200 = present

install-native.sh wraps build + this upload + registration into one command if you'd rather not do it by hand.

Nano runs the whole Exasol database as a single local container, so reach for it when you require Docker (or Podman) as the execution environment — local development, CI, or a self-contained demo. It reads language metadata under /exa/slc and launches UDFs in /exa/sandbox — mount the same unpacked rootfs at both paths (this is the "copy it to the directory" step; a read-only bind mount is the copy). Persist Nano's database files in nano-exa. On first init, pass builtinScriptLanguageName=slc/mojo (persisted in nano-exa/exasol.conf; omit init params=… on later starts). --security-opt unmask=ALL is required for SLC/UDF execution.

mkdir -p nano-exa

podman run --rm -it --name exanano-mojo
--security-opt unmask=ALL --shm-size=512mb --pids-limit=-1
-p 127.0.0.1:8563:8563
-v "$PWD/nano-exa:/exa"
-v "$PWD/mojo-rootfs:/exa/slc/mojo:ro"
-v "$PWD/mojo-rootfs:/exa/sandbox:ro"
docker.io/exasol/nano:latest
init params='builtinScriptLanguageName=slc/mojo'

Wait for Database is now up and running!. Default SYS credentials are sys / exasol. If port 8563 is taken, map another (e.g. 18563:8563).

Registration differs between deployments — this is the step that most often trips people up.

Personal — nothing to do here: exasol slc custom install --alias MOJO already registered the alias and restarted the database, so the language is active immediately. Confirm with exasol slc list (the container shows as available).

Nano — the SLC is mounted and named via builtinScriptLanguageName, so the alias points at that built-in name:

ALTER SYSTEM SET SCRIPT_LANGUAGES = 'MOJO=builtin_mojo';

Enterprise — the alias is a localzmq+protobuf URL into BucketFS, ending at the executable (mojoudfclient, no leading slash). SCRIPT_LANGUAGES holds all languages, so preserve the existing ones (read them first, then re-set the whole value):

SELECT system_value FROM EXA_PARAMETERS WHERE parameter_name = 'SCRIPT_LANGUAGES'; -- then, keeping the builtins that returns: ALTER SYSTEM SET SCRIPT_LANGUAGES= 'PYTHON3=builtin_python3 JAVA=builtin_java R=builtin_r MOJO=localzmq+protobuf:///bfsdefault/default/slc/mojoslc?lang=mojo#buckets/bfsdefault/default/slc/mojoslc/exaudf/mojoudfclient';

builtin_mojo is only valid for the Nano mounted-SLC mechanism. On Enterprise it must be the full localzmq+protobuf://…#…/mojoudfclient URL — using builtin_mojo there yields "No usable script language container".

New sessions pick up ALTER SYSTEM (reconnect first). Use ALTER SESSION with the same value to test in the current session only.

The native client dispatches on the SQL script name (there is no %udf_object); the script body is ignored. The container ships baked-in UDFs — DOUBLE_MOJO_NATIVE (SCALAR), SUM_POSITIVE (SET), PY_SCALE (Python interop), MIRROR_MOJO (EMITS). The scalar one carries the _NATIVE suffix

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →