Zirql: Extracting a Shared Query Language
When implementing Zincite1, I wanted a query language similar to the one I had built for Zirium.
以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。
When implementing Zincite1, I wanted a query language similar to the one I had built for Zirium. As with Zirium, I took inspiration from tools such as awk and jq, whose operations suit the data they work on: text records and fields for awk, JSON values for jq. I was particularly interested in writing checkers and transformations for MiniZinc data files: checking assumptions about an instance, changing its parameters, or making a smaller instance to investigate.
I soon decided it would be more fun to unify the two query languages than build another implementation. Selecting MiniZinc declarations and following MLIR operands need different operations, but filtering collections, naming intermediate results and combining conditions need the same language machinery. I also wanted improvements to those parts to be available in both tools.
The result is Zirql, a Rust library that both tools use to parse and evaluate queries. Each application supplies its data and the operations that make sense for it. I’ll call that application the host. The examples here use Zirql 0.2.0.
The documentation’s word example starts with a list of strings. I like it because the data needs little explanation, so we can concentrate on the language.
ash apple birch apple
The host splits the file at whitespace and exposes the resulting array as words:
["ash", "apple", "birch", "apple"]
Both occurrences of apple stay in the array. The host leaves case and punctuation as written.
To find the words beginning with a, filter the array:
words | filter(w => starts_with(w, "a"))
starts_with(w, "a"))">
The result is:
["ash", "apple", "apple"]
The predicate is a lambda: w names the current word, and filter includes it when the lambda returns true. Filtering keeps input order and duplicates. For a short predicate, _ can stand for the current argument:
words | filter(starts_with(_, "a"))
Zirql supplies starts_with, so the same operation works on filenames or names read from source code. It compares the strings exactly, so starts_with("Apple", "a") is false.
Instead of using a pipeline, a list can be constructed using a comprehension. Here is the same example using the MiniZinc comprehension style:2
[w | w in words where starts_with(w, "a")]
I prefer the filter when selecting entries without changing them. A comprehension becomes useful when constructing a different result for each entry, or combining several generators.
To put the selected words in alphabetical order, add a sorting stage:
words | filter(starts_with(_, "a")) | sort
Sorting gives ["apple", "apple", "ash"]. Add unique to remove duplicate spellings:
words | filter(starts_with(_, "a")) | sort | unique
["apple", "ash"]
Without the preceding sort, unique would keep each first occurrence in input order. For statistics, tally produces a dictionary of occurrence counts:
words | filter(starts_with(_, "a")) | tally
{"ash": 1, "apple": 2}
The keys follow first occurrence in the input, so ash comes before apple. These stages come from Zirql; the word host supplies the input.
The order of the input lets us ask which words occur next to each other. Suppose we want to count neighbours regardless of which word comes first. Build a set for each adjacent pair, then tally those sets:
[
{words[i], words[i + 1]}
| i in index_set(words)
where i < length(words)
] | tally
The result is a dictionary from sets to counts. Here it is in Zirql notation:
{
{"ash", "apple"}: 1,
{"apple", "birch"}: 2
}
Array positions start at one. index_set(words) gives the valid positions, and the condition skips the last one, leaving room for words[i + 1]. Each position produces a set containing the two neighbouring words. The second and third pairs contain apple and birch in opposite orders, so tally counts them together.
Dictionary entries follow the first occurrence of each pair. Set equality compares membership, so {"apple", "birch"} and {"birch", "apple"} identify the same key. If we name the result neighbours, then neighbours[{"birch", "apple"}] gives 2.
Tuples, arrays and dictionaries can also be keys. JSON objects require string keys, so a host writing JSON must choose how to encode other kinds of key.
Sets also discard duplicate members. Two adjacent occurrences of apple would give the singleton set {"apple"}. Empty input and a single word both give an empty dictionary, {}.
The indexing makes the pairs adjacent. Two independent generators over words would visit every combination, including pairs that never appeared together.
The same filters can work on records with named fields. The CSV example loads each row as a record, using the header names as fields, and writes query results as JSON. We can use it to make a packing list from a couple of files.
Suppose we are bringing games to a friend’s house. games.csv records who owns each game, whether we are bringing it and whether it is packed. Kingdomino is already in the bag; Patchwork and NMBR 9 still need packing. Everdell and Cottage Garden are staying home (bring is "no"), although both are unpacked. The columns are title, owner, bring and packed.
With one input file, rows names its array of records. To find the games still to pack, select rows where bring is "yes" and packed is "no". Step through the filters to see which cells they check and which rows they return:
Choose a step to follow the underlined query text.
rows | filter(.bring == "yes") | filter(.packed == "no")
The header supplies the field names. All five games are available, including Everdell and Cottage Garden, which are staying home.
Choose “Return rows” to see the first result.
rows | filter(.bring == "yes") | filter(.packed == "no")
The first filter selects the three games we are bringing. Everdell and Cottage Garden have bring equal to "no", so they drop out here.
Choose “Return rows” to see the first result.
rows | filter(.bring == "yes") | filter(.packed == "no")
The second filter checks the three chosen games. Patchwork and NMBR 9 are unpacked; Kingdomino is already in the bag.
Choose “Return rows” to see the first result.
rows | filter(.bring == "yes") | filter(.packed == "no")
Return the two games still to pack. Kingdomino is already packed; Everdell and Cottage Garden are not coming. All three stay visible as context.
[ {"title": "Patchwork", "owner": "leo", "bring": "yes", "packed": "no"}, {"title": "NMBR 9", "owner": "mira", "bring": "yes", "packed": "no"}]
✓ selected row
The header supplies the field names. All five games are available, including Everdell and Cottage Garden, which are staying home. The first filter selects the games to bring; the second selects those still unpacked. _.bring and _.packed read fields on the current row. Checking only packed would also select Everdell and Cottage Garden. The result contains the whole Patchwork and NMBR 9 records, in file order.
The CSV host keeps cells as strings. It neither turns "01" into a number nor interprets "yes" as a Boolean.
To run this example, use Rust 1.88 or newer, clone the release and save the linked games.csv file in the checkout:
git clone --branch v0.2.0 --depth 1 https://github.com/zayenz/zirql.git
cd zirql
cargo run --locked --example csv -- '
rows | filter(_.bring == "yes")
| filter(_.packed == "no")
' games.csv
Look up the owners in another file#
The game file identifies owners by ID. A second file, people.csv, maps these to names: mira is Mira and leo is Leo. Save it beside games.csv. Pass the files as games=games.csv and people=people.csv, and those names become bindings in the query.
Choose a step to follow the underlined query text.
let pending = games | filter(.bring == "yes") | filter(.packed == "no"); [ { "game": g.title, "owner": p.name } | g in pending, p in people where p.id == g.owner ]
Save the games we are bringing but have not packed as pending. Everdell and Cottage Garden are excluded before matching owners.
Choose “Return pairs” to see the first result.
let pending = games | filter(.bring == "yes") | filter(.packed == "no"); [ { "game": g.title, "owner": p.name } | g in pending, p in people where p.id == g.owner ]
Follow the first pending game: Patchwork’s owner is leo. The matching id in people.csv supplies the name Leo.
Choose “Return pairs” to see the first result.
let pending = games | filter(.bring == "yes") | filter(.packed == "no"); [ { "game": g.title, "owner": p.name } | g in pending, p in people where p.id == g.owner ]
Next, NMBR 9’s owner mira matches Mira. These steps illustrate the pairs visited by the comprehension.
Choose “Return pairs” to see the first result.
let pending = games | filter(.bring == "yes") | filter(.packed == "no"); [ { "game": g.title, "owner": p.name } | g in pending, p in people where p.id == g.owner ]
Each matching pair produces a new record with a game title and an owner’s name. The games determine the result order.
[ {"game": "Patchwork", "owner": "Leo"}, {"game": "NMBR 9", "owner": "Mira"}]
✓ selected row selected cell
Save the games we are bringing but have not packed as pending. Everdell and Cottage Garden are excluded before matching owners. let names the filtered array pending. For each game, p in people visits the owner rows, and where p.id == g.owner selects the matching row. The comprehension builds a record with the game’s title and the owner’s name.
Run the complete query with both named files:
cargo run --locked --example csv -- '
let pending = games
| filter(_.bring == "yes")
| filter(_.packed == "no");
[
{
"game": g.title,
"owner": p.name
}
| g in pending,
p in people
where p.id == g.owner
]
' games=games.csv people=people.csv
The result names Leo for Patchwork, then Mira for NMBR 9. Each owner ID occurs once in this file. A missing ID would omit the game; repeated IDs would produce multiple matches. With several files, the query must use the table names rather than rows.
To turn these records into reminders, replace the record expression with p.name ++ ": bring " ++ g.title. ++ concatenates strings, giving ["Leo: bring Patchwork", "Mira: bring NMBR 9"].
The same file can supply a checklist for all three games we are bringing. Kingdomino needs to appear too, with its box already checked. Step through the construction to see how the source fields become checklist records:
Choose a step to follow the underlined query text.
[ { "label": g.title, "checked": g.packed == "yes" } | g in rows where g.bring == "yes" ]
The checklist includes all three games we are bringing, including the packed Kingdomino. The two games staying home are excluded.
Choose “Compare text” to see the first result.
[ { "label": g.title, "checked": g.packed == "yes" } | g in rows where g.bring == "yes" ]
For the three chosen games, compare packed with "yes". Kingdomino gives true; Patchwork and NMBR 9 give false.
[true, false, false]
✓ selected row selected cell
[ { "label": g.title, "checked": g.packed == "yes" } | g in rows where g.bring == "yes" ]
Use each title as a label and the comparison as a Boolean checkbox. The table underlines computed values; JSON keeps them as true and false.
[ {"label": "Kingdomino", "checked": true}, {"label": "Patchwork", "checked": false}, {"label": "NMBR 9", "checked": false}]
computed value
The checklist includes all three games we are bringing, including the packed Kingdomino. The two games staying home are excluded. The where condition selects the games to bring. Each record uses the title as its label and the comparison g.packed == "yes" as its Boolean checked value. This gives true for Kingdomino and false for Patchwork and NMBR 9. The host writes the new records as JSON for the checklist’s presentation code.
Run it with games.csv as the only input again:
cargo run --locked --example csv -- '
[
{
"label": g.title,
"checked": g.packed == "yes"
}
| g in rows
where g.bring == "yes"
]
' games.csv
The
正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。