
Sherlock – Financial Crime Analytics
A transaction-analysis assistant for investigating online financial crime: investment scams, romance scams, authorised push payment (APP) fraud, account takeover and money-mule networks.
以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。
A transaction-analysis assistant for investigating online financial crime: investment scams, romance scams, authorised push payment (APP) fraud, account takeover and money-mule networks.
Load bank or payment-provider exports and Sherlock will:
It is a single Java file with no dependencies.
java Sherlock.java demo demo.csv java Sherlock.java demo.csv sherlock> analyze sherlock> trace ACC-VICTIM-01 sherlock> report case.html ACC-VICTIM-01
Sherlock produces investigative leads, not proof. Every finding must be verified against original bank or provider records and handled under your force's evidence procedures.
What's new in 2.0
Version 2.0 is a substantial rewrite. Every change below is covered by the built-in self-test (java Sherlock.java selftest, 59 checks).
Strict amount parsing. Version 1 silently read 1.234,56 as 1.23 and (500.00) as +500. Ambiguous amounts are now rejected with a reason, and --decimal-comma reads European formats correctly. Currency symbols and codes (£1,234.56, GBP 100.00) are understood.
Time zones. --tz Europe/London, or tz= in a mapping file, sets the zone that zone-less timestamps are read in, and every output says which zone its times are shown in. Sherlock warns:
when zone-less times were read as UTC;
when a file mixes times with and without a zone;
when a local time falls in a clock change, either repeated in October or non-existent in March.
Selectable tracing rules. tainted-first (the v1 behaviour, and still the default), clean-first, fifo and prorata. The rule is printed on every trace and report, and the new compare command shows all four side by side.
Loops must happen in time order. A→B→C→A is only flagged when each hop follows the previous one within cyclewindow. Two-account round trips (A→B→A) are now detected as ROUND_TRIP. Small payments below cyclemin are ignored. If the search stops early, the output says so instead of failing silently.
Scores no longer inflate. Each finding type counts once, at its highest weight, so an account sharing three devices scores 25, not 75.
CSV parsing. Quoted fields may contain line breaks. Files that are not valid UTF-8 are read as windows-1252, which is common in UK bank exports and is what breaks £ signs. An unclosed quote is reported.
Exact money. Every amount is held in pence (long) and converted from BigDecimal with banker's rounding. Each trace ends with a reconciliation line that checks every penny is accounted for.
Evidential output
Every finding cites transaction IDs, and every trace line cites file:line.
Audit log. Every command, setting change, file load and file written is timestamped, with the SHA-256 of each input and output. audit shows the log, and it is appended to every report.
Report stamps. Each report records the tool version, Java version, analyst name, time zone, base currency, tracing rule and the full settings table.
Every skipped row is listed with its reason (the skipped command, and in reports), not just the first ten.
Every output file's SHA-256 is recorded in the audit log, and printed when a report is written.
New detectors
Finding What it means
LIKELY_VICTIM / POSSIBLE_VICTIM Repeated payments to one payee, with scam-typical memo words ("fee", "tax", "unlock", "release", "invest", …) or a payee that is itself flagged. Listed separately for safeguarding, never as suspects.
ACCOUNT_TAKEOVER / ATO_PAYEE A new device or IP on an account with an established history, quickly followed by a payment several times the account's usual size. The account holder goes on the safeguarding list; the payee gets a suspect finding.
DORMANT_REACTIVATED An account silent for dormantdays that suddenly moves money.
ROUND_AMOUNTS An intermediary account whose payments are mostly exact round sums (weak signal).
REPEAT_AMOUNTS The same exact amount again and again in a short window (weak signal).
WATCHLIST An account, IP or device from a loaded watchlist (for example, from previous cases).
STRUCTURING (extended) Now checks incoming as well as outgoing payments, and accepts several thresholds (set threshold 10000,25000).
New features
Multi-victim tracing with attribution. trace ACC-V1 ACC-V2 ACC-V3 keeps each victim's money separately labelled and ends with a table of how much of each victim's money sits in each account. This is useful for restraint orders and compensation.
Reverse trace. reverse EXCHANGE-A shows where the money an account received originally came from.
Several input files with de-duplication. Load statements from several banks. When the sending and the receiving bank both record the same transfer, the two records are matched (same payer, payee, amount and currency, within dedupewindow hours) and merged rather than counted twice. Every merge is logged (dupes). Loading the same file twice is refused.
Column-mapping profiles for real bank exports, including statement-style files (one account's statement with Paid out / Paid in columns). java Sherlock.java maptemplate writes a commented template.
Currency conversion from a rates file, using the rate on or before each payment's date.
Graph exports.
Graphviz DOT, of the whole case or only the traced money.
Entity and link CSVs for i2 Analyst's Notebook or Gephi.
A normalised, de-duplicated transaction CSV.
HTML report with a Sankey diagram of the money trail, an activity chart, a cumulative-loss chart for victims, and light and dark themes.
Shell conveniences.
Account names can be typed in any case or partially (account m10), with "did you mean" suggestions.
findings <TYPE> lists every finding of one type.
Detectors can be switched on and off (detectors, disable round-amounts).
config and saveconfig load and save settings files.
Arguments can be quoted.
Restructured code. Detectors sit behind a Detector interface. All state lives in a Case object instead of static globals. A known-answer self-test is built in.
Upgrading from 1.x
Most v1 commands and CSV files work unchanged, but some results differ:
Java 11 or newer. No build step: java Sherlock.java … compiles and runs the single source file.
java Sherlock.java demo demo.csv # write a demo case (investment scam + mule layering) java Sherlock.java demo.csv # interactive shell java Sherlock.java demo.csv analyze # one-shot: any command can follow the file names java Sherlock.java selftest # run the known-answer tests
The demo contains:
java Sherlock.java [options] <file.csv> [file2.csv ...] [command args...]
Option Effect
--tz <zone> Time zone for zone-less timestamps and for display, e.g. Europe/London.
--map <mapping.properties> Column-mapping profile for the files that follow it on the command line.
--decimal-comma Read amounts like 1.234,56.
--base <CUR> Base currency (default GBP).
--rates <rates.csv> Exchange rates for conversion to the base currency.
--watchlist <file.csv> Known accounts, IPs and devices.
--config <settings.properties> Load settings; other options on the line override it.
--analyst <name> Name stamped on reports.
--no-dedupe Do not merge cross-file duplicates.
Example with two banks' records:
java Sherlock.java --tz Europe/London --analyst "DC Smith"
--map bank-a.properties victim-statement.csv
--map bank-b.properties mule-statement.csv
provider-export.csv
Input formats
Standard CSV
A header row is required. Column order doesn't matter, and extra columns are ignored.
java Sherlock.java maptemplate bank.properties # write a commented template
A profile maps Sherlock's fields to the export's own column headers and describes its date format, zone, delimiter, decimal style and any title rows above the header. A statement-style export holds one account's statement, so the profile names the owner:
name=Example bank statement owner=12-34-56 12345678 counterparty=Counterparty account debit=Paid out credit=Paid in date=Date time=Time date.format=dd/MM/yyyy time.format=HH:mm tz=Europe/London memo=Description+Reference skiplines=2
Instead of debit/credit, you can use a signed amount column (negative means money out), or amount plus a direction column with direction.out=DEBIT,DR and direction.in=CREDIT,CR.
In statement mode, ip/device values are kept only on money out, because on incoming rows they describe the owner's session, not the sender's.
Unknown keys are rejected so typos don't pass silently.
Rates file
date,currency,rate 2026-03-01,EUR,0.85 2026-03-01,USD,0.79
rate is the value of one unit of the currency in the base currency. Each payment uses the latest rate on or before its date; Sherlock warns if it had to use a later one. Payments that cannot be converted are kept, and Sherlock warns that totals mix currencies.
type,value,note account,ACC-M01,Op Example mule device,DEV-77,seen in a previous case ip,203.0.113.50,
Commands
Data
Settings: settings, set <name> <value>, config <file>, saveconfig <file>, help, quit.
Finding Weight Signal
WATCHLIST 40 Account, device or IP on the watchlist
PASS_THROUGH 30 ≥ passratio of incoming money forwarded within dwell hours on average (FIFO-matched)
SHARED_DEVICE 25 ≥ devicemin accounts send from one device
STRUCTURING 25 ≥ structmin payments in or out just under a threshold, within structwindow hours
FAN_IN / FAN_OUT 20 ≥ fanmin distinct counterparties within fanwindow days
ATO_PAYEE 20 Received the large payment that followed a device change on the payer's account
CIRCULAR_FLOW 15 Part of a loop of three or more accounts, hops in time order
DORMANT_REACTIVATED 15 Silent for dormantdays, then a burst worth ≥ passmin within burstdays
ROUND_TRIP 10 Money sent to an account and sent back
SHARED_IP 10 ≥ ipmin accounts send from one IP (weaker: Wi-Fi, VPN, mobile NAT)
ROUND_AMOUNTS 5 An intermediary account moving mostly exact multiples of roundunit
REPEAT_AMOUNTS 5 ≥ repeatmin identical payments within repeatwindow days
Score = the highest weight of each finding type, added up. HIGH is 60 or more, MEDIUM 30 or more, LOW below that. Scores are a triage aid. Businesses, charities and shared households can match these patterns.
Safeguarding findings are never added to the suspicion score:
Sherlock replays every transaction in time order. Each account is a pool of labelled money, and each payment takes money out of the sender's pool according to the chosen rule. Untraced money in the victim's payments becomes that victim's label.
The choice matters. On the demo data, the collection account ACC-M01 receives £38,000 from VICTIM-01 and £10,000 from five other victims, then pays out £47,700:
sherlock> compare ACC-VICTIM-01 Account tainted-first clean-first fifo prorata EXCHANGE-A 19,000.00 9,600.00 17,000.00 15,041.66 CRYPTO-WLT-9 0.00 9,300.00 7,400.00 7,362.50 EXCHANGE-C 9,200.00 9,300.00 7,500.00 7,362.49 EXCHANGE-B 9,200.00 8,700.00 5,200.00 7,283.33
The legal rules for tracing mixed funds differ, so confirm the approach with the prosecutor before relying on any one column. Pro-rata splits use exact integer arithmetic with a largest-remainder rule, so pennies always reconcile.
For reverse, every pound carries an origin label: the account that first paid it out without having received it in the data. tainted-first and clean-first therefore behave pro rata there, and fifo remains distinct.
HTML report: the same content as summary tiles and tables, plus:
It is a single self-contained file that follows the viewer's light or dark preference.
Graphviz: dot -Tsvg traced.dot -o traced.svg. Nodes are coloured by level, victims are green, exits are double-bordered, and edge width scales with value.
i2 / Gephi: entities have Id, Label, Type, Score, Level, Received, Sent, Findings. Links have Source, Target, Weight, Count, First, Last, TxIds.
Type settings to see every value. They can be changed with set, and saved or loaded as a properties file.
The default £10,000 structuring threshold is only a starting point. The UK has no fixed reporting threshold for transfers, so the limits worth testing are usually the banks' own per-pay
正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。