Investigate IPs, domains, hashes, and CVEs across 6 free threat intel APIs — without switching between browser tabs.

Quick Start · Usage · Architecture · API Keys · Screenshots · Contributing

🚀 Proudly featured in the official Awesome OSINT repository.

ThreatLens is a single command-line tool that unifies threat intelligence lookups across the most trusted free OSINT sources. Instead of pasting an IP into five different websites, ThreatLens queries them all in parallel, normalizes the results, and gives you a clear verdict — in the terminal, or in a polished, color-coded Excel/JSON/CSV report.

Built for SOC analysts, incident responders, threat hunters, and anyone who wants fast, reliable IOC enrichment without leaving the shell.

pip install --require-hashes -r requirements.lock

🧰 Usage

Investigate a single IP

python main.py -i 45.33.32.156

Basic single-IOC lookup

Investigate multiple IOC types at once

python main.py -i 45.33.32.156 -d malware.example.com
-s d41d8cd98f00b204e9800998ecf8427e -c CVE-2021-44228

Mix and match IOC types in one run

Parse a log file — all IOCs auto-extracted

python main.py --file /var/log/apache2/access.log

Bulk investigate straight from raw logs

Output JSON instead of Excel

python main.py -i 8.8.8.8 --format json

Machine-readable output for pipelines

Use only specific APIs

python main.py -i 8.8.8.8 --apis abuseipdb virustotal

Restrict enrichment to selected sources

Generate every report format at once

python main.py --file access.log --format all

Excel + JSON + CSV in a single run

Lookup a CVE — no API key needed

python main.py -c CVE-2021-44228 --apis nvd --format json

CVE enrichment via NIST NVD (free, no key)

Verbose / debug mode

python main.py -i 8.8.8.8 -v

Full request/response logging for troubleshooting

Check a CVE against CISA KEV + EPSS, with an asset-aware decision

python main.py -c CVE-2021-44228 --apis nvd cisa_kev epss
--import-assets assets.csv --decision-cards

Patch / Isolate / Monitor / Not-affected recommendation

Parse a Suricata eve.json and export to Splunk

python main.py --file eve.json --log-format suricata
--export splunk

SOC log ingestion → SIEM export

Build a hash-manifested evidence pack for the investigation

python main.py -i 45.33.32.156 --evidence-pack

ZIP with a SHA-256 manifest for chain-of-custody

See all CLI flags

Flag Description

-i, --ip IP address(es) to investigate

-d, --domain Domain(s) to investigate

-s, --hash File hash(es) — MD5 / SHA1 / SHA256

-c, --cve CVE ID(s), e.g. CVE-2021-44228

--file Path to a log/text file to auto-extract IOCs from

--log-format Format of --file: auto (default) | text | zeek | suricata | sysmon | jsonl

--apis Restrict enrichment to a specific set of APIs (now includes cisa_kev, epss)

--format Output format: excel (default) | json | csv | all

--output Directory to save reports (default: ./output)

--no-report Print results to terminal only; investigation history may still be stored

--import-assets Import an asset inventory CSV (hostname/ip, criticality, internet_facing, owner, product)

--decision-cards Produce a Patch/Isolate/Monitor/Not-affected card for every CVE result

--export Send results to one or more SIEM destinations: splunk | elastic | sentinel (opt-in, must be configured in config/keys.env)

--export-insecure-tls Disable TLS verification for SIEM export (testing only; logs a loud warning)

--evidence-pack Package the investigation into a SHA-256-manifested ZIP

--cache-path SQLite path for local cache (default: .threatlens/investigations.db)

--cache-ttl Cache lifetime in seconds (default: 3600)

--no-cache Bypass IOC-result and provider-feed caches; history and asset inventory are unaffected

--no-history Do not save this run to local investigation history

--privacy-mode External lookup policy: standard | passive | approved-only | local-only

--approved-apis Provider allow-list required for approved-only mode (combined with --apis, if supplied)

--privacy-preview Show original and sanitized IOCs, providers, and destinations before enrichment

--watchlist-add Add provided IOCs to the local watchlist

--watchlist-list List active local watchlist entries

--watchlist-remove Remove one or more TYPE:VALUE entries from the watchlist

--watchlist-scan Scan active watchlist entries and show changes since the previous stored observation

--watchlist-ttl-days Watchlist expiry on add (1-3650 days; default: 90)

--max-requests Cap on external API calls per run (default: 250)

--max-iocs Maximum unique IOCs per run (default: 1000)

--allow-private-iocs Allow private/loopback IPs (disabled by default)

--delay Delay between API calls, for rate-limit tuning

-v, --verbose Enable debug logging

🧭 Vulnerability & SOC Triage (v2.2)

ThreatLens can go beyond IOC lookups into lightweight CVE triage and log ingestion:

--privacy-mode passive uses the configured read-only provider lookups. --privacy-mode approved-only --approved-apis ... restricts lookups to the named providers; only providers both approved and selected with --apis are used.

--privacy-mode local-only disables external requests and uses unexpired local enrichment cache entries when available. A cache miss is reported as Unknown.

URL query parameters with names associated with credentials or tokens are stripped from external lookups. The original IOC remains local for display and reports. Use --privacy-preview to review the original value, sanitized value, provider, and destination before the scan continues.

--watchlist-add stores supplied IOCs locally for 90 days by default (--watchlist-ttl-days changes this). --watchlist-list and --watchlist-remove TYPE:VALUE manage entries; --watchlist-scan bypasses IOC-result cache and compares fresh results with the last stored observation.

Watchlist membership and observations are local SQLite data. --no-history prevents a watchlist scan from saving its observation, so its next persisted scan compares with the last saved observation. --no-cache bypasses both IOC-result and provider-feed caches.

Add an IOC and review its sanitized value and provider destinations

python main.py --domain "https://example.com/path?access_token=secret" --watchlist-add --privacy-preview

Manage and scan local watchlist entries

python main.py --watchlist-list python main.py --watchlist-scan --privacy-mode approved-only --approved-apis virustotal urlscan python main.py --watchlist-remove IP:8.8.8.8

Use local cached intelligence without contacting providers

python main.py -i 8.8.8.8 --privacy-mode local-only --no-report

The watchlist stores IOC values and change snapshots in the local database. Protect the database file according to your incident-response data-handling policy.

threat_intel_tool/ ├── main.py # CLI entry point & argument parser ├── requirements.txt # Runtime dependencies ├── requirements-dev.txt # Dev/CI tooling (ruff, bandit, pip-audit, pip-tools) ├── requirements.lock # Pinned lockfile with SHA-256 hashes ├── pytest.ini # pytest configuration (marks, etc.) ├── config/ │ └── keys.env # API keys (copy from keys.env.example) ├── output/ # Generated reports land here ├── src/ │ ├── engine.py # Main orchestrator (collect → enrich → decide → report → export) │ ├── models.py # IOC, EnrichmentResult & AssetRecord dataclasses │ ├── storage.py # SQLite cache, feed cache, investigation & asset history │ ├── parsers/ │ │ ├── ioc_parser.py # Regex-based IOC extractor with validation (plain text/logs) │ │ ├── common.py # Shared streaming/validation helpers for log parsers │ │ ├── zeek.py # Zeek TSV logs (conn/dns/http/ssl/files.log) │ │ ├── suricata.py # Suricata eve.json (alert/dns/http/tls/fileinfo) │ │ ├── sysmon.py # Sysmon JSON-exported Windows Event Log (EIDs 1, 3) │ │ └── jsonl.py # Generic, schema-agnostic JSON-Lines extractor │ ├── enrichers/ │ │ ├── base.py # Abstract base — safe HTTP client (redirect-block, budget, retry) │ │ ├── registry.py # Enricher dispatcher │ │ ├── abuseipdb.py # AbuseIPDB (IP) │ │ ├── virustotal.py # VirusTotal (IP / Domain / URL / Hash) │ │ ├── otx.py # AlienVault OTX (IP / Domain / URL / Hash) │ │ ├── shodan.py # Shodan (IP) │ │ ├── urlscan.py # URLScan.io (URL / Domain) │ │ ├── nvd.py # NVD / NIST (CVE — no key required) │ │ ├── cisa_kev.py # CISA KEV (CVE — cached bulk feed, no key required) │ │ └── epss.py # FIRST.org EPSS (CVE — batch-fetched, no key required) │ ├── decision/ │ │ ├── cve_decision.py # Deterministic Patch/Isolate/Monitor/Not-affected logic │ │ └── asset_correlation.py # CVE ↔ asset-inventory matching heuristic │ ├── assets/ │ │ └── importer.py # Safe CSV asset-inventory importer │ ├── exporters/ │ │ ├── base.py # Allow-listed, retrying, TLS-verified HTTP POST client │ │ ├── splunk.py # Splunk HTTP Event Collector │ │ ├── elastic.py # Elasticsearch _bulk API │ │ ├── sentinel.py # Microsoft Sentinel — modern Logs Ingestion API │ │ └── dispatcher.py # Builds configured exporters, isolates per-destination failures │ ├── evidence/ │ │ └── pack.py # ZIP evidence pack with a SHA-256 manifest │ ├── reporters/ │ │ ├── excel_reporter.py # Color-coded Excel reports │ │ ├── other_reporters.py # JSON & CSV output │ │ └── terminal_display.py # Rich terminal tables │ └── utils/ │ ├── config.py # API key loader & runtime config │ ├── logger.py # Rich logging setup │ ├── banner.py # ASCII banner │ ├── quota.py # Per-run request budget (thread-safe) │ └── security.py # IOC validation, formula neutralisation, secret redaction └── tests/ ├── conftest.py # pytest fixtures & --run-e2e flag ├── test_core.py # IOC parser, verdict logic, cache round-trip ├── test_enrichers.py # BaseEnricher HTTP edge-cases — mock only ├── test_reporters.py # Excel/CSV formula protection + SQLite integration ├── test_vuln_intel_enrichers.py # CISA KEV + EPSS enrichers, feed cache ├── test_cve_decision.py # Deterministic CVE Decision Card logic ├── test_asset_importer.py # CSV asset import + storage persistence ├── test_asset_correlation.py # CVE ↔ asset matching heuristic ├── test_log_parsers.py # Zeek, Suricata, Sysmon, generic JSONL parsers ├── test_exporters.py # Splunk/Elastic/Sentinel exporters — mocked HTTP ├── test_evidence_pack.py # Evidence pack ZIP + manifest integrity └── test_cli_e2e.py # Full CLI run against real NVD API (opt-in, --run-e2e)

Design principles

Run all unit and integration tests (no network required)

pytest tests/ -v --ignore=tests/test_cli_e2e.py

With coverage report

pytest tests/ -v --ignore=tests/test_cli_e2e.py --cov=src --cov-report=term-missing

Run the end-to-end CLI test (makes a real NVD request)

pytest tests/test_cli_e2e.py --run-e2e -v

What's tested

Test file Coverage

test_core.py IOC parser (all types + edge cases), verdict logic, SQLite cache round-trip

test_enrichers.py BaseEnricher.get() — redirect blocking, budget exhaustion, 429+Retry-After, API-key redaction in logs, non-JSON response, invalid JSON, host allow-list, HTTP scheme block

test_reporters.py Excel & CSV formula-injection neutralisation (7 prefix variants), numeric passthrough, SQLite TTL expiry, upsert, investigation recording

test_vuln_intel_enrichers.py CISA KEV feed caching/TTL, EPSS batch prefetching, missing/unexpected-field tolerance

test_cve_decision.py Every Patch/Isolate/Monitor/Not-affected branch, the KEV hard-floor rule, and validation

test_asset_importer.py Column validation, formula neutralisation, size/row DoS limits, storage persistence

test_asset_correlation.py Product-token matching, criticality/exposure aggregation, "no product data" vs "not matched"

test_log_parsers.py Zeek TSV, Suricata eve.json, Sysmon JSON, generic JSONL — malformed-line tolerance, private-IP rejection, oversized-line limits

test_exporters.py Splunk/Elastic/Sentinel — mocked HTTP, allow-list enforcement, secret redaction, retry/backoff, dispatcher fault isolation

test_provider_response_validation.py Mocked provider payloads — response field allow-lists, bounded scores, CVE v