Published: 2026-05-24

A single OSINT investigation against one target takes 30–90 seconds: WHOIS lookup, subdomain enumeration, IP reputation checks, breach lookup, username search. That is fine for individual investigations. It becomes a bottleneck at scale. A threat intelligence analyst reviewing 50 suspicious IPs from a SIEM alert, a security team auditing 200 contractor email addresses for breach exposure, or a researcher investigating a botnet's C2 infrastructure spanning 300 domains — all of them face the same problem. Sequential execution at 60 seconds per target means 50 minutes for 50 targets and 5 hours for 300.

The solution is not faster tools. It is parallel execution: running investigations concurrently rather than one after another. The constraint is rate limits on the APIs being queried, not computation on the local machine. With proper concurrency control, the same 50-target investigation runs in under 5 minutes.

Python's asyncio is the right tool for I/O-bound OSINT work. All OpenOSINT tool functions are async def coroutines. To run multiple targets in parallel, wrap them with asyncio.gather():

import asyncio
from openosint.tools.search_whois import run_whois_osint
from openosint.tools.search_ip import run_ip_osint

async def investigate_domain(domain: str) -> dict:
    whois_result = await run_whois_osint(domain)
    return {'domain': domain, 'whois': whois_result}

async def bulk_investigate(domains: list[str]) -> list[dict]:
    tasks = [investigate_domain(d) for d in domains]
    return await asyncio.gather(*tasks, return_exceptions=True)

results = asyncio.run(bulk_investigate(['example.com', 'test.org', 'demo.net']))

return_exceptions=True is critical in bulk runs: it prevents one failed target from canceling the entire gather. Each result is either a value or an exception instance, which you filter and log separately.

Unbounded parallelism gets API keys suspended. The correct pattern uses a semaphore to cap concurrent requests per API:

import asyncio

# Limit to 5 concurrent requests to respect API rate limits
sem = asyncio.Semaphore(5)

async def rate_limited_lookup(ip: str, lookup_fn) -> dict:
    async with sem:
        result = await lookup_fn(ip)
        await asyncio.sleep(0.2)  # 200ms between releases = max 5 req/s
        return result

async def bulk_ip_check(ips: list[str], lookup_fn) -> list[dict]:
    tasks = [rate_limited_lookup(ip, lookup_fn) for ip in ips]
    return await asyncio.gather(*tasks, return_exceptions=True)

Tune the semaphore value and sleep delay to match each API's rate limits. AbuseIPDB free tier allows 1000 requests/day with no per-second limit documented; use a semaphore of 10 and no sleep. Shodan free tier: 1 request/second; semaphore of 1 with 1.0s sleep. VirusTotal free tier: 4 requests/minute; semaphore of 4 with 15s sleep between batches.

For multi-API pipelines, run a separate semaphore per API. A domain investigation that hits Shodan, VirusTotal, and AbuseIPDB simultaneously needs three independent concurrency controls, each tuned to its API's limits.

OpenOSINT's built-in multi-target runner handles the concurrency and rate limiting automatically. Create a target list file, one per line:

# targets.txt
192.168.1.1
10.0.0.1
suspicious-domain.net
attacker@example.com

Run the investigation:

# AI-assisted, all targets in parallel, JSON output
openosint multi targets.txt --json > results.json

# Direct tool, no AI, IP reputation only
openosint multi targets.txt --tool ip --json > ip_results.json

The multi-target runner type-detects each target (IP vs domain vs email vs username), dispatches the appropriate tool set, runs concurrently with built-in rate limiting, and writes structured JSON results. Each result object contains the target, the tool outputs, any errors, and a timestamp. See the tools reference for the full result schema.

Parallel investigation produces a list of result objects. The next step is aggregation: identifying cross-target patterns that indicate related infrastructure or coordinated activity.

import json
from collections import defaultdict

with open('results.json') as f:
    results = json.load(f)

# Group IPs by ASN to find infrastructure clusters
asn_groups = defaultdict(list)
for r in results:
    if r.get('tool') == 'search_ip' and 'asn' in r.get('data', {}):
        asn_groups[r['data']['asn']].append(r['target'])

# Flag targets sharing infrastructure
for asn, targets in asn_groups.items():
    if len(targets) > 1:
        print(f"Shared ASN {asn}: {', '.join(targets)}")

The same pattern works for any shared attribute: registrar, nameserver, creation date window, or abuse report category. Finding 15 domains that registered on the same day through the same privacy-protected registrar service is a signal that no single-target investigation would surface. This is the analytical value of parallel investigation over sequential. For generating structured reports from aggregated results, see the OSINT report generation guide. For the API-level details of each data source, see the OSINT API automation guide.

• Automating OSINT with APIs — per-API rate limits, authentication, and Python patterns
• Generating structured OSINT reports — JSON, PDF, and Markdown output from bulk results
• OpenOSINT tools reference — multi-target runner documentation and result schemas