← Back to Blog

AUTOMATING OSINT WITH APIs

Published: 2026-05-24

CHOOSING THE RIGHT API FOR THE QUESTION

Each major OSINT API answers a different question. Shodan answers: what services are running on this IP or across this organization's infrastructure? VirusTotal answers: is this URL, file hash, or IP flagged by any of 70+ security vendors? Censys answers: what does this IP's TLS certificate say, and what subdomains has this organization ever issued certificates for? AbuseIPDB answers: has this IP been reported for malicious activity, and how recently? These are distinct questions with distinct data models. Understanding which source answers which question prevents redundant queries and ensures you're using the right tool for each investigation phase.

All four expose REST APIs with API key authentication. Free tiers are sufficient for investigative use; paid tiers are needed for bulk automation. Rate limits vary significantly — understanding them before building automation prevents failed runs and temporary key suspensions.

SHODAN: INFRASTRUCTURE SEARCH

Shodan's API is available via both REST and the official Python library. Install: pip install shodan. The Python library wraps the REST endpoints cleanly:

import shodan

api = shodan.Shodan('YOUR_API_KEY')

# Host lookup: ports, banners, TLS data
host = api.host('8.8.8.8')
print(f"Org: {host['org']}")
for item in host['data']:
    print(f"Port {item['port']}: {item.get('product', 'unknown')}")

# Search: find open admin panels in an org
results = api.search('http.title:"Admin" org:"TargetCorp"')
for match in results['matches']:
    print(f"{match['ip_str']}:{match['port']}")

Rate limits: free API keys get 1 query/second and 100 results per search. Monitor credit usage via api.info(). For ASN-wide queries, asn:AS12345 is more reliable than org: which matches on free-text org name fields. OpenOSINT's search_shodan tool wraps both host lookup and keyword search with timeout enforcement and error handling.

VIRUSTOTAL: MULTI-VENDOR THREAT INTELLIGENCE

VirusTotal's v3 API uses Bearer token authentication. The key endpoints for OSINT are: /api/v3/urls/{id} for URL analysis, /api/v3/ip_addresses/{ip} for IP reputation, and /api/v3/domains/{domain} for domain analysis. Each returns a data.attributes object with last_analysis_stats (counts of malicious/suspicious/clean/undetected verdicts) and last_analysis_results (per-vendor verdicts).

import requests, base64

API_KEY = 'YOUR_VT_KEY'
headers = {'x-apikey': API_KEY}

# URL analysis (URL must be base64url-encoded without padding)
url = 'https://suspicious-site.example.com'
url_id = base64.urlsafe_b64encode(url.encode()).decode().rstrip('=')
r = requests.get(
    f'https://www.virustotal.com/api/v3/urls/{url_id}',
    headers=headers
)
stats = r.json()['data']['attributes']['last_analysis_stats']
print(f"Malicious: {stats['malicious']} / {sum(stats.values())}")

Rate limits: free tier is 4 requests/minute, 500/day. For IP lookups, the last_analysis_date field tells you when VirusTotal last scanned the IP — important context since stale data can produce false negatives on recently-compromised infrastructure. OpenOSINT's search_virustotal handles URL submission, result polling, and timeout for analyses that haven't been cached yet.

CENSYS AND ABUSEIPDB

Censys uses HTTP Basic Auth with an API ID and secret. The primary OSINT use case is certificate search: every subdomain that ever received a TLS certificate is indexed. Query the certificates index:

from censys.search import CensysHosts

h = CensysHosts()
# Find all IPs presenting target.com in their certificate
for page in h.search("services.tls.certificates.leaf_data.names: target.com"):
    for host in page:
        print(host['ip'])

This catches hosts that never appear in DNS aggregation because they're not linked from any public page — exactly the forgotten infrastructure worth investigating. For OpenOSINT users, search_censys wraps this with CENSYS_API_ID and CENSYS_API_SECRET from your environment.

AbuseIPDB's v2 API returns a structured abuse confidence score (0–100), total report count, last reported date, and the reporter's categorized abuse types (brute-force, DDoS, port scan, etc.):

import requests

r = requests.get(
    'https://api.abuseipdb.com/api/v2/check',
    headers={'Key': 'YOUR_KEY', 'Accept': 'application/json'},
    params={'ipAddress': '192.168.1.1', 'maxAgeInDays': 90}
)
data = r.json()['data']
print(f"Score: {data['abuseConfidenceScore']}% | Reports: {data['totalReports']}")

AbuseIPDB is most useful for threat context: an IP with 400 abuse reports in 90 days is high-risk infrastructure regardless of what geolocation or ASN data says. OpenOSINT's search_abuseipdb exposes this directly; see the IP intelligence guide for combining AbuseIPDB with IP2Location for full context. For bulk IP reputation checking across a target's entire infrastructure, see the parallel OSINT guide.

BUILDING A UNIFIED INVESTIGATION PIPELINE

The pattern for multi-source API automation: collect all target identifiers first, then dispatch queries in parallel, then correlate results. Python's asyncio with aiohttp handles concurrent API calls efficiently without threading complexity:

import asyncio
import aiohttp

async def query_abuseipdb(session, ip, key):
    async with session.get(
        'https://api.abuseipdb.com/api/v2/check',
        headers={'Key': key, 'Accept': 'application/json'},
        params={'ipAddress': ip, 'maxAgeInDays': 90}
    ) as r:
        return ip, await r.json()

async def bulk_ip_check(ips: list[str], key: str) -> dict:
    async with aiohttp.ClientSession() as session:
        tasks = [query_abuseipdb(session, ip, key) for ip in ips]
        results = await asyncio.gather(*tasks, return_exceptions=True)
    return {ip: data for ip, data in results if not isinstance(data, Exception)}

OpenOSINT wraps this pattern in its multi-target runner. The openosint multi targets.txt command handles rate limiting, retries, and structured JSON output automatically. For custom pipelines that need direct API access, the pattern above is the starting point. The tools reference documents which API keys each OpenOSINT tool requires and what structured data each returns.

SEE ALSO


Home · Blog · Tools · GitHub