OSINT

# RULES OF ENGAGEMENT (always re-read)
#   - All OSINT activity below assumes authorised testing of YOUR OWN assets,
#     or assets you've been contracted to assess under signed scope.
#   - 'Public' data is gathered passively — but using it against an org without
#     authorisation can still constitute reconnaissance for an unauthorised attack.
#   - For LEARNING: practice on intentionally-vulnerable targets (HackTheBox, TryHackMe,
#     OWASP Juice Shop) and OWN domains.

# === Why OSINT? ===
# OSINT reveals the attack surface without sending a single packet to the target.
# - Subdomains the org forgot about
# - Leaked credentials
# - Employees + tech stack (for social engineering risk)
# - Past breach exposure
# - Cloud assets, code, secrets

# === Domain + DNS ===

# 1) Whois — domain registration info
whois example.com
# Look for: registrant, contact emails, name servers, registration date

# 2) DNS records
dig example.com any                # all record types
dig +short txt example.com         # TXT (SPF, DMARC, verification keys)
dig +short mx example.com          # mail servers
dig +short ns example.com          # name servers
host -t any example.com

# 3) Subdomain enumeration
# Passive
amass enum -passive -d example.com
subfinder -d example.com -all -silent
assetfinder example.com

# Certificate Transparency
curl -s 'https://crt.sh/?q=%25.example.com&output=json' | jq -r '.[].name_value' | sort -u

# Active brute force (uses DNS queries — quietly)
amass enum -active -d example.com -brute -w wordlists/subdomains.txt

# Cloud DNS lookups
cloud_enum -k example -k example.com

# 4) Reverse DNS / IP range
shodan host 192.0.2.1                # who lives on this IP?
shodan search 'ssl:example.com'      # certs containing example.com
shodan domain example.com

# === Certificate Transparency monitoring ===
# https://crt.sh                — search by domain
# Cert Spotter (free)            — alerts on new certs for your domains
# https://transparencyreport.google.com/https/certificates

# === Web archive ===

# 5) Wayback Machine — find old / removed endpoints
curl -s 'http://web.archive.org/cdx/search/cdx?url=*.example.com/*&output=text&fl=original&collapse=urlkey' > urls.txt
wc -l urls.txt

# Filter for interesting endpoints:
grep -i 'admin\|debug\|test\|api' urls.txt

# Or use gau (get all URLs)
gau example.com

# === Source code repos ===

# 6) GitHub — leaked secrets / sensitive files
# Manual search
#   site:github.com example.com api_key
#   site:github.com example.com password
# Tools:
trufflehog github --org example-org
gitleaks detect --source ./repo --report-path report.json
githound --org example-org

# Look for:
# - .env files with secrets
# - config.yml with passwords
# - Hardcoded API keys, tokens
# - Database connection strings

# 7) Code spillage detection (your own org)
# - Add gitleaks / trufflehog to pre-commit hooks
# - Run scans across all org repos weekly
# - Set up CT log monitoring for unexpected certs

# === Employees + people ===

# 8) Email format detection
hunter.io / RocketReach / Apollo
# Find email pattern: firstname.lastname@example.com

# 9) LinkedIn — employees, departments, tech stack hints
# Useful for understanding org chart + targeting social engineering tests
# Tools: PhantomBuster, Apollo, theHarvester
theHarvester -d example.com -l 500 -b google,bing,linkedin,duckduckgo

# 10) Email breach lookups
# HaveIBeenPwned (free)
curl -s 'https://haveibeenpwned.com/api/v3/breachedaccount/email@example.com' \
    -H 'hibp-api-key: YOUR_KEY'

# DeHashed (paid, more breaches)
# These show which past breaches contained an employee's email.
# Use the info to assess credential stuffing / reuse risk.

# === Tech stack fingerprinting ===

# 11) Identify tech
wappalyzer-cli https://example.com
whatweb https://example.com
builtwith.com

# Headers
curl -I https://example.com
# Look at Server, X-Powered-By, set-cookie patterns

# 12) Shodan / Censys — search internet-facing assets
shodan search 'org:"Example Inc"'
shodan search 'ssl.cert.subject.cn:example.com'
shodan search 'http.title:"Example login"'

# Censys (alternative)
censys search 'services.tls.certificates.leaf_data.subject.common_name: example.com'

# === Mobile + apps ===

# 13) Mobile app stores
# - Search for the org's apps
# - Download APKs / IPAs (legally) for reverse engineering
# - Look for hardcoded URLs, keys, debug endpoints

# Tools:
#   apktool — decompile Android APKs
#   jadx — decompile to Java
#   class-dump / Hopper — iOS reverse engineering (typically requires jailbreak)
#   MobSF — automated mobile analysis

# === Cloud assets ===

# 14) S3 bucket enumeration
cloud_enum -k example -k example.com

# Look for buckets named:
#   example-prod-backups
#   example-staging
#   example-dev
#   example-uploads

# Check for public read on each.

# 15) Azure storage / GCP buckets — similar enumeration

# === Social media ===

# 16) Public profiles + posts
# Helpful for:
# - Identifying employees + roles
# - Tech stack hints (job postings)
# - Conference talks revealing internal architecture
# - Photos with badges / dashboards visible (rare but possible)

# Tools:
#   Sherlock — find a username across many platforms
#   Maltego — visualise OSINT relationships

# === Pastebin / dump sites ===

# 17) Search for leaked data
# - pastebin.com searches
# - Telegram channels selling/leaking data
# - Underground forums (requires authorisation; legal risk if unauthorised)

# Defensive tools:
#   Recorded Future, Flashpoint, DarkOwl — commercial threat intelligence
#   HaveIBeenPwned — for employee email exposure

# === Workflow ===

# 18) Typical OSINT engagement
# 1. Define scope (in-scope domains + IPs + org names)
# 2. Get authorisation (signed)
# 3. Phase 1 — passive recon
#    a. Whois + DNS
#    b. Subdomains (CT + passive enumeration)
#    c. Wayback Machine for historical endpoints
#    d. GitHub for code + secrets
#    e. Tech stack fingerprinting
#    f. Cloud asset enumeration
#    g. Employee enumeration (LinkedIn, theHarvester)
#    h. Breach + leak databases
# 4. Phase 2 — active recon (sends traffic)
#    a. Port scanning
#    b. Service enumeration
#    c. Web app exploration
# 5. Report findings + recommendations

# === Defensive use ===

# 19) Continuous OSINT for your own org (Blue Team)
#   - CT log monitoring (Cert Spotter, your own subgraph)
#   - Brand mention monitoring (Google Alerts, Mention)
#   - GitHub org-wide secret scanning (gitleaks/trufflehog in CI + nightly)
#   - Breach exposure alerts for employee emails (HaveIBeenPwned monitoring)
#   - Dark web monitoring (commercial services)
#   - Domain typosquatting detection (DnsTwist, urlscan.io)
#   - Honeypots / canary tokens (Thinkst Canary, CanaryTokens.org)

# === Tools cheat sheet ===
#   Whois / DNS:     whois, dig, host, nslookup, dnsx
#   Subdomain:        amass, subfinder, assetfinder, sublist3r, crt.sh
#   Wayback:          waybackurls, gau, gauplus, paramspider
#   GitHub:           trufflehog, gitleaks, github-dork, git-hound
#   Tech detection:   whatweb, wappalyzer, builtwith
#   Shodan/Censys:    shodan, censys-cli, zoomeye-search
#   Cloud:            cloud_enum, s3scanner, gcpbucketbrute, AzureHunter
#   Email/People:     theHarvester, hunter.io, RocketReach
#   Breach:           HaveIBeenPwned, DeHashed (commercial)
#   Mobile:           apktool, jadx, MobSF, frida
#   Aggregator:       SpiderFoot, recon-ng, Maltego

# === Authorised practice environments ===
# - HackTheBox     — OSINT challenges + lab
# - TryHackMe      — OSINT rooms + walkthroughs
# - PortSwigger    — Web security academy (web-side recon)
# - OWASP Juice Shop — vulnerable web app
# - VulnHub        — downloadable VMs
# - OffSec Labs    — OSCP / OSEP prep environments

# === Legal + ethical reminders ===
# - Public data is generally lawful to gather, BUT:
#   * Using gathered data to attack without authorisation = unauthorised access
#   * Some jurisdictions criminalise even reconnaissance (read your local laws)
#   * Privacy laws (GDPR, Privacy Act) restrict storing personal data without basis
# - Bug bounty programs typically define scope + authorise OSINT within it
# - When in doubt, ask before acting; document everything

# === Reporting template (OSINT findings) ===
# Finding: Sensitive S3 bucket discovered
# Severity: High
# Evidence: https://example-backups.s3.amazonaws.com/ — directory listing enabled,
#           contains 'production_db_2024.sql.gz' (847MB)
# Discovery: cloud_enum -k example-backups returned the bucket name
# Risk: Production database backup accessible to unauthenticated users
# Recommendation:
#   1. Block public access immediately (AWS S3 console)
#   2. Rotate any credentials stored in the backup
#   3. Enable bucket logging + monitoring
#   4. Implement IAM Access Analyzer to detect future public exposure
# References: AWS S3 security best practices, OWASP Cloud Security

# OSINT = Open-Source Intelligence — what an outsider can learn legally.
# Sources: WHOIS, DNS, archive.org, GitHub, LinkedIn, court records,
# breach databases (HIBP), public sub/issue trackers.
# Helps clients understand what they're already leaking before you scan a port.