Threat Intelligence Tools
OSINT Tools Library
A constantly updated list of web-based OSINT tools and techniques from across the open-source intelligence community, curated by Flashpoint.
Table Of Contents
Thank you for visiting our OSINT Tools Library. Here you will find information and links to some of the most useful OSINT tools and techniques from across the OSINT community. We encourage you to bookmark this page and refer to it to support your investigations. This page is updated regularly, so be sure to check in from time to time!
Generative AI
ChatGPT
ChatGPT is an AI-based conversational chatbot developed by OpenAI. It is designed to generate human-like text responses to user inputs, making it capable of engaging in conversations on various topics.
Whisper
Whisper is a free transcription and translation tool from OpenAI. It can be used offline and supports various languages.
RTX by NVIDIA
NVIDIA’s Chat With RTX program lets you personalize a GPT large language model (LLM) connected to your own content, such as documents, notes, or other data.
Perplexity.ai
Perplexity is a free AI search engine that allows users to ask any question and get answers, including the internet sources used to develop the answers.
Text-based Search
Boost your Googling skills
Google is an incredibly powerful free resource for anyone doing online research. Learn how to get the best results out of this search engine with the company’s guide to Google search refinement.
Use search engines from around the world
Access a list of global search engines to gain a more diverse understanding of global topics and uncover sources that may not be readily available through mainstream search engines.
Determine a domain’s external threat landscape
theHarvester is a command-line tool included in Kali Linux that acts as a wrapper for a variety of search engines and is used to find email accounts, subdomain names, virtual hosts, open ports/banners, and employee names related to a domain from different public sources (such as search engines and PGP key servers). theHarvester is designed to be used during the reconnaissance stage of a red team assessment or penetration test.
Search for email addresses
The service Hunter.io is popular among offensive security professionals and OSINT practitioners who want to find email addresses for members of an organization or obtain additional information about an email address. Hunter.io has a free plan with an API allowing 25 searches and 50 monthly verifications.
Search for source code
If you’re looking at a unique piece of malicious code within a script, it can help to check if that same unique snippet is found elsewhere. PublicWWW is a public search engine for source code that will help you do that. For a good example, find a small portion of the Magecart skimmer, put it into the search engine, and look at the results.
Find deep web data
90% of the internet isn’t indexed by search engines. This includes many internal links and deep websites. If you’re looking for a file shared by a company between two people or organizations, you most likely won’t be able to find it in Google or any other search engine. URLScan has many of these links indexed in their historical scans. If you type in a domain for any website into their search engine, you’ll find multiple links from that domain in the search results. It’s a great way to do deep web research without needing access or logins.
Search for Bitcoin addresses
Blockchain and cryptocurrency investigations have been a relatively recent development in the OSINT space and are gaining relevance as more threat actors and scams begin using cryptocurrency as their transaction method of choice. If you’re looking into a crypto wallet related to ransomware, blackmailing, fraud, etc., check out chainabuse.com. It’s a public database of addresses used for nefarious reasons. They also have an API to build an OSINT tool for crypto.
Power up your search
You.com is an AI-powered search engine. It’s privacy-centric (like DuckDuckGo), but it has built-in functionality for social media and video searches, and it separates the results by category.
Search Pastebin dumps
Pastebin is a great resource for OSINT collection. A great tool to search for Pastebin data is PsbDmp. At the time of writing this, PsbDmp had 25990293 Pastebin dumps in its archive. You can easily use its search engine to check if the data point you’re investigating is in its archive.
Search Shodan
If you’re doing technical OSINT collection or investigation, you’ve probably heard of Shodan. At first glance, Shodan can be intimidating, and it is hard to find what you’re looking for. Fortunately, Jake Jarvis put together a guide with multiple use cases and examples for searching on Shodan. From prison pay phones to industrial control systems, there’s a ton of untapped potential inside of Shodan’s index.
Look up businesses using Secretary of State data
If you’re doing business-related investigations for entities in the United States, you’ll often be requested to pull Secretary of State documents for state filings of that business. These can be hard to track down if you visit each state website and pull the information that way. Fortunately, Cobalt Intelligence made a tool that pulls Secretary of State filings for businesses in all states from one search engine, saving a lot of time on business lookups.
Collect website metadata
If collecting metadata is important to your OSINT workflow or are building a URL or domain-specific OSINT tool, finding a reliable, fast way of grabbing that information from a target URL can be tricky if that’s not your area of expertise. Fortunately, there’s an API called Website Metadata that can easily grab metadata for any URL you specify.
Reverse cleartext and hashed passwords
Similar to using Google Dorks for Pastebin, you can also reverse cleartext and hashed passwords to find more information. If you have a cleartext password, use an online hashing tool like MD5 Hash Generator to turn it into a hash; if you have a hashed password, use hashes.com to turn it into cleartext.
From there, you can use tools like Dehashed or IntelX to turn those cleartext or hashed passwords into emails, phone numbers, names, etc., to further expand your ability to pivot across different data sources and build a full digital record.
Find leaked emails on Github
If you’re doing OSINT investigations into GitHub repositories, you might be interested to see who is all making commits to that project. Fortunately, finding email addresses in these commits from Github users is pretty simple. The hard part is grabbing all that content, specifically on larger projects. GitRecon will do that for you in one motion.
Example: You find a GitHub repository that says it’s a PoC of an exploit for a software vulnerability. Upon further investigation, you find that it’s a honeypot for malware. You can use GitRecon to determine who is involved if that information is available.
Gather OSINT from hashes on Pastebin
If you want to collect a series of hash values such as passwords or IOCs, you can use a specific Google ‘dork’ to locate them on a website like Pastebin.
Example:
site:pastebin.com intext:“SHA256”
You can replace SHA256 with any other algorithm, including MD5, SHA1, MySQL, NTLM, or SHA512.
Once you have a hash, check it against places like VirusTotal to see if it’s malicious.
Look up phone numbers with Phomber
Phomber is an easy-to-use, free Python tool for reverse phone number lookups.
Find a person’s name behind any Gmail address
1. Create a Google document
2. Share the document with the Gmail address
3. Untick the “Notify people” box. The name will be disclosed.
Search usernames across multiple domains
Developed by a group of OSINT practitioners, the WhatsMyName tool allows you to enumerate usernames across many websites. The results can be exported as a CSV, PDF, or Excel file.
Search social media username availability
Namecheckup.com enables you to search for social media username availability across 253 platforms and apps. It also helps investigators locate additional profiles for individuals who used the same username for several profiles.
Check domain availability
Namechk checks 36 domain name possibilities and 100+ social media websites and online platforms. Sites include Facebook, Twitter, YouTube, Blogger, Twitch, Tumblr, TikTok, WordPress, eBay, Yelp, Flickr, and PayPal.
Search multiple online marketplaces at once
SearchTempest is a search engine for online classified ads that delivers results from Facebook Marketplace, Craigslist, eBay, Amazon, and others.
Gather information from Instagram
Toutatis allows you to extract information from Instagram accounts, such as e-mails, phone numbers, and more. (Python is required.)
Search Facebook
Who Posted What is a Facebook keyword search for people who work in the public interest. It allows you to search keywords on specific dates. Donations are encouraged.
Search through a digital library
Internet Archive (AKA Wayback Machine) is a non-profit library of millions of free books, movies, software, music, websites, and more. Its mission is to provide universal access to all knowledge.
View historical web data.
Common Crawl is a free resource that contains petabytes of data, which has been regularly collected since 2008.
Reverse email search
Epeios is a search engine that allows you to perform reverse email searches, find related Google reviews, and see which websites are associated with any email address.
Analysis
Investigate URL-shortened links for OSINT
If you’re doing OSINT data collection at scale, you’re likely to come across a lot of shortened links. If you try to investigate each one manually, which is a viable method, it will take a lot of time. Fortunately, there’s an open-source tool that will expand those links for you at scale. Links like bit.ly, adf.ly, lnx.lu, linkbucks.com, and adfoc.us can be expanded automatically with urlExpander. Whether you have a million links or a hundred, check it out.
Visually investigate cryptocurrency
If you’re tracking a lot of illicit actors, you’ve probably had some exposure to cryptocurrency investigations. A common challenge in crypto investigations is making sense of extremely large datasets. Many tools will show you a single connection and rely on you importing that information into a different tool like Maltego to form visual connections. Breadcrumbs is a blockchain analytics platform accessible to everyone. It offers a range of tools for investigating, monitoring, tracking, and sharing relevant information on blockchain transactions. It solves the problem of connecting the dots between transactions and wallets. The free tier provides limited access to almost all functions, including monitoring and graph creation. Whether you’re new to this space or looking for a budget-friendly solution, this is worth a look.
Leverage this domain OSINT multi-tool
For investigations, it’s helpful to keep a running list of found data points in as many tools and resources as possible. Sometimes, a slight change in configuration from one tool to another can offer a different angle or set of information. Investigator is an OSINT multi-tool for domains. It has a variety of modules to help dig deeper into specific categories. The web app format makes it possible to use with limited technical knowledge or in controlled environments.
Try Bitcrook
Bitcrook is an OSINT multi-tool that might check a couple of your boxes in a single script. Bitcrook is unique because it factors in court cases from case.law and data from Melissa. It also has standard modules like IP and username lookup.
Find out how a site was built
Builtwith enables you to examine any website’s code to understand how it functions and what frameworks/technologies it is using.
Automation & Efficiency
Shorten repetitive tasks in terminal
If you spend a lot of time in the terminal for OSINT collection and analysis, you probably run the same commands repeatedly. There’s the classic meme of pressing the up arrow a million times until you find the command you were looking for. What if you could create a shortened version of that command to reference quickly and use at will? PCMD lets you do that. Type in a long, specific command, save it as a shortcut with PCMD, and use it repeatedly without typing it out.
Discover the best OSINT threads
Some of the best OSINT techniques and workflows can be found on Twitter. The issue with Twitter is that they break them up into multiple messages, so discovering threads can be difficult if the post doesn’t specifically say it’s a thread. If you’ve read a thread on Twitter before, you have likely seen people tagging apps like ThreadReaderApp to consolidate the thread into one post. What you might not know is that you can query all threads they create on their website just by searching for #OSINT.
Collect onion URLs automatically
Because the dark web doesn’t have a free, unified search engine like Google, OSINT researchers who don’t have access to a service like Flashpoint have to either hunt for it manually or rely on static databases like dark.fail to find more. OnionSearch aggregates the search results of multiple dark web search engines that index these links, allowing you to search for things in multiple places simultaneously. If you’re looking for a free tool to discover new dark web URLs, OnionSearch is worth checking out.
Remove web elements more efficiently
Many OSINT workflows require adding and removing web elements to reveal certain details. Snip allows you to permanently remove certain web elements so that you never see them again. This is very useful for frequently visited websites.
Build workflows with Automa
This Chrome extension allows you to create a workflow using a drag-and-drop interface and save it as a playbook to replay at any time. From auto-fill forms to repetitive tasks, taking screenshots, and scraping data from a website, Automa is pretty flexible. If you want to automate some of your OSINT workflows, give this a spin.
Extract data from any website
Instant Data Scraper is an automated data extraction tool for any website. It uses AI to predict which data is most relevant on an HTML page and allows saving it to Excel or CSV files (XLS, XLSX, CSV).
Automate scraping with Browse AI
If you’ve ever used Instant Data Scraper, this tool will be very familiar to you. What Browse AI does that Instant Data Scraper doesn’t is set up automated scraping tasks and export the results automatically to a Google Sheet. If you want a no-code way of scraping websites or want to build quick proofs on concepts before committing to writing a custom web scraper, give this a shot.
Automate writing tasks with Tango
Tango is a very handy tool for writers of blog posts or documentation articles. It allows you to write step-by-step tutorials with screenshots in a few seconds. Tango works by recording your actions on the screen while enabled and converting that into a process-based how-to guide when the recording stops.
Manipulate datasets from Terminal
OSINT practitioners frequently run across TXT and CSV files. Often, the formatting for CSV files is incompatible with your investigation, and you need to remove certain columns, add others, change column titles, etc. If you’re looking to remove all columns except for a select few, use this trick:
$ cut -d, -f<column number> –complement input.csv > output.csv
Example:
$ cut -d, -f4 –complement osint.csv > oshint.csv
This will remove the fourth column of osint.csv and save it to oshint.csv. If you remove –complement, you’ll remove all columns except column 4.
To learn more about this Linux command, check out this article.
Search through large datasets with ease
Leaked credential databases are a key component of investigations into individuals with a small digital footprint. Sometimes, these datasets can be hundreds of gigabytes in size, making it difficult to open them in any software or browser and look for applicable information.
Enter Ripgrep. It allows you to recursively search a database—regardless of size—for a specific data point and see if it exists. There’s a learning curve, so you might want to check out this video before getting started.
Data Organization
Export or print full-page screenshots
GoFullPage is a Google Chrome extension that captures an entire webpage as a screenshot. This is useful for storing or archiving pages in a specific state or when you need to share a webpage with someone who might not have access to it.
Archive digital evidence
Save as MHTML is useful, but it can only capture one page at a time. If you want to download multiple pages or an entire website as MHTML, Enter SinglePage. SinglePage lets you capture the current tab, a specific selection, multiple tabs, highlighted text, etc. It also has a SHA256 module to hash your evidence.
Manage CSV files with CSVKit
If you’re dealing with spreadsheets on a regular basis, it can be hard to manage really large datasets. CSVKit is a suite of command-line tools for converting to and working with CSV files. It allows you to convert file types (like CSV to JSON), select certain types of information from the dataset, or import the data to PostGreSQL. The lookup capability is especially useful.
Example: csvgrep -c phone_number -r “555-555-\d{4}” data.csv > new.csv
You can look up a phone number, take it from the main dataset, and add it to a new one.
Convert URLs to PDFs
PDF my URL allows you to download a website to a PDF for offline viewing. You can convert a single page or an entire website to a PDF and remain anonymous because the app’s IP address requests the information from the website, not yours.
Save webpages as HTML files
SingleFile is a Google Chrome extension that helps you to save a complete page (with CSS, images, fonts, frames, etc.) as a single HTML file.
Image & Video Tools
Reverse image search with RevEye
RevEye is a Firefox add-on that allows you to perform reverse image search with Google, Bing, or Tineye.
Try a visual search with Microsoft Bing
Bing visual search is simple and intuitive. You can drag and drop any image, type any URL into the search bar, or even take a photo directly in the home dashboard.
Verify content on social networks
The InVID verification plugin is a Chrome extension designed to help journalists debunk fake news and verify image and video content.
Find and download all images on a web page
Imageye Image Downloader is a tool that helps you find the images published on a web page.
Get better Instagram data
A common method for finding connections on social media is by looking at followers, followings, and engagements (comments, likes, etc.). Instagram is becoming more and more difficult to collect data from. Sterra lets you get information about an Instagram account through its followers and followings. Here are some things it considers:
- 📊 A probability function to determine the close social circle of your target
- 📥 Export of the followers / following lists (with their details) to excel and csv
Capture YouTube metadata at scale
If you’re collecting video OSINT, it’s important to document the metadata. For YouTube specifically, YouTube Metadata Bulk allows you to capture metadata for a single video or for a list of videos. That metadata can be exported as TXT, JSON, CSV, or PNG. Entities like tags, geolocation, links, and upload times can be grabbed instantly.
Build your own reverse image search
Reverse image searching is a major technique used in OSINT collection and analysis. The issue with many reverse image tools is that you rely on the algorithms of Google, etc. to find a positive match. While these tools are often reliable, wouldn’t it be great if you could build your own? Well, you can. This Github project provides the foundation for building your own reverse image tool. It’ll find exact and near matches out of the box; however, you could combine it with facial recognition to make it more sophisticated and accurate.
Scan and detect deep fake videos
Deepfake videos are becoming a huge problem on the internet. From Tom Cruise to Barack Obama’s public service announcements, the future of disinformation looks pretty grim with the ability to create fake videos that fool the masses.
Deepware provides a web app and API that help identify deepfake videos. You can send them to Deepware for review, whether stored locally or hosted on a URL online. The API is straightforward to use if you want to add this capability to an OSINT tool.
Try Facebook’s built-in OCR
You can search for specific alphanumeric strings, such as license plate numbers on Facebook and comb through images to find a match. Facebook automatically tries to use OCR within its images to display relevant results. I tried a search for “MWP-46-99” (a license plate) and got the following result:
View image exif data
EXIFdata.com is an online application that lets you upload an image and view the associated data such as shutter speed, exposure compensation, F number, date, time, and more.
Geolocation
Improve your geolocation skills
GeoGuessr has become the official primer for developing your geolocation skills. It gives you continuous exposure to different scenarios to train your deduction skills. Give it a read or three, and watch your scores improve. Take what you learn on GeoGuessr and apply it to real challenges.
Bootstrap geolocation
AllYouCanRead is a collection of local newspapers to a specific city, county, etc. If you collect all of the newspapers you’re interested in and pull their RSS feed, you can use keyword filters to look for keywords of locations within publications of papers local to that location. So, looking for “San Francisco” in a San Francisco-based newspaper will likely yield results in that area. Scale that across multiple cities and countries, and you have bootstrapped geolocation.
Detect ships
The Ship Detection Tool, built by Bellingcat, is a satellite imaging tool that uses Synthetic Aperture Radar (SAR) imagery to identify ships in port, at anchor, or at sea.
View GPS interference
GPSJAM provides daily maps of possible GPS interference based on aircraft broadcast information. The map shows low, medium, and high levels of aircraft GPS accuracy to signify the likelihood of GPS jamming or other potential navigation system interference in the area.
Geolocate images using shadows
Bellingcat’s Shadow Finder Tool, developed with its Discord community, helps you quickly narrow down where an image was taken by reducing your search area from the entire globe to just a handful of countries and locations.
“Open-source information is going to be of incalculable value both to our intelligence community and military in the future.”
Glenn gerstell, former general counsel of the national security agency
Identity Protection & Safety
Identify breached accounts
Have I Been Pwned is a free web-based tool that allows you to quickly assess whether you may have been put at risk due to an online account being compromised or “pwned” in a data breach.
Check data points against spam
If you’re investigating a lot of data, specifically IP addresses, emails, or usernames, make sure to check Stop Forum Spam to see if any of the data points you’re dealing with are found there. If you find a positive hit on any of your searches, you can pivot on the found data points to find more information. Example: If you have an IP address and get a match, it’ll give you an email and a username. You can reverse those back into Stop Forum Spam or any other OSINT tool you use.
Create credit cards for your sock puppet
When building a sock puppet (alternate persona), it’s important to create as complete a profile as possible. Many websites require a credit card to sign up. Using Card Generator, you can make a card tied specifically to your sock puppet and maintain consistency across the web without compromising your identity.
Education & Training
Level up your Telegram investigations
If you’re collecting data from Telegram, there are a ton of tools you can use to find new users, groups, channels, etc. Awesome Telegram OSINT is a collection of tools, tactics, and techniques for extracting information from Telegram. If you’re a seasoned Telegram collector or just getting started, this is your one-stop shop.
Read OSINT case studies
Reading case studies is one of the best ways to learn about new tools, tactics, and techniques. “Exposing the Invisible” lists case studies on its website that can inspire you in your investigations.