The Best Open Source Intelligence tools

What are the best OSINT tools to get valuable information? Are you looking to track people, identify domain owners, IPs, servers and find confidential company data?

OSINT is often gathered during a reconnaissance phase. The reconnaissance phase of a cyberattack is one of the most critical steps in the process.

During this phase, an attacker strives to gather as much information as possible about an intended target in order to plan and execute a successful attack.

This can include leveraging various intelligence gathering techniques such as port scans, IP address enumerations, network fingerprinting, and vulnerability scanning.

Attackers can also use social engineering – gathering information from social networks or emails – in order to obtain valuable details about their target’s environment.

Through open source intelligence reconnaissance, attackers gain the knowledge needed to compromise their target’s security and access confidential data or cause disruption.

What is OSINT?

Open Source Intelligence (OSINT), has gained considerable attention in recent years as an effective tool for gathering crucial data efficiently.

It provides quick and reliable insights into a wide variety of topics, from political campaigns to national security threats.

The open source nature of the data makes it easily accessible by anyone with access to the internet, allowing both organizations and individuals to make decisions quickly and effectively. By harnessing the power of technology, OSINT provides information on threats or outside vulnerabilities, people, adversaries, domains, sensitive information etc. Essentially gathering actionable intelligence.

What is OSINT in Cybersecurity?

OSINT tools plays an ever-expanding and integral role in cybersecurity. OSINT combines multiple analytical techniques to obtain information from publically available sources such as news reports, social media platforms, and broadcast media.

This data can then be used to form insights into organizational risk patterns and security threats. Generally considered a subset of the larger field of intelligence collection, OSINT is beneficial for discovering vulnerabilities that may not be found through traditional threat intelligence techniques.

Utilizing open source intelligence proactively can help organizations stay ahead of potential data attacks by gathering external information related to their environments or on a specific threat actor. Security teams can also use an OSINT tools search engine to obtain information on specific threat actors or to identify indicators of attacks and indicators of compromise associated with an IP address.

Is Open source intelligence Legal?

OSINT is a legal and legitimate activity as long as it is conducted in accordance with the laws and regulations of the country where it is being carried out.

Given, it involves collecting, analyzing, and disseminating information that is publicly available and legally obtainable. It is a commonly used.

However, it is important to note that certain types of information, such as personal data and classified information, may be protected by laws and regulations, and it may be illegal to access or use this relevant information without proper authorization.

Additionally, gathering intelligence using an OSINT tool, for illegal purposes, such as stalking or malicious intent, is also illegal. Because of this, it is important to understand not only the framework of standard espionage laws and regulations, but also those governing access to domestic databases at both federal and state levels.

Is Open source intelligence Free?

Data collection involves, analyzing, and disseminating information that is publicly available and legally obtainable.

This information is often available for free from a variety of sources, such as news websites, social media platforms, government websites, and other online resources such as the dark web, dark web search engines the deep web, social media accounts, or via a Google search.

However, it is important to note that there may be costs associated with accessing certain types of information, such as subscription fees for access to certain databases or paywalls on some news websites.

Additionally, while most of the information itself may be free, there may be costs involved in the process of collecting, analyzing, and disseminating the information, such as the time and resources required to gather and review the data.

What are OSINT techniques?

Popular methods of open source intelligence include:

1. Search engine dorks
2. Social media analysis
3. Website analysis
4. News analysis
5. Document analysis
6. Data mining
7. Human intelligence
8. Public records research
9. Geolocation analysis
10. Image and video analysis
11. Phone number and email analysis
12. Network analysis
13. Metadata analysis
14. Advanced Google search techniques
15. Online forum analysis
16. Domain name system (DNS) analysis
17. Whois database analysis
18. IP addresses analysis
19. Technical web analysis
20. Deep and dark web analysis

All in all, there are a multitude of different OSINT tools and techniques being widely adopted by anyone looking to gain greater intelligence on their respective domains and to connect data points.

How to Detect and Identify Company Critical Data?

1. Determine the scope of the research: Identify the specific information you are looking for and the type of company you are researching. This will help narrow down the sources you need to search and the techniques you will use.
2. Search websites: Many companies make information about their products, services, and operations publicly available on their own websites.
3. Check government databases: Many government agencies maintain databases of information about companies, such as business registration records and financial statements.
4. Search online news sources: News articles and press releases can provide valuable insights into a company’s operations and performance.
5. Use social media: Social media platforms, such as LinkedIn and Twitter, can provide information about a company’s employees, products, and services.
6. Check industry databases: Industry-specific databases, such as those maintained by trade associations or professional organizations, can provide information about a company’s operations and performance within a particular industry.
7. Search online business directories: Online business directories, such as those maintained by the Better Business Bureau and Dun & Bradstreet, can provide basic information about a company, such as contact information and company size.
8. Use search engines: Search engines, such as Google, or a dark web search engine can be a useful tool for finding public information about a company. Simply enter the company’s name and see what information is available.
9. Analyze the data: Once you have gathered the data, analyze it to identify patterns and trends. This will help you identify the most important and critical data points about the company.

The top 30 open source intelligence Tools

These are the best open source intelligence tools out there

1. OSINT Framework Even though OSINT Framework isn’t a program that should be installed on your servers, it’s a very helpful way to access free tools, resources, and search engines that are made available to the public online. They are committed to providing the most relevant links to useful OSINT data sources. Although the initial focus of this web service was IT security, over time it has changed, and you can now get information from other industries as well. The majority of the websites it utilizes to query the data are free, however some can have a little price.
   
   
   
2. Prowl Search Engine Lupovis Prowl is an both an API and a Search Engine that can be used to enrich IP data in an automated fashion, this is backed by a custom implementation of the Snare platform, which allows gathering of data on a massive amount of threat actors globally. By searching an IP address you can obtain indicators of intelligence, indicators of attacks and indicators of compromise associated with the IP address. It is a new tool, gaining traction in the intelligence community.
   
   Once the data has been enriched, the insights acquired offer valuable intelligence for any organization wanting to stay updated on user actions or vigilant against adversaries or intruders.
   
   
   
3. Censys is a platform for internet-wide scanning and data analytics. It allows users to search for and analyze data about internet-connected devices and the services and protocols they are running.
4. Wappalyzer is a browser extension that analyzes the technologies used on websites. It can identify the software and frameworks that are used to build a website, such as content management systems (CMS), web servers, and programming languages.
5. CheckUserNames, allows you to look for usernames on more than 170 social networks. This is especially helpful if you’re conducting research to find out on which social networks a person is using the same login.
6. HaveIbeenPwned (HIBP) is a website that allows users to check if their personal data has been compromised in a data breach. The website maintains a database of data breaches and the data that was compromised in each breach. When a user enters their email address or username into the HIBP website, the website searches its database to see if the user’s information has been involved in a breach.
7. TherHarverster Another excellent option is theHarvester, which can retrieve important data about any subdomain names, virtual hosts, open ports, and email addresses of any business or website.This is very helpful when you are just starting a penetration test against your own local network or against networks belonging to third parties with permission. Similar to earlier programs, theHarvester is a part of the Kali Linux distribution.
8. Shodan is a search engine that allows users to find internet-connected devices and the services and protocols they are running.
   
9. Nmap (Network Mapper) is a free and open-source network scanning tool that is widely used by security professionals, network administrators, and hackers. It can be used to scan networks to identify the devices that are connected to them, as well as the services and protocols that are running on those devices.
10. WebShag An excellent server auditing tool for checking HTTP and HTTPS protocols is WebShag. Similar to other tools, it is a component of Kali Linux and is very beneficial for penetration testing and IT security research.
11. Jigsaw Information on any corporate employees is gathered using Jigsaw. For businesses like Google, Linkedin, or Microsoft, where we can simply pick up one of their domain names (like google.com), this method works great since we can then acquire all of their employee’s emails on the various corporate divisions.
12. Fierce is a PERL-written IP and DNS recon program that is well-known for assisting IT security experts in locating target IPs linked to domain names.RSnake and other members of the previous http://ha.ckers.org/ originally wrote it. It is mostly used to attack local and distant corporate networks.Following the definition of your target network, it will conduct a number of scans against the chosen domains in an effort to identify susceptible network configurations and locations where private and important data may subsequently leak.
13. UnicornScan One of the best intelligence-gathering tools for security research is unicornscan. Additionally, it includes a built-in correlation engine that attempts to be effective, adaptable, and scalable all at once.
14. Google Dorks Google dorks are advanced search operators that can be used to find specific types of information on the internet. They are often used by security professionals, researchers, and hackers to find information that is not easily accessible through regular search queries.Some examples of Google dorks include:
    1. site:<domain> – search for a specific domain or subdomain
    2. filetype:<extension> – search for a specific file type
    3. intext:<phrase> – search for a specific phrase within the text of a webpage
    4. inurl:<phrase> – search for a specific phrase within the URL of a webpage
    5. link:<URL> – find pages that link to a specific URL
    Google dorks can be very useful for finding specific types of information on the internet, but they can also be used to access sensitive or confidential information that should not be publicly available.
15. Spiderfoot is an OSINT tool and reconnaissance tool that automates the process of gathering information about a target from the internet. It is designed to help users gather data about a target’s IP addresses, domain names, email addresses, and more.SpiderFoot is written in Python and can be run on a variety of platforms, including Windows, Mac, and Linux. It includes a large number of built-in modules for gathering different types of data, such as:
    1. IP addresses: SpiderFoot can gather data about a target’s IP addresses, such as who owns them and when they were registered.
    2. Domain names: SpiderFoot can gather data about a target’s domain names, such as who owns them and when they were registered.
    3. Email addresses: SpiderFoot can gather data about a target’s email addresses, such as the domains they are associated with and when they were created.
    Overall, SpiderFoot is a useful tool for gathering information about a target from the internet. It can be used for a variety of purposes, including intelligence gathering, reconnaissance, and security assessments.
16. Creepy is part of those awesome osint tools for social media analysis. It allows users to gather data about a target from social media platforms and other online sources.Creepy is primarily used for intelligence gathering and social media analysis. It can be used to gather data about a target’s social media profiles, online activity, and relationships. It can also be used to visualize data and identify patterns and trends.Creepy is available as a desktop application for Windows, Mac, and Linux. It includes a number of built-in modules for gathering different types of data, such as:
    1. Social media profiles: Creepy can gather data about a target’s social media profiles, such as their username, profile picture, and activity.
    2. Geolocation data: Creepy can gather data about a target’s location, such as the latitude and longitude of their posts or the location of their IP address.
    3. Network analysis: Creepy can visualize the relationships between a target and their connections on social media platforms, such as friends or followers.
17. Recon-Ng is a web reconnaissance tool that is written in Python. It is designed to help users gather information about a target from the internet, and it includes a variety of modules for different types of data. Recon-ng is primarily used for information gathering and reconnoissance in the initial stages of a penetration test. It can be used to gather data about a target’s employees, networks, domains, and more.
    
    Recon-ng has a modular architecture, which means that it can be easily extended with additional modules. It also includes a number of built-in modules for different types of data, such as:
    1. Employee data: Recon-ng can gather data about a target’s employees, such as their names, job titles, and contact information.
    2. Domain data: Recon-ng can gather data about a target’s domains, such as who owns them and when they were registered.
    3. Network data: Recon-ng can gather data about a target’s networks, such as IP addresses and DNS records. Overall, Recon-ng is a useful tool for gathering information about a target in the initial stages of a penetration test.
       
       
18. Grepp App How do you conduct a search over 500,000 git repositories on the internet? , you could use the unique search boxes provided by GitHub, GitLab, or BitBucket, but Grep.app works incredibly well.
19. Leakix is the a platform combining a search engine indexing public information AND an open reporting platform linked to the results.
20. OpenVas OpenVAS (Open Vulnerability Assessment System) is a free and open-source vulnerability scanner and management platform. It is designed to help organizations identify and manage vulnerabilities in their networks and systems.
21. Foca ElevenPaths created a technology called FOCA (Fingerprinting Organizations with Collected Archives) that may be used to scan, analyze, extract, and classify data from distant web servers and their concealed data.
22. IVRE Although often disregarded, this infosec tool can significantly improve your infosec discovery and analysis procedures. On the foundation of well-known programs like Nmap, Masscan, ZDNS, and ZGrab2, IVRE is an open source utility.
23. Maltego allows users to search for and visualize relationships between people, organizations, websites, and other public data on the internet. Maltego is primarily used for intelligence gathering and threat intelligence. It can be used to identify relationships between different pieces of information (i.e., a graphical link analysis tool), such as social media profiles, email addresses, and IP addresses. It can also be used to uncover hidden connections and identify patterns in data.
24. Exiftool is a command-line tool for reading, writing, and modifying metadata in a wide variety of file types. It is primarily used for working with metadata in image, audio, and video files, but it can also be used with other types of files, such as documents and executables.ExifTool is widely used by photographers, digital asset managers, and other users who need to view, edit, or manipulate metadata in files. It is known for its ability to read and write metadata in a wide variety of formats, and it includes a large number of built-in tags for different types of metadata.ExifTool can be used for a variety of purposes, including:
    1. Viewing metadata: ExifTool can be used to view the metadata in a file, such as the date a photo was taken, the camera that was used, or the author of a document.
    2. Editing metadata: ExifTool can be used to edit the metadata in a file, such as changing the date a photo was taken or the title of a document.
    3. Removing metadata: ExifTool can be used to remove metadata from a file, such as removing personal information and public data from an image or video.
25. Zoomeye is a search engine that allows users to find internet-connected devices and the services and protocols they are running. It is similar to other search engines such as Shodan, but it is focused specifically on internet-connected devices.ZoomEye is used by security professionals, researchers, and other users to identify trends and patterns in internet-wide data, discover new devices and services, and monitor the security and compliance of their own systems. It can be used for a variety of purposes, including:
    1. Vulnerability management: ZoomEye can help users identify vulnerabilities and misconfigurations on their own systems or on systems they are responsible for securing.
    2. Compliance monitoring: ZoomEye can help users ensure that their systems are compliant with various regulatory requirements.
    3. Research and analysis: ZoomEye can be used to conduct research and analysis on internet-wide data, such as identifying trends in internet protocols and services.
26. OWASP Amass is undoubtedly one of the best market-available reconnaissance and network mapping tools, and it was originally created by Jeff Foley.
    
    It is frequently used for network discovery, DNS enumeration, and general attack surface mapping tasks, with a strong emphasis on information collecting and data scraping on HTTP, SSL/TLS, and DNS protocols.
27. AllinOne Search engine is as it’s name indicate an aggregator of search engines.
28. Carrot2 uses machine learning to cluster information searched for across multiple sources
    
29. Waybackmachine – The Wayback Machine is a digital archive of the internet that is maintained by the Internet Archive, a non-profit organization. It allows users to access historical versions of websites and other internet resources. It keeps an active collection of websites.
30. SearchFTPs allows a security researchers to search through real time data across indexed FTPs. This is an excellent tool for ethical hacking and osting gathering data sets and gather personal information from publicly available data and potentially sensitive data.

How to detect OSINT?

Cyber deception can be used to identify someone conducting reconnaissance against your company. Cyber deception involves creating fake data or systems that are designed to lure attackers into revealing their presence or intentions.

For example, you could set up a fake network or server that appears to contain sensitive information, and monitor activity on that network or server to see if anyone attempts to access it.

By using cyber deception, you can identify reconnaissance activity and gather information about the tactics, techniques, and procedures (TTPs) being used by the attacker. This can help you better understand the threat and take appropriate countermeasures to protect your company’s assets.

Conclusion

Want to try one of the best contextual intel tools in the market?  Give us a shout ! We built it for InfoSec professionals who need to gather information about adversaries targeting them.

Or start using our free search engine today and our API to integrate your apps IP reputation and a de-noiser for your SOC.