The following text is a list of 25 well-known open-source intelligence (OSINT) libraries worldwide. I have personally used 11 of these tools and found them to have good information mining capabilities. I recommend them as worthwhile to have.
OSINT Framework, as the name suggests, is a web security framework that is a collection of OSINT tools to make your intelligence and data collection tasks easier.
This tool is mainly used for digital fingerprinting, OSINT research, intelligence gathering, and reconnaissance for security researchers and penetration testers.
It provides a simple web-based interface that allows you to browse different OSINT tools filtered by category.
It also provides excellent categorization of all existing intel resources, making it a valuable resource to understand what information you may have overlooked in the infosec field or for the next recommended OSINT steps in your investigation.
OSINT Framework is categorized based on different themes and objectives, which is easy to see when viewing the OSINT tree through the web interface.
Website: https://osintframework.com/
On the internet, websites are attacked and user data is stolen every day. This data often includes usernames, passwords (encrypted or even plaintext), email addresses, IP addresses, etc., posing a great threat to user privacy and security.
CheckUserNames can check the usage of usernames in 160 social media platforms and detect if your username, password, or email address has been leaked.
Website: http://checkusernames.com/
You can check if your registered website information has been leaked online. The website does not display your leaked information but provides details about the data breach event!
Website: https://haveibeenpwned.com/
BeenVerified is a website that allows you to search for background information on other people. By entering some personal information about the person, you can find out their family information and whether they have been involved in any illegal activities in the past.
Website: https://www.beenverified.com
Censys is a new search engine used to search for information about internet-connected devices. Security experts can use it to assess the security of their implementation, while hackers can use it as a powerful tool for reconnaissance and collecting information about their targets.
Website: https://censys.io/
BuiltWith is a website profiler, lead generation, and competitive intelligence tool that provides information about the technologies used on the internet, e-commerce data, and usage analytics.
It tracks technologies including widgets, analytics, frameworks, content management systems, advertisers, content delivery networks, web standards, and web servers, to name a few of the technology categories it covers.
Website: https://builtwith.com/
Google Dork is a fast scanning tool that enumerates websites.
Website: https://pentestit.com/google-dorks/
Maltego is an open-source intelligence (OSINT) and graphical link analysis tool used for collecting and connecting information for investigative tasks.
Website: https://www.maltego.com/
Recon-ng is a full-featured web reconnaissance framework written in Python. It provides a powerful environment for fast and thorough web-based open-source reconnaissance.
Recon-ng primarily plays the role of information gathering in the penetration process, but it can also be used as a penetration testing tool, although it has very few related attack modules and requires self-expansion. The biggest advantage of Recon-ng is its modularity, allowing for unlimited functionality expansion. With enough imagination, it can become a powerful tool.
Website: https://github.com/lanmaster53/recon-ng
TheHarvester is a tool that can collect email accounts, usernames, hostnames, subdomains, and other information. It gathers this information by searching through public resources such as Google, Bing, PGP, LinkedIn, Baidu, Yandex, People123, Jigsaw, and Shodan. This information can be extremely useful in later stages of penetration testing.
Website: https://github.com/laramies/theHarvester
Shodan has servers worldwide that crawl the internet 24/7 to provide the latest internet intelligence. It provides tools to answer questions about the internet on a global scale, such as who is buying smart TVs, which country/region has the most wind farms, and which companies are affected by Heartbleed.
Website: https://www.shodan.io/
Jigsaw is a tool for developing the US market. With this tool, you can search for detailed contact information for US companies, including email addresses, contacts, websites, and phone numbers.
Website: www.jigsaw.com
SpiderFoot is a free and open-source web information gathering tool written in Python. It supports cross-platform operation and is suitable for Linux, *BSD, and Windows systems. It also provides an easy-to-use GUI interface for users.
SpiderFoot allows users to gather various information about the target, such as website subdomains, email addresses, web server versions, etc. Its simple web-based interface allows you to start scanning immediately after installation - just set the target domain and enable the corresponding scanning modules.
Website: http://www.spiderfoot.net/download/
Creepy is an application that collects geolocation-related information about users from social networking platforms and image hosting services. Creepy displays the collected information on a built-in map and provides context-related information about the specific geographic location.
Website: https://github.com/ilektrojohn/creepy
Nmap, also known as Network Mapper, is a network scanning and sniffing toolkit originally developed for Linux.
Nmap is a network connection port scanning software used to scan open network connections on computers on the internet. It determines which services are running on which ports and infers which operating system the computer is running (also known as fingerprinting). It is one of the essential software tools for network administrators and is used to evaluate network system security.
Website: https://nmap.org/
Webshag is a cross-platform multithreaded tool for security auditing of web servers. Webshag collects useful features for web servers, such as port scanning, URL scanning, and file fuzz testing. It can be used to scan web servers via HTTP or HTTPS with proxy and HTTP authentication (based on authentication or digest authentication).
Website: https://tools.kali.org/web-applications/webshag
OpenVAS is an open vulnerability assessment system, which can be described as a network scanner that includes related tools. Its core component is a server that includes a set of network vulnerability testing programs that can detect security issues in remote systems and applications.
Website: https://www.openvas.org/
This tool is a comprehensive domain scanning tool. It can quickly obtain the DNS servers of a specified domain and check for zone transfer vulnerabilities. If the vulnerability does not exist, it will automatically perform brute force attacks to obtain subdomain information. It will also traverse the surrounding IP addresses to gather more information. Finally, it will segment the IP addresses for later scanning by other tools such as NMAP.
Website: https://github.com/mschwager/fierce
Unicornscan is a port scanner that obtains information and associations by attempting to connect to user systems (User-land) through a distributed TCP/IP stack. Its main features include asynchronous stateless TCP scanning with all TCP variant flags, asynchronous stateless TCP flag capture, active/passive remote operating system, application, and component information retrieval through feedback analysis.
Website: https://tools.kali.org/information-gathering/unicornscan
FOCA is primarily an information gathering tool that checks and scans files for metadata and hidden information. These files can be found on web pages, and FOCA can download and analyze them.
FOCA can use search engines such as Google, Bing, and Exalead to search and download relevant files. By stripping the information from these files, attackers can obtain the machine name, operating system, software installation path, and version information of the file uploader.
Website: https://www.elevenpaths.com/labstools/foca/index.html
ZoomEye is a search engine that retrieves network space nodes. It analyzes the global nodes through a backend distributed crawler engine (just like any other search engine) and determines the characteristics of each node, such as device type, firmware version, location, open port services, etc.
Website: https://www.zoomeye.org/
Spyse is a complete Data-as-a-Service (DAAS) solution developed for internet security professionals, companies, remote system administrators, SSL/TLS certificate providers, data centers, and business analysts. All Spyse online solutions are presented in the form of themed services, which together make up a platform with information gathering, processing, and aggregation capabilities.
Website: https://pypi.org/project/spyse/