Top 10 FREE OSINT tools (with demos) for 2024 - And FREE OSINT course!

everyone it's David Bumble back with Micah and Griffin guys welcome thanks David David great to have you guys yeah I'm really excited about this video Just For Everyone hasn't seen our previous videos I've linked them below free course that you can get from your website is that right that's right the uh introduction to oan training um has a huge amount of people that have started taking it to to get into the oent world uh it's free two hours of great content that Griffin and I made yeah we really wanted to be welcoming to the huge audience of people that are interested in open Source intelligence right and you know not not put that content in a place where you can't access it or you have to pay for it and things like that so this is out there for everyone to check out uh we really want to show you what's possible and get people interested and excited in the craft in a in an investigation at one point I was tasked with locating an individual from a series of photos and I had photos from an event and while I didn't have any information about who that person was I was able to take contextual information from that event and the other things happening there and locate a a Nexus of people that were associated with the event and through that research I was trying to discern who the person was that was in these photos with all these different people you know my my target of interest and as I started going through people's social media accounts and getting names of family members and Associates and spouses and working my way through that Network to try and identify this one particular guy I was having a hard time finding data on the person that I ended up feeling very confident was my suspect and the reason is because he had very little online digital presence and I wonder if that was by Design I'm not sure but in doing the the investigative steps that I take going through different search engine research and different online databases and things like that I was able to identify a social media account that had his given name on it and a username attached to it no other information no photograph nothing useful very Bare Bones and most people would probably close that Tab and move on I took that username and I prepended it to various gmail.com hotmail.com and things like that as I was checking to see a presence of an email email address associated with that person's possible username using the epos email tool I was able to find an email address associated with um accounts that this service detected by making that guess with his username and one of those Services had a profile page that had his photo on it that was a dead match for the guy I was looking for in fact he was wearing the same shirt in a surveillance image that we had so I was able to put those pieces together and connect the dots on him in a very roundabout way but knowing that a tool like epos could do this kind of search for me allow me to take my instinct my My Hope right that I might be able to guess this email address and do the work to find the associated accounts and make that validation really grateful that you guys have done that you know for helping the community so for everyone who watching link below for the free course and the other videos but I'm really excited about today's video we've been talking about this for a long time it's nice to talk about ENT but I think a question a lot of people have is okay what tools are they like practically you know is there something that I can start using today and I believe you guys have got like a top 10 or something Fantastic Tools to use in ENT is that right absolutely yeah so ENT is definitely enabled by tooling right and and real open source intelligence work involves the the analyst mindset and the intelligence cycle and sort of the collecting processing and analyzing of that data but the part that everybody really likes is the tools right you you want to you want to have that cool website that finds the piece of information for you or you want to have that awesome browser extension that you know creates efficiency in your work for some kind of compatable process and so when we put this list together we really tried to think about what's going to be accessible to all of you right everybody who's watching this video it doesn't do me any good to show you some awesome tool that I pay a lot of money for that you know it doesn't fit your type of work so we really focused on tools that are freely accessible um and we tried to cover a wide range of topics uh in open source intelligence as far as what their functions are and how they can help your work uh but we want you to walk away from this video with 10 things you can do today or try out in your open source intelligence work people need to understand what a tool is in Ence I know that coming from cyber a tool was a script or an app or something that you ran on your system and tools in in ens can be like that too but in general a lot of them are up in your browser uh it might be a website that you visit or it might be a browser Plugin or add-on or extension like Griffin mentioned so please understand that that uh osen Tool uh goes by a lot of different names and we're going to show you mostly ones are up in your browser with a couple of exceptions and uh they're all free even though Griffin and I do use paid tools some amazing paid Tools in our daily work so that's great I mean I think it's important to differentiate between like people who are studying or starting out and people who do this professionally so it' be great to hear like I can Sol with this tool but then also give us like the the nuclear option for lack of a better word you know like what you guys really use out there so I'm going to keep quiet now you guys take it away all right so our first tool is going to be um this is our number one tool and partly it's because of of the usefulness in the field and partly because I helped make it um this is what's my name and if you go I know you got to promote your own stuff right David of course so this is I just say this for everyone who's watching sorry to interrupt Micah Micah is way too humble he like comes across as this humble guy but I this mic is amazing and it's written a whole bunch of stuff so cyber background you've done coding you've created all these amazing products that people can use for free so I'll just say that M I got to plug your stuff because you're way too humble my advice to everyone is you know don't make M or Griff upset because they'll find you um just like in the movie right but what I do have are a very particular set of skills I have certain set of skills David yeah absolutely well thank you thank you yeah um what's my name. app was originally created by me as a python script and it got like 50 people using it and then osen combine Chris polter did an amazing job making and hosting this website and now it gets 50,000 views a day it's a free tool and it's amazing the the simple uh way that you use it is just take a username or usernames that you are are looking to find where they are on the internet and you put it into the little field here so I'm going to just put in here mad for miracle Max just this random name if I wanted to put other ones I could um it'll do multiple at the same time very very important piece of it is this categories and filters we've got all of the 600 plus sites that this will check through your browser categorized so if you only want to do a subset of that like coding ones or or uh political sites you can select that or if you want to do all of them including the ones that are not safe for work choose all we do that because some of the sites that this tool checks uh are pornographic in nature there's PornHub and x hamster and other ones because some of the places where people have accounts that we might need to find for missing exploited trafficked people maybe on those types sites so yeah all we do is we hit the green button here and then let my browser do the work my browser is now visiting 629 different sites and it's found 16 possible hits for the uh username mad for miracle Max the way this tool works is really simple it just takes those URLs like twitter.com mad for miracl Max and it puts that username in the URL so if you have a tool that if you have a user name that would go into a URL like for Instagram or for GitHub this tool May check it and it does it in your browser and we do not collect any of your data sorry I just wanted to say because you said something and I want to make technically sure about it did you you said it it uses your browser but if I am using this tool does it mean I my the traffic at my home is going to all these dodgy websites or is it it's actually I'm just connecting to your website and your website is doing all those connections nope it it's the it's that your browser is making all of these requests and that's why by default we keep this as all excluded cluding not safe for work so you're if you just come here and you put in your favorite username or your username it will not visit um all of the not safe for work sites it will visit a bunch of other different sites from your browser um and that keeps it it easy for us because we don't have to proxy 600 requests times x number of users no I'm glad you said that because it's it's something that people need to be aware of because if they're at work you don't want to you don't want that in the logs right oh absolutely you don't want that network security person coming knock on your door saying hey why are you visiting all these sites and be like I'm not I'm just running this oh yes the nice thing about this tool David is that the output is made for open- Source intelligence people or cyber people um here's all of the positive results that came back and what we need to do is visit every single one of these and see if that person who has a profile at like BuzzFeed or let's say GitHub or let's see there's a keybase in here there's even some like the Twitter archived here let's see that one so all I do is I open each one of these off and I look at the next profile and I see hey is this my mad for miracle Max is there anything that ties this person to my target person could be the profile picture or the bio or something else and in many cases we there we see the profile picture again so these might be related him his biggest fan we have an air we have a a name of a of a city but it's wrong so that might be important here we have a a link to his website so what we do in ENT is we will oh this one has a person's name Gavin interesting um the idea is that we find all the places where this person's mad for miracle Max name is and then we gather it up and do some more queries like here we see a Twitter account a GitHub account we see a device um and in some cases we can even go back in time like this one here it looks like on archive.org there's one hit yeah I'm glad you mentioned that because I mean you could run this against real people right and a lot of confidential or personal information can be found yeah absolutely this is one of the this this username enumeration is one of the ways that Griffin and I and ENT people all over the world are finding people is that we run their usernames we pull up other links to them and then we follow those links um in this case here in archive.org we see an old tweet that mad for miracle Max made and there's his picture as well um and so we can take this data and then do other things with it and was amazed when I talk to you guys and other people in ENT how easy it seems I mean obviously you guys are experts and you've done lots of work to do it but like how easy you could find like stuff that that's kind of like you wish was hidden well and David that's that's what we teach in our classes all the time we teach people how to bring order to the the chaos of everything online how do we find it and how do we look through it and a lot of it's not the finding it it's how do we look through it and how we figure out if this is relevant and related and that's where tooling comes in so handy because you first get that kind of spark that idea that you know hey mad for miracle Max has a profile on this website I wonder if they are mad for miracle Max in other places and you could manually go out and do that research and maybe use search engines to be more efficient right but what if someone built a tool that would do all that searching for you and of course Micah does that because that's what Micah does well and the other thing is honestly how many times have youve been like I know I had an account on I think Myspace but I forget what it was and you enter in try to find your username here if you just put in this tool it'll find a lot of your accounts and actually that's the first thing that that I tell people who are interested in finding their online personas and finding what their their what's online about them search for your usernames because in this tool at the bottom of the page it also runs a Google search with your username with an username that you put in there and does some other things so you can really help find all of the the places that your username is finding I even found somebody else that was using my web Reacher username yeah very frustrating our second tool is going to be the internet archive and Micah has shown how we've gotten a result there based off of pivoting from a username using the what's my name Tool our first favorite tool now we're in the internet archive the Wayback machine archive.org it goes by several different names that's our second tool that we're going to show and so you can see here that um Micah showed a result for the mad for miracle Max Twitter account that was grabbed by the Wayback machine and if you look up in the top left part of the screen there you can see that it's grabbed that particular URL for that status but let's talk about what the Wayback machine does the Wayback machine is Archive of web pages um different types of software projects books and all kinds of other things and from a research standpoint in open source intelligence it's a potential Gold Mine of previously captured information about websites and we use this all the time to explore things like um the profiles that Micah mentioned if those profiles are accessible to the crawler um from the Wayback machine uh or for example if you do research into a company um and so on the homepage here you have the option to type in a URL or words related to a site's homepage as it says and it's going to go ahead and check their archive to see what they might have available for you and I've completed a search here for a site called infos cis.com which is a major company with a longtime web presence and so if you were tasked with exploring the the web presence of this company or maybe um you know that that company has uh changed uh leadership over time or maybe it's a personal website and and you're hoping that they leaked contact information earlier on in the in the cycle of the web page something like that you get this amazing calendar of all the years and all the times that they've captured those pages so from a research standpoint I could spend a lot of time going through these old captures and this is obviously an example with a lot of content and I could look for those uh moments where maybe they they had information on the page relevant to my investigation or it was different than it is now and that's useful for the question I'm trying to answer um I love that they put it into a calendar like this so that you can get as Gran as the day look at multiple captures and see how those things compare now if we wanted to go back in time for example let's say 2001 I've got that pulled up here and you can look at captures that happened in that particular year maybe that's of interest to you because things have changed with the company around that time or you're looking for information about a previous person who's associated with that company and clicking on one of these captures will actually take you to what the Wayback machine grab at that exact date and time and so this is an example of what the infosis website looked like back then and you may see different features on here from before different information on uh on the site as before like I mentioned this is a really great and powerful tool just for this one specific use case that's very powerful and they also make it easy so you can see this has 976 captures which is quite a lot of data they actually will give you if you switch over to changes instead of that calendar view that we had the first time you get a year-long Archive colorcoded of how different the site was each time they captured it so they're measuring the difference looking at that differential so maybe you have tons and tons of content over the years on this site and you really need those major changes maybe you're trying to prove that you know this company used to be branded as a different company and they've completely Rewritten their online image well guess what maybe that happened you know round about this time here where you see the darker color blue massive changes to that website so from an investigative standpoint I'm very interested Ed in what was happening around that time on their site and we' used this in many cases to to look back at how things were even a day a week a month ago it doesn't scan the entire internet but it does capture a huge number of sites on the internet so um there was a a murder that was being investigated I didn't investigate it or neither did Griffin but there was a murder that was being investigated um and the uh potential murderer was on the church board and associated with this church and the church website stripped all of that off and was like no he's not a part of this and you could literally go back one or two days and see well it used to look like this on your website now it looks like this you definitely changed it so really helpful for going back in time so the third tool is going to be search by image now this is a plug-in for your browser and what it does is it takes an image that you want to search for see where it was used where else on the Internet it's been and it sends it off automatically in your browser in other tabs to a whole bunch of different image recognition sites or image reverse image search sites and I'm going to show you that here and then we'll we'll take a look at the tool itself so remember this from one of our other favorite tools if we right click and open this image in a new tab you see this is some kind of baseball stadium maybe you're interested in finding out what baseball stadium this is and so you right click on it and you can send it to Google Google if you want to search image with Google what we're going to do is search by image and then we're going to search all the search engines that I've configured it to do so when I click this what it's going to do is it's going to open up five new tabs here and take that picture URL and send it off to Google and Bing and other ones and here we see Google has already recognized that as orial park at campden yards over in Baltimore Maryland and that's exactly where the picture was taken the other places like being here it doesn't actually find it uh Yandex which is a Russian search engine does say Orioles wallpapers so this is very similar to other things this gets into how do we do things more efficiently you absolutely could copy the URL to this image right like that copy that and then open a tab and go to Google and put the URL in then open a tab and go to tinai and Yandex and other places but the reality is is that if we can use this tool to send it to a whole bunch of different sites that saves us a lot of time and then we focus on the analysis so here this is shutter stock the stock photo site you can see that this looks very similar to this right and so if we go back here um you can see this is the right field line at C Camden Yards if we click there um one of the best things that I've used this for in cases is something that I always encourage people to do when they're getting a new tool check the setting what can you do with this thing um for instance if we come over here to this little red camera that's what it looks like when you put it into your Chrome or your Firefox even Edge and Opera you'll see this little red camera go to there options and look at this we have over 40 different websites that it will send data to so when I'm looking up some picture of a canal over in Asia I'm going to probably turn on like Buu and soou and some of these Asian focused search engines that have that type of imagery um if I'm looking for more a brand a picture of a of a something on somebody's shirt a logo or something I might enable some Getty and iock and Adobe stock photos or I could even go down to like the global brand database trademark view Australian trademark search because people submit logos like this to those places and we can search that using this tool for free so that tool is it can you just mention the tool how do I get it ABS so this tool is search by image and search by image is available as you can see on this GitHub page you can get it from the Chrome Store Firefox store or whatever so you just in your browser you just click and visit the Chrome web store and then in here or in Firefox store we just type search by image and then we look for that camera right here by Armen Dev um and that's what we would install so very simple easy works with whatever browser you're using for ENT and free you prefer Firefox right I prefer Firefox but I use Chrome a lot more than Firefox for open source intelligence and there's there's a really good reason for that is a lot of the really cool tools like one of the ones we're going to show you in just a little bit only works in Chrome so can't use Firefox so I'll use Firefox I'll have Firefox up and running I'll do my my research queries and stuff but all my ENT I'm doing in Chrome or chromian based browser you're basically geoca an image right is is that what you're doing there well we're finding that Associated data for the image it might be geolocation it might be who owns it it might be um other things associated with it where I can buy that product uh Griffin had a a case that he worked on where he W had a picture and it had a whole bunch of different uh products in it household products like cleaners and other stuff and we use reverse image searching to find out what stores sell that product and then we can based upon that maybe identify what area of the world that that picture was taken in or um where somebody shops because that might be important also to a case yeah I used the search by image tool recently for an investigation where we had a a missing child and um through some social media research I came across a series of photos that depicted people that were on a sports team together and I was trying to localize that sports team to figure out where they were and and see if that was going to be a friend network of the person that I was looking for a social media presence of and use the search by image tool I was able to um quickly search a number of different uh services for that the image that I had but crop it down to the logo on the team shirts and be able to identify the the team and then subsequently the location and the town that they're from which helped me in social media research to connect those dots between the the people in the photos and the person that I'm looking for and obviously other things helped you find the the missing person right exactly yep those those types of tools can help to when when you're investigating getting a missing person a lot of times finding their presence online can yield helpful information whether it be active online presence that maybe people don't know about um additional accounts maybe under under Alias names and things like that or Associated persons or locations that might help you um maybe catch up to where that person is depending on the circumstances of their disappearance and all of those things can be enabled by doing reverse image searching and really the search by image tool creates that efficiency that we talked out about at the beginning where where you have to perform a task like reverse and image searching but you need to do it repeatedly over lots of services search by image does that at the push of a button all right ready for number four yeah that's great this is brilliant all right so the fourth tool that we're going to uh talk tell you about that can help in your ens is a little bit different it's not one that's going to go out there and get information but it is absolutely one that's going to help you organize the information that you have to collect open source intelligence the the flashy tools are the ones that go out and grab data and organize it and sort it and show it to you this one is one that's going to help you take notes um it's one I feel very passionately about it's called obsidian and you can get it for free at obsidian DMD uh obsidian is essentially a tool that allows you to take notes on your computer it saves everything in text files or image files or PDFs whatever it is that you're putting in there and it allows you to see Connections in that stuff it's kind of like multi it's got multigo functions it's got uh kind of VIs functions but it also has word functions and other things and it's all in in this one tool I could take literally two hours to go through this tool because I've done that I have a video of myself going uh uh showing how to use this tool I've also created some resources out there some free osen templates that that people can download and we'll show share those links with you but essentially you have text documents all of these things over here are text documents and then you can edit them hyperlink them all using a technology called markdown the great thing about this for using with ENT is something that we can see right up here in the example case many times what we have is we'll have a primary person that we're researching and that person will have a car associated with them or we work at a certain job or we'll have other people they're associated with and trying to keep track of all of that in just notepad or word pad or or even like Co vs codium or something like that or word it's it's linear it's sequential and it doesn't represent those relationships really well obsidian definitely does that because what we can do is have a page for Alister Kemp this madeup guy I I created and then we can say hey this person is linked to Gabriella Parkington who has her own page so as you start creating these other entities we create another note for them and those other notes then contain data just about that person now once I was doing when I was doing uh hacking website attacks and stuff this would have been incredible because you know you scan an entire subnet and then you find certain hosts those hosts have certain vulnerabilities certain opportunities to attack them or to test them here we can actually separate each one of those into their own note then note the exact things that were wrong how we went ahead and exploited them and how the customer can fix it so this is a general note-taking tool that has some really cool built-in features for instance since I've linked uh Gabriella Parkington and Alistar Kemp together by by by this method right here just showing their their notes are connected I can come to the built-in graph View and we can see that these notes are touching somehow now we can do this in other ways too think about this you are looking at a subnet of computers and you've noted on each one of them that they're hosted in a certain place or owned by a certain person or all are tagged to the certain dark web onion address well those types of of patterns and connections come out in this tool really really interesting and this is just the tip of the iceberg I use this every single day for taking notes for meetings for connecting ideas and thoughts within my daily tasks and to frustrate Griffin because he doesn't like using it that much obsidian has so many Advanced features and plugins available um that it really can be a little bit overwhelming on the surface and Micah does have um both a free and paid course um regarding how to use obsidian or getting started with using obsidian but you can kind of customize it to your heart's intent I do use obsidian in my in my stuff as well you know when it comes to investigative research you have to have a professional way to collect and organize your notes if you plan to do professional ENT work that's just the bottom line and this is a free tool with many professional features that anybody can access so that's why we like to recommend it one of the things that I constantly hear is I would love to be able to have my standard operating procedures for what I do or what my team does uh when I have an email address or a phone number you know there's there's a process that we generally go through to find out where it's being used who owns it I would let people tell me I would love to have that closer to where I'm recording my stuff with the way that I've built this free ENT Vault um for obsidian you have that right here so we have the standard operating procedures here that says hey for a person's name make sure to go and search for it in a person's search in the people search engines use that's them and true people search and so when we create these these things we can say hey make sure to do these these these things by going to this part of our standard operating procedure so imagine that we have three people that are working on a Case whether it's investigating a domain or a person or a business they're all working from the same standard operating procedures that is a live document inside of your Vault it's really powerful for standardizing Content too okay but hold on a minute so there's obsidian that's that the tool that you would use but you've created this Vault thing is that right yeah that's right um so what it is is Obsidian saves all of the files that are associated with whatever you're doing into what they call a vault and a vault is just a folder or directory on your computer I've chosen to take that vault which has all of the settings for this all of the plugins and extensions and has all of this content and I've zipped it up and put it onto a GitHub repository that's on my github.com we breacher and it is the obsidian osen Vault templates and I'll give you that link people can download that Vault onc compress it and then just point obsidian at it and it will go ahead and open that up and they'll be able to use this sample Vault yeah because the problem I'm seeing is okay you've shown us a few tools already it's like okay how do I even start and I mean that's great right because you've given us kind of like a road map or a path like do this do this do this do this right yeah and and that's the thing is that these are our favorite tools um when we came on we told you it was our favorite tools and and and some of these favorite tools are used in certain places and not in others with like obsidian I'm I use it literally every single day for ENT for other things and and some of these tools are really easy to use like search by image you pop it in your browser and you're going obsidian I will warn you does take a little bit B of time to to actually like get into and learn there's a learning curve there um and this is my uh web Reacher GitHub there's obsidian ENT templates right here and if you come here it tells you all about uh how to use this what to do I've even got a blog post a YouTube video and other stuff to help people get into this and as Griffin mentioned um myos and training has an obsidian course that's pretty darn popular that teaches people uh obsidian in Frozen so the the YouTube video is free and then that's a paid course is is is that you got it YouTube video is free paid paid course down here yeah I I think it's it's important to differentiate this just for everyone who's watching I love it that Micah and Griffin have given us like the free stuff to get started with um and it's you know if you're a student or you're just interested you don't necessarily have the money but then you're also giving us like the professional version don't be shy to like recommend like professional tools the one I'm thinking about is um Justin's hunch Hunley is it Hunley Hunley yeah yeah is that is that kind of the same thing or is that more right yeah it's a paid tool and it does a lot of really good thing I think I would more uh reference it to like burp site Pro and burp site uh with they have a free Community Edition you get hooked on the Community Edition and then you see all the power in the Pro and you're like all right one day I'm gonna have my employer buy that for me and then you get the license and now your capabilities have skyrocketed same thing with these things Griffin's gonna be talking about a tool some tools later on that have a free version but then if you pay them you get even even more so yeah y that's a great way to approach learning about these different tools and it you get a chance to experience the use of it see the capabilities of it understand how it's going to enhance your work and your capabilities and then you or your employer you know the people you work with can make that buying decision about do we upgrade this do we go to a PID tool the reality is you know in these types of environments the folks that are making the bigger and better capability tools and adding in data repositories and doing all these cool uh features an enhancements all the time that cost money so they have to charge for their products eventually right our number five tool here is urlscan.io and we could probably do an entire video about all the capabilities of URL scanio we're going to keep this focused on osen practitioners okay there's lots of capabilities Beyond just what you could gather from an Osan perspective but this is a site here where um people can have uh a domain scann a website scanned and look at all kinds of different functions and features and Technologies and things things related to it so let me tell you about kind of how I use this uh there's lots of times in my investigative work where I come across the URL maybe it's part of the evidence in a case maybe it's you know the initial contact between a scammer and a person that they're trying to extort uh or maybe it's uh it's left over in an archive of something that we've scraped that we're trying to use to investigate a a person who's gone missing or something like that I never go and visit a website just right out of the gate by clicking on the link or typing it exact to it never I just never do that right and I've got to have ways to go and sort of remotely visit that site ahead of time so that I can get a sense of is this going to be okay I have lots of tools that do different versions of just that type of thing but urlscan.io offers some really complete capability if my investigation led me to a to a URL that was my ent. LLC and I was curious about that site I needed to go there and visit it but I don't want to actually touch it I can use this site to go visit it now before I perform this search I want to stop and give everybody a warning okay what I'm doing on here right now is publicly viewable everybody's going to be able to see that I scanned my. LLC so if you don't want that to happen this is what you need to do when you have what you're going to search here you have some options you can see it's defaulted to private scan sorry I have it set on private scan right now if you click on options it defaults actually to the public scan I recommend that you change it over to private if you need to so that other people can't see it now there's other configurations here that you can play with you can change the user agent and so on but we're going to stick to the very simple basic use of this I want to know about my ent. LLC and I kind of want to look privately and I'm going to solve a capture here because what's better than solving captures on a live demo all right so I've performed the search and it's taken me to this now if you look I'm not on my. LLC it's actually detected that there is a redirect sequence here where I've submitted the URL my. LLC and the effective URL the place I ended up is my ent. trining so this is another Safeguard feature that I'm using a URL scan for and that is to make sure that I'm not being redirected to someplace malicious down here you can actually see the page URL history of the Hops and where you're going um and I get this helpful screenshot right here I can click on it and it's going to give me a a full page screenshot to look at what's right there on the homepage right with they right what they've captured and I can see that here in the browser without having visited that myself now this might be enough information for me to know that I can proceed to that site safely you know maybe I have to do additional research to look at what technologies are running or what strips are running on the page and and that goes beyond the scope of really what I want to talk about here but um I've now detected that I'm being redirected and I've been able to put my eyes on that front page of the site so that I can see sort of what the content is right now there's a few other useful features here you can see that this particular site has been scanned nine times on URL scan. IO now similar to how when we look looked at the Wayback machine as one of our earlier tools you were able to see previous captures it might be useful for you in your investigation to see those previous captures on URL scan of this particular um URL right um and you can do that you can search that if you go to this search icon here at the top of the page and perform a search for my .t trining you'll actually get a list of all of these different times that my ent. trining has been captured but if you notice here it's also capturing subdomains okay yoga. myos and. trining right so this can also help you expand your research into the subdomains of particular site um maybe see things that you didn't already know about right that could have been hidden information see how far back people were looking at it and look at those changes over time really from an osen perspective just these very basic features of urlscan.io are super helpful to someone like so uh for an osen perspective many times what we need to do is actually analyze a web page like break it down what's running this where is it running this in a single query has given us all of that you see over here we've got IP addresses we've got Cloud flare Amazon Google you can see where the parts of this website are being hosted and if you need to send subpoenas if you're law enforcement or you just need to understand is this going to be inside or outside of my scope for uh some type of penetration test or security testing you've got a lot of that right here and it hasn't been done from your computer the urlscan.io system has reached out to this Plus on the right hand side you see detected Technologies so if you know that hey you you're going to be working on mix panel and jQuery and recapture you can create you can gather those tools so that you can do your assessment even faster me it's also great that you you don't go to some dodgy website and get like malw or something installed stuff like that is that right oh absolutely yep and in fact one of the things that this will do I is um farther up uh this site absolutely has no malware because it's our site but it will actually oh wow okay yeah it'll run the the URL through things like Google and other places that have that block list of hey this is a known malicious site or a known malware hosting site and it'll flag it right there in that page and let you know this is fishing or something else like that yeah like I was saying earlier mic you guys are way too humble right and you create all these crazy tools well we I would love to to claim that I created this stuff but um these tools are are just ones that we've collected and our number six tool is another one focused on website and infrastructure and that's going to be DNS dumpster.com dnsdumpster uh is a site that's going to go out there and check the structure of a site grab all those interesting nuggets that we want and give it back to you in a very readable form so I'm going to look at a site here called madfor miracl max.com if you remember earlier in the video we looked at the username Miracle Ms for miracle Max we've maybe discovered that person has a website and we want to take a look at what's going on in that particular site we run that query for mad for miracle Max we get some information here about maybe the hosting locations right could be indicative of where they're coming from the world degree you can see their DNS servers you can also see things like their MX records are they running email Services through that website and who is the company that provides those Services the most interesting part A lot of times is right here in the text records now if you're going to look at mad4 miracl max.com you're going to find some interesting text records right this first one here reach out to Maxwell at madr miracl max.com if problems sometimes you might find people leaking information contact information email information things like that this here looks like some base 64 encoded text maybe you have basic4 decoder you could run that and find out if there's a message there for you lots of different things here you can find and validate that maybe they had Services running on those sites that required a text record at some point you can see apple domain verification Google verific a and so on and that can help you understand the capabilities and Technologies I like this kind of thing because when um I'm investigating a more organized Network that may have multiple websites set up to run maybe a fishing campaigns or other things a lot of times they're using common Technologies across those different sites and you can detect that okay there's a lot of similarity between the stack that goes into this website and this website and this website and really it's starting to look like it's all the same kind of network also from a cyber perspective uh we know a lot of you out there are probably either coming from cyber or currently in cyber this is one of those One-Stop tools where you do one thing the tool does a whole bunch of things faster than you can for instance if if we wanted to do DNS lookups we could do that there are special tools that allow us to do that we could do command line tools or online tools if we wanted to do DNS enumeration or passive DNS checks we can do that as well but this tool with a single entry does all that for us for free and it's so quick I mean this is an example from a a domain called panerabread.com it's a restaurant chain over here in the United States just to give you an example of what a a professional company and not a capture the flag endpoint might look like here we've got the DNS servers we've got some mail records there that tell you where they are getting their mail you can see that on the right hand side ironport system so this company uses ironport if you're a penetration tester or you are somebody that is doing reconnaissance or or ENT on a certain domain and you can understand that the mail the MX records for that company are being sent to or that mail is being sent to ironport systems then you know that you're going to have to avoid iron ports filtering mail filtering if you have to send a fake fishing email to test their systems or something but Griffin mentioned the tax records if you scroll down here we see that Panera is connected to Google that's no that's a no-brainer but they also have Facebook at lassan who makes conf influence jira we even have some other things in here like there's a Dropbox domain verification so just with this information here the text record crazy we start to put together a picture of what external services this company may be using I say maybe using because this could be really old or it could be for future stuff the David the really cool stuff some companies make their a records their IP and host names really re verb so if we scroll down a bunch further so here we start seeing some of these really interesting host names on the left ftp2 any connect to David I bet you know what any connect is yep yeah so if I was trying to Target this company's external interfaces and I needed to know where their company VPN endpoint is there I have a host named any connect 2 and it's on AT&T's network so in just one query of panera.com I get all of this data for free it's an incredibly powerful tool that saves me so much time and that's what I love about you guys doing these videos it's people aren't always aware companies aren't always aware of how much is available out there and how much is leaked I love that these tools expose not only the technical pieces of information but also for me anyway they spark that sort of creative curiosity where I think hey wait a minute how can I connect that to this other piece of data or what might that lead me to or what pattern is that showing me if I look hard enough and that's really the the power of Open Source intelligence is this data is all out there and it means different things to different people to an investigator it might mean something totally different than a person who's very um Technical and looking for a different you know use case and not only that going from cyber to ENT it pushed me from command line and and scripts all the way up into the browser and so you know at first I was like no I'm going to continue running my GPS with a really amazing you know pipe string but the more more I'm I'm recognizing that these tools are available the more I see the value in them I mean you absolutely could run some kind of script or some kind of tool on the command line to dig these different subdomains and domains and then grab those and recursively check them out but this tool does it for you in your web browser and it's so much easier to just offload that Mike you're gonna be in trouble now with like the most of the audience because you know if you don't use C you like a reject right yeah you know when I first started doing uh open source intelligence I I injected a lot of command line stuff GPS Cuts SS ax into my my course and I I recognized that when I had moved away from cyber students and cyber um people uh the law enforcement started coming in uh normal people that are not cyber came in and they're looking at these GPS and SS and a and cuts and and they it was just horrendous so I embraced the browser based tools and I love CLI as well I mean it's like use the best tool for you for what the job right so use what makes you most efficient oh absolutely and there are some python tools that do an incredible like across the board job uh it's just it's much easier to just pop open a browser type and fill in a field than remembering python 3-h host equals you know all of that stuff yeah the output is often a lot more um more digestible and presentable yeah all right let's move on to our next tool this is tool number seven tool number seven is a browser plugin and earlier I mentioned that I use Chrome and chromium based browsers for more of my ENT work because there's some tools that are amazing and they only work in Chrome or chromium based tools instant data scraper is one of those instant data scraper you can literally go to the Chrome web store as I've done here and install it in your Chrome chromium Brave Microsoft Edge browsers that are all chromium based and what this tool does is just just makes again makes life really easy and I'm going to use uh your video for an example here David one of the things that we commonly have to do is is scrape contacts or or scrape friends list or activity or feeds um and to do that I mean it's it's incredibly painful to scroll down take a screenshot or copy this with my mouse and then scroll down more and do that so what I can do is I can scroll down a page and just do just grab have my my browser load a whole bunch of the comments and then hitting this little it looks like a Pokeball to me I don't know how they're getting away with this with the copyright things I hit the Pokeball the tool is in my Google Chrome and it's now checking the HTML on the page looking for tables it's pulled a table here um and what it will do is it'll pull tables I'm going to try another table you can see this it's found this table here so if I wanted to download all of those next set of videos I could do that I'm going to have it do another one let's see oh and these are the comments now so each one of these is a row in the table and it looks like I've got all of the different things from who made the comment to what the comment was URL yeah and then David we just download it to XLS and when we do that we come over here and here's all the comments for free w yeah plus the tool has an amazing ability to scroll down the page automatically so we can for those longer pages that are on like Twitter or YouTube whatever it'll scroll down recognize more things it needs to download it'll download those and scroll down and do that again and again again it can even hit the next button to go to the next page of search results really cool David I'd like to say that our next tool number eight is a new tool that I'd like to introduce to you but the reality is we've been using it this entire time it is hasm yeah you knew it right I was going to ask you about them so that's great tell us about it yeah so um many of you that are in cyber security recognize that we use Virtual machines for a lot of stuff right we we have our hacking machine and then we also have our our virtual machine that maybe is our Target and and we have uh them separated and we can set them to be reverted so that we can do whatever we want and then it reverts back to a known good State and that's all on your system with ENT that works a good amount of time and there are some really good OS VMS but what I'm finding is having that overhead of buying and running of virtual machine software or just using free ones like virtual box it can be expensive both in the resources on your system and in the funds that it costs for a VMware or something like that what about leakage you know that's another problem people always worry about like if you download some trash on that VM it might get into your actual system might not be isolated like you think it is yeah right go having that isolation is really important and I was just going to say that in the ENT world having that isolation from me my computer my browser my IP address really really useful and so this entire time whenever I've been giving a demo I've actually been using the chasm workspace to do that and so this is inside of a browser you can see up here I have myos and trainings this is my Chasm uh interface and I'm running a new buntu Linux desktop computer in my browser and so I can do I've done all of the demos I'm running an a spreadsheet here I can use all of the the features of this upload and download files and it's all doable and then when I log out I can since I have the paid version here I can set it to either actually when I log out it may keep my changes or it might not depending on how I started this I started this to maintain all my changes that persistent mode um and it's is really really helpful in my investigations it's like a doctor container or something right it is yeah it's a set of doctor containers and the workspace the cool thing is that we can go ahead and go to other workspaces so while that Ubuntu desktop is running right now it's it's running and doing whatever it is it could be uh running some scripts for me or just paused right now I can go ahead and launch a persistent or non-persistent Google Chrome browser and this is being launched on Chasm servers and here I have a Chrome browser so I can surf whatever I want I can go to whatever site I want the IP address that I'm using is Chasm server this browser if it gets infected with malware when I delete the session and trash it none of that comes back to my computer at all it just deletes it um it keeps me safe allows me to not use my home IP address or my work IP address for my work and it allows me to do a whole bunch of stuff in my browser so if I just had an iPad or or a a mobile device I still can do things remotely and one thing I noticed is are you and Griffin using the same device or how you doing like is you're flipping from one to the other it looks like so uh we actually were going to try to do that David where we can share a workspace but we didn't work it out in time so we actually have yeah this kind of thing is becoming really important because it's like when you get an email or something I mean you get paid Solutions but you just want to be able to click on something like this where it's totally isolated so if it doesn't get infected you don't you know doesn't doesn't affect your computer so I think this is fantastic it's it's really interesting to see you guys using it with ENT yeah and and you know David there there's that maliciousness but there's also just suspicious sites out there you never know what you're going to get when you go to certain sites so being able to come over here and to to our Chasm instance and just do whatever it is that we want without having that fear of oh it might infect my company's computer or whatever really really useful um and I should mention that Chasm has both paid and free system so if you have no budget and you want to try it out go over to Chasm web and go to Cloud personal and then down here it tells you all about it and all about the features and stuff that are involved with it you can just sign up for the free version the free version does not have some of the features I showed like the persistence and other stuff but it is free and it gives you that virtual browser in the cloud uh one of the ways that I will use instant data scraper is when I have a a great deal of results um and things that I want to capture and analyze so this is an old blog that I wrote about a a series of fictitious profiles on LinkedIn that I determined to be connected based on the context of what they placed on each of those profiles so um I noticed I was being targeted by a number of fake profiles that had you know Gan generated imagery the the generated ADV serial Network photos of fake people but as I examine the profiles I noticed the consistency in things like biography language and and other text based things and I thought I wonder wonder if Google has scraped those results in a way that if I run queries using Google Dorking to kind of isolate those particular profiles in the Google results then I could scrape it and what I found was running sections of those bios that they were constantly repeating on this whole Farm of fake profiles was just giving me hundreds and hundreds of results and it allowed me to do things like create graphs of the different types of profiles so I could see what the proposed backgrounds were of these people that were trying to do fake recruiting and and things like that so just a lot of U in that tool and it's all about the creativity and how you want to use it there are loads of different ways that you can approach open source intelligence Research In some cases you might be looking at a company business entity or a person connected to a company and things like that and open corporates is is one of those repositories that allows you kind of that that Peak behind the curtain with who is involved in a company maybe where they're physically located it also helps you make those connections between maybe businesses in other parts of the world um or other parts of the country and other associations between the officers and people when you land on this page it gives you the option right off the bat to search their 222 million companies that's a lot of results right you can also search officers by switching over the search right here 311 million officers so they have a massive data repository um of stuff that you can search those officers are the people that run the companies not like Law Enforcement Officers just to throw that out there exactly the the people associated with the company so if I'm going to look for an officer for example Elon Musk and you can filter it by different jurisdictions if you really need to narrow it to a country you're going to get a results page that looks like this and these are all the corporate um entities that they've found Elon must be connected with as a company when you switch over to officers you can run it as an officer surch and so you kind of get that that either or look at the data and how they capture you can see in the name of the company they have Elon muskin Associates but maybe you're interested in the people in the company specifically because maybe Elon musky is associated with intelligence artificial limited and that doesn't have his name in it and you can open each of these and see further details for example um when I look at Elon Musk as a person I get all of these different roles that someone named Elon Musk is associated with so chairman chairman of the board president things like that maybe I'm interested in the company space exploration and I open that and I can see here information about the company when it was incorporated what type of business it is maybe a registered address do they have an agent down here where is where it gets interesting from an osen perspective for me who are the directors and officers associated with that business who are the people that are in the circle of maybe the person that I'm interested in looking at or the business that I'm interested in looking at that can help me pivot out to other companies you know when you think about the amazing research done by some uh investigative journalists into uh folks that are trying to hide assets or you know things like that around the world how how do you spiderweb that whole network well one way that you can do it is examine their business Holdings examine the companies that people are associated with find those personal business associations and then spiderweb out into that Network and here you get a list of all kinds of different directors and officers one thing that I really like this site for and this is a a feature that requires you to be logged in as a user it's not a paid feature or anything like that it's still free but you do have to be logged in is that they'll direct you to the registry page of where the data comes from now this is really important for me because a lot of times I want to see that original document ation and lots of states and countries and provinces around the world will scan and upload that information to their local database you know in the United States it might be a Secretary of State website and I can actually physically examine the documents that were submitted to the state to incorporate this particular business those documents are going to have things for me like email addresses names phone numbers addresses and other things perhaps don't appear in these records or have been changed or offis scad over time that original piece of paper might have the one nugget that I need it's a it's both amazing and scary at the same time to see how much information's out there open corporates for those of you that are programmatically inclined has API key that you can register for and you can do a lot of this work of looking up people looking up businesses and getting those references and those connections uh via python or some other tool as well so really really helpful to act quickly being able to assess out who's associated with what organization and what other places did you find programming help a lot M because it seems like you create a lot of your own tools or integrated with a lot of other tools I think programming allows me to do things faster and at more at scale although nowadays with chat GPT doing what it's doing it's making it much much easier to do that complex programming of analyzing huge amounts of data very quickly or grabbing a lot of data and that's really is why we've picked up some of these tools is that these tools like instant data scraper allows us to in our browser grab all this content whereas I would normally have to write a python script using uh requests or using you know some other things like beautiful soup or something to scrape out that content parse it then export it to a CSV don't need to do any of that anymore because it's all pushed up to these Cool Tools so I mean is chat gbt one of your tools or is that like an extra prep that we're saving for another video with you no that's great I look forward to that because it's like I can imagine if you dump data into chat gbd you can just ask it questions about data and it does all that hard work right oh absolutely that plus image creation uh image analysis some of the tools nowadays are really really cool because you can throw an image into a chat GPT if it's a non-sensitive image and chat GPT or some of these other AI tools will tell you exactly where that the city that it was taken in or other places about the image so that's some some stuff for a future video so for everyone who's watching do you want Mike and Griffin to come back with AI for I think that'll be a good video all right so our last tool is u a bit of a twofer right so we're going to we're going to include a second um option here at the end but the epos tool os.com is an amazing tool it was kind of first to Market in this in this little niche that's now grown and become actually very competitive in terms of options and and paid tools versus free but what epos originally did was took an email address and look for the presence of a Google account associated with it why is that important well an open source intelligence finding things like a person's Google account if you're a law enforcement officer uh allows you to do things like get all kinds of information via legal process from that company the connected Google account if you're an open source practitioner and you don't have access to that legal process can give you things like their profile photo that you could reverse search or glean information from or connect back to social media accounts their Google reviews that they've left for businesses while they've been logged into Google so that time they hated that Chinese food that they got from that restaurant might help you narrow down the area that they live so there's there's pivotable information there now the tool has expanded and grown and I'm going to show some more of what it does now um but originally it was checking for those Associated Google accounts here on the homepage you can see that you have the option to search by email and search by phone which phone is going to be a paid feature of this service email is going to be free and we do recommend that you set up a free login account on epos while it doesn't cost any money to perform the basic queries if you are logged in they will give you more unredacted information than what you would get if you were logged or were not logged in if you're not logged in they do blur some of the results out um but you can hear search for example John gmail.com you know just kind of an innocuous email address that's going to have a lot of results and as you can see they're giving the option right off the bat to download the data which I really like that's a new feature so you can grab that and work with it offline um get it for your reporting but it starts out now with hitting have I been pwned and have I been pwned is that Checker website that's going to give us you know hey have we seen this email presence in breach data right that can help with your validation and things like that as you go farther down though they're going to check other services for the Pres presence of that email address in use now this can be very useful um in finding social profiles in making connections and things uh here they're they're checking uh vieno Flicker and you've got the Google account now I mentioned before that there can be a profile photo right so this person doesn't have a profile photo set up on here but if they have one you can see that and open it via this link and get a look at their full profile photo on their Google account one thing that I didn't mention before is that you can have a Google account without having a Gmail address so I run every single email address that I'm researching through the epos email tool because you never know on someone's hot mail or Outlook or or company email or school email is going to be connected to a Google account that's going to give you more information you can work off of you get the user generated content about what their name is is that can be interesting and this is where you've got those Google Maps where you can look at their linked reviews right maybe they've left a business review and so on really valuable information here there's other services checked Trello Skype and some others gravitar so this tool is really built out over the years and that buildout has created some um competitiveness in the space uh one tool that we are't demoing because it is a fully paid tool now is ent. Industries and osen industries is a competitor now to epos that has really been Innovative with the addition of features and more modules and more search capabilities but it is a paid service if you do this kind of work all the time I would definitely recommend that you look into the paid tier of fos and the option of ent. Industries as a paid competitive option and figure out what tool is going to work for you one of my favorite things to do whenever I get a connected Google account to someone's email is take a look at this Google Maps link now what that's going to do is drop me in Google maps and show me any business review that that Google profile has left while that user's been logged in and that can help me to localize a person maybe I'm trying to find their physical location or maybe I have a list of addresses and I'm not sure what part of the world they live in now or what part of a country but one time one time actually um not that long ago I had an investigation on a fugitive and this fugitive was was taken off from a case where they had been um charged and indicted and they were on the run and we needed to find out where she was and as I comb through all the data that we had from the investigation and I was looking at all the options that I could explore with different things one piece of data stood out to me and that was an email address and I thought I wonder if that service is being used because I thought maybe we could follow up with Google or some other data provider but what I got was a surprise because this user had left a Google review for an apartment complex in a particular part of the country saying I just rented an apartment from XYZ apartment complex and so and so at the front counter the manager was the most helpful person and left a glowing review of this business that wow that the sheriff's visited you know 24 hours later knocking out the door to take her back into custody so you're you're always looking for those slip ups and sometimes they're really big slip UPS like these but people who you know are using these services and the stuff that we all kind of just take for granted and don't think about privacy in regards to are leaking that information in a way that can help you as an investigator connect dots track them down you know find things like that yeah I look at this from two perspec is right it's um it's great that you're able to use this and on the flip side if I want privacy I shouldn't use a Google account absolutely absolutely everything that you do online is potentially connected and it doesn't matter if there's a physical connection between things sometimes it's that contextual connection with your profile photo or the text based connection with your username and all the stuff that we've sort of jumped through in these in these tool videos all of those dots can connect together in different ways and that's where you're the investigator having the the capability that's enhanced by these tools but bringing their own creative mindset and their own curiosity is really going to be powerful and effective no I think one of the things that I most loved about getting into the Osan space was that it helped me figure out that that I wasn't as private as I thought I was online and it helped me to find all those places where my extra data my username my email my my old MySpace account whatever was out there and I could then identify that once I had it identified I could remove that from the internet now there are always copies of things and all but uh in general it it it really helped me increase a little bit of that privacy and reclaim some of those things that we just give away any tools that you recommend for like removing or getting rid of your data out there so there are some tools out there that can do this I think the best one that I can recommend is a PDF by Michael bazelle on Intel techniques.com this is it it's not automated there are companies out there that say that they will automate this I have no experience with that but I can tell you I had an intern a couple summers ago work for me and their job was to visit all of these different websites that pull your data out of it and they had a huge amount of problems because some websites that had my phone number and my email they wanted a picture of my driver's license so that they could verify that I was actually me and there's no way am G to give them more data to remove that little data so so I'm it's it's challenging and some sites won't remove if a third party asks to remove your data they won't even deal with them especially over here in the United States but Griffin what is this document yeah so um Michael bazelle of ENT Fame and privacy Fame and so on um created a digital guide for you to do self- removal of your online data and there are services that you can pay that will go do this um work for you if if you need that that time saved because it really is a big big time suck but um the reality is it's better for you as the person who knows the data to put your own eyes on it because to be honest leaving information out there that could be incorrect might actually be helpful in some cases um and then making the decisions about where to remove the pertinent information could be useful but the data removal guide by Michael bazelle presents you with hundreds and hundreds of sites that may have your data and ways that you can request that data to be removed um and you can review and as Micah mentioned you may come across situations where they ask for validation in the form of more information or a driver's license and ID and things like that and then you have to make a personal decision about what's right for you but in the world of of digital privacy this is a great place to start it's sort of the kind of thing that you're never really done with so having that that removal guide can help you as a checkpoint to go back and look at some of those Services again I will say anecdotally um after I had my intern remove uh our phone number and address and other things from the internet uh we got a huge increase in spam phone calls and email scams and fishing and other things so because we had essentially validated that these emails and these phone numbers were actually valid so your mileage may vary depending on where you are in the world but here in the United States there there was even a penalty for removing yourself from websites that's why I'm glad I live in the UK If I I don't want to get political but if I was living in say uh Switzerland or somewhere that would be even better from a data privacy point of view it's it sounds like in the US it's tough right very absolutely very tough great for investigators and people that are ethical and take the time to consider the ramifications of things not so great for the people that need to hide I know I mean we spoke about it last time there's a huge interest in privacy of privacy videos get crazy views on my channel but you want to balance that with like the good side so it's like it's always nice to get you your guys perspective on privacy for like good people rather than we're not trying to help bad people hide but just for people who are tired of Google chasing or you know selling their data it's always nice to get your guys input guys I really want to thank you for sharing that was amazing so top 10 tools I've put links below Mike and Griffin's training uh if you can afford it you know go and support them for creating all of this free training and extra training to help all of us guys thanks so much thanks for having us

Channel: David Bombal

Views: 52,296

Keywords: osint, geolocation, geolocation game, geolocation tracking, social media, cyber, hacking, osint tools, osintgram, osint framework, osint ukraine, osint tutorial, osint course, osint framework tutorial, osint investigation, osint tv, osint curious, osintgram kali linux, osint instagram, cyber security, information security, open-source intelligence, open source intelligence, cybersecurity training, kali linux, what is osint, ethical hacking, hacker, google dorks, dork, dorks

Id: PRqOj5qM1ic

Length: 68min 18sec (4098 seconds)

Published: Sun Mar 03 2024