Search Before Google

these days Google is synonymous with internet search to the point where the US Supreme Court is considering whether or not the word Google can be considered a generic term still it's hard to imagine that there was a time on the internet when the Olin incorporation didn't exist let alone have the world's most popular search engine ever since the beginning the internet by collecting information from around the world has had a problem simply put there is too much data the amount of web servers and more importantly web pages increased exponentially and without any sort of centralized organizational body it was easy to get lost in the sea of information the solution organize everything into web directories either hand sorted and organized or automatically collected and indexed by a machine they centralize the uncountable sources of information on the internet and make it easy to find what you're looking for to really get into the histories of some of these though we're gonna have to take a few steps back not just before Google but before the World Wide Web itself long before HTTP became the standard for passing around information on the Internet a much simpler protocol FTP was used and in some cases is still used today as networks started connecting to each other though a problem emerged there were just too many files available for download across too many servers for users to find the one they wanted a pretty simple solution emerged however a program named Archie written by Ellen Temptation 1990 would download all of the file listings from all of the popular servers onto his universities local computer on a monthly basis then users could search for a particular file in the local Westing's through a single command until which server hosted a Archie itself didn't perform any searches but it did embody one of the most basic elements of a search engine automatically forming an index of available files for download much like the problem Archie solved with the introduction of the World Wide Web it was necessary for users to know where to go in order to access web page either by knowing the direct link or by following a link on another page Tim berners-lee and a team of other expert Volunteers started manually compiling in 1991 what they called the World Wide Web virtual library the library much like Archie centralized information on the web though being created entirely by hand had the benefit of not only holding high quality academic information but also having the information be organized by category the only consequence of manually cataloguing web pages was that as the intern grew exponentially the library had trouble keeping up meaning that its listings remain fairly small it would be two years until a program automatically index webpages the the program the world wide web wanderer was never intended to be a search engine the wanderer considered to be the first web crawler a program that discovers and indexes new web pages was only supposed to measure the growth of the web by helping from web page to web page following links and storing addresses into an index called the windex the wonderer fulfilled two of the three main tasks of a modern search engine paving the way for a slew of different automated search engines in the following years the first of these early search engines was jump station the brainchild of jonathan fletcher were working at the University of Stirling in 1993 his intent was very similar to that of the virtual library making it easier to find new sites on the young web but also took cues from the wanderer by leaving the tasks entirely to automation what made his program arguably the first real search engine was that it automatically found web pages indexed them and allowed end-users to perform a keyword search on the index speaking of the index when Fletcher first launched his program it ran for ten days until it have visited every existing website called 25,000 of them by the end of jump stations Ron had collected information on 275 thousand pages in fact that significant growth was the reason the plug had to be pulled on the project it was too expensive even after Fletcher modified the program to only index the titles of web pages the database it was growing was beginning to take up too many university resources unable to find anyone interested in financially backing the project jump station the Internet's first true search engine was shut down in 1994 within the next few years after jump station was shut down several new search engines sprouted up and attempts to fill the void left between 1994 and 1996 we sought Lycos from Carnegie Mellon excite from Stanford HotBot from wired look smart from Reader's Digest and info seek the first search engine to sell advertising in its search results turning a profit for the service in the first to offer targeted ads based on user search habits a lot of these search engines were short-lived either merging with each other or being bought out by another company but probably the most advanced of the day was one going by the name of Alta Vista just like all the other search engines Alta Vista started as an experiment and Digital Equipment Corporation known for our PDP mini-computers of the 60s and 70s in an attempt to make finding data on the public network easier by 1995 Dec had the alpha processor a 64-bit chip which when paired with their multi-threaded web crawler made for an incredibly quick search engine the speed AltaVista offered became a major selling point surrounding his public lunch on December 15th 1995 the other big feature Alta Vista had to offer was that it contained a full text index of the 500 gigabytes of webpages it crawled previous search engines like jump station could barely afford the storage space to keep an index of the titles of every web page but Alta Vista having the backing of dec was able to keep an index for every word in every website increasing the likelihood that a search would find the page it was looking for Alta Vista didn't only do search though in fact you might be familiar with some of their other creations including Babel Fish and early online translation program and CAPTCHAs even though Alta Vista was bought out in 2003 and shut down ten years after that it had a good runner being the primary search engine for the web processing tens of millions of requests at its peak and even providing the search capabilities for another popular site on the web Yahoo compared to the other search engines of its time Yahoo was a bit of an odd duck mostly because for a long time it wasn't an entirely automated search engine like Alta Vista was Yahoo started out with two Stanford students Jerry Yang and David Philo who were simply building a list of their favorite web sites ala World Wide Web virtual library under the name Jerry and David's guide to the World Wide Web eventually switched to the backronym Yahoo rather than searching automatically crawled and indexed websites like Alta Vista Yahoo had a hierarchical directory of web pages which was created entirely by hand Yahoo started to grow they begin to charge for reviewing and listing sites on their page in order to turn a profit this prize as well as the man-made directory for a time worked in Yahoo's favor increasing the bar for quality in listed sites as well as being far less susceptible to spam ensure even the Yahoo results were usually the best of the web unlike many of the search engines on this list Yahoo is actually still around granted the directories been replaced with regular search and the company's currently being bought up by Verizon but that's a story for another day last but not least there's ask calm whereas it was originally known Ask Jeeves Ask Jeeves was powered by an automatically crawling and indexing search engine but more similar to Yahoo had human editors sorting through search results the idea was to create a natural language based search engine where you could enter a question like what is the speed of light instead of having to try to guess the right keywords to locate the information the human editors were critical and matching relevant websites to the more popular questions the search engine was asked though it could also return automatic keyword search results as well like Yahoo asked tocome is still around though Jeeves has long since retired and there we have it a pretty comprehensive list of the pre-google search engines but the question remains what happened that caused Google to come out on top over all the other more established services in short Google was able to combine the best parts of other search engines like Infoseek Google sold advertisements in its search results like Alta Vista Google had the full text index of the pages it crawled like Yahoo Google was able to organize content based on quality but unlike Yahoo Google could do it all automatically it all comes down to a little project Larry Page and Sergey Brin had been working on at Stanford University their algorithm backrub determined the importance of academic papers by seeing how many times a paper would be cited by others the more times the paper was cited the higher its importance this lends itself tolling some web pages were the more times a page is linked to the more important it was the basis behind an algorithm called PageRank when paired with web searching capabilities PageRank became the search engine we noticed today Google things to PageRank automatically sorting websites by the relative popularity Google was far more resilient to spamming a single search term to rise to the top of results a practice that plagued other search engines like Alta Vista also helping on Google was the fact that it was introduced rather late into the search engine game in 1998 at the time the dot-com bubble was nearing its peak and what the resultant pop that killed most of the competing search engines Google search had plenty of space in the market to take over during to its current state at about 80 percent of the search market nowadays Google is used interchangeably with web search and being under the same umbrella as other major sites on the web like YouTube it's hard to ignore it as a search engine that serves an estimated was 3.5 billion requests per day even so if it weren't for the groundwork laid by numerous experimental and commercial projects throughout the 90s we may never have had the giant incredibly useful slightly monopolistic and definitely not evil internet tool we so easily take for granted today [Music]

Channel: The Science Elf

Views: 363,313

Keywords: intermet, internet history, search engines, archie, world wide web, web crawler, altavista, yahoo, lycos, infoseek, looksmart, excite, askjeeves, ask.com, google, google history, computers, technology, software

Id: 1vQXHdlTMok

Length: 9min 18sec (558 seconds)

Published: Sat Feb 03 2018