Building a simple trivia game | Web Scraping with cheerio | Express + Apollo GraphQL | Vue

[Music] hello friends welcome to coding garden with CJ welcome to a stream where I don't know what we're gonna be doing yet but we'll figure that out let's say hello to all the people in the chat hello Cassandra who was the first welcome hello Assad hello Rennes oh hello Tom hello Subin hello sir um / a little Victor hello Kevin who says do it we're gonna do it we don't know what we're gonna do but we are going to do it hello Daniel garden hello mejia Stephen says golang stream maybe probably I mean I don't know if I'm in the mind letting go link tonight we could so um it's been a while since I streamed I mean not quite a week but it's been almost a week since I last dream I've been busy and busy with work I did some volunteering this weekend I got a little sick I'm feeling better and so we're here we have arrived yes I am feeling better Thank You Brandon and hello coding journey hello ran hello one pocket pimp what's up a little bit in hello I'm ed oh hello cool ATG her all says I like it living on the edge living life on the edge topic DVD and hello Brandon Douglas is saying Elm maybe I don't know neither Oh I'll save it but neither L nor going sound fun right now I don't know yeah I'm feeling better uh Stephens been busy saving the room I definitely haven't been saving the world yeah I was only like my stomach felt a little upset I never really had a fever I just didn't feel a hundred percent but I feel good now I'm feeling good yeah thank you a meadow Steven has a dozen suggestions yeah so um the thing is we missed code Coty's yesterday so we could do some Coty's we also could we could sprinkle things around we could like do one code kata maybe answer one QA question build one thing I really I want to build something I really want to make something I don't know what it would what that would be though I want something that I can build within an hour so I don't know but if you have suggestions feel free to throw them in the chat right now and then we'll vote on them in a little bit daniel says do something with MongoDB I think I think I think we could do that builds let's build Google no way no J's framework exploration maybe Mejia says a server in a course origin a front-end URL so only that your Elkin acts as the API oh no in server cores origin yeah so you want to add the the base URL of your front-end so HTTP colon slash slash some domain comm that's the origin and if you put that there then in the browser at least only that origin can access the your back in we're not gonna build Google something we can build in an hour build a Q&A I already have a Q&A website I mean we could build like a basic version of this to see how it was built that could actually be pretty fun but this site here we have a lot of questions to get there I'm gonna try to do a a Q&A stream tomorrow morning so we can get through some of these questions but a lot of these questions are actually really good topics for just a video in general so yeah but I kind of like the idea we could build like a basic site like that with voting something like that a Chrome extension maybe we will not build a search engine that takes at least a week ATD says I love the type ship stream I sympathize with the struggle you went through with the eslint prettier tooling these days yeah tooling is crazy which is like if I wanted to build an app just a basic app with that setup I would probably just use what I built as a base and go from there because it literally took me like over I mean the stream was like three and a half hours but if you cut out all the talking it's probably like two hours of just configuring everything which is crazy but yeah that's the thing Oh graph QL would be fun I haven't touched graph you own a very life in over a year but mejia is saying how about an Apollo server Apollo that would be fun yeah I've been looking into Keystone it's probably a bigger topic than tonight we could so the other thing is today is Thursday so we could do like a throwback Thursday I could build like an angular 1 site or like a site with knockout J s or something like that ah but Keystone is like a CMS interesting angular what yeah a little acid spark what's up rails I've only done a little bit of rails in my life but yeah Daniel says I saw someone on reddit scrape their councils API for determining which bins get put out and when that could be an idea yeah so I mean there's the the Denver government website that says when trash day is trash day for me was today that could be interesting they might have like an API or something a browser text RPG that saves to a local database interesting and hello Jacob welcome and hello Andrew welcome twitch channel scraper who follows who kind of thing that could be interesting we wouldn't really need to scrape we could just use the twitch API but you really could build up a big build up a big network graph because follower data is public so you could basically get the follower to spike screen just cut out come on oh there it is welcome back uh-huh but you could get all the followers of one channel and then for all the followers that are returned get the followers of each and like build out it like a dependency graph that would be interesting I'm feeling better Andrew we miss Cortes yesterday I didn't exactly feel like doing just Kato's right now so I thought you know let's let's figure out what we're gonna do I don't really know that Finder API maybe codeine garden Twitter we've done something similar but it would be interesting to see what we could do with with feathers instead of building it completely I mean mostly from scratch we can actually have login I don't know I like the idea just some kind of full stack app that we can complete in about an hour let's say an hour and a half and and I mean it this time because I'm literally gonna stop streaming and let's call it ten fifty to my time so in two hours I'm done let's go ahead and set a timer so I mean we won't we won't really even have two hours because we're talking right now but yeah and hello Sharpie 360 what's up I saw you were streaming earlier fact I hate reacting okay Django yeah Python Python might be interesting I hate mobile development yeah it'll take two hours to come up with the idea which means we need to be coding within 30 minutes that's that's what this means and you all have to help me out with that so uh hello dip okay answer it's the morning for you welcome welcome welcome build an interpreter in vanilla JavaScript we could we absolutely could I recommend you check out the YouTube channel low-level JavaScript because he does just that yeah angular one yeah Oh backbone and ie 11 compatible site no oh it's Friday that yeah it's Thursday here it's not quite Friday yep yeah angular 1 sick if we did throwback angular that's what we would do and Euler 1 6 yeah I saw you struggling through that tooling set up when I search for an easy solution then I found a poetic it's amazing now I'm using it every day it's amazing so you found an easier solution than what I came up with but if you did that's great yeah can I skip your message I don't think I did sorry b3x 360 no I haven't gotten there yet there it is I haven't gotten there yet orcs or Oryx twitch says I'm new I'm 11 old-school reference it would I think would be fun it'd be fun to do something throwback yeah I'm here I was just I was sick I was also out work has been busy as well so there's that web scraping would be fun to sharp e360 says teach us all about cores we could definitely do that I I too didn't to like I don't know if it was the tail end I feel like YouTube was have are you streaming right now because I feel like the the live stream was having issues rabbit works that's not it what's that rabbit works JavaScript there you are yeah you're not live right now your latest video I guess hasn't processed that's a rabbit rabbit twerks but yeah uh yeah I tried to tune in but it was like the livestream was being weird but um what I saw you do and I'll tell you really quick what in I didn't know if it was live or not which is why I didn't comment let me only pull up an editor new file tours so in your back-end code you were setting up the cores module and you have to look at their docks because that if you want to have a dynamic origin meaning two different ones you have to have like a function that does it or you could use an environment variable so what I saw you do is you had cores not that just cores and then you passed it an object and you said origin is something like this HTTP colon slash slash localhost 3000 or I don't know whatever your localhost is or HTTP colon slash slash some deployed site and I saw you do that and ultimately this won't work because in JavaScript the order sign when you're doing assignment is like if this is not a value then use this but this is always going to be a value so by setting up this way the origin is always going to be localhost 8080 you'll never be some deployed site what you could do is look at your node environment variable so something like if process that env node E and B is equal to production then use your this one this one the deployed site otherwise use that what about sticky keys give you something like this yeah this will never lock exactly a hello Alka thank you so but if you do something like this Shawn also known as rabbit works this should work because wherever you're deployed you you just need to make sure that when you're deployed node V and E is set to production and that should do it for you yeah yep welcome the mouth see I never graph to trace everyone back to Bill Gates mmm no way Apocalypto we were talking about my I'm behind on chat but that was my my screen huh yeah definitely check out MD in four courses good stuff oh the poetics setup is that literally what is called in px poetic okay hmmm automatically install and maintain yes lint prettier editor config every be in real store javascript pipes rip and react that sounds great cuz that's exactly what it was trying to do oh yeah that's pretty cool let's check that out mmm-hmm build that screen cutout effect that obviously would be actually really funny to make like a screen saver that looks like your screen is broken what will save that one yeah graph QLD I don't you I've only messed around with this felt so if you look at my channel I've done some videos on slow or I was just like learning it and building basic apps but I don't really use it on the regular yeah which is why we have to watch this clock when this clock hits zero I'm done streaming so we needed topic somebody has to remind me somebody in the chats got to be like we need to vote we need to start building right now yeah hijack the timer yeah I do remember that but it was a fun stream and a surprising number of people watched it yeah in bur octane hello bud rush I haven't used Ember in a long time yeah is there a reason you've been hearing so much about rust its popular it's a low level language it's time to build just do the thing already yeah we don't know we're gonna do yet though you're welcome for the welcome Jacob post rabbit works are you streaming right up you mean afterwards what what do you mean now you're not streaming but yes I will host you know twitch points yoke J s what's that I've never heard of it is this like an old thing come on a library for building asynchronous user interfaces this repository has been archived well eases rxjs there's some JSX in there I guess interesting Oh somebody was up who who carry if I was in affiliate nerd it whose nerd it the nerd it'd say something I didn't see a message from nerd it oh no twitch boy oh yeah yeah I don't know I don't have any monetization enabled oh yeah mm-hmm hosting rabbit works when I'm oh yeah for sure if I would have went live earlier we could have but yeah yeah yeah I don't have any monetization so you can't you can't give me stuffs yeah hello welcome course can only be sent in the response headers yeah so on the back end your back-end can set the access control allow origin header in the response stickykeys how do you set the envy flag when starting the CLA you could do that or you can set up your env file but also when you're running the command you could say like no D and V equals production node index page is and this will set that environment variable for this process when it starts up the JavaScript file rabbit twerks yeah test what's that about yeah and hello Chad welcome welcome welcome thanks for the follow a spurious programmer welcome to the show works which says scraping is very interesting if and when what's the best approach and resources I have I have quite a few videos on scraping weak admit scraping would be fun we're gonna do we're gonna vote let's say we vote in five minutes I'm gonna catch up on the chat we're gonna vote in five minutes and we'll decide what to do we'll be able to maybe build an app well maybe do some web scraping we could do data structures and algorithms I had that thought before I started the stream no eighty [Music] wait did did Chad what happened with Chad I don't know a global leader board for the drop game this is not a bad idea cuz I have all of the the leader board data stored locally it wouldn't be global but I could make I could put it online I could backup the data and I could make a website where you could see historical drop game scores for the coding garden so yeah people do like web scraping it's a yeah it's a thing I'll do it you can trust me what somebody mentioned a graphic you all back in so we'll definitely have that in the poll twitch extension that could be cool we got to figure out what to do yeah and hello Lee from Bangladesh well welcome this is not your average coding garden stream I mean it kind of is at this point because we really do just talk for a while but there's no predetermined topic we're gonna are gonna figure it out ari is asking how can you embed server-side react code into a legacy app that's a very big fight it depends on like what is that legacy app is it like a PHP app because then that would I mean somebody's probably done it but I don't really know yeah our open source names trademarked like objection J yes for example not unless they got a trademark I wouldn't think I'm not a lawyer I don't know oh gee no point yeah I don't I don't have that I don't have any any other twitch stuffs send me money yeah so you might have seen on other twitch channels you can get points for watching yeah well you know we could do we could create a graph QL in point that actually does scraping under the hood that might be interesting yeah and hello guy dev welcome play code play slash code elevator saga what is a levator sock it sounds familiar the elevator programming game huh I've never seen this before interesting [Music] and hello Lee and thanks for the follow Shu Hikari welcome I have this res dot set header access control allow origin for each response you don't really you don't need to do it for every response that's that's why you would just you use if when you use the Korres middleware this automatically sets it for every response so you don't need to do it it never every Handler and that's if you're using the Korres plug-in you could technically create your own middleware that does that yeah I need it I did get some compressed air so I need to clean my clean my keyboard yep toast I was saying you can set environment variables in your package.json - for sure and there's also like your dot e and B file or whatever deployed environment you're using might have a way to set in bar I've not met variables rabbit twerks yeah ignore the chat I see you Lee I'm just behind on chat yeah they do like web scraping yeah okay data structures and algorithms with typescript so I could try poetic why not recreate my drop pot that would be too easy I could be done with that in like five minutes rabbi twerks come on thanks we is web scraping a sort of legal gray area I guess most companies wouldn't mind for simple request and assume most people don't really yeah so ultimately if you're scraping script is behaving like a human would and isn't making more requests than a human would most sites well notice or care some sites explicitly say in the Terms of Service that you can't do it some lawyers would say that's not actually enforceable I don't know I'm not a lawyer like I said yeah G RPC would save my life double vote well basically we're gonna take these questions we're going to narrow them down into a poll and then we're going to go from there because there's what they're they're are repeats yeah so there is like an undocumented twitch graph QL API for sure yeah hey man she was saying water s XSS and C surf and how to avoid them um XSS is cross-site scripting that's where somehow a malicious user is able to execute scripts on your site and C surf is cross-site request forgery this is an attack where someone can make a request on a different site to your site but it impersonates the user that's currently logged in how to avoid them with C surf you can set tokens and for cross-site scripting you just need to make sure that you see a no ties user inputs and sanitize anything that goes into the DOM little where is great way to some yeah for of course for sure I get all my legal advice from the stream don't say that a graph QL backed collective paint webpage that'd be cool I'm down for that Oh Thank You Lee I appreciate that yeah there's some graph QL on that list all right we're at we're at time it's time to make a poll and I will be made it we made it through through through the chats create a trivia game that scrapes trivia questions websites the stories in my graph QL database whoa that's great I like that yeah I've got to make a poll okay so these are all the things that we saved let's just do a live poll truffle with all the suggestions that came up gonna do a little bit of this and a little bit of that okay what should be built well we could just say learn going learn um playing so LM is like a front-end framework but it's all functional and it's interesting something something something MongoDB this isn't that specific so I'll skip it simple Q&A site and I would say to demonstrate feathers dais though we could probably do it with like a graph QL back in a Chrome extension something with graph QL something something something graph QL Keystone I guess it's like a CMS I don't know the thing is if we have a ton of options it's the votes are gonna be pretty diluted I think we'll do a sec we'll do one poll we'll take the top three or four and we'll do another poll after that somebody's at rails yeah elm Ling is on there for sure oh okay let's say a scrape denver website for trash collection times and denver just because I live in Denver but you could extrapolate whatever I build to to do that a text RPG always put that twitch follower graphs a simple Twitter something something something web scraping glitch slash broken screen effect cool Apollo server in client I think I would have apply if we do graph QL what even is in brockton I'm just going to look that up really quick [Music] a new Fed of set of features when taken together represent a foundational improvement to the way you use ember there are a lot of options yeah interesting all right I'll put it on there build something with yoke Jess I don't even know what it is yeah so we have web scraping on their data structures and algorithms intro / basics could we do that I don't know drop the game leaderboard website twitch extension elevator saga collective paint graphic UL trivia game it's great trivia questions from somewhere okay and it looks like some more so there 4016 that's just since I started making this poll let me just see if there's anything new I should add hello Irwin who says ugh what about the drop game I'm yeah I can allow multiple pull answers I'll do that because there are a lot of options rust and wasm that could be interesting something something something keep keep distance keep keep some distance from Elm and intro down yeah we could do that PWA Nightmare on Elm Street and thanks for the youtube subtitler speaking of chrome extensions I've been thinking around twitches were restriction to the graph QL already uses gql requests as a browser extension Oh interesting oh and hello deep sense we could you know if we use graph QL we'd need some sort of thing behind it we could use madhu MongoDB this I believe is that custom feathers authentication which is what Andrew just linked to there are a lot of options something something something engine X build a pure data ramp code visualizer I don't know what that is we're gonna okay alright and hello Bob who says this is a late one yeah I guess it is Postgres stolen what so we're gonna improve spam prevention but you can vote on multiple answers don't vote for more than five we're gonna do one more poll after this so the the top five thingies we're gonna do another poll and vote from there that's the plan we've been streaming for 32 minutes we want to get coding in about seven minutes so only you can help this process go faster so see what we got put this back close that and we can watch the results come in vote now um Peck says I hereby nominate myself as your future project manager oh thank you I'm heck we'll figure it out we'll get there we've got time plenty of time and hello Osama from Pakistan welcome is engine X the doctor won so engine X is typically used like on your server to manage multiple different applications running on your server whether they be like PHP apps or apps running in docker containers or just static files different things like that yeah okay says I have no idea how to implement a graph QL server I did it like a year ago so it'd be it'd be fun to do gr peace out no the poll has already begin contribute to some very famous library that that could be interesting too just like picking an open source library and trying to solve an issue we're already in it though 95 there's a lot of votes name I did allow people to vote on multiple but like is that don't vote for more than 5 because you're really just diluting the vote anyways but yeah orcs which says I miss cap Hill oh cool yeah I don't work near there but I've been there I live in Denver and hello so GERD welcome it is a lot of options pick five put all these ideas in the poll into one app yeah okay uh ten per 10 percent votes for learn going let's let's just make a note of this the ones that are at the top now you you can you can vote more than once but on different things I believe I believe that's the thing so if you try building voting for the same thing twice it should only give you one vote let me know if that's not the case it should be the case let's see wait no I don't want that what out of all these things what do I want to do this one sounds the most fun to me a trivia game where we scrape some trivia answers all right I can do this that's a bus that's part of a bus that's a crosswalk very good okay cool vote has already been counted that's good to know okay so we're gonna grab the ones that are all at the top and we're gonna do another poll so we have learn going data structures and algorithms something he's something with graph QL where's the trivia game it only got five votes that's lame yeah it does you could vote on multiple things though yeah check the things you want we could do it an idea generator yeah do the trivia game I think I might but it would be fun to add some scraping in there right all right I guess we'll take these already we're gonna take these top four and the trivia game idea and we're gonna do a new poll and we'll go from there people really want to learn go okay learn going data structures and algorithms intros and basics do that something with MongoDB let's say MongoDB plus Express because we need something to talk to MongoDB something with the graph QL and MongoDB or we could do like a basic graph QL API and show how it can talk to a MongoDB and a Postgres DB and scraping or we'll have that just this two separate options and then a trivia game scrape questions and answers from the web okay you can only vote for one here we go could I do a trivia game with all of those and Himanshu with the super chat thank you so much I Manchu who says rust in wasum or PWA should I add it to the poll here can i edit it we're gonna add it new poll well that's not good um something with MongoDB and Express something with with graph you oh and then the suggestion that just came in now we're not setting a precedence here you can't just give me money to add things but tipsy ninja has been watching the show for a very long time so I'm gonna add this to the pool yeah the time we have we have 1 minute in 11 seconds left so you better vote quick um PWA let's go title is required it's also fun to just sit here and talk to you right isn't it no you can't vote on multiple you can only vote on one I'm gonna vote for the trivia game I'm not a robot I swear vehicles there's one you're over there two vehicles that's a vehicle Oh Thank You Stephen can a topic be being a streamer oh like like what it's like to be a streamer like just asking questions about answering questions about it is that what you mean meta cool well if someone's super chatted to have it not be added I would still add it you can't negate something yeah a trivia game with all of those and Dylan with the basic pizza maker so that's a joke about the right we're already cutting into our time but that's you all My Week was good I just I was busy with work I got a little bit sick this past weekend but I'm feeling better I think they not only want to learn goaling they want to see me learning gogo lane which is a different thing I think a trivia game using go you know I think we might try to build this trivia game with graph QL I don't know what can we do in an hour and a half all right once we get five people saying pole is done the pole will be done right we're gonna use note which I can't like I can't I only have an hour and a half we got to use node pole is done oh it's done po is done alka says I'm way down for learning go with CJ okay that's five five things in sacred gaming says sorry I can't hang around have a great night thank you for stopping by a secret gaming all right so we're gonna build a trivia game here's I need I need your help what you what you need to do right now is to find websites with trivia questions and answers we'll pick like two of them and then we'll scrape then that's the first thing they need to yeah CJ wins so trivia will be fun it definitely will be okay so we'll call this quick trivia because we're gonna build it quick I guess it'll have a client and a server yeah the server can probably have graphed ùl in it Quizlet might be the way to go but yeah so in the trivia can be like maybe it can be programming related things but it can really be anything but we we just need to make sure the site has like question and then maybe multiple-choice answers because then we can just build a simple simple game yeah let's make a server directory let's make a client directory we could vote on what we build it with I'm just gonna make an executive decision and say if we build the client with view because that will be easy so let's just do a view create client nope nope communities new old and busted NPM new hotness view create client we could use poetic no i'm i've been using typescript in react all day today I'm kind of over it so for right now we couldn't use typescript in view no we need to get this thing done okay there's that SAS air B&B config let's go okay so we got a view app being generated called polar head there are 36 votes I could use typescript and never declare types okay so we have a few things J app click has open TDB API what is this what are you what are you linking me to Sharpie 360 linked it as well so what is this Oh open trivia database interesting the thing is like it is just an API right which probably gives you back JSON what I want is a website that doesn't give you JSON because we're gonna scrape it we're gonna scrape it to to get the questions out of it because that we're gonna cover multiple topics right we're gonna we're gonna scrape we're going to we could actually make the scraping endpoint behind a graph QL in point that could be a thing trivia fYI Alyssa Milano starting what fantasy television drama about a group of sisters that discovered their witches is it I don't know charmed I knew that did I don't know so we could we could like this is a perfect site for scraping because we could pull out the question we could pull out the answer I like that were there other ones The Simpsons archive we could do a Simpsons quiz who is Homer Simpson's brother this would be a really weird scraping job because basically we would just take this and then try to match it up with the answers it's like in two different scrapes we could do it though yeah any already-built api is different describe yeah for sure for sure science trivia bird trivia okay what bird has the largest wingspan we could build a very simple trivia site with just these bird trivia questions but we could scrape them what it what is Jay sir this a set of simple trivia service that serves the answers and questions pulled from Jay archive you yeah so this is already an API but we could see where they're getting their questions from and then directly scrape that website what is J archive charmed bewitched nice you guys got it why is everyone so Jasmine yarn is cool like yarn was cool when it came out but NPM has caught up to all the things that yarn was doing for the most part but yeah yeah absolutely so captures are definitely used for training machine learning algorithms bird trivia now this is my project manager who's telling me to get started alright what's this about a fan created archive of game and player information is beginning of the show to the president oh the Jeopardy archive Oh [Music] though jeopardy is a little tricky because it's in the answer question format 50 trivia questions that only geniuses whatever trivia question 1 what is the only u.s. state that only borders one other it's not Florida it's not Washington it could be Maine I'm gonna go with Rhode Island oh no it's the answer was right there right there this this actually might be perfect because we could just scrape this one page use these trivia questions and in the app that we build and I think I'm okay with that um does npm cast a version of packages now I think it tries to it's still slow everybody got it made [Laughter] [Music] okay so I think this is will day we're gonna scrape this website straight up like if we look at the source of it that's a ton of JavaScript but if we just search the page for like trivia question number one yeah it's like in an h2 all is gonna be hard though this is like a really weird HTML format let's see so we have a div with class listicle page you okay you so we could probably generalize that and then let's see where this one is yeah and then this also has a listicle page and the answer is in the same one and then that one's there ha so I mean this will be like an interesting exercise in scraping just because the format of this page is just so weird let's see if this was added with JavaScript though because if we look in the actual oh thank you well somebody brought it up but my my break timer actually isn't running so there it's running now yeah so actually this might work because for each listicle page element there are there are 30 listicle page elements this says there are 50 questions do we have to go to a second page oh and then view slides 11 through 20 so this could be interesting because the scraping code we write for the first page there's only five questions on this page how many pages do we have to go through well this is horrible but this is this would be a good exercise in paginating scraping because we'd have to scrape each of these pages it is great so first off this what this website sucks like but yeah so I mean and the reason they put it on so multiple pages is because it what do you call it it gives them ad revenue I haven't adblock around but yeah let's do it ads ads but yeah let's see what happens if i turn my ad blocker off here we go I think I have to disable yeah look how many script us tracking scripts are disabled sixty-four tracking scripts all right here we go yeah there's an ad there's an ad over there there's another ad look at these ads oh man okay this is perfect though yeah it is Reader's Digest my mom used to read those we used to get them in the mail okay so that's trivia questions and then if we look at this this is trivia questions page / - if we do page slash one well that actually work will it break no that's great okay so we're gonna use this URL let's go ahead and get started on the back end I think initially so yeah we're gonna go into the server gonna be a super basic Express app once we get all the scraping working then we can put that behind a graph QL API I think that's the plan so let's let's make this a server for an install Express install cours morgen for logging cheerio for doing the selecting this is like jQuery on the backend and we'll install node fetch for making the requests do I need anything else I'll do that ammount per page I don't think so Sharpie 360 I mean maybe but I don't think so it's purposefully split up into multiple pages cool okay we have all those things let's make a source directory and then let's also install es lint as a dev dependency and we'll use the Airbnb config we could have used I thought about it using Axios for like a hot second but we're not gonna NP excellent - - in it - many T's in it we use require no react no typescript we're running in node we're gonna use Airbnb there we go let's go poetic comes yeah we're not going to use typescript a future straight a future stream where I'm more prepared we will use typescript we will yes typescript yeah we could maybe figure it I don't know about right now though okay so let's open this up in vs code actually do I have other code yeah what's closed vs code open up just this one I'm sorry imma Hedra future stream definitely in a future stream let's just create a straight shot Jes will get this working and then we'll think about actually like making an API out of it our URL is that and let's just make the request so we're gonna need node fetch to actually make the request and then we'll need cheerio which is a selector to actually get the data out of it so we're gonna bring it in cheerio we'll create a function that's like get trivia page we'll pass in the page number and so the URL will be just this and then we'll call this like the base URL and so when we do the fetch will create the URL out of that will just say URL well base URL plus page now let's template string it because we'll put a trailing slash on the end so we'll say base yeah but it's complaining about camel case okay we'll do camel case cuz I am using the Airbnb config base URL slash well know the slash is already on there let's put this lash I like having the slash slash page numb / right let's just log the URL and then later we'll say get trivia page will pass in page 1 and let's see what happens when I run this this is the page we're gonna scrape great and hello wings in the sky we just voted we spent way too long trying to figure out what we're gonna build but yes and actually since I'm thinking about it let's update my title on twitch and YouTube here we go what are we doing we are building trivia oh god what just happened did I just close Chrome I couldn't know right I had what did chrome just crash how is that even possible there we go yeah cuz like I have like if I try to quit yeah it'll say hold Q to quit I didn't do that here we go what are we doing building a trivia trivia game so web scraping with cheerio graph QL and view yeah this is what we're doing hmm yeah yeah the time yeah the timer timer still going though that's still good if we but we got plenty of time I'll be done with the scraping in 13 minutes plenty of time command queue to change feet name is that a thing trio isn't necessarily just for scraping I'll pull up there I think okay if you look at cheerio on EDM I had you until the word view well that's we're not gonna touch you for like 30 minutes so you could stick around at least till till view shows that or don't it's up to you but cheerio is not it's not really meant for specifically for scraping but it basically gives you jQuery like syntax on the backend so we're gonna use node fetch to request the raw HTML of this webpage and then once we have the but the raw back the raw HTML we're going to use cheerio to pluck out the parts the parts that we need so that's that's the plan okay so we've created that URL let's go ahead and fetch it make this an async function oh we're gonna fetch that URL and then we're going to call that HTML that's going to be response text so we just want the raw HTML text of this trivia web page and then I'm just gonna log it out go it works cool so when we were looking at the source of this page what I saw was each of these elements has this listicle page class and that ultimately has I believe the question and the answer yeah so if we look at the listicle page container I believe the container has the question and the answer in it yeah so each listicle page container has a question and an answer so let's do this yeah and the listicle page container has two listicle pages inside of it the first listicle page is the question the second listicle page is the answer so here's what we need to do we need to select all listicle page containers and this should give us five because there's only five five questions on here so do we have zero one two three there so that it was just four why is there not five let's let's just see let's put this over here the question oh goodness I thought it was text we could we can pull out the image if it's only images but I thought it was text overlaid on top of the image maybe it has an alt tag oh look at this it has data attributes so we could look at image has a source set it's just an image isn't it so we can we can pull the image because we actually don't have the question as texts oh goodness oh goodness me let's let's let's just search to see if it's in there somewhere state that okay yeah so it does have alt text somewhere only u.s. state that only borders one other so that's there it's not exactly the text from the image let's see oh this is so this is so bad because look at this one the question is actually here name the number that has three more than one-fifth of one tenth of one half of 5,000 and then this says what is the missing number this is super Jake each number is the previous two numbers added together the eighth number is the sixth and seventh numbers if you can for these wait wait know that that was the answer and this is horrible I didn't do enough research I think we're still gonna go with it though we'll just like choose the image that's why it is an interesting scraping challenge oh we could use OCR okay so we could we could okay that's what we're gonna do we're gonna at least get the image and we'll potentially use OCR well scrape what we can yeah we gotta go like we look at look we got we got we got to get going okay so what I wanted to see though was the fact that there are only four listicle containers so that's the first one that's the second one oh the second one actually has two in it okay the other thing is if we select all the elements that have the listicle page class we should have ten elements every two is a question and an answer I believe yeah so we have ten and so like this is what we can't see it but if we go down here that's the question and then the next one is the answer so I think that's the way we're gonna go so we're gonna say first first up we create or cheerio instance use a different site we're already too far down this path so we'll say cheerio load and we'll pass in the full HTML content and then from there we can use this dollar sign just like jQuery so let's say the question answers let's just call this containers that's gonna be this thing select all of the listicle page elements and then I'm just gonna log containers that link then we should get back ten let's see yeah so there are ten of them cool thirteen lines of code we're too far in I don't know it's gonna be a fun challenge because so many people are in so many times I've scraped and the sites are just so easy to scrape so this will be fun figuring out what we need to do okay so if we look at the basically every other one is going to be oh man this is this is jank this is every other one is the question itself so all like zero two four six eight those are all the questions so if we look at that element though so we're looking at at listicle page we could then let's do a quick stretch it's Drinkwater so okay so right now we're inside of the listicle page element I would say I really want to select all P tags that do not have a class name on them really wait is that super checked whoa Thank You rabbit works who says why did the blind dev lose his job because he couldn't see thanks thanks so much man much appreciated and everyone definitely check out how it works if you haven't he was streaming earlier today on deployment is this oh yeah so this is your still your full stack off series check them out thank you so much okay here's what I'm ready okay so we're just gonna create a basic for loop we're all even-numbered items is 0 mod 0 zero zero mod 0 nan no no 0 mod 2 should be 0 1 mod 2 is 1 2 mod 2 is your okay so we all even number ones let's just do a regular plain old for loop I think we can do a for loop let a equals zero while I is less than containers that link and we're gonna increment by two each time because we're gonna say the question is going to be containers at I and the answer is going to be containers at I plus 1 that should do that just just to see what we get here let's create an array of questions just like an array like that and right now all I'm gonna do is I'm gonna say questions not push we're going to push an object for the question is question dot text yeah so really what you need to do is if you want to get those cheerio message methods you need to pass that element back in to cheerio like that and then you can do things like dot text so here's already I'm gonna say question dot text and then answer we're gonna do answer dot text just to see what happens and then right after all of this we're gonna log questions okay so we're sort of kind of getting somewhere but we can see that the question here the inner text has like the options right and then the answer gives us the answer photo credit and then the description of the answer which which I think we'll keep now for this next one that's like the question okay and I think for all of these ABC and D are the beginner of the answers so let's do this we need for this given question we need options options is going to be truly just be an array and then what we need to do is say for the question we need to find all P tags and then for each one of those if it begins with a b c or d we're gonna push it into options so we'll say each which is built into that this gives us the index and the element and then we can do we need to sum like this if that element that matches so we just want a regular expression that says starts with can we do a like a group A or B or C or D so if it starts with a b c or d case-sensitive then it's an option and really we should we should just put this in a variable like the text itself that dot text so then we can use it so if it starts with that then i'm just gonna say options dot push the text like that so now we should get options let's see yep we get we get the options so now we can see a Rhode Island B main c washington d florida we can we'll trim the text to get rid of like any new line so you can see that the new lines are showing up there you could use starts with yeah that would also work but i'm okay with this and actually what we could do is extract out just the options so instead of putting a B C or D let's make sure that the answer that they give us always has the actual answer in it so like answer C answer B answer D answer a so yeah I think what we'll do for the options because in our game we could make it so that it actually shuffles these so they're not always ABCD yeah so look for the period and then remove it so I that this is what I'll do with the regular expression so let's call this change so now instead of matching I'll execute this regular expression to extract out the part that I want so I'll do that dot exec on the text but it should be followed by a period and we want to escape that and really I don't I don't know what we get back with I think is gonna give us back like an array we should be able to say well period and then followed by any number of whitespace and then this part right here everything else we want as a matching group so that'll put that in there yeah so Oh starts with eight starts with A to Z then our period can I do a 2d because it only it should only ever go up 2d like that and then that should give us our matching group let's do it yeah so this is an array let's just log it because really I think we're gonna want to say like array at 1 because that gives us our actual value yeah so arrey at one gives us the actual option so we're gonna push matching at one there we go so now we have an array of options that are just the option itself it's good to go and now we can pull out the answer the question is gonna be harder we already talked about that because it's not always in the text but we'll figure this out so but at least for the answer we can do a similar thing so we're gonna iterate over any P tag that begins with answer and then we'll extract out the actual answer from that thing we'll do that so I gives us our options let's call this answer container because what we and we can also call this question question container and then really my answer we're just gonna start off as nothing and will say answerer container find P tag and let's just look at the website to confirm but I believe when you look in here oh it's no it's an h2 we actually could just extract out the h2 that's easy so find the h2 that's in there grab the first one because there should only be one use a selector like this grab the text and then do a similar thing that we did up here so I want something like this teamfight is 56 minutes so the thing is like I could do this it's just this is gonna break seriously but like not it's not gonna break but it could break answer what's the format answer : any A through D period and then the item itself text that at one shot trim now this is a big mess and if I wanted something better like two I probably wouldn't do that let's just see if this works if this works that's amazing it works but because I have this big ol thing if any of the other pages don't have the layout if these in the exact same way this could totally break so if the answer isn't always in an h2 then this could totally break but that that does it right so we have our options we have our answer options and answer now we need to actually get the question for that will at least get the question image yeah I'm not looking at chat at all I mean are we in math class No did I did I do i plus equals to I did yeah cool I went to college yeah I have a computer science degree yeah and everyone definitely check out wherever works he's doing some good stuff over on his YouTube channel okay so what I think we'll do first up is we will extract out this image so if we look at it and hello Mutasa welcome to the show listicle image wrapper we can at least select by that mystical image rapper grab the image inside of it and then it has these data attributes um we don't necessarily need the source set we actually could just grab the source itself set actually we can grab will grab the alt text let's do this let's just grab the alt text and see see what we get Rhys is complaining because it's Reda cleared let's do that okay so in the question container so we'll say a question container find the element with the class listicle image wrapper and I want the image that's inside of it and then that whole thing will put in varying instance and we want to grab the alt attribute and that should give us the question so let's just see if every single image has a good alt attribute undefined did I do the right thing listicle image wrapper IMG yeah so that should select the image let me catch up no tesseract interesting oh is that is that an OCR package we could do that I don't know if if we're up for that though we only have 51 minutes left and hello Thea queen whole welcome and hello barrique coding is it depends I like to I like to make coding easy a lot of people make it convoluted or hard and I like my code to be clean and easy to understand for the most part so it really depends optical character recognition so what we could do is we could run this image through an optical character recognition text to get the question but yeah hmm is my break timer yeah and that's a good point Alka so you can right click and it's click things and say copy selector we're really you can even just right click this and say copy selector but let's see what it gives us I think that's what we need at least let's see what I did find that and the immediate image it should have still worked the alt tag is what we're looking for right the alt attribute um yeah oh let's just see if that changes anything no it doesn't if you look at the source that we get back oh I mean the image itself has some classes on it really though we could just find the image let's just do this instead of like a more specific selector we're just gonna find the image that's inside of the question container and then get its alt attribute yeah what is that so this doesn't work only u.s. state that only borders one one other that's fine the name the number that and then the actual question itself was in the the lower area so I think what we need to do is if it's a scenario like this where there's a lot of text that is actually the question what's the missing number oh man for this one you absolutely need the image okay let's let's at least just get beat let's just at least get the image so instead of getting the alt attribute we'll get the source let's see if that's actually set it is that's a data image we don't want that we want to grab the data original source this attribute here and let's call this the question image so for this one wait are these all the same oh no one oh two oh three oh four they're all different but that gives us this image I think that's fine one last thing is we can actually grab the the answer text we're like the answered descriptor yeah so it's just the first P tag in the answer it should all not the first it's the second P tag but it doesn't have a class like in this one it's the second P tag okay here's we're gonna do so in the answer container call this the answer description we're gonna say answer container oh and I'm missing chats I'm sorry do that and thanks for the follow Dracula I just did some OCR on the stream what is the only ghost good job Alka if you do exclamation more break you can see a link to my break timer that's the plan Mejia oh yeah where we just got to finish the scraping tesseract OCR okay I might check it out do I know how to remover of aid page blockers not exactly I don't know yeah I don't I don't really it depends on the situation so okay so I'm gonna find P tags I'm gonna grab the first one and that should be the answer description so let's do this we'll make it a selector it wants a text and we're going to trim it yeah I always trim it just because you could have new lines so let's see what happens if I don't trim it you might see a new line on the end and so trim just removes any extra whitespace yeah I didn't necessarily have to trim this one though but the thing is the site is so ski it's so scammy but like every single answer links to some other place if you can figure these trivia questions out see if you can solve these tricky math puzzles oh well it's fine this is fine I could remove the last sentence sometimes I don't need to remove the last sentence okay so at this point we should be good now we can just throw this algorithm at the Oh actually let me make sure this is all in one paragraph oh no this one's not this one has multiple paragraphs in the answer okay here's what we need to do we need to grab all paragraphs except for the first one because the first the first paragraph is the image wrapper does anyone know a CSS selector that says give me everything except for the first one if not I can just iterate but is that even a thing CSS select all P tags except for the first one but otherwise here's what I'll do I'll say answer description initially is an empty string well then iterate over not first child something like that but here's the other way to do it so find all P's and then for each one bring it the index and the element not first of type this could work let's try it because so we'll say find P tag not first oh no no P oh and then I can pass in a the class that I don't want because I actually don't want this listicle image wrapper class so any P tag oh no do that any P tag that is not listicle image wrapper and then grab the text from that let's see if let's see if that fixes it yeah the only thing is is it grabs this one too grabs the P tag in here that's the credits overlay no it's a span though that actually didn't work you oh gee Rio has a nut let's check it out I probably shouldn't be spending this much time look we only have 40 minutes left not selector remove elements from the set of matched elements okay let's try that so we could do P tag like this give me all P tags better not this class let's try that after some reason it's still getting the the credit 13 minutes of scraping has elapsed thank you thank you deep sense do I have the class name Oh maybe they just have like malformed HTML the credit is inside of the P maybe I do not image credit this isn't even a reactive this is just backend that's not working okay I'm gonna do we could probably figure this out but right now we're not gonna figure it out so what I'm gonna do now is just iterate over all the P tag except for the first one so initially answer description is an empty string and then we find all P tags for each one we get the index and we get the element feels like one hour and then we're gonna say if index does not equal zero then we'll say answer description plus equals this element oh we could use slice let's see that that would be easier let's see slice okay we're gonna do that thank you thank you thank you we'll go back back to where we were that's actually great I haven't used a lot of these cheerio methods so find that and then slice one which should remove the first one there we go that did it yeah I did it okay so last thing I guess is scraping multiple pages we still kind of need to do OCR oh no we need to get this question I totally forgot about that we need to get the the first paragraph if it is there let's see what it looks like you yeah I think again if we so if we select the paragraphs in the question container skip the first one and if the first paragraph isn't doesn't start with a Vande it's the question I think that's what I'm gonna do now we have 38 minutes left yeah so if it doesn't begin with that so let's see for this one if we selected the first P after this P it would have a so we wouldn't choose it this one if we selected the first P there that's okay that should do it Cody says they use test rack before it works okay but one issue though is some of the images like this one kind of weird fine and they like this one find the distance we actually need this text so we're gonna we're gonna grab that text so let's just say this question actually we'll just say let question equals we could default the question to the alt text I don't never mind let's just do this do they all start with no they all they all don't start with quotes so we'll say a question container we want to find paragraphs we don't want the first one because that's the image and then if we slice it and then actually mean technically we could just say find all paragraphs and then get the first one because not the zeroth one the first one it's a possible question because that question Part 1 image text + part 2 the alt text yeah I think if we have time if we have like 10 minutes at the end I might play around with tesseract but for now I think this is good so we'll say if possible question that text actually we'll just we'll just throw text on there right now if possible question dot start - starts with except a regular expression no starts with I want that one starts with a period if it doesn't start with that then we found the possible questions so we will say question equals possible question like that okay and then we put the question in here okay question two people are standing back-to-back they walk each yeah so that back up that one a question that one didn't work what's the second to last one is this like a what country is this oh no it's just this what's the oldest continuously inhabited city in the world yeah so we we would have to do OCR on that one what is the missing number for that one you need the image also so I think we're pretty much always going to show the image on whatever trivia site we create name the number that is three more than one-fifth of one tenth of one half of 5,000 that's good and then that one again you need the image okay I think I think this is this is as good as it gets so what we'll do now is we'll scrape multiple pages so I already have this set up so that it scrapes page one let's see what happens if we try to scrape page two so basically the URL is now going to change to be page / - we're gonna use the same scraping logic and hopefully nothing is weird and it works so let's try page two cannot read property one of null great I knew something was gonna happen so on line 30 let's yeah so this is that thing I mentioned earlier why I probably shouldn't do this so let's look at page two and see why that broke so we're gonna go to page two and this specifically is happening let's just log the index it's happening on the fourth - yeah on the fourth question on the fifth question so if we look at the fifth question which is the last one this is saying you answer : 8th Oh No look they messed up well no they didn't mess up this is a question that's not multiple choice oh man okay so that's multiple choice this is multiple-choice that's fine this one's multiple-choice that's great oh this one's tricky let's see this is horrible it's so bad because honestly we need to get the entire like after this we need we would have needed these clues because we're missing these clues in this one but we would have gotten the multiple choice cool this is not going to it's not going to get that answer and then this one is not multiple choice what's the only u.s. state whose name ends with three consecutive vowels so it doesn't say like insert ABCD which is why my regular expression broke so here's this is what I'm gonna do we're gonna say no ultimately we wouldn't have found any any options so I'll say this let answer equals nothing and if options that link so if we have options then that means our answer is going to start with a b c or d else our answer is going to be a little bit different it's not going to have a through d it's gonna be like that cool so this gives us page two and now the questions totally for that one at least this is so bad he is horrible ah okay that's good no question there you're trapped in a room with two doors only one door will lead you out one guard does all that okay that's fine I think I feel like it's good enough for now we can always revise this we could do OCR later on all right let's see if we can get page three we got page three can we get page four well this one didn't get any options all right I'm gonna go back to page three all right let's see what happens on page 321 230 fill-in-the-blank James mom has four children their names are April May June and I see so there are no options for that one that makes sense then you yeah so there is an alt text but it's not always correct oh yeah click here for this one weird web scraping tip yeah this is a horrible accessibility for sure a function to check if the question is multiple choice yeah so ultimately at this point the way it works out is if there are no options it's not multiple choice we could just skip anything that's not multiple choice but that's fine all right page four okay page for the very last question says which is the definition of peruse to read or examine something carefully okay that's good what is the maximum number of words that a single dog can understand 200 and then this one we didn't get the question how many legitimate children did King George the third have the answer is six what is that the right thing dogs can understand words I guess so oh no no that was that 115 okay we got that one and then the one after that is about King George - how many people stood in line of succession between King George the fourth the fifth I don't know Roman numerals okay so that one's good okay page four is good try page five it's probably okay page six yeah page 7 ok how many pages are there Ivy is fourth okay page seven there's only five per page what's 50 there should be ten pages yeah let's go all the way to page 10 oh it's good and if we go to page 11 it should break entirely because it doesn't exist yeah cool so here's what we're going to do here we're going to return the questions we are going to create a function that says get all trivia you know what no we're just gonna have this one this one function and we'll have a graph QL in point where you can pass in the page number maybe no we need to get all of them I think I think we'll get all of them okay get all trivia questions I kind of want to do one at a time so you can do this we can make it an async function and put in a weight inside of a for loop a Roman numeral decoder I did that in JavaScript a while back going would be fun yeah so we just need a for loop goes from 1 all the way up to 10 we'll just say questions is an array this let a equal one o is less than 11 I plus plus it's complaining but that should be fine and then we can just log all the questions know await in Oh get trivia page they should just be complaining about Noah wait and loop that's fine the the reason it's complaining is because when you put in a wait inside of a for loop it actually waits maybe it waits before it it goes around to the next generation on the for loop well that's okay with me because I want to get each page one at a time instead of bombarding their site with requests but I also could sleep I could just do something like this I wait a new promise set time out so that way it waits one second between requesting each page and that way again I'm not mum barding their site with requests let's just log requesting page I and then it will log out all the questions all right here we go I got a call the function one two three four five six seven eight nine oh I love the fake stretching when you get into what you're doing it happens right because I want to keep coding so we did it now we have a big long array of really messed up trivia questions now let's put a graph QL API in front of it we made an app that counts yeah like if we did anything we at least counted up to ten look at us where is it here look at this we at least counted up to ten oh man thanks for the follow destock dude we could do that to do them all at the same time the main reason I did this is just because when you're scraping you don't want to bombard a website with requests because we could request all ten pages simultaneously but I do want to actually wait between each one yeah hello Alex in your welcome Irwin and thanks for the follow toasted cheese I miss anything else all right requesting a pager number should employ if the page number is divisible by three and five yep okay I think we have what 30 minutes left we don't have long all right let's just put like a simple graph QL API in front this cuz it should be simple right I have used graph QL yoga I don't know if it's still a thing and if people if it's still like maintained let's look at it deploy to AWS - okay so graph QL yoga was last committed in August they do have some issues open but what I know about graph QL yoga is it is super simple to set up a graph QL API it's it's built on top of Apollo server yeah so it is built on top of Apollo server but it just makes it super simple to get up and going you basically just do this you have your type definition you have your resolver and and you're done we could look at a how tricky is it to get things up and going with Apollo server yeah I spent way too long on scraping we should have picked a better site stand alone so install Apollo server and graph QL require it in add your schemas add your resolvers and then you're ready to go okay so what is graph QL yoga do better than that this is kind of all that I need anybody watching have you ever made a graph QL server what's the best way to go in hello Alex hello [Laughter] well thank you or when we did our best on the scraping we really did Apollo is the way to go okay I'm curious though how is it anyhow what is graph QL yoga dude oh you know so um graph QL Yoga adds the graph graphical interface automatically automatically adds subscriptions Apollo is lame says Steven and then Apollo so do I get graphical by default by using Apollo you okay and let me look at a Paolo Express server which was just suggested Paulo Server Express so require and Express do this create an Apollo server you what's the difference I've never used react lazy or suspense should I just go the Apollo route can we get something set up in five minutes let's try it I don't know we're gonna try it we're just gonna do Apollo's rumor we're not gonna eat we're not gonna do Express or anything like that so let's do this we're not we're already in a server folder so well I technically already have Express installed we'll do a Palo Server Express it was published 14 days ago Shiva Wingo oh this is Apollo server this is like an official package oh no it's not you Oh Server Express okay it's it's official it's from the graph QL team yeah okay that's what we want we're gonna install Apollo Server Express this is our basic graph QL in point Cody's asking what does graph QL actually provide versus a REST API is it just buzz tech no it's pretty cool actually so restful api is have multiple URLs right they have multiple endpoints that adhere to the rest standard so you have like a get in point on a resource that returns everything you have a get in point with an ID in the URL and that returns one so rest is all about defining endpoints in the URL whereas graph QL has a single end point so from from a client perspective you still could technically make a post request to your graph QL server but there's only one endpoint and your client sends a query that could potentially select multiple documents it could like query documents it can do multiple things but just on a single in point so it's not really a buzz word it's a lot of people build their API is like this okay so this should give us a basic hello world let's get it going I'm gonna insert times run it out okay the stall node maaan yeah I think Apollo server is basically Apollo server express it's just like specifically for express all right we got node Mon let's add some scripts we're gonna create a dev script for a node mug and then run it Oh source index all right so we have this graph QL endpoint and by default we get this graphical UI where we can interact with it and what one cool thing about a graph QL endpoint is that it exposes its schema typically production graph QL servers don't expose their schema because then anybody could like query all of your data but we can see the the schema that was provided by the backend so there is a query type called hello that's going to return a string so I don't know how this works query hello go did I do it right I didn't dear I don't know how to do this let's see how do we actually query it in see you later client telephone thanks for hanging out with us yeah twitch has so much data the graph QL is pretty useful if it was available yeah definitely remove that remove the parentheses I should probably just look this up but like this go go syntax error you oh do I not need query do I need do I literally just do hello well no I do need a query remove the parentheses on query hello hey how could how could I build something even know how it works but okay so our back-end exposes this in point called hello and then this would be like the the query that you would send from the client the back-end receives that and then it uses its resolver to return hello world great alright what we need to do is we need to define a schema that represents all of our scraped data so right now we have a query type hello that returns a string but I'm going to do a query type of questions that returns an array of trivia question probably like that we'll look it up and then I need to define the trivia question type like this and if we look at our data I'll do go to let go laying eventually scram help me it's been a while since I did it yeah yeah here take a look at a graph QL request for some twitch data let's see it's tight query I think it's o-type query is basically saying we're defining what actually this is like the initial query type I think so because all of your resolvers will go in here but here's the interesting thing about graph QL which we're about to demonstrate is your resolvers can literally return anything so graph QL is not like a database technology it's really just an API layer so you can create resolvers that talk to a database or talked to a Postgres database or create resolvers that talk to external API so I'm actually gonna create a resolver that talks to this scraping thing that I just did so what I'm about to do is define my types so basically I'll be able to query for all trivia questions and so I'm gonna define the like the type of a trivia question which has all of these properties like this and I think there's some online converters right let's just see jeaious object to graph QL [Music] can I go to google utilities not introspect convert object to graph QL schema generate graph QL schema graph QR code generator there there's got to be a way to do it [Music] you rath ql code generator try it out [Music] graph QL compose get started no I mean I could I could actually just create the schema myself but I know there are tools that will do it I know there's like a JSON schema generator see if we get JSON schema dotnet and there's one converter that has just like a bunch of built-in conversion types time to write my own generator well we only got nine minutes left [Music] we could we could just say this stream was all about scraping JSON to graphic you won't thank you nice la let's this should work let's see nice okay so you pass it in some JSON alright now this is technically not JSON this is a JavaScript object we need to convert it to JSON let's see if it'll work it'll probably give us like a syntax error it's gonna be a good game convert ya invalid JSON let's say a J s object to JSON it could technically like json stringify it all right we're gonna json stringify it this alright so that gives us back some JSON now we plug this into there um wait what oh I have to click convert okay question oh that's he's done this bye so our trivia question has the following uh so question is a string question images of string answer is a string answer description as a string and options is an array of string cool so now we have our questions resolver and ultimately that really just needs to resolve the array of trivia questions so I can import in this get all trivia questions we're gonna word we're gonna remove the timeout between requests so it should request a little bit faster so he's going to return questions and we can basically create like a little caching resolver so this thing is going to run once to request all of the pages and then we're gonna use that in this resolver here to resolve the questions so let's just say module exports equals get all trivia questions well let's do this exported in an object and then over here we'll import it in import straight not from require that scraping file and then for our resolver let's create a nice little function this is going to be an async function let's do this we'll say let questions equals null and right here we'll say if questions is a thing and then just return the questions otherwise I don't really even else but otherwise we're gonna say questions equals get all trivia questions well scrape get all trivia questions not using pure functions no mainly because I want to have this cash I don't want to I don't want to scrape every single time we're only gonna scrape the first time so there needs to be a trivia app yeah but we only have 5 minutes we could but you make it happen watch me watch me but Pranjal asks if graph QL is back in technology how can it replace Redux they have a different library so Apollo client is what would replace Redux in like a react app this is gonna be scrap get all trigger your questions and we're going to return questions like that all right so we have ourselves a very simple graph QL API that can resolve those questions which came from a scraping API is it still running no it's not [Music] syntax error unexpected name found oh how do i oh I did this wrong this should be a ray of trivia question like that there we go okay so now yeah we can go to my graph QL front-end will refresh this page we should see a new schema definition yeah so now we have a query for questions so I'm just going to query questions and of those I want the options and I want the question let's just say I want those two things go it's gonna take a second because we do have to scrape on the back end right we had to scrape all the pages but now it's actually cached in memory and I can do I can can you continue to do more queries so now let's say I want the options the question and the question image and it should come back like it comes back instantly because it already cached the data but now I'm just plucking out the properties that I want I mean and technically I kind of want all of them so a question image answer we get that to answer description get that too and yeah that's great wait we have a graph QL endpoint that's awesome I believe we could query it too I'm not familiar with the query syntax but we could say overly give me questions where the answer has the word main in it or something like that yeah we have five minutes hold my water okay we have three minutes left let's build a basic View Client to pull these in and show a random trivia question yeah you came in lace late sigh we tried to scrape this website which is just absolutely horrible and we didn't really realize until we were like 10 minutes in that some of the questions are only in the image Oh is bad but here I'll link you to this trivia page but it was a fun exercise in scraping really badly formatted web pages because we did our best we did our best okay so what's great well we already have our view app I think we're good to go there so I'm gonna keep that running and go into the client which is a view application and is there an Apollo client for view view Apollo client protecting against blind pew of people for sure a view Apollo in a great graph QL interview apps getting started [Music] installation do you add Apollo I can do that and then let me go into basic usage to clear all Apollo queries in your view component as the Apollo object and the component options okay cool so we can just in our in our component we give it an Apollo property with the query that we want this is gonna be too easy watch me add graph QL API server no thank you configure Apollo engine yes Apollo sir graph QL I think it can work over either I mean by default it just uses HTTP so by default it's literally just a post request to slash graph ul like if we watch the network traffic here it's clear it whenever I submit this it's just a post request to slash graph QL but the body of the request includes my query and then the backend handles that okay ad example code no no no okay we're gonna do it I think we have one minute two minutes we have one minute we're gonna do this I swear I swear we're gonna shoot we're gonna at least get the oh wait do I need to add graph QL tag I guess I'll already have that but it's gonna be great I swear okay it really didn't like some of the stuff that it added but we have our app dot view let's just do this we're gonna try to show the questions show them in a pre tag probably do like json.stringify the questions put them in a pre tag get rid of all our styles we'll add a script don't know why that isn't working export default we're gonna throw this on there we're gonna say Apollo we want our questions query we need to import in GQ Oh time is an illusion did it not does it not install this do we need to install that and then I can just copy my query from the playground so I can take this and now this is going to be the query in my view component here like so and then we should be able to see the questions let's look at how it got set up so it created the view Apollo provider I need to pass in I do need to pass in the URL to my server right installation HTTP link so we have times out times up we're out of time but I bet I can show you a random trivia question in say five minutes only five minutes of extra time only five minutes but we're gonna we're gonna figure this out okay but we're adding view Apollo I think I need to create an instance of an Apollo client because I don't know here let's see what it's doing HTTP endpoint localhost 4000 no where's mine my endpoint is oh it is it's already set up when is this complaining about you plug in a how is that not listed in the client dependencies you see oh i plug in apollo it is there what are you talking about oh it's a dev dependency that's why okay it should just work watch this watch this you're gonna be amazed it's just gonna work well that didn't quite work what it wouldn't I replace we have our app dot view there's nothing in here oh I didn't save it that's why save it save it yeah we literally have a view app connected to our graph QL API beautiful beautiful okay so now what we'll do is we'll just show a random question so let's just add a computed by computing not a computed we'll have like a data property that is the next question or like get random question data so it's gonna return an object we have current question which initially is null and then we'll have a button to set the current question so we'll add a little button we'll say next question and then when you click this thing we're gonna call next question cool so now we need a next question method next question and we'll just choose a random one so we'll just say this current question equals this dot questions at some random index so math dot four of math.random times the spot question is that link wise is complaining what should be fine unexpected token expected a comma oh I need a parenthesis because of my math top floor like that okay so I think I have the view dev tools No so we can look at app we can see that current question is null if I click next question something breaks you so we can say this dot current question equals this questions this should work right so when this thing is clicked call the next question function let's just make sure it's being called and also let's look at this dot questions here oh ok so we do have this stock questions it should work with with the pronoun oh it works it works for some reason it just didn't update okay so mm-hmm it's it's totally fine yeah you can do it with or without the parenthesis I've always done it with the parenthesis because I think of this as more of like an expression that will evaluate it versus like in the react world this is more of like a reference to a function but in view that's not the case this is an expression that's evaluated okay so it should work so now that we have the current question we can display some stuff so we'll have a P tag and really we'll just put this all on a dim and if this thought-current question is a thing we want to show it so we'll just do a via just not this stuff but current question so if it's a thing then right here we're gonna say current questioned question if it actually shows up then we'll just show an image tag where the source is current question question image like that and then we need to show all of the options but let's see if what this gets us next question boom we can adjust the Styles later probably how many legitimate children did King George the third have next question which is not one of the US states that does not allow the lottery next question in which movie did the song white Christmas make its debut next question okay so let's do this let's put the question text below the image and then below that we'll have a div that's gonna have each of the possible answers inside of a button I guess so we're gonna repeat this button for option in options the key is just the option itself it's not how that works like that and then inside the button we're just going to have the option next question oh yeah current questions just thank you that's that's why that's breaking so current question is the thing and we want its options buzzer did go off oh it's done all right I swear I swear I'm not gonna stream for another hour but here we're gonna start counting up we'll see how long this takes me I think I can finish it in a couple of minutes okay there we go so we get our buttons now when we click a button we'll determine if that is right or wrong so oh geez what happened there no this is bad is very bad but when you click on this option I will say guests with that option and then we'll just have a simple guest function that takes in the option and we'll say if this current question dot answer is equal to option then we're going to alert correct and then we'll just go to the next question LS will alert wrong correct answer was and then we'll just throw in this current question answer that's it this is this is our quiz game that's all you get and this needs to be a template string yeah template string here we go all right it's ugly next question which okay I'm gonna add some styles before I play this game let's just call this the class is current question so current question has a width of 100% and then current question image also has a width of 100% let's say the font face is sans-serif font size not font face font family who are you what is this font size two rims and text the line is Center and then let's say any button has 100% width so current question button 100% big font center the text all right here we go and actually max width nah are we fine let's just style all buttons that way and it should be display:block all right here we go nice okay we got ourselves a trivia app next question which original voice which original Winnie the Pooh voice actor attempted to create an artificial heart I'm gonna go in you're wrong the correct answer was Tigger all right next question how many illegitimate children did King George the third have I'm gonna go with 15 correct next question we did it um I can add a little bit of margin on this on this button down here or actually just let's just add a margin on the current question altogether margin one rim Oh Thank You Alka oh no that didn't do it well then we'll just do like with 90 percent there we go good enough is it mobile-friendly though of course it is look at that you play this game on mobile all day so good those don't really look like buttons on mobile for some reason all right next question which of these US presidents didn't earn a four-year college degree anybody know I'm gonna go with Rutherford beat Hayes wrong the correct answer was Harry s treatment okay what was the original name of the band Led Zeppelin I'm gonna go with the rain wrong and the correct answer was the new Yardbirds wicket Lester alright I appreciate that everyone yeah so geon's has learned a lot Irwin says love the scraping I learned a lot cool ok I'll push this coat up I'll put the link in in the description yeah always go see I think that's a good plan you can make this so much better you could make it you can have points you could have a timer we could add real time where multiple people are answering it and the answer with the most votes wins I'll you Kyle except pull requests absolutely yeah there's so much we could do with this but for a roughly two hours of coding a little under two hours of coding read chat is YouTube I'm sure me on both but in two hours of coding we scraped a very weird website this thing we built a basic graph QL API that does the scraping underneath the hood and then we built a view Jay s application using Apollo client that can grab those questions and give us a random trivia question so I say that's a win what is the most common street name in the US Main Street wrong correct answer was second ah what color are the not so golden arches what color are the not so golden McDonald's arches at the McDonald's in Sedona Arizona okay first of all this trivia set is called 50 trivia questions only geniuses will get right what makes you a genius for knowing what color the golden arches are in Sedona Arizona um but I'm gonna go with red and the answer was turquoise oh I know this one Tigger correct okay no how can you it have you been there okay all right I'm gonna go but yeah I'll push the code up I'll put it in the description I'll link it on the discord thanks to everyone for hanging out with me this was a fun stream we didn't know what we were gonna do so we did some cool stuff and it was a good time so let's find somebody to raid on Twitch okay is a genius no one I follow is online the Promethean has zero viewers how is that right are they ending their stream Oh No it's 53 what prime ajan prima Jin jee-in stands for genius oh okay this alright you caffeine overloaded jhin's let's just search for it anybody doing web development you react j/s and link of legends are you streaming right now X metrics can't really see their screen Wow we could just write wish it was should we raid code phobia yeah they're working on a twitch extension that'd be fun I'll read them just go to sleep is he doing going really all right so we're gonna read code phobia and if you're watching on YouTube head on over to twitch dot TV slash coding garden I'll give you a minute or two to get there and then we will raid code phobia altogether go oh yeah it says go it literally says go oh yeah and who has who has the raid message send it no copy and paste it we want the seedling we want the heart yeah there you go Shaggy's got it hello shaggy yeah so copy and paste that message when we go over there we're going to send that message we're gonna share the coding garden love are you streaming right now okay working on a big project oh go on the back end okay cool alright okay alright we're gonna raid code phobia thanks everyone for watching this is super fun like I said I'll push up the code I'm open to pool requests and I have a feeling I'll never touch this code again but maybe we'll revisit it in a future stream so wherever you are in the world have a wonderful morning afternoon evening or night and until next time here's this [Music] you

Channel: Coding Garden

Views: 6,594

Rating: 4.941606 out of 5

Keywords: full stack web development, vscode, mechanical keyboard, live streaming, html, web, apollo client, beginner, frontend, news, web scraping, graphql, vue, javascript, cheerio, full stack, css, q&a, learning, lesson, apollo, live coding, educational, full stack javascript, node.js, tutorial, learn node.js, debugging, programming, backend, qa, live stream, express, learn javascript, web development, node-fetch, coding

Id: oD-sS4sLApk

Length: 152min 5sec (9125 seconds)

Published: Thu Jan 23 2020