Laravel Worldwide Meetup #8: Using Vite in Laravel & How to optimize your DB indexes

here we go hello everybody welcome to the 8th edition of the laravel worldwide meetup like in the previous edition we have two guests that will talk on two very interesting subjects later on my colleague sepp will join us to talk on how to use vit in larval but first up we have a talk on db indexes let me bring on the first guest guys sasnovski and here's guy hello okay how are you doing i am pretty good i am terrified by him i'm fine i'm sure i'm sure everything will go okay man don't don't have too much nerves i guess uh it's too late it's okay you're going to talk about indexes right uh yes um but it i think it's going to be a bit different than what people might expect from the title so we'll see how it goes okay uh the stage is your man i'll let your screen be visible now and take it away i guess have fun cool so hello everyone hi laravel worldwide my name is kaisnowski and um i am here to answer the question of how should i index this query and the answer is it depends so thanks everyone for having me you can find me on twitter on my website or on github uh shout outs to freak for having me enjoy zeb's talk next so i'll just assume that everyone laughed now and i'll just move on so here's the origins of this talk if you don't know already i am currently writing a book on database indexing and i recently sent out this tweet asking people about things that i've always wanted to know about database indexing and there were two reasons why i sent a suite one was well i was actually interested in what people want to learn about indexing but i was also interested in the kinds of answers that i get like what do people associate with database indexing if i ask a question like that just the type of answer i get is very revealing to me and the answers actually were pretty interesting to me most of the answers were some variation of does it make sense to do dot dot dot something when should i do x and most commonly how do i optimize this query so these answers were really interesting to me and it might not be immediately obvious to you why why they're interesting and i don't want to spoil it at the beginning instead i want to show you a few examples of a query and then i'll ask you like okay how do we optimize this and we'll play through a few variations of the scenario and see how our answer might change and then hopefully by the end you might understand why i think these and these questions are kind of interesting uh but i'm also going to reveal it in case uh this talk is so bad that you don't know what i'm getting at okay cool so here another side note that all the examples in this talk assume that there are currently no indices anywhere that's just a simplification to remove a few variables right so assume that any query you see there aren't any indices on the table at that point all right just a side note okay so here's scenario number one you have the following query you want to select star from table where some column column one in this case is greater than equals to seven that's the entire query pretty straightforward and now the question is how do i index that and i actually want you to think about how you would index this query yourself if you were asked this question here's the query how do i index that in fact i want to go a step further and i actually want you to go to this page that will hopefully appear in the chat in a second as well and if you go to the site there's going to be a small straw poll where you can vote on how you would actually index this query and even if you don't know please still vote there is a i don't know option um the more answers we have the more fun this is going to be i hope um so to account for um street delay and letting people think about this for a second um i'm just going to like awkwardly try to buy time for a minute or so until we have enough votes i don't actually know how many people have voted yet this is like what you see is what i see i don't have a presenter's screen for some reason because i couldn't figure out how to set it up so 20 votes already okay thanks that's uh wow what just happened yeah so anyways how's the weather guys cool right yeah right on so um it's loading slow someone says all right you know what i think fray can you give me one more update on how many people have currently voted yes we have currently 29 votes 30 so okay so you know how many people are watching so i know like roughly uh let's see i think the counter is now at 61. so okay so half of you still haven't done your job so please that's not what we're paying you for all right so okay i'm just gonna move on to the next slide and uh i really hope that this works so in theory okay cool so 33 words um i don't think if i can actually refresh this i might just oh it's live refreshing oh wow this is the future and also this is some pretty pretty nice sanitation going on here okay cool so let's look at what we have 36 votes um and we have 28 votes so the majority said put a regular index on column one followed by i don't know and then there's two people who said put a descending index on column one all right cool so like i think the trend is visible um i'll just leave this page open and while i talk so this makes sense right like if we put an index on anything it has to be column one because it's the only column that is even mentioned in the query um we do a select star so there's no column there and our where clause only has a single condition and that's on column one so if we were to put it anywhere it has to be column one and so a regular index meaning not a unique index or descending index makes total sense right that's a pretty straightforward query except that it's not the correct answer you have activated my trap card the correct answer was in fact two three i don't know so congrats to the eight or so people who voted i don't know you were actually correct you don't know because i haven't given you enough information to answer this question all i have given you is a query without any context whatsoever right so asking how do i index that you can't actually answer this question let me show you what i mean you see there are a few things that i kind of forgot to tell you um i just didn't think they were interesting so first of all the table we're talking about has about six million rows okay so it's not a it's not a gigantic table um but it's it's respectable right secondly oh yeah that's sorry my bad i forgot to tell you that the the values in this column in column one they can only be between one and nine that's sort of a domain thing right it has to do with the kind of data we're actually saving in this table so we have this very natural constraint that it just it can never it will always be between one and nine um sorry i just i forgot to tell you that and thirdly so i recently decided to have a look at the data that i have in this table and analyze it a bit and specifically what i was interested in was how often in these six million rows does each of the values between one and nine actually appear right so that just basically a very straightforward histogram um just counting how often does each value in column one appear in these six million rows and it turns out that it looks like this so what does that tell us so without even having to look at any of the specific numbers here one thing we can immediately see is that the data is very much not evenly distributed right it is heavily skewed towards the top end in fact almost all of the data clusters around the seven to nine range so if we have six million rows total in this table like if i just look at this um these three columns probably would sum up to i don't know 5.8 5.9 million right so the vast majority of our data is actually in this upper range so why is this interesting so one thing that you need to know about indices is that they don't work well for queries that return lots of data and i'll explain why that is in a second but that is just that is one of the properties of an index it does not do particularly well if your query will return lots of data so why is this interesting here let's get back to our query and i want to ask you another question there's no audience participation this time if we look at this query i have removed these seven down here and just replaced it with just a placeholder so what first of all what's our comparison operator so in this case there is a greater than equals right it is not an equal it is greater than equal so right off the bat we're selecting for a range right that can match multiple values also this is a range without an upper bound right it's just anything that's equal or above this number so the question now is if this is my query and given that i have the data that i just showed you that's my data distribution six million rows and it's distributed like in the previous slide what's the minimum number of rows this query could possibly select so if you wanted to select the fewest number of rows possible that is barring any nonsensical values right if you say greater than equals 10 obviously you're gonna get zero back but um we know that this is our domain so we'll pick valid numbers here so if i want to select the fewest number number of rows possible i have to maximum value right give me anything that's nine or above and there is nothing above so basically this would just get me all the numbers that have a nine in column one and if we do just a set of count to see how many that will give us okay that is about 1.9 million rows so where does that leave us uh we have a query that in the best case it will select 1.9 million rows any other value that we put in this weird where condition is going to select more because we have a greater than equals right it will include everything above it as well so in the best case we will still select 1.9 million rows and indices don't work particularly well if you return a lot of data and 1.9 million rows is quite a lot of data so why don't indices work that well for for queries like this this is a bit beyond the scope of this talk so i'm going to go through this pretty quickly i would suggest you watch my talk at lara con eu 2018 where i go a bit more in depth on this but here's the really quick explanation so first of all an index only contains data from the columns that you actually put the index on so if you were to put an index on this column one from our table the index would only contain the values from column one so it will only have like ones and twos and threes and nines right so that means any additional data that our query needs needs to be fetched from the table because it's not on the index and in our case we're doing a select star so we want to know every column but we don't have every column on the index we only have column one if that's where the if that was the index we were going for so we have to go back to the table and read the row from disk to get the remaining rows that we're interested in sorry the remaining columns that we're interested in and we have to do that for every row that is matched and we just saw that the best case scenario is we're matching 1.9 million rows so that would mean in the best case the database would have to perform a little over 1.9 million round trips to the table 1.9 million reads from disk that is just absolutely monstrously going to overshadow any benefit that the index is going to give you because this reading from disk is going to be a bottleneck so if you were to put an index on this table the database would never use it so okay given all this information we know what the data looks like we know how many rows we have we know that indices don't work particularly well for this type of query so how do i index this you don't that's in this case the answer is you don't because this is not a problem that an index solves right the query might be slow but slow query does not necessarily mean the solution is an index right because in this example the problem isn't that we it takes a lot of time to find these rows right that's really what an index is used for you have a very small subset that you want to find and you want to find it quickly that's what an index does if your query matches basically all of your data the index is not going to help you very much because what you need to realize is an index does not make a query more selective it is not going to change the result that will be horrible right so if your query matches three million rows and index is not going to change that so in this case the solution is you don't index but you could only really know that this answer will only make sense for the specific context that i have shown you if my data is distributed this way and i have this query then the answer is don't index so here's a like a exercise for the listener something food for thought i'm not going to answer this in this talk so how how would the answer change if it would change at all if my data distribution was skewed the other way so i still have 6 million rows but instead of it being clustered around the 7 to 9 range it is now clustered around the 1 to 3 range would that change anything right because we have a greater than equals query so depending on what number we pick we might actually select only a very small part of the table so would that change your answer i'm going to spoil it for you it still depends right because there's still more factors that go into this but this is an interesting thought experiment for you to go through and if you you can reach out to me on twitter if you want if you have any questions about this but that's it for this example so we just saw that i just gave you a query without any context but the answer to how do i index is is actually only partly dependent on the query and it's just as much depending on what does my data actually look like okay so let's move on to another example so let me give you some context first imagine you're working on a on an e-commerce platform right you're some shopify knock-off or whatever um and your platform has many sellers and many buyers hopefully and all orders get saved in the same table so you have like a single table multi-tenant kind of thing going on so your orders table would have all the orders from all the sellers and then probably have like a seller id or tenant id to map it back to the specific seller and let's assume that your business is going pretty well so you have quite a lot of sellers and they're selling quite a lot of things so you can imagine that this orders table can get pretty big pretty quickly it's very easy to imagine depending on that car in that table that this table can have tens if not hundreds of millions of rows right this is a pretty beefy table so this is the context and now here's the query that we want to optimize right so we have this orders table and we want to calculate the sum of the orders that are placed for a given day uh in a particular date range so this is like the uh in in one month the month of march um and since we're doing an aggregation we have to group by the date field but actually that's not what we're trying to do we really want to do like enrolling average so we use this as a derived table and then we select from that doing an average over the total column with this window function that i defined on here so this is very obviously a three day moving average and then of course we want to order descending by date right so very basic stuff okay so obviously this query is not basic right if you just nodded along and went like yeah that's pretty basic stuff and you're either much better than me or you're a liar okay so the specifics of this query aren't really that important uh just think of it as a stand in for you have a pretty complicated query that does a lot of number crunching it does aggregations and there's a window function whatever that is and the data it's operating on is very large right this orders table can have hundreds of millions of rows so this is kind of like i don't even know if this query would work i think it does i think i've tried it once but it doesn't matter right it's just a stand-in for a complicated query in your app but now you're asked to um to optimize this and again you basically have no context so one question in this example that you can ask because what we're going to do in in this example in the previous example when we talked about the greater equal to zero thing a seven thing we didn't change our query right the query stayed the same we just fiddled around with the data a bit like assume the data is distributed like this assume it's distributed like this and that actually changed our answer in this case we're going to go one step further and we're not going to change the query and we're also not going to change the data we're just going to like look at change other parameters from the context that this query lives in and see where that takes us so one question you might ask if you see this query and you're being told it's slow is well how is this query actually being used right i i just this is just like a random query that was presented to me and let's assume that you have i don't know 3 000 sellers and running oh no i have to go through the animation again running this query for a single seller let's say it takes two seconds which for a database is glacially slow and so if you have 3 000 sellers i'm attempting to do math on youtube and it's going to be recorded so if you have 3 000 sellers and it takes 2 seconds to run the screen for a single uh seller then running it for all of them will take 6000 seconds which i think is 100 minutes right so i think we can all agree that like over one and a half hours is pretty slow so you might want to know like okay how is this query actually being used more interestingly how often is this actually being run so let's look at two possible answers to this question how often does this query run and see where that takes us so case number one this query is used to generate a report that gets sent to each seller at the beginning of each so let me highlight a few interesting bits about this so the query is used to generate a report the report gets sent to each seller so maybe in an email or something at the beginning of each month so what can we learn from this we know that the query is only used to generate an automated report which means it is not triggered by user interaction that means it is very predictable how often this query will run right there's not going to be any sudden spikes or something and it's it's like an automated report that gets generated probably at two in the morning so it doesn't really matter if this query is slow or fast right performance doesn't really matter here this is not exactly the critical path because the query only runs once a month at night so who cares let it run two seconds right so in this situation oh yeah and we also know that the orders table is pretty large right so what do we do with this information if you now were to uh you were asked to index this query again the solutions don't index that's not your because let's think about what would happen if you were actually going to index this first of all you have a table with potentially hundreds of millions of rows so any index you create even if only on a single column would be pretty damn big first of all it's going to take quite a while to build it's going to take a lot of whoops it's going to take a lot of storage and it's only going to be used once a month for a query where the performance doesn't even matter but because of the way how indices work as soon as an index exists on a table it will affect write performance to the table basically all the time right you only use it once a month but you basically just sabotage the right performance of this table the rest of the time as well it's like these trade-offs aren't really worth it right so now you might say okay but i might just hog the database connection for like 100 minutes that sounds like it could be a problem fair enough but again one of the points i want to make is slow query does not equal better put an index on that right because our trade-offs so what could you do in this situation with the knowledge we now have to tackle this problem assuming this actually was a problem so here is a non-exhaustive list first of all if you know that the query is only being executed in this scheduled job that only runs at a specific time like you have all the information there's nothing surprising about that so you can just spread out this report generation of a multiple hours you don't have to cue them all at three in the morning because no one cares if they get their automated email at three in the morning or at four in the morning like no one's going to sit there refreshing their mailbox at three in the morning right just waiting for this automated report so you can spread them out over like i don't know eight hours right who cares doesn't matter and like spread the load that way so it doesn't really matter that is a single query takes two seconds as long as you spread them out enough another thing you could do is you could pre-compute this instead of like doing the whole calculation at the very end you could sort of keep a running total of whatever metric you're interested in during the month and then basically this report generation just selects the final computed value right none of these solutions have anything to do with indices okay so how do i index square well you don't not in this case let's look at another case so now this query is used to display a real-time chart on the setters dashboard page again same query same data probably even same application so let me highlight a few things it's used to display real-time chart on the starters dashboard so what does that tell us well now the query is actually triggered by user interaction if no one ever opens their dashboard the query is never going to be run okay which makes it more unpredictable because if you have a black friday or something there might be a pretty big spike in how often this query gets executed it's used to display real-time data so let's just go with the most naive approach possible and say that this chart refreshes every five seconds like it just pulls right it's just a horrible solution but just for the sake of argument so as long as you have the dashboard open this this chart component is just going to pull the back end every five seconds and every one of those polling things is going to rerun this query which means now depending on how many sellers have their dashboard open this query can now be run dozens if not hundreds of times per second so we went from once a month to two in the morning to holy crap this is the critical path this is going to kill our app but we didn't change anything about the data or about the query so what do we do in this case well in this case well you absolutely need to do something right this is going to murder your application performance so what's the solution here well it's still complicated because there are still other factors to play into this i want to give you just a few examples of what you could do in this particular case one is yeah you could actually index this if you do you should probably index it pretty aggressively and by aggressively i mean usually the index becomes very specific right it has a lot of columns so you have like this composite index that goes over multiple columns because you want to avoid having to go back to disk right because you're selecting something that's not on the index and if you have an index that has data from all the columns that a query needs that is called a covering index so in this case the index has all the data the query needs which means we don't have to go to the table at all which means there are no reads from disk at all the entire thing is performed to memory it's pretty damn fast but the index becomes very specific and that usually means it becomes less reusable right and that's a trade-off but it might be uh worth it in this case so what else could you do well you could cache the results right instead of actually recomputing it every five seconds you might just cache it for an hour or so and then basically return the cache result because does it really have to be real real time or just like real time enough or you could again pre-compute the results like have a separate job running that just calculates this value in a completely separate job and then this end point that your your chart component is hitting basically just fetches this pre-computed result so similar to what we did with the with the generated report okay so there's a bunch of options only one of them is actually put an index on it but these solutions are very different but we didn't it's the exact same query and the exact same data we just in this case we just looked at well how is the query actually being used right so there's so much that goes into designing a good index that really the only answer you can give if someone asks you how do i optimize this query is well it depends man it depends on so many things i can answer this question so let's go back to these questions that i showed at the beginning that i got as responses to my tweet right there were all some variation of well how do i do x when should i do this when does it make sense to do that and i hope you now have a like you understand a bit more why these questions are kind of interesting like they're pretty revealing to me because they're kind of missing the point or i guess they're they show that there is a pretty large like knowledge gap there because that question really can't be answered if you only provide me with just a query right but the fact that that's the kind of questions i got to me was very very interesting because it means that there is there's still much more to learn about this topic and it's like one of those unknown unknowns things right where people might not even realize that uh they don't know that they they don't know that they don't know something right so this is kind of the intention with this talk i haven't really given you any um hot tips and tricks on like here some really cool indexing tricks because they're going to be so specific that they might just be useless to the next person right so that kind of talk would probably never really work but instead i wanted to show you that there's actually a lot more that goes into this question and a lot more that goes into designing a good index than just looking at the query so to sum up indices and queries don't exist in a vacuum right you can't just look at the query and nothing else because we saw that the answer could still be very very different it's based on so many other factors secondly and this is a big one i think not every slow query can be improved by adding an index it is not a silver bullet right our first query with a greater than equals wasn't slow because it took a lot of time to select to find the data the problem was we just selected too much but the query wasn't selective enough that is not something an index is ever going to solve thirdly it's developers that need to know the stuff why is that it's because we just saw in the last example you required like application knowledge right where is this query being called from how often does this run no it's a scheduled job it runs at two in the morning blah blah blah these are all like application things right they don't have anything to do with the database or the or the query itself that is just application knowledge which usually developers have right but developers also usually have the knowledge about well what does our table look like what does the data look like that we're storing in this table you might be able to do some like basic analysis on well how is is distributed so it's developers that kind of have access to all these bits of information that go into designing a good index which i think which i which is why i think this is so important for developers to know so context is everything also when it comes to uh indexing so next time that you have a query that you that is slow um so instead of jumping right into let's just throw a bunch of indices at this table first of all like verify that you actually have the problem that you think you're having right slow query is not slow query right not every slow query is the same it could just be because it's selecting too much and then keep in mind that you should also like examine things outside of the database that might factor into designing a good index so i hope this was useful i hope that you learned something so thanks for real this time all the other stuff hasn't changed you can still find me on all of these things i forgot to put the link to my mailing list for the book on here because i'm an idiot so i'll guess i'll post that in chat or something and i will publish these slides of soonish on github uh so you can look through them again so yeah thanks a lot frank for having me uh i hope this was useful and uh enjoy the rest of the meetup yeah this was uh certainly useful uh thank you for the stock guy really enjoyed it um maybe one more thing that i can ask you when do you expect that your book is is coming out because you're well it's been working the worst question you could possibly yeah sorry sorry about that but i had to it's yeah it's it's difficult to answer um because there's like initial release and then like the thing is done like the thing is done is going to take years um but the when i first publish it it's still like i'm in this phase where i'm like doing one step two steps forward one step back so um i think there's still quite a lot of work to do um so it's it's still going to be a few months out i think um okay okay but i yeah i plan to release it sort of in in chapters um and the way the chapters are structured is kind of uh self-contained so they solve a very specific um problem so you won't have the issue of like chapter two won't become useful until chapter nine is released but that it's gonna take like five years right it's like everything is self-contained so it's going to be useful but yeah all right cool yeah uh i guess like our slow queries we'll just wait patiently on your book as well so that's a nice segue man that's a good joke to uh to end this part of the meetup so thanks again guy and yes i'll see you uh see you around okay we are going to take a little break now uh we're going to take a five minute break or something and then we'll be back with uh sebastian the diana who will be talking about using feet in larval enjoy the little break and see you in five minutes okay welcome back everybody i hope that you enjoyed that little break um now it's time for the second guest and let me bring him on i have here for you uh sebastian de diana my colleague at sparci oh especially welcome it's nice to see you in another context [Laughter] now i'm not going to spend too much uh of uh of the time here uh i'd say take it away man all right i'll share your screen like that hi everyone so today i'm going to talk about using veet with laravel veed is a bundler so a tool that bundles your applications front-end assets so front-end assets is mainly javascript files but also things like css images all of that veep is a tool created by avenue who also created vue.js but it might be a common misconception that is a build tool for view but it's actually a build tool for any framework or stack you're using so even if you're using react or even the tall stack with alpine and stuff veep might be useful for your applications so before we dive into how veed specifically works and with laravel i kind of want to do a quick overview of how bundlers work in general well i guess webpack specifically because most people are already using webpack through laravel mix but bundlers are a black box to a lot of people so to really understand what the vit does different i want to have a quick overview of how bundlers bundle things so i try to summarize bundlers in one sentence like define them and what i got to is bundlers recursively resolve and transpile modules so these are a lot of fancy words which essentially mean modules are any kind of single items mostly mapped to a file or a dependency in your javascript packages for example app.js could be a model module app css could be a module but it could also be an image or something else so bundlers also handle transpiling modules transpiling means in this context transforming it to a common language of javascript so for example if you're using single file view components a bundler doesn't know what how to read an app.view file so it needs to transpile it to app.js so it knows what to do with it recursively resolve so a bundler is going to go through a lot of files and through the or modules actually and through those modules it's going to find that those modules have their own dependencies and import things so it kind of creates a tree or a graph actually of a whole bunch of dependencies and it resolves and transfiles them all one by one so in action this is a typical app.js file for let's say a laravel application that also uses vue router and in this app.js file i'm importing view the framework and an app component so like a base app view component this is something our bundler in this case let's say webpack understands so it's going to start recursively resolving its dependencies so importing view then we see that we have a view dependency module that gets added there we import app.view but here the bundler is kind of stuck because it doesn't know what how to read that view file so right now the bundle will transpile app.view and we see it imports view router and it imports a few more components specific to our application so let's say we're building something forged like here we have server sites billing these are all on different pages um bind bound together with view router so we also see we have two different import statements here view router is imported with an import keyword while server size and billing are imported with import functions the differences in import when you import with an import keyword and javascript you're adding it to your bundle as a fixed thing if you import with an import function it's actually only going to be imported or loaded more specifically when you request it in this case on the screen so the servers module will only be loaded when you go to the servers page through view router and same for sites and billing but either way in this context they all get bundled to a single they all get combined to a single bundle but in the browser we might load more or less of that bundle so traversing the tree further the servers have a select box on them so we require if you select view select has a dependency on low dash for example our site start view also has dependency on low dash so it all gets intertwined and this all gets grouped together in a single bundle so let's say we run npm run dev to create a dev build with webpack or level mix more specifically of this or let's say we want to start watching for changes also the first thing webpack is going to do is it's going to actually go through all the steps we just went through so it's going to create a bundle of everything this takes yeah these days like in a something this size would be probably five to ten seconds with webpack but in a real application with a lot more modules probably in the hundreds um you easily can go up to like 20 60 second build times that's because webpack has to do a lot of work these are a lot of modules a lot of transpilation a lot of resolving and it always has to do all this work before you can actually get going in your browser so veet does this differently first of all it makes a difference between two kinds of modules in your application on the one hand you have the dependencies which mostly live in node modules and on the other hand you have your application source code so when you create a death bundle with feet or actually when you start developing with feet so you start watching dependencies for changes too the first thing veed is going to do is it's going to create a bundle of only your dependencies because your dependencies don't change often only when you when you install more modules so this is something you can just do once at the start and then it probably won't touch him anymore while you're developing your app so vt does this very very fast because under the hood it uses es build to bundle those modules for development yes build is actually also a bundler but it's written in go and that makes it very very very fast so like veet bundles these modules in probably like a few milliseconds like less than a second that you don't even notice that it's happening so that's one thing that makes feed very fast already now the second thing is it loads application modules only on demand so with webpack you run npm run dev and it starts building everything with feet it's not going to build anything besides the dependencies until you actually request them so only when you open your browser veed is going to start building app.js because you're probably loading apps yes through your browser so you load it it loads app.js and then it starts traversing it says okay we also need app view for this page let's say we're on the sites page so it also loads the site's component through your browser so with feet it's actually not v that's bundling all these in a single file in development feed just make sure that your browser correctly understands all these import statements and has your browser import all of these in separate files so if you'd open a network tab for this with feed and development you would see that first it would be importing app.js it would also be importing the bundled vendor modules and then it would import app.view and site stop view but it wouldn't be loading servers or billing yet because they're not requested since with the import function we're able to code split at these places so because vita has less work to do than webpack that also makes it faster now in production this is handled a bit differently because es build is a very fast and powerful tool for bundling things but it's not as good at some more specific cases like handling css files or correctly code splitting applications so veet uses roll up under the hood for production builds so if it actually does development bills and production builds in two different ways another difference with production builds is that while in development we offload all the work of loading these modules one by one to the browser in production we don't want that because that would be a lot of http requests for a single page because it would have to load every single module one by one of your application which is not something noticeable in development but in production this could slow down the performance of your app especially the loading performance so veed essentially brings together a few specialized tools so es build and rollup and kind of binds them together in a single development workflow all right so now that we understand how vt works i want to do a demo of transforming a level application that uses laravel mix to a level application that uses v so this is going to go pretty fast this it's i used an existing application to do this so it's not a green field application where everything's gonna perfectly transform to feed we're gonna come across some special edge cases and stuff that i think is interesting to see how they get solved so i chose ping crm which is a demo application for inertia.js so this is a view heavy javascript heavy level application so this will be we'll go through this pretty fast but i have a whole series of blog posts that go through all these steps a bit slower that you can refer to afterwards all right so i want to get started right now we have a working version of our application and it is built with webpack so we're starting from scratch i'm going to open my code editor the first thing i want to do is clean up package.json so we're going to remove laravel mix and we're going to update npm script to get them ready for feed so we can get rid of our mix and we also can get rid of view loader because all dash loader files are web spec webpack specific loaders and first we're going to install veet itself so we're going to install version 2.2 and since we are using view in this project we will need to install the view plugin we actually this project actually uses view 2 so we're going to install create plugin view 2 which is inversion roughly 1.4 here we go i'm already gonna run an npm install so we can fetch those dependencies in the meantime i'm gonna clean up these scripts so all i care about now is dev and production dev is going to call v production is going to call feet build so with feet there's no difference between watching and hot reloading everything is hot reloaded by default so npm run dev is going to start a dev server which is going to enable hot reloading for your application all right so these are installed now we want to set up veed itself so veed is set up with a v.config.js file which exports an object with the configuration in it so first thing we want to do actually before i create this file i'm going to dig into the mix file we have for this application to see like what what kind of things does this application use so level mix this application has a single javascript file app.js it uses view it uses post css there are some changes to the webpack configuration it has some different junk file names but that's not really relevant to us right now and it has some aliases for scripts which we'll take over in v2 okay so the first thing i care about are these entry points we want to build app.js and app css with v so how do you define an entry point with build rollup options input and this can be a string or an array if you want an array because you have multiple entry points so we are going to build resources.jsapp.js and resources css app.css so this looks kind of weird how hidden a setting this important is but you have to remember veet is built for single page applications in a different context and in the context that vit normally works these would be auto discovered from your source code so because we're using this with laravel we have to do some additional configuration compared to a v project from scratch this is what we're gonna have as an input and we want to build these two or two yeah to the public folder so build has an output directory of public build so it's important that you use a fresh directory for this because whenever builds assets it's going to clear this directory so if you would say i want my access assets directly in public it would delete the contents of public every time you build which is not what you want in the context of a level application so we want them in public build this is also easy to add to get ignore you can just add a public build you're getting north file and your assets will never be committed to get history all right what else do we need here since we specified a specific output directory we need to change the base path of our assets so the base path of our assets will be build this is not important for so in our blade file later on we're going to import something like build app.js for example this is relevant for the chunks that app creates so apple will also probably have something generated like build slash selectbox.js for example this is something that vt is going to import from app.js and that's the difference between base and outdeer out there is where does your bundle go and base is tells v to prepend this to all modules that your your main application is going to import now when we're running a development server we're not serving from public but we are serving from localhost localhost 3000 to be exact by default in this case we don't want this build thing to be prepended we only want to do this in a production build so to make a difference between production and non-production and v we can actually turn this into a function that returns an object so this will work the same now and here we have some arguments and we care about the command argument so here we're going to say if the command is serve then in this case we're using the development server we don't want any base path if it's not serve it should be question mark if it's not served then we won't build so in this case build will only be used for the production build now there's one more thing we need to add in this configuration and that's the public directory so these are kind of confusing because they all three look the same the public directory is something else entirely when you build with veep it is going to look for a directory called public and it's going to copy all those contents into the final build this is useful when you're building a single page app in a totally different context and you want to add some static assets to the final build of that app like some images or fonts or whatever we are now in the context of a laravel application and we already have a public directory so we don't need vee to do any additional copying for us that can be exposed from the web server later on so we need to disable this but unfortunately there's no way at the moment to have v actually disable this there's a pr open for it but it hasn't been merged yet in the meantime there's a little work around you can just specify a fake directory doesn't matter what this could be anything you want and since viet is not going to find this folder it's not going to copy anything over so this is just a quick hack to make sure that v doesn't start copying your public folder all right now back to this mix file there were two more things we cared about um those are these two aliases here so we want to make sure that v that view uses the runtime build and we want to add this at alias so this add alias for those that don't know how this works you can use to specify the root of your application so we have an index.view file here and it imports from add shared layout so if i look here in my file tree we have resource.js here we have shared and then the layout so instead of digging with relative paths all over the place you can specify a route with this could be anything but in this case it's at so we want to mirror this in our v configuration and actually it looks exactly the same so we can copy this resolve paste it here there we go all right v does not use regex here so we can get rid of this dollar sign and the last thing we need to do is import path so what does it get mad at us import path from path all right so this is already a good starting point i'm now i'm going to start building this i'm going to start the production build and things are still going to be broken but let's look at the errors and fix them one by one until we have a working bill so npm run production let's go all right so let's start with the warning invalid import import add page name all right so this is actually an inertia specific thing i'm going to go to app.js so for um you don't need to understand inertia completely for this but the way energy works is the server returns a string so this is kind of off topic now the server returns a string and that string is used to discover a component on the front end and render it so in this case the server returns dashboard index it's going to be passed as a name and then it's go we want to import add pages dashboard index which is something we see that exists here pages dashboard index now veets doesn't support this template literal variable dynamic import syntax so we need to solve this in a different way with veet you can do something that's importing globs of a bunch of dependencies and then those dependencies you can start yeah you can start querying for this in that glob okay that was a bit hard to say but let's just look at it we can do import meta glob now we're going to go pages and then specify glob so we're going to scan all of these files so we're starting here we're going to go to pages directory and we're going to scan all view files and this we're going to put in a pages variable so what vt is going to add build time replace this with some kind of objects and as keys this object is going to have stuff like pages dashboard nexus view for example this is going to contain something so the next thing we need to do here is use this name to find the right key here so that is pages and we're going to say pages this is actually this thing we need here dot view all right so this is glob generates dynamic imports so if you remember so import functions essentially if you remember that from the slides this is going to code split this bundle at every single separate page so you want to load this on demand so to load something from a glob you need to call it so this is these dot dots are actually a function that return a javascript module so we're going to call this and then same as here we can call then on it and return the default export of the module if you want to return this all right let's see if this solved our issue let's check the token where is this is this here roll up time something is not happy okay i'm gonna ignore this for now and solve some other issues first okay um next issue is css so we're not using post jss yet because in our mix file the voice css settings are specified here but vt supports postcss out of the box but requires a post css configuration file so we're going to create one processor config.js and we are going to copy these plugins back to proceed as configuration of yes module.exports you want to set some plugins those are the ones from our webpack file we're going to copy the imports of these two all right so now veet is going to discover this configuration file and use it for our css again and we have a full build so now we have to fix the previous issue i assume i don't know what is going wrong here resolve component name okay we have a better issue we have a better debunk here now the problem is it does not understand login dot view so the reason it doesn't understand login not view is because we installed the view plugin but we haven't told veets to use it yet so in our configuration again config.js we are going to register a plugin in a plugins array so we need to import the plugin we're going to import it from veet plugin view 2. we are going to import this is a weird api but we need to import create view plugin from here and then we can call it here right so now the view plugin is installed and vt should understand this file and we'll get rid of our pesky unexpected token error okay we have an unexpected token again plug because it has to be plug ends perfect okay we have more errors but this time it is a new error it cannot find the select input module so this is kind of an annoying one especially for older applications but when you're working with view and veeds it actually doesn't support importing view files without specifying the view extension this is something that vue is apparently going to change in the future that is going to be become mandatory everywhere but now you already need to do it in v so i'm not going to go through all files i have a regex ready here that's going to add dot view to all of these imports so i'm just going to run this here we go right so now all of the imports have dot view explicitly after it i'm going to fix this one here because it's double all right so we got we added the view extension let's see how it goes this is taking longer which is a good sign that means we're getting further next vit's dynamic import polyfill is enabled but it was never important so for dynamic imports we need a polyfill specified by feet itself so this is just something you need to add to your bundle when you're using feet and kind of forget about it so in our app.js all the way at the beginning i'm going to add this dynamic import polyfill import and i'm going to run the build again all right this is good news we have our first successful build the files we care about as entries are here we have app css and we have app.js so as you see v did a lot of code splitting on the one hand you have these pages like edit pages index pages and on your end you also have components this is stuff that vee intelligently starts extracting like if it sees that multiple pages use the same component it will extract a component to its own file so those other modules can share these ones all right what's up next we want to dive into laravel now now we're going to look how we can load these into our blade view so but before i go too deep into that i want to see if my build actually worked so i'm just going to copy these paths so we have our css path i'm going to go to our blade view app.play.php i'm going to replace this asset app call with our generated with our file name with the hash in it let me do the same for app.js all right so technically if i save this and refresh we should be pretty good all right this does not work because we have an unexpected token error so feet doesn't output normal classic javascript files but it outputs javascript modules so when you add a script tag referencing an asset for v you need to make sure you add a type module so your so your browser can understand how to parse it module refresh and we're here so this is now exactly the same application the same code but generated by feet if i look at the network tab i can see that it's all being served from our public folder and it uses it uses our newly generated assets all right so now that we have this i yeah you don't want to update these every time you do a build of course so let's see how we can in laravel kind of auto-discover these files so there are two ways you can do this traditionally i would solve these things by generating a manifest file that's also how laravel mix works so a manifest file i don't know if it still exists here a manifest file is a json file that maps like add.css to this but the problem is the manifest that v generates aren't always as easy to parse as they should be and kind of randomly move things around especially with css so i haven't always had the most success with it if you want to learn more about how to do with a manifest then you can check out the blog posts but now i want to look into an alternative way of doing it and that is just looking into the file system and based on a pattern discovering these files so to do that i'm going to write a helper function first thing i want to do is create a helpers file in my app folder php and i'm going to create a veep function kind of like we have a mix function it's going to have an asset parameter that returns a path so the input of this function will be like something like vt app.js what we want to get back from it is build assets app dot and then for example right so the first thing i need to do is register this helpers function helpers and source no app all right so now i'm gonna dump autoloader and we can use this file all right so the first thing i want to do is transform this to something that we can use to discover this and we can use a glob for that so i'm going to dump here spring replace i'm going to replace the dot with dot asterisk dot inside the asset i'm gonna in blade i'm gonna replace this one with beats app dot css the script with veeds app.js all right so now if i refresh we should see an error on the find function defeat i is this dumped it app helpers.php demo gods aren't happy today okay fine i'll just do it here then all right um so we are going to dump this now we have app.astrix.js the next thing we need to do is add the public path there so public path and these files are going to be and build assets concatenate that all right let's see what this does all right so this already looks like a glob we can use because it uses the right path so now we're going to glob it's going to return an array of all files it matches and here you can see it found the javascript file we were looking for so we care about the first match here because technically there should only always be one match here all right so let's store that in a path variable and the next thing we want to do our last thing we want to do is get rid of all of this part so we're going to do another string replace we're going to replace the public path with nothing in our path let's see if this looks good all right that's the path we want and we're going to return this now if i refresh okay we have our assets and including them from the right location and our script tag and link looks good here okay now that we got production working i want to dive into development assets so to start the dev server instead of production we run npm run dev as you see this is ready in 180 milliseconds so in 180 milliseconds vt already bundled all of the node modules that it needed to bundle and got a server ready for us to start developing so as i said before with the slides it's only going to start actually building the application once we visit an entry on this server so i'm going to comment these out for a second and i'm going to replace these with dev server equivalents so the first thing you want to do for dev server assets in v is we want a script don't forget the module and we are going to include the dev server client itself so localhost 3000 slash atv slash client after you added the client you can start adding your assets which you add by referring to them through their entry points so here you have resources js app.js the second one you want to add is a style sheet so this is a weird one but in development you want to add your style sheets as scripts so here we just say resources css app.css now you want to do this because vit is going to wrap these any css file you add here in some kind of javascript box and that and it adds hot reloading to your css so if you modify some css it will nicely live reload in your browser okay so if everything goes right and we refresh network tab okay see this now it takes longer than before because it was building the assets as we requested them so opening it the first time takes a few seconds but this is still very very fast i can see now that all the assets are coming from localhost if i refresh again now it is fast and whenever i go to a different path it's going to start generating more assets that it only needed once you open that path all right so now something really fun i want to show just to show how fast feet is in vs code you have this feature to auto save so i'm going to enable autosave let's say i want to auto save any changes i make after a specific delay and i want to set the delay to one millisecond so this is pretty much going to auto save my files whenever i change any character now i'm going to go to that dashboard file and i'm just going to start typing hello dashboard so this is how fast vt is recompiling and hot reloading this page just on every single keystroke it's recompiling as you can see in the console here it gets hot updates so this is amazing when you're developing an interface or especially like when you're working with tailwind or something you can just start modifying here and every single keystroke you do text red 500 is going to be reflected immediately so this is really where view where vt starts to shine when it comes to developer experience where you get the hot reloading setup good and it even adds an overlay like if you have a syntax error it really gives you a lot of information of what's going wrong so with or without autosave this is a really useful feature so i'm going to turn this off for now all right so there's one more thing i want to do that's pretty important me at least and that is i want to have something in place that decides whether production assets need to be loaded or local host assets because we don't want to be commenting and uncommenting this the whole time so what i'm going to do is i'm going to create a directive somehow i'm going to call it dab server that is going to be true if the dev server is running so if the dev server is running i want these dev server assets if it's not running i want these production assets so i can take these out of comments and we're going to say and dev server so we are going to do this by adding a custom blade directive an app service provider right so we're going to say blade if dev server so this is a quick way to generate one of these if statement directives and we're going to add a callback here so the first thing i want to add is if we are running a production app environment for running in production the dev server is off we're not going to do any additional checks or something just return false as soon as possible if we're running in production now if we're not running in production what i want to do is i want to ping this url or this domain and see if it returns something because if we're not able to connect that means the dev server is not running so we can use laravel's relatively new http facade for that let's say http get http localhost 3000. so when this can't connect then laravel is going to throw an exception so we want to try and catch this try catch it's going to throw a connection exception so if level through an exception we are going to return false because if laravel threw an exception it wasn't able to connect so we know that the dev server is not running if laravel was able to connect we don't care about the response we just care about that laravel did connect with this host that we're going to return true so we know the dev server is running so right now the dev server is still running so if everything went right we should see dev server assets which we know because our network is loading from localhost now if i turn off my dev server you see it already immediately refreshes because when you disconnect from a dev server on a feed it's going to refresh the page and you actually already see now that it uses the original assets from the public folder instead of the dev server so that directive did its work and that is all i had to show for vit and laravel um that's the end of the demo so like i said this was pretty fast if you want to go through it on your own time i'd say visit my blog sebastiandeddine.com for these posts specifically if you're interested in v or have any questions or whatever hit me up on twitter at sabdini and that's all i have to say for now and thanks for watching that was pretty cool i saw some emojis with like stars in their eyes and love in in their eyes when you did like the hot reloading thing that was uh was pretty amazing i i guess people like that that workflow a lot um i maybe have more one question for you um is there a way to get yeah what you showed us in laravel to uh to just pull that in via packager or something because i thought that there was all already a larval feed package by made by someone is that interesting to check out um definitely interesting to check out um i've played around with it myself um it does require some configuration i don't know by heart what um i personally am someone who really likes to have ownership over the entire front and pipeline of an application because it gives you more room to add optimizations and stuff compared to using like something like laravel mix for example but if you're interested in just getting started i'd say um give the package a look i i forgot the name of it but it's also doing in my blog posts okay cool uh thanks again for uh yeah letting us uh feel the power of feet uh seems glad to be here pretty cool okay and yeah see you soon man see you soon okay we are already at the end of the meetup i hope that you enjoyed both of the talks we'll be back next month at the last tuesday of the month i think it's the 25th and i will have two guests i've already prepared a little card here so the first speaker will be tom witkowski he recently launched a interesting project called open door dot me and we'll take a look at how that looks behind the screens and our second guest will be uh robin malfey robin is a uh i guess wizard at tailwind labs uh he uh worked on creating headless ui for tailwind and we'll take a look at those components in his talk so that is all that i have for you i hope that you liked this meetup and i'll see you again next month bye bye everybody you

Channel: Freek Van der Herten

Views: 5,798

Keywords:

Id: wLwVr9ToNIs

Length: 86min 2sec (5162 seconds)

Published: Tue Apr 27 2021