JavaScript Modules Past & Present

[Music] good evening everyone thank you first of all to Vijay for having me back I was with a group of developers who worked on a react open-source project having to do with server-side rendering and we had a lot of fun I'm speaking together it was five of us passing off the speaking roles back and forth for the course of the meetup so this time I'm holding down the fort alone but I'm excited I'm very excited to have a chance to nerd out with you guys for a little bit about JavaScript modules and this is a top that of topic that I think is particularly fascinating because when we look at the history and the development of modules in JavaScript we're kind of going in parallel with we're tracing the history of JavaScript growing as a language from kind of humble origins as this lightweight scripting language that was running in Netscape to really becoming a powerhouse that gives us the modern web with all of its interactivity and everything that we love about it so now we're basically in an era where we have JavaScript running everywhere Universal JavaScript JavaScript on the client and on the server side on we have it in our thermostats even flying Jones for us so needless to say javascript has come a very long way and so have modules oh and the pronunciation of my name for anyone who might have been wondering is meetin rhymes with region so just anyway so the first thing that I would like to posit is that modules are a fundamental part of our lives as JavaScript developers they're kind of part of the scenery so I do a lot of my day-to-day work in a react codebase so when I fire up my code editor many of the files I'm looking at have kind of a predictable structure I see some import statements at the tops and export statements at the bottom I almost don't even think about them I mean unless I've put a typo in my file path or something and I than I am thinking about them but I imagine there are other folks in the room working in other frameworks and working in other modular type code bases and environments so modules and modular thinking are kind of second nature to us and that's really how it should be modules are supposed to make our lives easier as developers they do this by giving our code structure and boundaries so that way we don't have to think about all of the moving parts I don't have to have the react source code mingling with my component logic I can take a grant I take for granted that it's going to be available for me and do what I wanted to do most the time so for now it resides elsewhere out of sight and out of mind and this is especially important as code bases get bigger so when we hear about terms like modularity and separating concerns that's really about our sanity as developers who are working with each other and working with a lot of code so let's start out and get on the same page as far as our concepts and terminology and we'll start with the basics what's a module anyway I like to think about modules as building blocks of data and functionality and we can bring them into a program and we can also make them available to other programs and ideally you can kind of treat them like you would build building blocks and hopefully you can add or remove pieces from the whole without the whole structural integrity falling apart so when we're talking about modules we're really talking about the higher-level design of our software we're often thinking about modules in terms of a single file because that's typically how we organize our modules one module one file we can also talk about modules in terms of their relationships to other modules so a module has an interface and that's it's exposed bits that let us have data and if I available for use in other programs I kind of like to think of the module interface as the little bumps on the top of Lego brick so you can try to put your Lego bricks side-by-side and they won't really stick together but and those little bumps which I actually just found out are called studs Lego studs or how we actually get our structures to stick together and so module interfaces are a little bit like that and a module can also have dependencies and and often that just means that we're dealing with other functionality that we're relying on to have the functionality of whatever module we're working in and so just a hammer a hammer at home in case there's any ambiguity about my personal physician on this modules are awesome they help us to keep organized code bases especially as those code bases are growing in complexity they help us resolve our dependencies making sure that any pieces of code are supposed to be there and we don't have to worry about getting them all in the right order they also help us to share and reuse code so we have a wonderful thing called NPM that gives us access to literally I think 700,000 packages right now a lot of packages of code and we ourselves can be part of a community contributing to that global repository alongside others so kind of cool but before we go any further if there is an elephant in the room and we should acknowledge it so these modules which I've been talking about and that are so fundamental to modular to modular and modern programming are actually just barely a part of JavaScript they weren't built into JavaScript pre es6 so for most of its years as a language javascript didn't have any kind of built in native module system so this is unlike Python for example and so modules only really recently became part of the language and they've actually only just landed modern browsers like as of this year so oftentimes when we're working with modules we're not working with the built in module system at all we might be using require statements if we're working a node or comma J yes we might be using import statements like I showed earlier from react code we might be using Favell to transpile them down to es5 and then to bundle those files that were all supposed to be separate into a single one for consumption by the browser so what's going on well this is where I started getting curious like what how did we work for so long without modules and JavaScript and as I started to take a peek into the history I found that there was this overarching storyline that JavaScript was really built disadvantaged when it came to modules it wasn't built with any kind of modularity in mind when it was first invented it was just supposed to be lightweight scripting inline scripting and over time as more demands were made of JavaScript as a language to do more for us we see an increasing need so running in parallel for usable module systems so I like to think of the history of JavaScript as unfolding in four phases I made these up so I call it the pre module era the do-it-yourself module phase the specification phase and the standardization phase and because it is something I made up and because it is a system of chronology it is not perfect not precise but what I was aiming to do and kind of splitting our timeline into these four segments was to bring some order to overall trends that we were that we're taking place so I'll do a little flyover right now before we jump in and get into more detail for each of those phases but let's start at the beginning in the pre module era when modules were not really a thing at all Java scripts didn't itself have much of an identity or much respect let's be honest at that point it was nothing like it is today and that didn't last too long before we started realizing that we needed some kind of modules but in the absence of those built-in modules we were improvising we were making module type structures ourselves so roughly in 2011 we start to see module specifications growing in popularity the JavaScript community especially the service server-side JavaScript community was really starting to take off and a lot of work had to be done just to make JavaScript the language people wanted to develop in and a big part of that was just coming on to the same page so that's also kind of our era of single page apps and frameworks like backbone and angular starting to come into vogue and finally today we have arrived in the era of standardization and that's the phase that is currently underway and I hate to spoil the plot right at the beginning but javascript is still pretty far from having a standardized module system we do have a standard system in place but we're still working out the kinks and we're still making things hold together in the ways that we hope that they will so that one day we will all be gloriously developing with modules and not having to think about all the quirks that go with with trying to do that all right so let's begin our tour we begin in a time before modules Java scripts age of innocence and as we peek into the yearbook don't-don't-don't we come across gems like this so totally 90s and I'm calling this the Dark Age that really I think we were probably having a good time anyone had a geocities website I sure did I think it was called like the raspberry garden or something it's like 12 anyway it had all these cool neighborhoods like Soho and Sunset Strip and Times Square and I guess you can go to the wayback machine if you want to have some nostalgia but anyway as I'm looking at this specimen from the archives I think I'm seeing at least for sure one definite use of inline scripting which is probably like this fun little animated status bar that's going it's crossing the screen for your entertainment any one spot any others that's the only one but I feel like kind of confident on what a time what a time but anyway yes a Wild West of inline scripting and script tags no modules to speak of really a question I want us to start thinking about is is this scalable what are some of the problems we encounter in this Wild West scenario well I'll say more about that in a moment but first let's get one thing straight at this stage in the game no one was really think about thinking about scalability that was not the name of the game and if you don't take it from me take it from the inventor of JavaScript himself mr. Brendan Eich I hope he pronounces it that way too I could I could be bettering it but he did say in an interview with InfoWorld that no one thought JavaScript would be used at the wild wide scale that it is not just reaching lots of people on the web but large applications like Gmail to write large code you don't just want this little snippet language that I made for beginners beginners you want strong api's and ways of saying hey this is my module and this is your module and you can throw your code over to me and I can use it safely so quick trivia question the anyone know what your JavaScript was invented yeah 95 or did you say 95 that is exactly right over the course of 10 very intense days for mr. Ike and I think it's an original instructions from Netscape we're actually to have a language that was a lot more like scheme for the browser so if anyone's worked with scheme which is kind of a dialect of Lisp you know it looks completely different from the JavaScript we know and love those instructions those marching orders changed significantly and I feel grateful for that so in the in 2006 we get the first of several manifestos that we will see so this particular manifesto came from the Yahoo user interface team in a blog post with the title global domination and they had a good point I would say and the point that they were trying to make is that global variables are a terrible risky practice that we're all engaging in this unsafe behavior that we're all engaging in but in growing code bases we really had to cut cut cut that out and so what's the problem well global variables are notoriously unreliable and insecure and there are a few reasons for that and the first has to do with variable naming collisions so assuming Mary and George the you why on the Yui dev team are working on the same code a name to the very their variable the same thing they were likely to run into trouble pollution in the global namespace and globally scoped variables aren't just well they're just really easy to break or to forget that they even exist and sick secondly because they're exposed they're not secure so it's not just coworkers who might innocently bump into each other in the global namespace dance floor third-party scripts and malicious code join in the fun and global variables are a vulnerability that we generally want to avoid especially if we're Yahoo and were the big guns in the game in the mid-2000s so the Yui team actually went as far as to say that an objective measure of the quality of a JavaScript program is how many global variables and global functions does it have a large number is bad because the chance of bad interactions with other programs goes up and a few other issues we're cropping up in this era in this pre module era whenever you wanted to reuse code you'd likely be doing some kind of copy/paste job and then following the trail of breadcrumbs wherever it led you and any time you tried to update your code it would likely be a logistical nightmare as you tried to find all the pieces the places where you needed to update it this dependencies were a hassle to manage especially if you were dealing with version control you had to worry about putting your script tags all in the right order and basically as code bases we're getting more complex we needed a way to manage them better so let us set foot into the next era do-it-yourself modules and in this second phase of JavaScript history not all of the problems from the pre module era were going to go away right off the bat but we'll start to see some of them being addressed particularly the problem of global variables because that was an immediate concern so this insight about needing to limit global variables actually spurred a ton of innovation with an improvisation really with what was possible within the constraints and flexibilities of the JavaScript language and this was really an era of transformation for the web we're really moving away from the geo cities model of static websites we're moving toward more interactivity and Ajax requests were becoming more of a norm during this period jQuery hit the scene during this time and the web was really starting to grow up a little bit from websites to apps and so let's just keep that in mind as we start to look at some of the ways that people addressed the shortcomings of living in a world without modules we'll pick up the story where we left off with our friends at Yahoo had who had just declared that global variables are evil and their answer to this problem of global variables was actually just a restrict them let's let's stick with a single object with godlike status the Yahoo object and the aim of the Yahoo global object was literally to contain all of Yahoo their state their and all of their web state and being in all caps you can tell that it was very important it was the global so if you wanted to do anything you would say Yahoo dot util Dom dot get or whatever it might have been you were somehow crawling through this object tree to find find what you wanted and jQuery might be a more familiar example you might remember having to have the jQuery object in available in global but as as much as this approach was touted as great practice there were definitely downsides to it too so it was still relying on a global variable a variable a single one that could have been rewritten with a single line of code and then poof potentially everything could be gone there was still no code reusability and really no help with dependency management or versioning but you were starting to mitigate some of the risks of just having free-range global variables so the basic structure of the namespace approach looks kind of like this we've got an object literal here and we've got some properties on it some data some functionality and we can assign those additional properties additional data while we're declaring that namespace object or after the fact with our handy-dandy dot notation and let's look at another approach that started coming up in this area era and that's the module pattern and the essence of the module pattern is being able to use an immediately invoked function expression with the power of closure so the power of closure is that it would shield private variables from being accessed outside of that function scope and as most of you know closure is what allows us to have access to the variable variable environment a function of a function even after we've exited its scope so you'll continue to see this pattern as we move along that function scoping is how we're going to start to get these boundaries and start to collect our variables in places where we can kind of protect them and not have them hanging loose everywhere control yourself JavaScript so here's a rather simple example of the module pattern in practice so this my do-it-yourself module we're assigning to it the return value of this if e and in that if e we've got in our closure scopes and private variables and then we've got our exposed functions that are returned via an object with the increment in print properties and so as one might expect you can call the print function and find out what your counter variables value is at any point you can call the increment function increment your counter if you printed it again its value is 1 but you cannot access that counter directly that is the important thing thanks to our function scoping it is it has lived it is gone you can't touch it and here's another popular implementation from back in the day also the module pattern just a flavor of it called the revealing module pattern which is almost exactly the same but in this case both our public and private variables are declared outside of the return statement so pretty much everything all of our variables are being defined in closure scope and we're selectively revealing what we want to and I my understanding is that this revealing module pattern was mostly just readable it was easy to use it was easy to work with you could work mainly within that returned object and decide what you wanted to have and what you didn't want to have included you didn't have to use this keyboard as much when you at all really you could simply refer to variables by name potential disadvantages of this approach this was actually considered by people who were thinking about it one of the most fragile ways of actually implementing the module pattern you might imagine that if you had some public variable that you were trying to change so you were trying to reassign it after having declared it and suppose you also had a private variable in here which somehow used or depended on that public variable well you might have a little bit of trouble and not get what you expect because because these variables are defined in this function scope what's accessible to those private variables and to private methods is the closure functionality is what was in this function scope at the time at the time this if he was invoked and so you might basically this is just to say that you might get some unexpected results it wasn't wasn't the best of the module pattern but it was out there quite a bit in the wild and just so we're clear namespace namespaces in the module pattern are not mutually exclusive approaches they often did go hand in hand which is something the late Doug Crockford advised and so here we got a variation of the module pattern that takes a namespace object as a parameter you can also sometimes find this out there with the dot Col function and you're using the dot call function and passing in the namespace object and then you just use the this keyword instead of having that namespace in there as a function parameter but anyway same kind of idea we pass in the namespace add some data and functionality to the namespace but we've got privacy we've got that privacy that that immediately invoked function scope provides us nice - closure so JavaScript is kind of cool okay so that's a wrap on our quick blazing tour through the early module era and as you can see the tools within JavaScript did take us a long way towards solving the global namespace pollution problem but we still feel like it's we still weren't where we want it to be if he still felt a little like a hack job we were still dealing with this giant global god object and we still had some problems that remained we never got to the point that we were able to actually swap modules with one another module sharing was not made any easier by this so and related to that the copy/paste problem with reusing functionality so even with the module pattern we didn't really get what we mean when we say modularity we didn't get all of those benefits that we wanted so that brings us to the next phase specification so what's getting specified here is a format a format that modules should take so we can consistently know and reliably know how we're going to load the and send them out into the world and as in the last era we start with a manifesto around 2009 or 2010 people were feeling - strong emotions about JavaScript on one hand people were very excited about JavaScript potential as a language and folks who were especially working on the server side we're excited to see that javascript could be used on the on the server side so great excitement on one hand great frustration on the other frustration about the limitations of the language and all the things we wanted to do with it but remember a little baby JavaScript coming from mr. Brendan Ickes womb wasn't really built for any of the things any of these expectations that were piling onto javascript so kevin dan gore another name that I may be butchering my respects to him wrote a blog post in January of 2009 that was called what server-side JavaScript needs and he outlined a proposal that included among other things a model system so there are some other things that he proposed and many of them we actually know and love today so he talked about having a package system for deploying and distributing modules he talked about having some kind of package repository for finding and installing these distributed modules and none of this existed at the time this was all kind of a pipe dream like wouldn't it be nice wouldn't it be nice if we as a JavaScript community had these resources available to us he talked about cross interpreter standard libraries also being important so api's that could be used across different browsers like Dayton math already were at that time so he was you know wouldn't it be great if we had an API for the file system that was consistent across browsers anyway the bottom line is that Dan Gore felt that what he was describing was not actually a technical problem it was an organizational problem it was about finding consent this among the JavaScript community bringing people together so they could make a decision and step forward and start building something bigger something better together so like adopting some a common currency or a common language his blog post which you're welcome to check out ultimately led to the formation of a grassroots group of similarly fed up developers who called themselves eventually or commonjs and they decided that they would start setting some standards for JavaScript including standardizing module loading not too long after that note came out just a few months later so this was May 2009 tangora wrote his blog post in January and they created a module implementation which has since taken the name Commons a s but it's distinct from that original group as we'll see that original group didn't end up meeting all of their goals they kind of created some things that they didn't mean to but anyway note had a number of competitors at the time that were also in the business of creating server-side environments for JavaScript but node 1 and the CJ s the commande s specification name stuck with that the node implementation so that's what we think of when we say Commons a yes today we're talking about nodes implementation of modules and the Commons is spec stipulates support for module loading via a require method and assigning exports to this property on the module object called module exports and this was really designed with server development in mind that will become important in a few minutes you might be working in node already but if you're not and if you're not hanging out in CJ s regularly we'll have a quick look at the syntax we've got an awesome jeaious file here and we are attaching we are exporting as its module interface this awesome person function that says so-and-so is awesome I wish I had people affirming me like that all the time and in some other file we are using that wonderful require method to bring in that awesome person functionality and that require method is going to have you load a module and now you can have dependencies the require function is actually really powerful it is something that will block notes thread of execution module loading in node is synchronous not a big deal actually because on the server side you're working typically with file systems and your files are there already it's not a huge tax or a huge cost to be dealing with with synchronous module loading and node but I'll walk through a few fun pointers related to the require method one thing that I learned actually this year while working on that react project was that node actually caches its modules and it looks in the cache first to see if the module has already been created as a module instance and so the fun Reaper could repercussions of that are that you could potentially modify that module after caching it and then when you're working with that same module again it might have some additional properties that you might have assigned to it it may have it it can change or evolve and have all of those properties that you you give it and then this module dot underscore load function that's kind of under the hood of the require method then calls this module dot underscore compile function and that's where the magic happens this method creates a special require function actually it's specific to every module instance that's being created and it generates a wrapper function so remember just like with the module of pattern where we had to scope things scope the contents that we wanted sort of protected in a module within a function so that we could have that little pile of data and functionality and properly scoped variables same trick here a little more sophisticated but we're still using function scopes and then it actually runs that wrapper function on the spot and I think I want to highlight here that all of this kind of happens at once you start wherever your entry is into your you start wherever with whatever file you're entering into you hit your require statement node goes off to another file does its thing doesn't move on until it comes back and finishes its trail of dependency unwinding so anyway so in case you're curious here's what the wrapper function roughly looks like I just grabbed this from the node docks and github they've got a nice a nice little readme type page on modules and yeah kind of made this point already but function scoping is a powerful tool in JavaScript nothing outlives the execution of that wrapper function except what's actually exported from that module and that's yep so that function will that wrapper function will assign whatever is being exported to that file to the module dot exports and off you go you have whatever you wanted that was exported from that file file name and durnais Mar just convenience variables so you don't have to type out the whole path and directory that's nice that node did that for us so there are of course options for using the commonjs module format on the browser many of us are again doing some kind of translation to whatever newfangled JavaScript code we're using so that we have continued compatibility anyway so CJ s wanders me on the server every now and again if we wanted to we can use those require methods if we want to so uh remember our friends at Commons is like the group of fed-up developers not the not the specification so in a strange plot twist even though they were really talking about JavaScript for the server side they didn't succeed in offering any kind of usable spec for the server side but they did create one for the client called asynchronous module definition or AMD quick look at the syntax there this define method takes as its first parameter this array of dependencies and then it has this function that it's going to fire off once those dependencies have been loaded true to its method it is asynchronous it is better suited to the browser and requirejs is one of the more popular libraries folks are using with AMD and then UMD the universal module definition which I heard a developer or I read that a developer was joking that when you try to please everybody no one leaves happy so this was an attempt to support both the CJ a spec and the AMD spec and with this Hulk of boilerplate as you might imagine it never really caught on no one wants to do that no one wants to do that so so rest in peace um D I'm sure there are people or I'm sure there are some code bases that are using it anyway so we may say that specifications really flourished in the early part of this decade but while they were a wonderful thing and while there was a lot of innovation and a lot of effort to come up with specifications that we all use it is possible to have too much of a good thing check Glowacki man all these names are hard he was a member of embers core team and he wrote in 2014 that unless all dependencies used the same module format dependency tree of depth greater than 1 is so painful nobody does it nobody agrees on a solution to this problem we are all so anyway specification hell and finally that brings us to the last part of our tour where are we now well I really wish I could tie up this presentation in a nice bow but the fact is that we're still kind of in the awkward teenager phase of Java JavaScript really finding its adulthood and finding oh it it's way of living up to the expectations we put on it we've seen a lot of far from ideal solutions but we finally do have a built-in module system and that's a huge leap forward in the development of a language that was never supposed to come this far so think about where we started yes modules actually are supported by all major browsers as of this year I think it was last month actually that Firefox just released their es modules compatible version and there's a lot of talk about how ES modules are asynchronous that's not exactly right really depends on the loader well we can talk a little bit about that but I think that's probably beyond the scope of what we can talk about tonight but other cool cool things include support for cyclic depend see so those are dependencies that either direct depend on each other directly or with a few degrees of separation in between um not going to go too deep under the hood with es modules today but let's at least look at this beautiful and consistent and concise syntax that it offers us we've got to export types on the Left we can have default exports some of this may be review for those of you who are doing this on a daily basis we can have named exports so here I've got a default export coming from a message that J s file I've got three little trivial arithmetic functions that are coming from a library J s file and then their corresponding import methods so we can import whatever we export it as a default export using the import statement like so with the word from when we're dealing with a file that has multiple names exports then we have the option of importing all of them we can even name give a name as to the containing object so importing all of these little functions is live or I guess I could have called arithmetic or something from libe j s we can selectively decide which parts of the interface that we want to bring in using I guess this is kind of it's not exactly yes that's kind of object this D structuring um yeah I would call that object this restructuring cool and in the browser we still have our trusty old script tag that has survived many years of JavaScript but now we have a new special attribute called type that we can call type and we can assign it module to let the browser know this is not regular JavaScript that you're going to be parsing you're going to be looking at a module file and that has to be handled differently so javascript can begin its process of module loading build building out a dependency tree and all of that we also have a fallback for older browsers so when you have the no module attribute this is going to be ignored by those modern browsers that are using modules natively the no module attribute says ignore me to those browsers but if you are not in one of those browsers if you're in an older browser you've got your trusty bundled regular javascript file in all of its global variable glory one of the conflicts the one of the current issues and JavaScript is that while most browsers are up to speed with yes modules node is not and you might be asking yourself why node and es modules can't just get along well the implementation actually is pending I was able to play around with it a little bit and just make sure you have the most current version of node and just use the I think it's called experimental modules flag or something experimental something flag and you can absolutely play with them in node but really what's going on is that yes modules in common jas modules these are really different specifications so when I was talking about node earlier doing the synchronous loading and execution of of those wrapper functions that was creating with require that's a really different approach to modules than es modules it's just they they don't play nicely together they are two different concepts of modules so one of the things that's caused a lot of contention is the dot MJS I think it's modular javascript file extension which is basically two node what the type equals module attribute on the script tag was to the browser so this dot MJS file extensions that says to note hey you're about to parse and yes modules an es module file some people have called it the michael jacksons script file extension anyway as I said I wish I could just wrap this up in a tidy bow but there are still a lot of questions about what the future is going to look like there's a lot of thought that es modules in HTTP 2 will be very complementary to each other with the possibility of server push and being able to sort of chunk out bits of bits of code quickly on demand and yeah I think we have yet to see what's to come and we're kind of in an exciting phase in the life the not so long life of JavaScript I think it must be and it's I'm terrible at arithmetic it's a twenty third year it's going into its twenty-third year just it's young adulthood and we all get to be a part of that and maybe at the end of that we will get to the promised land thank you [Applause]

Channel: freeCodeCamp.org

Views: 11,434

Keywords: javascript, js, javascript modules, js modules, es5, es6, npm packages, es6 modules, mejinleechor, girls who code, modular javascript, es6 tutorials, js tutorials, codesmith, codesmithio, codesmith tutorials, javascriptla, modules, modular programming (programming language paradigm), wesleywoo, javascript (programming language)

Id: GQ96b_u7rGc

Length: 44min 50sec (2690 seconds)

Published: Tue Aug 07 2018