Forget Cloud: The Peer-to-Peer Web

so to get into this let's talk a little bit about the basics of how the web works I'm going really back to the initial conception of the web in its most basic form the web is a way to publish documents alright so if I wanted to publish document what I would do is upload it to a web server and then that servers gonna host it and distribute it out to my audience right and that's the publishing flow now of course the big deal with the web early on was the fact that you linked from document to document right but the hypertext that's the big idea the server was just sort of an implementation detail it's how we did it but that wasn't necessarily the key of this thing the key was that it was the web documents link to each other and without a complex top down a hierarchy you can be a free and open publishing platform alright and that worked pretty well and it got popular after a while we started want to want to do things we couldn't do originally on the web we wanted to have things like Wikipedia where you could edit the page inline we want to have things like Twitter where you can put a little status updates outright so deal with that we started to put an application code and I think my computers froze on in a second there we go started to put our application code into the server and along with that application code we began to put databases onto the server alright and this is how we began to have more dynamic behaviors on the web and we moved away from the web 1.0 conception of documents and links to the web 2.0 idea which is that we're going to have rich applications running on top of the web platform but these applications live in the server your application code your database your source of truth all going to exist inside the server and these computers with their web browsers are what you call a thin client they're just rendering a UI and the core of that application is up on the server now the good thing is that works really well the cloud has been in a lot of ways an enormous step forward for computing both for developers and for end-users everybody benefits from the fact that it is always available these servers are kept online all the time there man had sent managed centrally and they update automatically right so you don't have to go and update individual computers everybody's just on the same page handles a lot of data scales up really well as an end user I don't need to worry about synchronizing my devices or losing a hard drive because I can just reconnect my account on a new device there are a lot of benefits to the cloud so we got to the point where we started to say well you know what let's just do everything this way right so here we are now with these enormous servers up at the top and it's gotten to the point that if you're going to do something that even in its nature is a peer-to-peer transaction like for instance sending a file from one computer to the other well how are you going to do it of course you're gonna use a service right you're going to stick it on Dropbox or Google Drive and that's how you're gonna get the file over to somebody so we have what I would call a an architectural monoculture everything is done client-server regardless of the actual needs of your application all right well what's the problem if it works it works with the big deal well first of all the answer to that can't just be that you don't like Facebook politics are a good reason to set yourself out on a project but you actually need to have deliverables right what are we trying to fix well if you talk to web users nowadays probably the number one thing they'll say about the web if you're asking them what they don't like about it is that it's a little bit creepy and that you don't own your data you don't own your network let's drill into these individually first of all for whatever reason the web has never evolved a way for users users to send information from one device to another with complete in to in secrecy if you're really advanced you can figure it out using something like PGP but for the most part all the data we send goes in the clear through a third party so we're talking about emails and we're talking about documents we're talking about office information right electrical properties going into the cloud and a lot of cases it's sitting in a warehouse somewhere in plain text now a part of our obligation is software engineers is to protect users data and in that regard we're not doing a great job this is the security limitation in the way that we're doing things and we ought to move past it there's also a an element of a lack of trust I think that this sets his users at this point our custom to the fact that they expect to have their data mined and that is going to eventually cause our user base to lose trust in what we do and that is just something that we should not allow all right next problem lack of modify ability so show of hands who here recently has had a pull request merged into Facebook nobody right you can't do it the lack of end-user modify ability is a freeze on the amount of innovation we can use or put into work on the web right if users can't solve their own problems then they're gonna have to wait for companies to do it for them there's not a whole lot of a fine grained control about what your experience is like on Facebook everybody gets the same thing all right so we're losing opportunities to do interesting things or to make decisions for ourselves I'm a hacker I want to be able to edit my source code somewhat complementary to that is a lack of ownership of your actual data all the data is being stored into a database in somebody's server so even if I did have all the source code to Facebook and I deployed it on some server somewhere I still wouldn't be able to message my friends or send out a friend request or see their fees because actually all of Facebook is stored on the internal data structures and the internal data networks of Facebook it's not a federated system so they would have to complete the architect to support that they're not going to and so users lack ownership of their data they lack ownership of their networks and they lack ownership of their code so why talk about getting away from a client-server monoculture the reason is that we can improve the privacy of the web and we can also improve the ownership of the online experience for people talking about this a little more dryly and specifically you could say it's an architectural deficiency we've put a whole lot of work into making the client-server model work really well tons of resources and it's been doing a great job but the period of pure story on the web is pretty anemic now broadly speaking you can think of the web as an application platform for networked applications all right if you're going to have a lot of users working together you use the web and so it would make sense that we have not just one design for how our networking is done but some flexibility of this then not only can you do client-server but you can also have peer-to-peer architectures or even better a hybrid of the two to match exactly what you're trying to accomplish with your application so we want some flexibility in how this is done all right so that's the why that's where we're coming from with this project let's talk about what we've actually been doing broadly speaking the mandate has been if we integrate peer-to-peer networking into the web what do we get out of it what kind of features can we start to stick into the browser and so what we've done is we've take an electron which is basically a wrapper around chromium and use that to create our own Fork of Chrome we're not the only people doing this right now brave is a browser that's pretty well known that's run by Brendan Eich exact same story there's a couple others as well but for us the focus is on a specific kind of peer-to-peer networking now we're not talking about WebRTC and we're not talking about blockchains the kind of peer-to-peer network we're using is a variation of BitTorrent it's called the DAT project all right very similar to BitTorrent we kind of it's like a BitTorrent 2.0 the same basic principles are at work so if you have a bunch of users accessing certain swarm of data they're gonna Rijos those files and give pieces out all right so you're sharing bandwidth and as a result it scales automatically and I'll talk a little bit more about that in a second the way that we are using this is as a drop-in replacement for HTTP okay and we're now getting rid of HTTP but mechanically we've added a protocol that functionally is exactly the same as HTTP but it's peer-to-peer and as a result you have this automatic bandwidth scaling because users share bandwidth but also any device can become a server in an ad hoc fashion I'm gonna use this laptop make it a server if I wanted to use a mobile device all right anything also cloud servers as usual can access the network and serve on it as well so let's go back to our diagram with how publishing works on the web in our model rather than I want to publish my document here or publish some media or a JSON file or something I'm gonna do it directly I'm not gonna use a server to do it I'm gonna be the server and somebody that's requesting it will download it directly from my device okay now real quick show of hands who everybody here raise your hand if you are familiar with how BitTorrent works by the basic level you've used it may be okay so like 2/3 of the audience okay so we'll step through exactly what happens the person on the right they're the first recipient of the website that we've created and we've served it directly from our device now somebody new comes along a third party and they want to download this website what happens is that rather than just one server my computer that published the site in the first place we now have two servers we have my computer and the computer that downloaded it from me and the third party could receive those files from either of us or both of us and this compounds over time so that as new people come into the network they add in their resources to host files for other people and it scales up and up and up like that now participation is not permanent it's going to depend on how we tweak it but the way it works right now for a short period after visiting a website you will automatically riho stit give back to the person that gave you that content I have an altruistic concept that we call this the swarm and it's kind of like a distributed server it's like all of us are working together to comprise one server logically the swarm has some pretty interesting end results most of them have to do with cost because you are sharing bandwidth with the network automatically you don't have to pay the burden pay the cost of all that outbound bandwidth that's actually one of the most pricey resources for hosting online it's about 10 times as expensive as as disk space and if you're trying to get to a world where users are able to independently publish websites or content this actually gets pretty important because you can imagine somebody makes a video it goes viral you're talking about a lot of bandwidth if you just had an HTTP server at somebody's home even if they actually managed to handle all those requests we're talking YouTube kind of scale here you're talking a lot of requests a lot of bandwidth being used they're gonna have to pay for it probably depending on what their ISP situation is like so the cost reduction is actually an important enabler for making it possible for you to publish information that gets popular and not have to rely on a really big vendor like YouTube to actually shoulder the cost of that bandwidth right the network makes it cheaper the other thing that it does and that I think is actually little more interesting is it makes it really really cheap to allocate new websites okay because then this network each website rather be being addressed by an IP address like we normally do you're talking to a computer instead what we use are public keys these are cryptographically addressed websites so you can allocate a new website simply by generating a new random key it's very easy to mint a new website and you can make as many as you want and this really shifts the way that you think about how you build an application because now traditionally you would think you got your client server you got a server somewhere and you're gonna put a database up there and that's how you're gonna publish information well now it's very very cheap to allocate new websites so why not instead just create a new website and start putting the user data into that website anytime you have a new user account create a new website for it and start sticking the data on there as JSON files it really kind of shifts up the way that you think about how an application has to be built I'll step through it and a little more detail here before we get into the mechanics of building an app the first thing I wanted to make really clear is that it's a file based data model it's almost kind of UNIX II we're going back to the idea everything is a file all the data in the network is a file as a result of the protocol when you go to a site you can actually ask the browser to show me the file listing and it'll take you into an explorer kind of interface and I'll show you in a demo in a second what this looks like but you can actually get a full listing of the application or the website that you're visiting you can see all the files you can open up the folders and see what's in there and you can read and so this is at the core of it it's all just files and it's like a bunch of static websites really in addition to the Det protocol for browsing around there's a Web API that we've added which allows you to read these files and write the files to get the file listings and also do things like watch for changes on the network which ends up being really valuable if you want to make real time responsive applications that sit there and watch these sites for newly published information right now I'm hitting you with a lot of concepts but we're about to wrap it all together at the moment the way this works is when you create a site you become the author nobody else can change it is bound to your device all right so the wrap this all together let's go by an example suppose we wanted to make a Twitter clone traditional way to do that is we create a website and a web server put it up in the cloud somewhere get a database a user that wants to get onto our site would access the server and would say hey create a new profile for me the server would say fine it would create a new record for you in its database and then it would publish that information by merit of serving to clients that asks for the information that's the traditional model all right so how would we do a peer-to-peer version of our Twitter clone what we're gonna do instead is we'll ask our application hey please make a user profile for me and instead of talking to a server it's just gonna create a new website for you create a new pure site and inside of that site it's gonna write your information as JSON files now the code for this is on your device alright there's no server involved this is a peer-to-peer web site all the code comes down to your device it decides to mint this new website for you to store all of your user data and now the distribution mechanism is through the peer-to-peer network and what you're gonna find is that the peer-to-peer network is sort of like your new database through the files is how you're going to read and write data so now this is showing it graphically here you got your peer-to-peer application here and that's the our Twitter clone and it mints the user profile side where we're going to put our posts we're gonna put our lights we're gonna put our friends any information about what I'm doing on our little Twitter clone it's gonna live on that user profile side and all of this is happening on my device so we don't even have to be connected to the network at this point right because we're just writing files onto my in down to my my laptop here and then making it available on on the peer-to-peer network and we can do that later right so the writing of the files could happen whether or not we have an internet connection and we can distribute the files later when the internet comes back on so here's what that profile JSON might look like it's pretty straightforward I got my name I've got my bio I've got my avatar my picture and also I've got a list of who I'm following right and included in there is some information about who it is and a URL for them for their personal peer-to-peer site right it's just using peer-to-peer sites and using files so now you might ask let's make a listing of people that I know right I'm let's say I'm following three people and I want to get a nice list of all of our faces and all of our BIOS and things like that well first we'll say let's render mine so we'll access my profile JSON that's already on my computer so we can just read it easy and then we've got these three people on following so for each of them all find their peer-to-peer site on the network and I'll download the profile that JSON for each of them right it's called it got all the information I need and when will aggregate that information into a table maybe in memory or we can write it to a disk if we want to persist it so that we can don't have to access the network in the future and then rendering is pretty simple just like you would be doing this in a normal application you're just gonna iterate to your table and you're gonna pull the data out and put it on the page right and this pattern has worked well enough that we've actually standardized it into a database that lives in user land which we call ingest DB so here you have these peer-to-peer web sites and I'm showing here for the first time a little glimpse of these public key addresses this is a shortened version it's actually 64 characters long of these URLs these have files on them and Zoey and Jess TB says okay I know that on the post folder if there's a JSON file inside the post folder that's like a Twitter post to scan for those and when you find them ingest them eat them up and put them into a table for us and we actually store that table inside of index dB so that we can run nice fast queries against them right then let's say you want to make a change to one of those posts or you want to make a new post well you're just gonna write a file a new post post slash three JSON to put the new post in there it'll ingest its way back into our table alright and this is the data model that we've been using for this system to build applications and I'll show you an example application that does just this but the idea again is that it's nothing but files just like in the good old days of UNIX files are at its core when you have a record you just write a JSON file and that gets published on these peer-to-peer web sites so okay that's that's that's the data model we've been working with and you know what let's just get these two things I'll get to this later let's just get to it straight to a demo so you can all see what I'm talking about here all right this is beaker this is version oh wait it's not yet released let me move it down a little bits we're not cut off there all right so y'all are going to be seeing some stuff that it not many people have seen yet because this is all still in development so we may hit some bugs when they hit some crashes let's hold our breath now the basic browsing experience is more or less what you would expect you can go to traditional websites you can create multiple tabs you know we're doing our best to sort of keep the expectations of how the web works let's go over to our web page here and the first thing sort of unique is that we have this period of pure version available tag right so we have a mechanism of being able to see when there's a DAT version of a site so why don't we go ahead and go to it now all you're going to notice changing if things are working correctly is this right here the protocol is gonna change but we have the exact same content being posted on HTTPS that we do on the peer-to-peer dat version so all right there we go yeah there we go so this is exactly like you would expect out of the web like I say it's a drop-in replacement for HTTP so you can navigate around you use HTML you have embedded assets links and things like that it works completely as you would expect where you start to see some more interesting things as over here in the top right first of all normally there's a couple of people Rijo stting free hosting the signage to see that there's not any but there's your pure account right there that's how many people are on the network I guess I'm lucky I already had this downloaded on my computer and save permanently so I didn't have a problem accessing it we've also got this control here where you can start to take advantage of how this peer-to-peer system works where you can go and view the files back when we go ahead and do that so here we're inside of our files Explorer and there is that index.html right there and we can just browse around and see all the assets involved for this website okay so again it's all just files yeah go ahead binaries for our downloads oh right right we're not distributing it on yet but at some point we absolutely will put it on there ya know it still can work on demand so you can actually download subsets of a website individual files you don't have to download the entire website when you go to a new site okay so it's a good question but thankfully no you don't have to download it every time now related to that we also have this control here which controls whether or not your Rijo sling aside at all and this is a way for us to get into this idea of socially kind of altruistically giving away bandwidth and helping each other keep your stuff online and so this is a specific control usually it starts out over here you go to a site you're not gonna realize it you'll you'll temporarily give up some bandwidth but you're not gonna keep it online but you can adjust the slider and say yeah I'll give them a day you know I'll give back a little bit or maybe I want to give a whole month so I'll give a whole month and that kind of gets the timer going if we open it back up and let us know how much time is left on that time we get a month still right so that's a really key part of what we think is interesting about this is that you can start to really contribute to each other actual resources right this is kind of a little bit more cohesive a little bit more like we're helping each other out it will stay in to the swarm and watch for updates so the likelihood of you getting stale is pretty small all right now I'm gonna cheat over here and look at my demo outline Yeah right why don't I go ahead and create one of these pure sites so again very easy to publish directly from a device and that allows us to have this cool button right here in the menu just to create a new website the sitter face is a little janky we're still working on it but let's go ahead and make our PDX node b-sides alright alright it just dumped us into our little file explorer and we're looking at our site here once we're a little crowded for screen real estate I apologize for that but here's the site that I created and by default it sticks in this JSON file which is a manifest file it's including information that I just entered in as well as the URL now let's let's stick in some files here let's bring in a picture let's say so how about I jump over to pictures folder and alright some cute cats that's cool so open up the URL here and I could take this URL give that to anybody on the network and they'll be able to download it directly for my computer file sharing builder I did now you'll notice it's a pretty enormous URL we got there that's the 64 character public key all right we have the ability to use DNS which is exactly what you saw earlier so you don't have to look at those ugly keys for the most part what we do is we just shorten it so it's not too bad to look at and you know we'll see maybe people don't care at all but ok this is a website too so let's jump over to the root resource and see what we get alright the file listing you would expect that out of a website without an index.html so I prepared an index tell you HTML let me pull that in all right I'm gonna refresh the page there we go even got some marquee on there so I do all that to demonstrate that this all works very much as you expect out of the web oh there we go there's a broken interface let's see if we can do that all right that's good enough I don't know if you all can quite see that but this is just HTML we're just doing an image tag right there referring to a file on our website right soup familiar stuff alright back to the cheat sheet alright we got another application let's get into a little bit that data model I was talking about about how applications are able to share data in the first place to do that I'm going to open up an application that might crash us so let's give a shot cool can we survive this is a code editor that we built originally we built this code editor to be a part of the browser itself but we kind of realized maybe that's not the best idea was getting a little bit large so we moved it out into userland so now this actually lives in as an application this is a peer to peer application okay and what it's going to do is use web api's to read and write files and provide a nice kind of code under experience a little bit like a light ide and in fact we're using the monaco editor which is what vs code uses so it's actually a pretty nice piece of software so let's open up our site that we just created there alright there it is right there's our little PDX node set while we open it up again now let's make a change it's you know that marquee was bad ID let's get rid of that and add a few more exclamation points all right and I hit control s now it's gonna have to ask permission because it's about to modify another side on my device so of course you want to ask permission for that but I'll go ahead and allow it over here and refresh there you have right so it's all kind of like a universal web file system that we're working with okay should be pretty intuitive now let's see if you know we have a live reloading tool and they see where that's at yeah all right this may work you can see a lightning bolt here to indicate what I did is I just turned on live reloading through our tools up here it should be possible should see this thing reload without me having to touch anything well that's in question there all right there went the think about that live reloading to your question earlier is that that works over the network because whenever you enter into the swarm you're gonna sit there and you're gonna put out a UDP packet once every I think five seconds not a whole lot of bandwidth but you're sitting there asking anybody got something new and by and large somebody nobody'll say anything but if there is something new it'll come down and the browser can actually recognize that if you got live reloading on they'll reload the page for you just a kind of a neat thing that we found hey we could do that you know we have access to that information so we threw it in alright one other kind of cool thing about this data model can be demonstrated with the WYSIWYG II the way you see is what you wiki now only can you edit other websites with these api's you can actually do self mutation which may actually be a really bad idea but we did it anyway so let me make a copy of this site now forking is just like in github you can actually go to any site you want and say fork this thing for me you're gonna download all the files you're going to create a new site copy the files over just like that you now have a version of the site that you can edit all right putting stuff in people's control that's the idea so let's do PD X node wiki create our fork this up and fast all right done at the edit button and let's get some cool Styles on there hit the Save button and we did it we just mutated the site using its own code using the WYSIWYG controls inside of there we just wrote to the source code of it and change the content so actually you know we jump over to the File Explorer and look at the index.html and not gonna be able to know well I apologize I went and broke our view source tool so we can't see it but inside of there you would find that we actually just modified the index.html just wrote directly to itself why not we might at some point have to rein that back we're not really sure what the security profile is something like that it's gonna be but for now we figured it's it's kind of cool makes it easy to make nice self modifiable websites for a wiki like that it's actually a pretty handy use case all right now let's bring it all together with one last idea earlier I was talking about the JSON records and sitting there and watching them and ingest d-beam well after writing that for applications we actually realized that we wanted to put some application features in the browser itself so ingest DB is inside the browser as well and we've used that to implement some application sawyer's and the browser starting with the ability to create public bookmarks and apologies to the guy from pinboard but we've just been ripping off all these good ideas we've got tags on the bookmarks we've got notes so you can say why you bookmarked it in the first place and you can see here I'm following my co-founder Terah Bansal and I can narrow down to the bookmarks that she's made public so each of us has a personal site sort of a profile site that's automatically generated for you when you first run the browser and that is where we're publishing these bookmarks so I can jump over to a new site let's find some random links on Hacker News here cool that sounds great let's share that with the world so you just got out of this control and make it public and there you go what just happened I sent over to a file browser here let's go to my user profile and my bookmarks I was under Bloomberg so there it is right there there's the bookmark that I just published just as a JSON file on my profile side I was look at that side structure again real quick this is exactly what I was talking about a moment ago I've got a profile that JSON with my name and my bio and all that information about who I'm following and then underneath the bookmarks file folder we had these JSON files which are all my publicly published footmarks right so hopefully you kind of get an idea of how these applications work and how this is what's going on underneath just files but through that we're able to build applications that have you know the kind of features you would expect out of the web and we're planning on taking this to the point where first of all you can have sites that you publish and sort of like a shared wide area network file system or you can see I'm following terrorists so she's actually in my file system and I can look at the things she's published and we're gonna narrow these down by type so you can have applications or music album sites and things like that and then perhaps we can even get into an idea of helping each other automatically host the content you published like your profile and things like that so that we never need servers at all do you have a good ten people following you and contributing bandwidth then by and large your social network should stay online by the merit that one and ten are probably going to have their computer on right and we think we can take this pretty far before we really so a we're probably gonna put in the time line as well so you could do you know the equivalent of a tweet inside the browser and things like that all right so that wraps up the demo let me just wrap this all up with a really really highly biased timeline of the web both past and future this is kind of fuzzy but we'll go there look at the 90s you know what I'm going to call the academic slash hobbyist web it's kind of ugly it was cool everybody was into it and some pretty amazing things happen Amazon Google for sure happened at that time and Yahoo was pretty amazing when it happened too but it was also kind of I it was a web that only its mother could love it was you had to be thinking big to really care about it around 2000 we got kind more web 2.0 right that's what we had Ajax happened html5 happens I things get really really commercialized at this point and it's actually if you think back on it was pretty amazing to have slid some of the new products coming out like I was in maybe high school and we were printing out MapQuest Maps and that was just like the biggest thing in the world it's easy to forget how exciting those times were especially because up to 2008 it kind of slowed down by 2008 we kind of stopped experimenting as much with social stuff started be mooring about more about sharing contents and stuff like Pinterest and imager and some of the different a media platform Scott took off and really around that time these three guys won the show and that's why I would consider this the corporation driven portion of the web where they are deciding what we're going to do with the web platform for the most part not so much Amazon but definitely Google what we're going to do with the web platform what we're going to do with with our software and of course Facebook sets the content experience for most people on the web most people actually just think of Facebook as the Internet okay so where's this going to go next I'd love to have this be 2017 but let's give us three years of cushion before we can have this happen where we move away from the corporation driven mechanism and into a consumer driven web or perhaps a little bit more high-minded Liat a more democratic web because if we're able to put control over the software and of the data into the end-user devices that we should be able to move to a model where rather than having businesses build off our applications for us we can get really open source with it we can let users build applications from the ground up change things to fit their needs and as long as everybody has a fork button in their browser they should be able to express their preference for where they want their web experience to go so that is where I think we ought to be taking this both giving people more individual control over what they're doing but also at the same time solving some of the pressing privacy problems that the web has now and in addition to it being peer-to-peer I think we can get some Indian encryption involved and really lock up some of the deficiencies for how we share data so that wraps that up you can check out what we're doing at beaver browser comm o 8 will probably come out within the next two months we're not putting a hard time line on it because we don't want to make sure we really like what we're doing so that's gonna be somewhere the next two months so feel free to give it a shot and you can find us online all the time feel free to hit us on Twitter and we are there to talk to you so they go very very much for listening [Applause]

Channel: Coding Tech

Views: 26,292

Keywords: web, security, connection, file exchange, javascript, browser

Id: ZrUHx-bnfZI

Length: 37min 19sec (2239 seconds)

Published: Sun Dec 03 2017