DEVCON1: Understanding the Ethereum Blockchain Protocol - Vitalik Buterin

hello ever'one welcome to the third day of the conference deep dive into aetherium protocols so we started off with Fei Jie wonderful presentation from Alex on the protocol on top of which the other aetherium protocols run so a Kaveri makes sense to follow that up with a presentation on the etherium protocol so you know the idea behind ethereum blockchain first came around back at the end of 2013 when people were getting really excited about Bitcoin people were getting starting to get excited about blockchain technology and applications beyond currency and there is a public consensus that watchings are useful for stuff and not just money so you know what change could be used for asset issuance crowdfunding domain registration title registration gambling prediction markets IOT you know people are coming up with some new category of applications every week and so you know the problem at the time is that most existing blockchain protocols were designed like this single purpose tools that are that are specifically designed around one particular application so in bitcoins case currency in name coins case domain registration in in prime coins case you know trying to do scientific computing and in my opinion not doing a very good job of it so then we had second generation protocols that works like this so we had protocols that had somewhere around 55 different transaction types where one transaction type is something like you know create a binary option another one is join a binary option and one for settling a binary option one for making a bet one for settling a bad one for doing some kind of two party dice game and every time someone came up with a new kind of application that was deemed important enough to be worth including the people who are creating these protocols would just add yet another transaction type so at the time you know we had protocols that are fifty-five transaction types I even worked for one of them for a few weeks after I worked one of them for a few weeks another two transaction types got added good old contracts for difference so even still you know it's pretty obvious I think from especially from the standpoint of our current generation of this particular way of doing things it's just fundamentally limited so what's the solution basically something like a smart phone so you know the idea is that instead of so instead of having you know a a protocol that's designed around one very small set of use cases you just create a general purpose operating system and you let people build whatever they want as applications on top of it so you know if you do that then things suddenly become better you know people maybe when you're inventing the protocol you might be thinking that okay people are going to use this for binary options contracts for difference making two party bets doing domain registrations and whatever other 50 applications you are aware of at the time then you know release the white paper and six weeks later you realize that people are inventing applications in a completely different direction and guess what your protocol does not even have to change one single bit in order to handle them so that's the power of generality so what is the classrooms behind a theorem well it's of watching CNN for six years pretty boring with a few tiny additions so the first important one is this concept of a built-in programming language so in aetherium there is a kind of a built-in Turing complete scripting scripting language it's essentially a hybrid between kind of standard virtual machine architecture is a bit Bitcoin script and and a few other things and the point is that you know people can write programs in this script or more realistically in high-level languages like that can pile down to the script then people to take their compiled scripts put them into transactions and send the transactions off to the blockchain transaction gets confirmed an address gets generated and you now have a special kind of account at that address called a contract so two types of accounts user accounts controlled by private keys you know we all the same old elliptic curve digital signature algorithm we all know and love second kind of account contracts controlled by code so these two types of accounts in aetherium essentially have completely equal privileges anything that humans have the right to do BOTS have the right to do so anyone can create an application with any rules by defining it as a contract so this is essentially the basic principle so you know the whole world of a theory am i creating a decentralized DNS system so basically you know the idea here is you want to create a system where people can register addresses and if they you know addresses could be you know something something dot eath and if they own an address they should be able to set the IP address associated with that ad domain address so the way the system works is fairly simple so you see a sort of declaration at the top data domains with an owner basically a mapping between a domain name and a to bowl containing an owner and an IP address and you have two functions one of them for registering a domain which basically says if this domain doesn't have an owner yet then sends the owner to or set the owner to whoever sense that particular transaction and you have another function set IP address which basically means if you own a domain then you set the IP address to whatever you want to set it to so you know fear varies simple rule set this is you know in of course you know real-life DNS systems have substantially more more features to them but sort of the fundamental core basically stays the same and you know you can probably build a more complex DNS system on top of this just by turning the IP address into a hash and throwing everything else on ipfs if you really want to but point it's very very simple logic you can write it in ten line in 10 lines of serpent code and publish it through the theory earlier here in blockchain and there you have a system that's basically like named coin so you know this is basically the meaning of you know anyone can create an application with any rules once once this application gets created then anyone can interact with the application by sending transactions that specify the contracts address as their destination address address so you send the transaction to a contract and that's how you talk to the application that's how you call functions that's how you do you know whatever it is that you want to do so in Bitcoin the state is fair is very simple it's basically just a mapping of in the sort of while technically speaking this sort of list of unspent transaction outputs but what it basically means in practice is it's the waste of balance as the west of sort of the amount of bitcoins that every particular person has and if you want to do a currency system that's basically all you need in aetherium the state is kind of more complex it's this sort of key value mapping going from addresses to account objects and there's two types of account objects one of them is the sort of simple accounts controlled by a private key in that case all you have is an onset of balance the second kind is accounts that are controlled by code and in that case you have these extra two fields that get filled up one of them is the code hash so the hash the hash of the code of that particular account and the second is the storage tree root which any sort of fairly intricate way that I'll describe later basically represents the kind of the entire set of storage of that particular account so for example if I create a contract with this code and register a domain then the fact that one protects domaine got registered to my and got registered with me being the owner and then they set the IP address to some particular value that information all goes into any sort of persistent storage of this particular account so basically no mapping from addresses to what blocks of this is the entire state of the etherium system at any particular time so code execution every transaction specifies a to address that it's sending to so you know this is fear fairly simple it's the kind of logic that we all understand if you set the message then you generally specify who you're sending it to and you want to specify the contents so what happens if you set a transaction to the to address well two options one of them is that the destination address is just a private key control account in that case all you're doing is moving ether around so if you do that then it's basically just being a cryptocurrency what if you're sending to an account that has code then what that's doing is that's basically activating the code and it's letting the code and it's actually letting the code run for set for some number of steps so code in aetherium gives you three things number one you can just send ether to other contracts number two you can read and write from the contracts own storage so for example if you're going here then the way that I would call the register function is by sending a transaction with data formatted in a particular way and with that function is going to do is or rather well that turns actually going to do is it's going to start running the code that actually represents the register method and we are it specifies self domains address donor equals m HDL sender that's setting a variable what that actually gets compiled to is setting a particular key in the contract so in storage so then the third thing code can do is you can call or basically sort of create these sort of virtual transactions that go to other contracts so theoretically you know I could create another contract that has some different set of rules and that contract itself can actually interact with this contracts that contract itself can register domains again an IP address of domains I could you know theoretically do just about anything that an account is controlled by an address can do so every full node on the blockchain processes every transaction in stores the entire state just like Bitcoin it's bold because it's important so this is sort of one of those one of those points that's important to make because you know people often ask well if I publish transactions to aetherium you know who is actually wrote well you know who so who is actually going to be enforcing the contract and who end and how many nodes are running them and so forth you know fourth year in 1.0 the answer is very simple every node in the network runs every transaction and that base in the process of running a transaction in you know if that transaction goes to a particular contract it actually involves executing contract code and you know setting and sort of changing the state to whatever the state is after the contract gets executed so firm that's the end point you can actually think of it as being a rather fairly simple system you know you can think of the state as being a database and you can think of each of these contracts as being as as being programs that are sitting on one computer except the computer is massively globally distributed it's actually a highly secure Network backed by tens of thousands of computers around the world so in school because it's important so gasps this is one of those sort of key mechanisms underlying the security of aetherium so when you tell people that you have a turing-complete programming language and that you let untrusted parties send code that's that's executed in a turing-complete programming language by every single computer in an entire network the first thing that they're going to ask is well what about the halting problem so there is this problem there is this very widely known result in computer science which basically says that it is mathematically impossible to create an algorithm which can tell whether or not any given program is just going to keep on running forever so how does the theorem prevent attackers from creating programs that just keep on running on everyone's computers forever and the answer is this concept of gasps so what this basically means that if your em network charges a fee for every computational step that that a contract execution takes so the specific way that that's implemented is that every block in the etherion blockchain has a gas limit so basically a limit on computational steps right now the gas limit is set to three one four one five nine two which is which is a nice number that we picked because we're totally not math geeks and so the then you know if you're creating a book if you are a miner and you're creating a block then chances are you're going to want to fill it with fill it with the transactions that give you the most profit so trend every transaction specifies a gas price basically an amount of ether that it's willing to pay per unit of gas and it specifies a maximum amount of gas that's willing to take so a transaction might specify I am paying 50 Chanin which is basically 50 billion from an ether per unit gas and I'm willing again and I'm going to consume up to five hundred thousand gas miners see all these transactions coming in they select the ones that have the highest gas price and they include them in their blog so when each transaction specifies to know that it's willing to spend up to two hundred thousand gas than what happen there is two things that could happen one of them is that execution takes up less than 200,000 good gas in which case everything happens normally it's okay the transaction pays only for the computational steps that it uses second case that goes over the limit if that happens then all the execution gets reverted and the reason why that has to be done is because if execution could stop halfway through then you might be able to sort of perform weird attacks on contracts so we try to like set together the execution to expire just at the time when like one half of the changes it gets me but the other half doesn't get made and you know we kind of preferred developers and be able to just not worry about that and so all the execution gets reverted but the transaction still has to pay for the gas anyway so don't run out of gas there's also gas isn't just a sort of simple 1/1 computational step one gas thing there are some operations for example operations that increase the amount of storage done that every node has to store and those operations take up a very large amounts of gas so the gas system is also the sort of fairly intricate set of incentives that are designed in order to encourage basic you using the resources of the FM system responsibly so gas limit is sort of the counter parts of the block size woman in Bitcoin how is the gas limits set very fairly simple voting mechanism right now if you mine a block then you can upvote or downvote the guest limit by a factor maximum factor of 1 over 1024 there is a default strategy in that we currently sort of instruct all the clients to follow in the source code which is you have a minimum of three one four one five nine two and if the usage starts bubbling up and hitting close to the gas limit then we target a one hundred and fifty percent of the long term exponential moving average of the gas usage so basically sort of targeted so that if it does get filled up the gas woman is going to be sort of just a bit higher than what then what one blocks normally contain so there are alternative strategies that we're exploring so one of them is this for example target a particular home go raid so what uncle's are Elbit explain a bit later but the general concept is that the more the blockchain gets clogged up then basically the more blocks and the slower blocks propagate and so the more blogs kind of get lost and don't end up being in the main chain and as it turns out thanks to a future of the etherion protocol you can actually calculate how much this is happening and so you can kind of target a particular amounts to which it's happening and you can say if we're seeing more than let's say 20 percent waste from blocks being just following off the network then the gas limit should go down so there are other strategies that work in centering so transactions contain basically seven sets of values number one nonce so what a nonce does is basically every every transaction has to have a union has to have a unique incrementing nonce so the first transaction you send from an account has to have a non zero second transaction you send has to have a nonce of one third a seven ounce of two and so forth and the reason this is done is to prevent real replay attacks so if Alice and ten eternal Bob you don't want Bob to take that transaction and reinsert it into the blockchain ten times the year himself another 100 ether now lets us fix that gas price the amount of ether per unit gas are we already covered that start gas same thing to the address of the the account that you're sending the transaction to or in other contract you're calling value the amount of ether that you're sending data is just a field of data the transaction sender can fill the data field with whatever you want says just an array of bytes no limits on length but the more there the more data a transaction includes there is actually a fee for that in gas so basically you lose 60 you have to pay an extra sixty eight gas for every byte of transaction data that you encode VRS there are values in an elliptic curve signature so look if you're familiar with cryptography you've probably heard of seeing the RDS the V value is basically an extra flag needed to do public key recovery so basically instead of checking the signature against the public key you actually derive the public key from the signature so there's this concept called of receipts so receipts are objects that kind of get hashed into the etherion block internet here and blockchain and every transaction has a corresponding receipt so these receipts have a few pieces of data one of them is an intermediate state route which once again will describe the mechanism later but it's a kind of hash that represents the entire state after the transaction gets executed number two accumulative gas tiers so the total amount of gas used in that particular block including that transaction so those first two things are kind of needed to do this sort of step-by-step accounting that what that actually makes it possible to kind of check what the validity of transit of individual transactions very easily and the third thing is logs so logs are this special feature in etherium that is sort of it's kind of a different kind of storage so in general you know when you set variables that gets set in account storage and that can be read or written by contracts logs are basically append-only so if you if you have a transaction in one particular block and it creates some logs those logs get hashed into that particular block but they are not visible but they cannot be accessed by contracts and they appear just in that block they're not in any kind of persistent state so nodes if they wants to for example they can just not store logs and it can prune and they can prune them so logs logs are also 10 times cheaper in terms of gas consumption and storage and the purpose of logs is to allow sort of efficient white client access to event records so one example of where you want to use a log is if you go back to the DNS example you might want to add a single line to create a log in the register method and in let's say the set IP method and the reason why you wants to do that is basically in order to allow white clients to very quickly scan through all the logs and sort of see all the instances at which so mean you know the domains got registered IP address got changed and so forth so there's lots of applications that wants to be able to sort of have a history of things that happen and logs are a convenient way of do of supporting that so logs have up to have a data field and they have up to four topics and there is a protocol that uses bloom filters to make it very easy for white clients to kind of search through logs so basically if you create a log that has a particular topic then you can very easily search through the entire blockchain and sort of identify all of the logs that's that matched a particular topic so if you're in virtual machine so once again as I mentioned it's this sort of combination of stack based architectures traditional virtual machines stat so inside of the virtual machine you have the stack basically you know 32 but 32 byte fields up to a maximum of 1024 of them memory just in it's an infinitely expanding byte array but the more you expand the byte array the more gas you have to pay so most of the limits in aetherium they aren't sort of static their economic and that's just a standard thing you're going to see again and again storage permanent contract storage you can read and write to good environment variables so for example the virtual machine you can access the block number the time the mining difficulty previous block hash and a whole bunch of other data so one one possible use case of that is let's say if you have a some kind of financial derivative that can only be processed or you know called struck or whatever after some particular time then you can you can say in the contract you can check the current the current block time stamp against you know some particular limit that you said logs so the virtual machine you can create these logs and there is an opcode by which the virtual machine in 10 itself call other contracts so this is basically the environment within which this contract code gets executed now you don't have to write in virtual machine bytecode yourself there's high-level languages so and one of these high-level languages is called serpent so this is this is one of them there's also one called there's one called salinity which is being you with being used much more right now there's one called lol which is kind of much more low-level if you wants to like really go into low-level code and be as efficient as possible you might want to here's lol although you know in my own opinion you you should probably consider serpents as well because servant lets you go as well as lol does and it has high level features as well so you know basically you write code in these languages then you have compilers and you can pile the contracts into bytecode and you can publish them you can publish the by a bytecode in a transaction going to the blockchain the ABI so when you send a transaction let's say that I wants to call the register method of that particular DNS contract that I keep on using as an example function calls get compiled into a transaction data so in this case the function call gets the way that it gets encoded is the for but the first four bytes are a function ID the next 32 bytes are the first start the first argument so this was the the domain that I wanted to change the IP address of and the second or the second argument ends another 32 bytes which is the IP address that you want to set it to so every time you call a function of a contract in ethereum it actually gets compiled into a transaction which is encoded in sum in using this algorithm which you know in simple cases looks like this just four bytes 32 bytes keep on going with 32 bytes but there are some special cases involving sort of variable length lists and so forth everything in aetherium relies on this one serialization algorithm called our LP recursive length prefix encoding so it's very simple so basically you know if you you know it's recursive length prefix encoding so you take the thing so in this case if you take the word dog and you just hex encoded you get six four six F six seven and you before it you prefect you prefix it with the length so in this case the length is three and yes you can see a three here over here you know it's an empty string and so you have a length of 0 over here this is a list and so it still has a length of 0 but it has a different sort of start flag because it's meant to be a list and not a string this is the four people who were set theory geeks this is a set theoretic representation of the number two it gets compiled to this or encoded to this rather if you have a string then you know gets compiled it gets you realized into this so there are like five implementations of this there is a much more detailed description on the weekly but basically this algorithm is used for blocks transactions network protocol order we sort of everything in aetherium land aetherium also has a cool new mining algorithm so the goal of this so listen is called eat hash so the purpose of this is to be GPU friendly and ASIC hard so it's a target GPU is as much as much as possible and basically prevent them from the problem in Bitcoin where you have these sort of spouts these specialized miners that with extremely large mining forms taking up huge percentages of the network so the algorithm is this uses memory hardness to accomplish this goal so basically instead of traditional proof-of-work algorithms like Bitcoin where the bottleneck is always how much computation you can do here the bottleneck is memory access and the theory behind this is that the more that memory is something which is highly optimized in which people spend billions of dollars optimizing it existing computers already and so the opportunity to make further up the optimizations is theoretically smaller than it is for computation so the way that it does this is by using this sort of we clever multi-level deck instruction so basically the idea is that you start off with a seed then from the sea to use this memory hard function create a cache then from the cache you create a data set and the data set can be created in parallel so each individual piece of the data set actually gets created by hashing together 256 parents that come from the cache and if you wants to just actually run the algorithm once then what you need to do is you take the data that you're mining on turn that into a seed then you grab a random 64 nodes from the pieces of the data set and you get your result so what's the point of this the point of this is that mining with this algorithm actually is memory hard because if he wants to mine and if he wants to mine at any reasonable speed at all then you basically have to pre generate this entire one gigabyte data set so that every time you pick a different non so you can just sort of sample from it and that happens fairly quickly if you just wants to verify blocks then you don't actually need the gigabyte all you need is a cache and the reason why is basically that if when you get to this stage you don't need to create this entire thing you just figure out what 64 indices you want then from each of those 64 indices are just computed from the cache now this does require you to make 16384 hash calculations but or as it turns out that's actually not that bad if you're only doing it once so because verification only needs to happen once this kind of low memory stole verification process actually works so but if he wants to mine then you need to do it with a lot of speed and so he needs to have the gigabyte so this is kind of the basic principle behind sort of why mining was in the particular way that it was in etherium so uncle's so in general Bitcoin has a ten-minute block time aetherium has a 17-second block time why is the block time not something even shorter like one second the problem with fast block times is basically stale rates due to network latency so if the block time is extremely long like ten minutes you can think of network latency as being negligible by comparison you can think of it as basically being instant and so we basically don't have problems once you start pushing block x down then your problem is that you have this risk that let's say over here we have one block in 2059 then at some point someone creates another block 2060 that block takes time to propagate through the network and so someone else might actually make another block before hearing about this block and so we have two blocks both on top of the same block 2059 and they're kind of both competing with each other so this is called a fork and the problem is that one of the blogs has to eventually win and the block that doesn't win just kind of gets discarded and the person who mined it doesn't get a reward so this leads the centralization risks so basically the issue is that the more powerful you are as a miner the better you are at getting a high network connectivity and if you created the last block then you don't have that you actually don't have this risk because you can just immediately start making the next block so because of the more these network connectivity issues become important the more problematic the centralization issues become and so you have this effect where large miners end up being more effective than smaller miners so our solution is this mechanism of on goals which is basically that if this happens then sure one of these one of these blocks is going to win but whichever whichever block loses it can kind of get Rhian cluded in the blockchain and it still receives most of the reward so the point of this as it basically cuts the centralization incentives by about 85 percent and it kind of allows the ethereum blockchain to continue to biess to be relatively safe under somewhat harsher circumstances both in terms of fast walk time and as of high transaction load Merkel tree is so how many people have seen the Economist cover from Blythe master saying it's all about the blockchain well my opinion is it's all about the Markel tree so and particularly unlike Blythe masters dogs I like it so which is more so the point of Merkel tree is is that there are this construction that allows for compact efficiently verifiable proofs that a particular transaction was included in a particular block so the general principle is that you know in G this is actually exists in Bitcoin if let's say every block has a few hundred transactions then if you wants to verify that one particular transaction was included in a block then you don't have to download the entire block you all you need to do instead is you can kind of download just one sort of branch of this tree and the tree is this kind of hash tree where every single sort of node in the tree is actually the hash of the two nodes below it and so we need to do is just download this kind of branch just download the specific part of the tree that's going down to your particular transaction and you check the hashes that are just going up the branch you don't check all the hash it's just the hashes that are all relevant to your particular branch and if all those hashes check out fine then you know that your transaction action that particular transaction actually is in that particular tree and so actually is in that particular block so the reason why this mechanism exists has basically for white clients so what a white client basically is as a clients that accesses a blockchain without processing every single block without downloading every single transaction why do you want this well it's pretty obvious you know the Bitcoin blockchain right now is thirty gigabyte is 30 gigabytes and you know a megabyte gets downloaded every 10 minutes it might be good enough for a high-powered laptop it's probably not good enough for this and it's probably you know in 30 gigabytes is probably too much for the devices that's the walk thought IT is going to be creating so instead you use these clever what you use these sort of clever tree protocols in to allow these devices to only verify the absolute minimum of information that they have to that's relevance to their particular transactions so in etherium the Merkel tree mechanism is actually some quite a lot more intricate than it is in Bitcoin so the idea with aetherium is that it's not just transactions that get stuck into morkul trees it's also the state so if you hear about this concept of a State Route what that basically is is this sort of is this root hash and from this root hash you have another tree and this tree goes and this tree goes down contains hashes of hashes of hashes of hashes of data representing the state of basically every single account in aetherium so you have use account objects as somewhere at the bottom of the tree you know you have nonce balance code hash and storage and storage is yet another tree so the point here is that you know if one of these like one of these white clients whether it's my phone or a select IT device or whatever else wants to know what the status of one particular contract is it doesn't have to process the entire blockchain it doesn't have to trust anyone all than it needs to do is just ask for a vertical proof ask for this sort of branch of the tree and verify just that one particular branch so this is basically the sort of concept of Ethier and white clients where a result from fulfill foldy is actually going to be presenting quite a bit more about this I believe after lunch today so every blog header actually contains three trees transactions State and receipts so it transactions are transactions we know what those are the state is every you know the entire state of every account balance storage every all the sort of static information that you need to know and receipts are they consist of this sort of extra auditing information and they have all the logs so if you're white client wants to know what what wants to know you know what particular logs or let's say associated with a particular topic then the white coin is going to ask for local proofs of logs so future directions we talked quite a bit about this on Monday but the general prints also the general categories are number one proof of steak so we're proof of steak is going to fit in is basically just at the top so it's just a change to the consensus algorithm so instead of blog instead of users having to verify to verify proof of work they're going to verify validator signatures instead and the way that you would join the validate validator set is you know as Vlad describes by submitting a secure a submitting a security bond so that entire mechanism is going to either be part of a protocol or it's going to be sort of specified in one particular contract not fully decided yet and from a user perspective all that's going to change is that instead of validating a proof-of-work nonce you're going to have a list of signatures that you validate instead skaila blockchain rent so the idea with watching rent is basically that right now you can access storage you can your code can set storage keys and those storage keys will remain there forever but potentially this might present very presentative serious long-term risks because the amount of storage could that the ethereum blockchain stores couldn't just keep on going up and up and there's not enough incentive in some people's views to remit so kind of to kind of delete storage there is a bit of an incentive because if you kind of delete store if you delete storage then you kind of get a refund but even still there's sort of no incentive not to kind of use storage as a bank so blockchain grant us the idea that every contract is going to have to pay some amounts per month for every storage key that it has so instead of storage being permanent you know one could imagine it lasting one year two years five years you know if this is going to happen it's going it's going to be as reasonable as possible but that's the kind of basic principle virtual machine upgrades so that's just going to not going to change any of the other semantics is you're going to swap out the virtual machine that the code runs in so in practice for developers is going to mean very little because theoretically you could still program in the same languages will just have a different compiler that can parse a different bytecode but if you wants to do things faster then you could potentially I mean ideally even program etherium code and things like C++ in Java scaleability talks about that on Monday so you know the idea that kind of having not every note process every transaction but I mean right now as far as the etherium protocol goes all this is basically it thang you

Channel: Ethereum

Views: 214,061

Keywords: ethereum, smart contracts, blockchain, Ether, Homestead, protocol, Vitalik, Buterin, Gavin Wood, Wood, Solidity, developer, Crypto, Cryptocurrency

Id: gjwr-7PgpN8

Length: 38min 18sec (2298 seconds)

Published: Mon Jan 04 2016