ElixirConf 2021 - Mrinal Wadhwa - End-to-end Encrypted Messaging in Elixir with Ockam

[Music] [Applause] [Music] [Applause] [Music] all right we are back and we are here with renault he works with an open source project called ocam and he's gonna tell us all about it um hi everyone uh my name is mernal and my talk today is about end-to-end encryption in elixir with occam so occam's uh open source library actually collection of libraries in multiple languages elixir and rust are our primary focus at the moment and you can do all sorts of things with the occam and we'll in a second but before we do i want to sort of set a little bit of context and talk about why you might want end to encryption in your application so you know this this is my way of thinking about access control policies uh or decisions in an application and so you know you've got a subject you've got an object and the subject wants to do some action on that object and the application must decide whether it wants to allow or deny that action sometimes i like to call it subject action resource as well and um for this to work the application needs to be able to reason about the identity of the subject the identity of the resource and what rules apply in terms of in which scenarios a subject is allowed to do something to a certain resource so to connect it to a real real world scenario one of um one of the companies that is using occam and building with it is a diagnostics company that makes labs that look like this and go in hospitals and their their equipment does things like this where it's you know dealing in samples adding reagents to them running them through centrifuges etc so if we want to build um a secure trustworthy system uh we need to think about a few things so um to generalize a little bit the operating environment of a typical application any application really looks like this right there's the application there's the machine and the application's running inside a machine the machine's running inside a network the network is part of some infrastructure and then you've got the global internet outside of it and all of these points are boundaries where we can have certain checks in our system and you know the network may be all sorts of types right it may be a vpc a kubernetes network a net in your home uh or in a factory or in a hospital some kind of edge network uh maybe even non-ip networks industrial networks etc similarly the machine maybe all sorts of different kinds could be a virtual machine a container kubernetes pod um an industrial machine etc and these things may be operating in all sorts of environments whether it's cloud data center hospital factory et cetera or your own home um but broadly speaking these boundaries exist in in any environment and if we you know virtualization kind of complicates this picture a little bit but you know roughly speaking this is how we think about things and distributed applications work by sending each other messages across networks over the internet right and these messages may be uh data like you know reading from a certain sample in in that lab equipment or it could be a command that says add this reagent to this sample or add apply software update to that machine um etc or it could be an acknowledgment that software update was successfully applied or a reagent was successfully added it could be configuration etc um and to be secure both sides must check incoming messages for integrity and authenticity so we want we care about uh message integrity and authenticity sender identification and authentication and then of course authorization and this these three green check marks that i've drawn here are um very high level there's a whole set of sort of cryptographic detail that i'm hiding here but you usually want more detailed guarantees you know things like forward secrecy and protection against key compromise impersonation etc those are technical terms around designing secure communication but broadly speaking we need to know that uh who the who the sender of a message is uh can we authenticate that sender and can we be sure that the message wasn't tampered in some way um as it traveled between these two distributed parts of the system of course if you care about user privacy or protecting business proprietary data like you know um in a medical healthcare setting you definitely do um you also care about uh confidentiality and guarantees as um in these two different parts um and our traditional approach to sort of building these guarantees has been to do checks at the network boundaries right so we might have vpns in place or firewalls in place or in typical server deployments you'll have a um you'll have tls that terminates at some kind of load balancer at the boundary of the network right so all the guarantees of tls also then end at that boundary but this is sort of the traditional approach the problem with this approach is that usually within the network boundary are lots of things could be hundreds of machines or thousands of applications and if we are an application developer that's responsible for one of these like tiny little blue boxes we don't have much control over how safe or not safe the our operating environment our operating network is um so um for our application our tiny little blue box to stay secure all the code in both networks must have no weaknesses so that our work stays secure because if any one of those things has weaknesses it could get compromised and because our checks are at the network boundary um our application could get compromised and if you think about you know sort of real-world deployments let's take our lab equipment that we were just talking about actually uh the the company that makes that lab equipment has a machines that are operating in hundreds of labs around the world um and their cloud deployment is spread around various data centers and various cloud environments it's not just one vpc in the cloud it's several vpcs across clouds across data centers so for the for that application developers work to say secure all code in all deployment networks must have no weaknesses for um you know for for bad things to not happen this obviously gets untenable pretty quickly and in distributed systems um and more so um if you have you know intermediary services like um event streams uh pub sub cues gateways that are dealing in you know non-ip protocols um their operating environments must also stay secure um and this obviously violates the principle of least privilege so what we want to do is we to build dependable trustworthy reliable applications we want to take that big massive red surface of vulnerability and reduce it down to a tiny little surface of uh vulnerability right we're always going to have certain areas where we're wonderable but we want to reduce it down as much as possible and this is where especially if you have distributed applications with lots of things talking to lots of other things whether it's microservices in across data centers or it's machines in various edge environments um end to encrypted mutually authenticated secure channels can enable granular authorization decisions um at the application layer they give you con these channels can give you control um as an application developer uh whereas you know you may not necessarily have control over the network boundary of where your things are installed so um occam allows you to create dynamically these end-to-end encrypted mutually authenticated secure channels that are very lightweight um so the rest of my slides are actually code examples uh with certain uh pictures that show how these protocols actually work and how are we able to do general purpose end-to-end encryption um you may have heard about end-to-end encryption in you know applications like signal or whatsapp where it's doing end to encryption over two hops between my phone and your phone via some server but with ofcom you can do this in all sorts of different topologies so let's look at some good to get started uh we need to get occam we're currently not uh published on hex um so you have to install it from from our git repo this is a little bit of setup at the end i will point you to examples and you can go check these out later but there's a little bit of setup you need to do and this might go in your mix config if you've got an elixir project but then comes this notion of application layer routing so if we want to do end-to-end encrypted or end-to-end trust decisions and protocols we need a mechanism to move data across network boundaries whether it's from a non-ip network like bluetooth into the tcp realm and then into our cloud environment or it may be across two tcp connections right so like for signal for example is doing end-to-end encryption over two tcp connections and there's a tcp connection between my phone and the server there's a different pcb connection between your phone and that server and um the signal protocols enabling end to encryption in that two hop scenario um how can we build a system where it's not just two hops or two tcp hops only it can be any protocol in any number of hobs and that's what's enabled by this application layer routing protocol that is included in akam um this you can almost think of this layer this protocol as a overlay network on top of several different types of underlay networks okay how does it work right it's very simple every message in our com routing protocol has these three properties it has a onward route which is a list of addresses it has a return route which is also a list of addresses and it has a payload which is opaque to the routing layer so the routing layer just doesn't even look at the payload so if you've got a um and another thing that is needed to make this work is um addresses to things that are remote and um in inside auckland we have a notion called occam workers so app here is a worker uh it's much like an elixir process except occam workers are also available in other languages like like rust so app here is a worker echoer is a different worker and we're sending a message from app to echoer and then from echo back to app and if you look inside those messages this is what they would look like the onward route would be echoer the return route would be app payload and so forth so there's some basic rules of how this works um the basic rules are a sender says a sender needs to know where they're sending the message basically the route to the destination and it makes that route the onward route of a message and it makes its own address the return route of the new message the replier and takes the returned route of any incoming message and turns that into the onward route of an outgoing message and it makes its own address the return route of a new message um it's a it's a little bit of rules because this is a protocol but it's not a complicated set of rules a fairly simple set of rules so let's look at let's look let's make this work right so to start with we need a worker um this is occam worker macro here is really um when i say use auckland worker it's really just a wrap around gen server and elixir and on the rust side we have to do a lot more work we actually built an actor framework and rust to make it work potentially the end result is awkward workers have serializable addresses that are known across machines um so to write an auckland worker you'll be writing a echoer worker and we need to implement this function called handle message um in state you can find up the address of this worker and remember this is an echoer so the rules are make my i make the return route of the incoming message the onward route of the outgoing message uh so the code here does that and then make my address the return route of the new message that's it right this is an echo or worker how do we use this so uh the first line here is just me including some scripts but um on line number three over here i'm registering this this process this elixir process with the address app um and then i create the autumn worker which you can create by like this where you say echo dot create and we can specify an address if we don't specify an address if we get a generated address that's the value that's returned and then we're defining a new message right so the app wants to send a message to echoer so it's defining a new message where it makes the echoers address the onward route and its own address as the return route and it sends the message that's it right and it then waits for a message um so when we run this it will look like this um the echoer will receive the message and print it the app will then receive a reply and print it pretty straight forward so this is what just worked let's make things a little bit more complicated um let's say we have two hops happening so we now have a middle worker here we'll call it h1 um and the routes now look like this right remember our rules for make this guy already knows the onward route so he knows that he wants to go through h1 and to echo um so the rules for hop are a hop will remove its own address from the beginning of the onboard route and add its own address to the beginning of the return route this is pretty much the entire occam routing protocol um and it's really just enforced by convention in the system because there are certain scenarios where you can do more interesting things with it and i'll touch on that when we get to forwarding towards the end of the talk okay so hop worker looks a lot like our echo or worker except it implements that new rule where it says remove my address from the beginning of the onward route and add my address to the beginning of the return route and then everything else stays the same to use the hop worker this time around our code looks very similar in the beginning we register ourselves as app and we create an echo or worker but then we create this middle worker here which we call hop and we give it the h1 address and then we have this message and this time i we're saying hey route this message first to h1 then to echo and then return route is my own so when we send this message and wait for a reply this is what happens so we see that our message actually goes uh first to h1 then to echo then back to h1 and then back to app um so this picture worked but this is all still just happening inside one machine and i'll i'll soon get to how do you do this across machines across transports um but bear with me for for one quick second um this one hop thing that we just did is obviously generalizable to n hops right uh so we can have h1 to hn and the same rules same sort of three simple rules that we looked at uh apply and we can do any number of hops um and to illustrate here i have the the code in the beginning looks the same but i've instead of creating one hop created three hops and then i send a message along h1 h2 h3 to the echoer and my return route is back to the app so when i send this this is what happens right so it takes the path we wanted it to take it goes from h1 to h2 to s3 to echoer and then back in the reverse order so the protocol is kind of general um as long as we follow those basic rules now comes the question of doing this across machines and the way we do this across machine is almost exactly like we did it um inside one machine so um except in this in this scenario i'm going to have two programs a responder program and an initiator program i'm going to create a tcp connection which is this big gray line where the responder will be the tcp server and the initiator will be the tcp client so let's look at some code this is my responder code so the responder creates an echoer because that's where we want the messages to arrive and it then starts the tcp transport and starts listening on port 4000. the initiator does something similar to what we were doing in earlier programs it registers its own address as app but then it starts the tcp's transport doesn't say anything about client or server it just starts initializes the transport and then it creates a message that looks like this where our onboard route has a tcp address and then a worker address and then we don't do anything special we just do exactly what we were doing in all our other examples and we send our message and we wait for a reply and when we run this we start our responder uh we then start our initiator we send a message an app an echo app sends a message to echoer echo receives a message app then receives a response and prints the response so cool so it worked across one tcp hop what about two tcp hops and and you probably have guessed it by now it works exactly the same so our actually in this in this picture i'm creating a responder program an initiator program and a middle program and inside middle i'm going to have a h1 worker so i'm going to do tcp hop worker hop tcp hop worker hop so our responder program still looks exactly the same it has an echo or worker and a tcp uh transport uh listening on port 4000 our middle program has that hop worker called h1 and it has um it is listening on port 3000 um and then we have our client or the initiator of the communication and all the code looks the same except for this one line right in this one line it's basically saying hey go to tcp address 3000 first over a tcp connection then go to this worker then take another tcp address route then go to echoer if we run this it obviously you know you've probably guessed it by now it works exactly as expected but what's cool about this is that even though i didn't show an example of this this this route this that the message takes doesn't have to be all tcp it can have a first hop which is bluetooth a second hop which is tcp or it can have a udp hop a bluetooth hop or tcp hop any number of these transport layer hops uh can be taken and these transport implementations are very thin wrappers around underlying transport protocols to plug them into the occam routing layer um okay so why are we doing all of this right uh we're doing all of this because we want to create secure channels that are end-to-end that are end-to-end encrypted and end-to-end mutually authenticated so that we can have end-to-end application layer decisions about authorization um so secure channel um broadly speaking is a cryptographic primitive that follows a protocol that looks like this um there is a initiator of a secure channel there is a responder of a secure channel um the initial the responder is waiting for someone to initiate a key exchange the initiator comes in um sends message one to initiate the key exchange there is a response there is another response the end result is this shared secret um there is a set of cryptographic operations that are that are done during this handshake um in a very simple way this could be um this is based around diffie-hellman um diffie-hellman key exchange primitives but you never really want to just use a simple diffie-hellman uh you want to use a complex vetted audited key exchange protocol that has security proofs um in auckland's case we use a pattern from the noise protocol framework which comes from the signal team and has been heavily audited is used in several systems like uh whatsapp wireguard etc so he's just taken that primitive and implemented it inside auckland so you've got this handshake protocol the result of the handshake is this shared secret once you have a shared secret you can then do what's called authenticated uh encryption uh which takes the shared secret and encrypts the data using symmetric pre-cryptography so now the end result is the initiator and responder have a set of guarantees about each other because of the establishment of this channel the set of guarantees are complex as i hinted in the beginning there is a whole host of things you have to think about like uh management of keys authentic uh authentication of keys rotation of keys uh forward secrecy etc but all of that is kind of taken care of um by the handshake protocol and the ratcheting protocol and that is used in this setting but that's not our core focus of discussion today so let's go back to what we want to sort of touch on which is that we want to make this protocol already existed but goal with occam was make it easy to take this protocol and make it end to end and so here's how that end to end looks like you have an initiator and you have a middle and you have a responder this was the same as before except i've added this big green line in the middle this is my way of showing a secure channel that is end to end from here to here and the reason that works is because just like we were able to send hello over our routing protocol we can send all the secure channel messages over the application layer routing protocol so this is what that looks like um this is the uh responders program so the responder program creates an echoer uh initializes a vault which is our abstraction to sort of um cryptographic hardware and you then create an identity key pair and then you create a secure channel listener and it's listening on this address and then you you know just like our earlier example you create this sort of tcp uh transport and start listening on four four thousand you have a middle that knows nothing about the secure channel because it's just you know it's just passing data through and to an end-to-end encrypted data through um so it just looks exactly like our previous example it's just listening on port 3000. our our initiator has to do a little bit more work the beginning seven lines look the same they're exactly like our previous example but the initiator also needs a wall to store its keys it needs to generate its own identity key pair and then note this route this route is the same route we took to do our echo over multiple tcp hops earlier except the destination is not the echoer worker it's the secure channel listener worker and then we say hey create secure channel over this route r using this walton identity key pair and then we have to wait a little bit because this is an asynchronous protocol we didn't need to block on it but for the purpose of this example i block um so um this will do that entire handshake and establish the secure channel once the secure channel is established uh we can then tunnel a message to the echoer through the secure channel so note what happened to our route here earlier we have this whole tcp route over here when we were sending a message to the echoer but now we're saying send the message through the channel so when the message enters the channel gets encrypted and then when it arrives at the other end gets decrypted and is delivered to the echoer um and we then simply just like all other cases just say hey outcome router route this message and then we receive a reply so let's run this and so we start our responders start our middle and then we initiate the hand the secure channel and you notice a lot more happen this time but eventually our app received the echo back but if you notice like what's going on in the middle the middle node just saw encrypted data pass through um whereas the echo got the decrypted message and the app got the got the reply so we got a end to encrypted secure channel that passed through two tcp hops this is exactly what um signal's doing for example when it's sending messages from my phone to your phone right it's to endo encryption over two ccp hops but you'll notice because uh the occam secure channel just relies on a route underneath this route can be arbitrarily complex with any number of hops any number of underlying protocols uh it just works um so you can start to do some really cool things with this um you know our our initial example was uh this is a tcp server this is a tcp server but usually you want to flip that around the reason you want to flip that around is because you might have your initiator in a private network and your responder in a private network and you might have this public node that is only responsible for relaying encrypted data and so by flipping it around and creating this forwarding address we can tunnel a secure channel through this relay middle um and get end to encryption between two things that are entirely private and don't need to open any public ports this is a very common example uh this may be you know machine in a hospital cloud service in a vpc this may be device in my home my phone on the cell network um etc um so the core example for creating that forwarder looks like this um where it has to send there's a tiny little protocol here it has to send this register message but once it sends this register message to the forwarding service um it gets back a response saying this is your forwarding at us on that public node um and then our code example to forward a message looks exactly like what we were doing earlier where we say hey okay you know um the um the responder registers this forwarder and prints out its forwarding address and then the initiator says oh instead of having a direct connection to the responder i'm going to send my message to this public node on the internet at this forwarding address and we'll just see in a second that this message magically appears on the other side in the private network so i'm running my i'm running my responder then we run the initiator we copy over the forwarding address this could be discovered through various different discovery mechanisms and when we send the message we receive a forwarding forwarded message when this responder might actually be running in a private network so this work um and this is the toolkit that enables all sorts of different topologies so like i said you can have um a public node that only passes encrypted data and have communication between an edge application in a factory a hospital home and a private enterprise service in a vpc in a kubernetes in a data center environment without exposing any data to the public internet or having any dependence on network guarantees this is application layer control um on these guarantees you could obviously have device to device end-to-end encryption um which is pretty cool um you could have because the first hop could be a different protocol you could have a first hop uh be a radio protocol from an iot device to a gateway um and then have two tcp connections back to the private enterprise service and these guarantees are end to end so if you know if your gateway provider is compromised by some kind of you know mirai botnet or iot compromise your application is not compromised because you're having these end-to-end guarantees built in um you can also do something like move into an encrypted data through intermediary services like kafka or rabbitmq and in this scenario there are several tcp connections going on and there are examples of this inside the occam repo that i encourage you to check out later but in this scenario kafka only sees encrypted data which is very interesting in healthcare scenarios or for gdpr requirements where the requirement may be hey throw away the data in x number of years and then um it's very hard to do that if you've got the data in the clear and gets getting logged and partitioned etc inside this environment it's very easy to do if it's encrypted all you have to do is just throw away the key um all right so to conclude occam is this collection of cryptographic protocols this blue box in the middle that it has this pluggable model where it can plug into lots of different cryptographic hardware lots of different transport protocols and lots of different enterprise services but at the core has this application layer routing or routing overlay the secure channels and several other things you need to build end-to-end trust uh systems um so come check out awkam on github and thank you for listening

Channel: ElixirConf

Views: 586

Keywords: elixir

Id: QJm1zTQAyOc

Length: 34min 33sec (2073 seconds)

Published: Sun Oct 24 2021