(upbeat music) - Hello, hello. Hello, hello. Uh. Welcome to ICON. (audience laughs) (laughs) Oh am I in the wrong place? Oops (laughs). I didn't expect so many
of you to stick around. So thank you. Thank you very much. Hello RailsConf. Hello everybody. (audience oohs and claps) So I am going to introduce myself because I think that
there are many people here who don't know who I am. And the way that I found that out is because I kept trying
to airdrop people my cat and they declined it. (audience laughs) So that was me. Hi, I'm Tenderlove. That is my given name. I look a bit different than I do online. This is my, this is my online persona and I guess as Justin says
this has to be my brand. I'm stuck with this for
the rest of my life. So I'm sorry. I work for a company called GitHub. It's a website where you
can do and share code. It's a legit company to work for. (audience laughs) I went to the office and the office is actually
called Le Gite Hub. For french people okay (laughs). I'm on the Rails Core team and also the Ruby Core team. I have some cats. This is one of them. This is Ctac Airport Facebook YouTube Instagram Snapchat. It's her full name. We call her Choo Choo for short. (audience laughs) This is Gorbachev Puff
Puff Thunderhorse III. He's not a very good conversationalist. Frequently he ends up just making mistakes and putting his foot in his own mouth. So I have stickers of my cat. And hopefully you got them from me because this is the very last talk and I don't know if I'll be around. But I have stickers of them. So if you'd like a sticker come see me. But alright let's move
on a little bit here. Speaking, speaking of things that are orange and like to puke on my floor. I want to take a minute
to talk about Hacker News. (audience laughs) So I drew this slide. (laughs) So I like to be on the internet. I'm on the internet on the Twitters. And a lot of the people that I follow they think that Hacker News sucks. They're always saying, "Hacker News sucks. Hacker news sucks." And I'm like why do they say this so much? Are they actually reading it? Is that, is that. If you hate it so bad,
why are you reading it? But the thing is like I actually like to, I like to browse Hacker News and read the comments and stuff. I read them so you don't have too. But the reason I read them is because I like to think about, I like to think about
how they got it wrong and how they can fix it. So recently, like recently there was
an article that was like, "why Slack is inappropriate for
open source communications." Why is it? Is it? I don't know. And so I was reading the comments and one of the comments was you know. "We are, temporarily,
in a kind of dark ages "for end-user open source software." And I was like, "we are." It's actually true. In fact the other day my friend, he got bit by a rat. And he actually died of the plague. It's literally the Dark
Ages. (audience laughs) It's really the Dark Ages. I drew that too (laughs). So another article that I read was electron is flash for the desktop. And I'm like really. It's flash for the desktop. And slack it often idles at 5% of our CPU. It uses up 5% of the CPU. All the time. And I was thinking, "Oh that's interesting." So let's say you have your, let's say you have your CPU and slack is chewing up 5% of that CPU. Well one way to solve this problem is to actually just purchase a larger CPU. (audience laughs) You can see clearly now that it uses much less than 5% of the CPU. So the lesson here is buy
us a bigger CPU ya dummy. (audience laughs) I mean. I mean if Alan Turning
were reading these comments he would be so mad. He would be turning in his grave. He is a real person. (laughs) So I read another article
about language features that I thought was interesting. It said, "No generics and no sum types?" I think this was about Go or something. "These are no longer
groundbreaking features, "they're the minimum." So it means that if your language doesn't have generics it means your language is a failure. It means your language is a failure. And actually this comment is true. It's totally true. And the reason this comment is true is because insurance
companies only cover generics. (audience laughs) So. So if you take your programming
language to the hospital, they're just going to say, "No I'm sorry. Your
language has been denied." They will not take that. I drew that too. (laughs). So anyway. This is all I have to say, all I have to say about Hacker News. Just don't read it, I mean unless you want to know all the future episodes of Silicon Valley. (audience laughs) So. I'm from Seattle. We have a space needle. And I'm a huge fan, I'm a really huge fan of
supporting local businesses. I love to support local businesses. And in Seattle my favorite thing to do is I like to drink a lot of local coffee. (audience laughs) So I was trying to find out about different local things
to do here in Arizona. So I went and I decided to try
out some of your local brew. Which is Arizona tea. (audience laughs) So I actually had to go to three different convenient stores. Or I went to three
different convenient stores and none of them had Arizona tea there. So I started think like, "Well maybe in Arizona
they just call it tea." And you can't actually buy it here. And then I found at for sale at the CVS. So I went to the CVS and purchased some. And then I looked on the back of it and it turns out it's
actually made in New York. And I asked the teller about it. I'm like, "Oh this is made in New York? "I thought this was Arizona. "Like an Arizona thing." And she said, "No, no. "Only tourists drink that." (laughs) And I'm like okay well. I'm a tourist so thank you. So, alright I want to get to, I want to get to the next
section of my presentation which is one of my favorite,
favorite parts of presentation. Every year when I give a Keynote I like to have this section in my talk. Which is the Trolling DHH section. (audience laughs) It's an annual tradition. (audience claps) Unfortunately this year
it was really hard. It was very hard to come
up with ideas because I think like the main points
of DHH's talk was that you know he's gotten older. And as he's gotten older he's learned that other people also
have valid opinions. (audience laughs) And I mean there's not much
bad you can say about that. So I didn't know, I didn't know what to do about this. But okay I, you know I'm going to sew
a little bit of doubt here. So I've got a question. I've got a question for all of you. If Rails is so good, then how come there isn't
a city named after it? (audience oohs) (laughs) The other thing that he
mentioned in his talk was that there was no way, there was no way to scientifically prove which language is best. And actually this is completely wrong. This is a totally wrong statement. Just ask any Haskell developer
and they will tell you. There is a mathematical way to prove that Haskell is the
correct language to use. Anyway. So I'm a faith based programmer. (audience laughs) Everyday I say the programmer's prayer. (audience laughs and claps) I also religiously
listen to George Michael. The other thing that was
wrong with his presentation is that he clearly did not put the, he did not put the clear
winner of the console wars. Which was the Neo Geo. This is the winner. It had 24 bit audio. (audience laughs) Not this 16 or 12 bit crap. Come on. Anyways so. I realized this year that I've been doing Rails for over 10 years. I got my first Rails job in 2006. That's when I started
doing Rails professionally. And I wanted to tell you, I want to tell you a story about my first, not Rails program. My first Ruby program. My first like real Ruby program. My first real Ruby
program I wrote in 2003. So it was 2003. 2003. And in this year, this year Lord of The
Rings was coming out. Lord of The Rings Return of the King. And I was really excited about this. I was excited. You'll notice the release
date on that is December 17th. Okay. Now my birthday is December 16th. And they were coming out with this movie. This movie. They were gonna play this one and the other two back to
back in a movie theater. So it's like 10 hours of movies right? And they were gonna do this
a day early on my birthday. So I really wanted to see, like I really wanted to see it. Now at the time I was a J2E developer. And well this is a secret so. I thought there were only gonna be my five best friends in the audience. So I wasn't gonna say that part. I used to do J2E development. And it would take about like you know 10 minutes to compile our code. And like I have an extremely
short attention span. So 10 minutes I'm like, I forgot what I was doing. But anyways I'm like what should I do with this 10 minutes of free time? I thought to myself, well I wanna buy these tickets. And I go to the website and I try to buy the movie
tickets and the website breaks. It just doesn't work. It's like crashing under load. I mean it's 2003. People did not know how to sell
tickets online at the time. So I'm like okay. Well I've got 10 minutes. I'm gonna try writing a Ruby program that will go buy the tickets for me. Okay so I wrote this Ruby program during my compile time that
would go to the website, enter all the information, including my credit card number (laughs). And try to purchase these tickets. And the thing is I knew, so I knew what would
happen when it failed. When the website failed right? So I would go to the
website, I'd fill it out. And I knew what the page
looked like when it failed. But I didn't know what a
success case looked like cause I couldn't buy the tickets. So what I did was essentially is like okay if I don't get a failure, if I don't get a failure I'm
just going to log a message. Okay. And I thought well I'll
see it in the terminal. I'll see that in the terminal and then I'll know oh
I need to go check out. See what's up. So I go back to writing
some JSP or whatever it was. Like probably configuring some XML files. And later on I notice, I notice oh my god. The output on my terminal is different. What is going on? And I see all of a sudden it had actually been succeeding. And I scroll up through the terminal and I'm like oh my god. (audience laughs) So I'm like what do I do. What do I do. How many movie tickets did I buy? So I called the credit card company and I asked them like
the most I don't know like non legit question ever. I'm like, "Hi I was wondering "how many times have I
charged my credit card today?" (laughs) Like who asks that? I mean if there's a suspicious question, that is one of them. So the person was very helpful. They said, "Oh well you've only charged
your credit card once." And I'm like okay great. Just once okay. So I wanted to make sure that I actually got the movie tickets so I called the movie theater. And I don't know, looking back on this I don't know how they actually had
this data but they did. So I asked them like, "Hey I tried to buy some
movie tickets from you "and I'm just wondering, "you know you're site was down. "I'm just wondering if the
transaction went through "and if I actually got the movie tickets." Person looks up the data for me. And they say, "Yeah, yeah. "you were able to buy some tickets "but it looks like we've tried to charge "your credit card hundreds of times." (audience laughs) And I'm like oh really. They're like, "what are you doing?" I was like oh I was just hitting refresh. (audience laughs) I don't know (laughs). So that was my very first
serious Ruby program was back then. Okay so I want to talk, let's talk a little bit
about technical debt. And I should move a little faster. I have a lot of slides here. Let's talk a little bit
about technical debt. Justin talked a little
bit about it in his talk. But I want to talk about a new strategy for dealing with technical debt. That I've come up with. So we know that developers
want to pay off technical debt. So as a developer you want
to pay off technical debt. But you know many of us
in the Ruby community have grown up and become managers. I've been in the Ruby community
for an extremely long time. And managers don't
actually want to pay off technical debt right? Cause it takes time away. They'd rather be adding features or whatever it is that they want, you know they want to ship. So what's happening is we've got this community that's growing up and we're ending up with all these people who used to be developers
and now they're managers. And they had published all of these gems. And all these gems have some
sort of technical debt in them. So what I want to do, my idea is, I want to create a meta gem. And this meta gem will actually package up all of these gems that have
technical debt in them. And what we're going to do is we're actually gonna to
call this meta gem a Bond. And (audience laughs), now these Bonds are made
of gems that have classes. And these classes have
extremely low FICO scores. (audience laughs) So. What we're gonna do is we're
gonna take those classes and we're gonna shop them around to different companies like like Test Double. And if Test Double won't
give us good FICO scores then we'll go to a different consultancy. The one down the road and see
if we can get a better score. So we're gonna package all those gems. Try to get them better scores gems or as we call them tranches. (audience laughs) So the other, my other strategy for this is that you know what we can do is we can take dependencies from other communities. For example younger communities like the Javascript community. They have newer developers and they want to pay off their
technical debt more quickly. So they're packages are
probably like Triple A rated. So what we want to do is start pointing at things
like webpack for example. Which has a good credit rating. So that we can actually sell these gems a little bit easier. Now what I think this is doing, you know I don't want to
be too forward about this. But basically what this is doing is when we depend on
these younger communities, that's actually helping us feed off the blood of young people. (audience laughs) (laughs) Sorry. Rather than doing our, rather than being able to focus on our, we want to depend on
these younger communities so that we can focus on
our core competencies. Which is concatenating a bunch of strings. (audience laughs) Alright so. So earlier today Sarah
was talking about how the different talks at
RailsConf are gonna be, you know we have like forests and forests and trees and getting into the weeds. The individual discussions
are getting into the weeds. And I'm sure you're thinking, "okay well Aaron's from the northwest. "So we're probably going
to have this beautiful, "beautiful looking forest. "We're gonna have a forest talk here." Since this is a Keynote. But as I go on with this, I'm afraid you're going to see that we might start getting
into the weeds here. We're gonna get into the
weeds with this talk. But what I really, really want you to know is that this is actually a forest talk. Okay. This is a forest talk. It's just that it's a
dead forest talk okay? It's a dead forest talk. And you'll realize this
at the end of the talk. Hopefully, what I want you
take away from this talk at the end of it is where did my 40 minutes go? (audience laughs) Give me back my time Aaron. So before we go too further though. This is not a joke. I need to do a part that's not a joke. Okay so five. (audience laughs) Five. Five dollar. Five dollar footlongs. No okay alright sorry. 5.1. Rails 5.1. I'm very pleased to announce
that Rails 5.1 is out. (audience claps and cheers) I'm honored. I think this is the first time I've been able to announce
the Rails release. And I want to say thank you. Thank you to the rest of the team. Thank you to the contributors. Especially thank you to Rafael. He did so much work to make this happen. So please (audience claps) Alright so I originally
wanted to title this talk I don't have time for this. I want to talk about things
that I would like to do but I have time to do them. And I like this title
because it's non-committal. Like I don't. You know it's just some things. I don't know, I don't have time. There's some things I'd like
to do and I don't have time. It's also a humorous
use of a common phrase. Which I think is good. The problem though, the problem though is that
when you're a Keynote speaker the organizers tell you. "Oh you can talk about
whatever you want to." There's no pressure. There's no pressure. And of course whenever
anybody says that too me, it just means it's all
of the pressure for me. It's just extreme pressure. An extreme amount of pressure. And in fact I actually
took the Searls-Briggs test and somehow my result was stress. (audience laughs) I don't like. So (laughs). So I'm thinking I want this, you know I like this talk title. But then I'm overcome
by fear and self doubt. I just think to myself you know what if I totally screw up? I'm gonna sound like, I'm gonna sound like I was arrogant. Like I didn't prepare for the talk. You know I didn't do enough work. So I thought okay well. What if I call it like five
things I don't have time to do. So I'll change it to that
so it's more concrete. People will know what to expect. But then I go even more worried. I'm like what if I don't actually have time to present five things. Or what if I want to talk about things that I do have time for? Now I'm limiting myself. What can I title it now? So the real title of this presentation is some things that I don't have time to do but you should do them. So maybe like three things. But I have time to do the other things. (audience laughs) So that's the actual title of the talk. I've split into two, two segments. The probably important segment and the definitely not important segment. So we're gonna go through the
probably important segment. And then after that I want you
to all shut your brains off and just relax because trust me this presentation
is a huge waste of your time. (audience laughs) So we're gonna talk about HTTP/2 and Rack. And I'm gonna refer to
it as H2 from now on. This is a thing that I do have time for. Let's talk about what is H2. H2, well it's a version of HTTP that's one better than H1. Most of us I realize, most of you are using HTTP1.1. So it's actually .9 better than that. But still better. Still an improvement. So really, really what it is is this list of stuff. It's a binary protocol. It multiplexes, which means that you can
do a request and response over the same TCP connection. It's secure. It only works over SSL. That's not technically true. It will work over a
plain techs connection. But all of the, or an unencrypted connection. But all of the main browser vendors only support it over SSL. And the other major
thing is push responses. And I'm not gonna talk
about the first three because they don't really
impact us as web developers. I'm just gonna talk about the last one because that's the major change for us as application developers. So push response flow. In a normal H1 application, you're request and response
will look something like this. You'll have an html page. That html page gets returned. It depends on jquery and
say my cat's CSS file. Right? So we have to go back and
forth and get all of these. Now with H2, when a user requests that index.html page, we know that they're going to need jquery and the CSS file. So what we can do is we can say, "Okay when you make a
request to index html, "I'm also going to push down
these other assets as well." They're going to come down along with it without you needing to request it. So we're saving time by
pushing these needed resources down to the client. Now unfortunately this has a problem. There's some problems with H2. And the problem is that H2 is stateless, in the same way that HTTP1 is. So we don't necessarily
know whether or not somebody has received that asset. So we have a problem where maybe we send those assets down to the user and they already have jquery. So in that particular case, the client may actually cancel it. So we'll send them index. It tries to send them jquery. Tries to send the CSS file. And the client actually has to say, "No, no. I don't want that. "I don't want that." And in some cases it may not
say oh I don't want that. It may just take the asset. And now you've actually sent
the asset to the client once. And so the client can cancel or we may actually resend the assets. Both of these are problems
with this type of push, push service. So the solution is H2O and salt. (audience oohs) I feed on your pain. (audience laughs) It energizes me. No okay really the solution
is H2O and cookies. So let's talk about H2O. H2O is, and yes this is an annoying URL, that is example with L at the number one. Okay. Yeah I know right. Anyway H2O is a proxy. It's an HTTP2 web server. It's a proxy server. And I'm gonna talk about the solution that it uses for dealing
with this asset problem. And it sits in front of your application just like say Engine X would for example. So we have a client here. And the client talks to
our proxy, our H2O proxy. And then the proxy talks
to our application server. Now the proxy speaks H1 to our app server and it speaks H2 to the client. So if we want to do H2
pushes we actually can with an H1 web server. And the way that we do
that is we can use a link, what's called a link header. So when we send the response, for example this response. The proxy will actually get the response, look at these link tags, and say okay I'm going
to push those assets down to the client. So in this particular flow, our server responds with
some value like that. It sends it over to the proxy. The proxy says, "Okay I'm going to push these
assets down to the client." Now H2O, what it does,
what the web server does. Is it actually makes a
fingerprint of those assets and stores them in a cookie. Now if you've already pushed those assets down to the client, it won't push them again. So it knows whether or not it needs to actually push those
assets down to the client. And this is extremely nice for us because it means that in our application we don't have to do any state track. We can just say, "I'm just going to push these down. "I know it's an index. "I'm going to send you the link headers. "You deal with it." So the proxy takes care
of all that for us. Now there's one problem with this response and that's that if you
look at the response here. We actually have to send the link header after we calculate the response code. So what if we don't actually
know that it's a 200 yet? What if we want to send some assets regardless of what the response code is? So the way that we can
do that is with a new, there is a new proposal being proposed. Verbs whatever. Called the 103 Early Hints response. And what this is is
juts a new response code that basically has no body
and it looks like this. We just stuff our link headers into that. So that previous example
would change to this. So we can send a 103 with the link header and then go on and do the
rest of our page processing. And H2 understands this 103 response. So this allows us to send push responses without calculating the status code. So let's talk about Rack Support. And this is going to come into Rack 3.0. The main thing that
we're missing from Rack is we're missing this push response. So essentially what we're going to do is add another hash key. Wooo. Yeah. Yes. Now I'm not very excited
about this solution. Because I don't really like the API of just passing hashes
and stuff everywhere. But the win that we get from this is that it's backwards compatible. You'll be able to
upgrade your applications without breaking anything. And you can actually gracefully
upgrade your applications. So we can say, "Alright I'm ready to
take advantage of pushes "so I can that now with my web server." And this is all coming in Rack 3.0. Hold on a sec. Getting a little dry here. Ah. Oh local tea. Support local. Actually you know what I'm
going to set this down here. I don't want to (audience laughs). I like the joke, but I don't like it
enough to ruin my laptop. (laughs) So alright let's, this was the part that you
had to pay attention to. And now feel free to shut off your brains for the rest of the presentation. We're going to talk about code smells. So I want to talk about code smells. So I have been working
on a project at home. I've been collecting a lot of data. And this is some data
that I've been acquiring. I have digital signal that
I've been collecting at home. And I apologize that the signal is, it's not very good. The data is kind of crap. And I want to do some, I want to do some analysis on this data. Okay. So I want to do some processing on it. And I've been trying to
do the processing in Ruby. Now unfortunately it's a lot of data. I've been recording a data
point every second since what is that? January 23rd this year. So it's still recording as we speak. And I've been trying to
do data processing on it, but I need to do a lot of math. And I'd like way to increase or speed up mathematic operations in Ruby. So I was thinking lately about doing some Ruby VM Tricks. And I'm going to show them to you. Show you what I've been
thinking about doing today. So first we need to talk
about how a VM works. The way a VM works is it's pretty easy. We have a program and there's a stack. And then there's a program counter. And this program counter always pushes at the next instruction to execute. So the way these instructions work is they manipulate this stack. So first we'll push on a
number four to the stack. Then we'll push a number
three to the stack. Then we'll add and the way add works is it pops the two values off the stack and it pushes the new value
onto the stack so we get seven. Now the way that this is
actually implemented is something like this. We have a list of instructions. And we have a loop. We loop over the instructions. We get the next one. And we look up what that
method for the next one and we execute that method. And we just keep running through this loop over and over again. Now there's a way that we
can actually speed this up. And we can actually eliminate this loop that loops over the instruction. And we can do that by
generating the virtual machine. So if we generate the virtual machine and we can generate a bunch of functions. And each function, each function represents
the instruction sequence that we're going to execute. So this is what the VM might look like. We have a couple instructions, or a few instructions implemented there. And we have this small bit of code at the end of each function
that's been generated. That bit of code right
there was generated. And what that does at
the end of each function is it looks up the next
instruction to execute and then just calls that one. So we're able to eliminate
that infinite loop. This technique is what's called
a threaded virtual machine. So you can think of it as threading. We're going through one function and popping out of that
function into the next one. And we keep going and going. So we're threading
through the instructions rather than doing a loop. Now there's a problem
with these function calls. A problem with function calls is that we can end up with a very deep stack so. What we like to do is
eliminate these function calls. So we can actually do that by getting rid of the function lookup. So what we say here is we say, "okay this is our normal lookup process. "We actually lookup the function "and we call send. "And then that goes and
calls the function." Instead what we can do, and this is a trick that
you can't do in Ruby, it's a trick you have to do in C, is we end up with a switch
statement like this. A label with a bunch of go to's. I use the word jump because that is I don't know I guess it's more socially acceptable than go to. So what we do is we actually look up the address of the place
we need to jump to. So we have this generated bit right here. So we say alright we're
going to generate this. We'll look up the address
we need to jump too and then we jump to that. So we don't actually have to, we don't actually have to
call a function anymore. We still have a problem here. There's a way we can speed
up this VM even further. And that's eliminating that table look up. We can get rid of that table. So instead of looking up the address that we need to jump to, instead we imbed that address
into the byte code itself. So when we're compiling your code, we imbed that address into
the instruction sequences. So then when we look it up we just say okay jump to
this address directly. We don't have that jump table anymore. And this is what is called
a direct threaded VM. And this is the VM that's
used in Ruby MRI today. Where the interesting thing is that these addresses are just integers. They're just arbitrary integers inside of your computer's memory. And we just jump to them. So MRI's VM is direct threaded. So what I would like to do create custom instructions built from, and build machine code at runtime. Since those instructions
are just integers, they're just places to jump to, we could actually just insert our own. Our own address and
just jump to that thing and have it execute there. So I put together a very
simple example of this. Please read it carefully. This actually uses LVM
to assemble machine code. And we can execute that
machine code from Ruby. The only thing that's missing here is actually disassembling the VM code and inserting our own integer to jump to. So my first idea of stuff
that I don't have time for is creating custom instructions at runtime that we can actually speed up
our mathematical operations. So okay we've got one
way we can speed this up. Now back to the data. I've got, you know, a faster VM would definitely
help analyze this data but a fast VM isn't
going to do us any good unless we have algorithms that can help us deal with analyzing this data. So in order to analyze
this digital signal, I would actually like to
build a police scanner. So now you know I am 36 years old. I am soon to be a, I am now a home owner. I need to be sitting on my front porch listening to a police scanner. Because that is what old people do. (laughs) Ah yes you will be me in the future. I'm sorry. So I bought one of these. This is a very neat tool. It is a software defined
radio that you can, a radio you can plug into USB. Now I'm going to give you an example of what the software actually looks like when you use this one your computer. It looks like this. This is live. I'm actually scanning, doing some police scanning right now. That bar in the center
there is a digital signal. The digital signal indicates what channel people are broadcasting on. So the digital signal is used to coordinate between different channels. This is what's called a trunking scanner. So you can see up there I'm actually manually moving the signal left and right. So the audio voices are actually analog and the thing that
coordinates which frequency those analog signals are on is that digital signal that's there in the middle. Alright I'm going to mute this. So you get an idea. You get an idea what
we're dealing with here. So you can move it back and forth. This is an example of an FM band. FM band radio. So you can see it's a little bit wider. This narrow FM that police use, this is what it looks like. In the middle there is our digital signal. Our digital trunking signal. And over on these little bars that you can see jumping around
those are our audio signals that we need to listen too. So we have to be able to
decode that digital signal in order to find the audio
signals to listen to. So to do this we do
digital signal processing. And this is actually split
into two different signals. There's two types of signals. Continuous signals and discrete signals. Continuous signals are like maybe the light is coming to you right. You see light or you hear music. A discrete signal is what we
deal with on the computer. Cause we actually have to
take samples of those things. We can't store an infinite
number of points on the computer. So we deal with discrete computers, discrete signals. And I made this. Persona five people
please appreciate this. Thank you I worked very hard on that. So discrete signals are
just a list of points. So again we've got this
discrete signal here and I do apologize for this crappy data. It really is just crap data. I apologize. But this is a discrete signal
that I'm trying to deal with. So the way that we deal with these is through a process called convolution. And this is what, what convolution is is taking two signals combining them into a third signal. And this is kind of what we're doing here. I apologize for the star. That star there. The star is actually the mathematical notation for convolution. Unfortunately it's also the thing we use for multiplication in computers. So convolution is taking these two signals and coming up with a
third signal from that. The formal definition is like this. We're not gonna, I won't describe it here. We'll do it in pictures. So we have three signals here. That top one is our original signal. The middle one is the signal
that we want to convolute with. And the third one is our resulting signal. So the code for actually doing this is, this is the code for doing it in Ruby. And I want you to notice
that it's based on arrays. We're using array indexes in this. And our array indexes start at zero. This is important. So our convoluted signal length is actually the length of
the two signals minus one. And the way the algorithm works is we have a signal at the top there, that's our origin signal. The signal at the bottom is the one we want to convolute with. And we take those points
on the bottom signal and multiply them times the
points on the top signal. And then sum those to get the points in the resulting signal. So we actually move it along. It starts at zero like this
and we move it along like that. And keep calculating those. And that's how we come
up with the new signal. So we're going to look at
a few low pass filters. Low pass filters are
just these little chunks that we apply to our, apply to the signal. So this is an identity signal. It's just a number one. And if we take the original signal, convolute it with the identity signal, then we just end up with
the same signal again. We can actually use this
to define an amplifier. So instead of multiplying
by one we have a two. So it looks the same, but you can see that bottom signal is actually twice as high. We can use this to shift
a signal one to the right. So if we set point one to one, we've actually shifted
our resulting signal to the right by one. We can take these little kernels and combine them together to do interesting things to the signal. For example we can make an echo. We have a amplifier plus a shifter which comes up with an echo. We an even come up with an average. So we'll take these two signals and figure out the average of a signal. And the reason I want
to do this is because taking the average of the signal will smooth out that
random data at the bottom but keep those sharp edges. Which is what we're trying to detect. We can actually use
this for radars as well. So for example if we transmit a signal like the one of the left and we get a signal like
the one on the right that contains random noise in it, we can answer the question does one signal contain the other? And we do that by taking
the received signal and convoluting that with
the transmitted signal. And if that received signal
contains the transmitted signal then you'll get a very
high spike where it is. So we can actually detect
when our signal is there. Another neat thing that you can do is you can come up with
a Gaussian Distribution. So if you take one signal and you convolute it
with itself a few times, you'll actually end up with a Gaussian Distribution like this. And I just think that that is neat. So again here's our convolution code. And remember that it's based
off of zero base indexes. And I thought to myself, you know I've got to do
all these transformations. And I want to graph them. And I want to graph them in R. Because doing it in R is pretty cool. Like it makes you look smart and stuff even though the graphs are crappy. Everybody's like oh crappy graphs. You used R. (audience laughs) So I thought to myself, "okay well I'm gonna port this. "I'm gonna port this Ruby code to R "and then graph that data." So I learned about R the hard way. I took this, I just ported the code straight over to R. And it turns out in R, arrays are actually one based. They're based on one. Now what's really extremely
fun about this language is if you ask it for the zeroth element, it gives you the type
that's stored in that array. So in this case numeric. I have a list of integers. And what's even more fun
is that you can perform mathematical operations on that type and it just works. No error. So (audience laughs). I'm like why are these
calculations totally wrong? I was very frustrated. So I thought to myself, you know I don't want to figure out, I don't want to figure out
how to port my Ruby code to use one based indexes. I'm not that smart. There must be an easier way. I thought to myself there
must be an easier way. Alright so back to the data. Back to this data. Alright so I've got this data and I'm sorry it is crap data. We're trying to improve it. But it is crap. So I didn't want to figure out how to deal with these one based indexes. So my next thing that I want to do and I just don't have time to do it is implementing cons, car, and
cdr in different languages but using only lambdas. And you may be asking yourself why. (audience laughs) Why would you do that to yourself? And the reason is I like
learning new languages, but I don't have much time. I don't have much time to learn a whole bunch of new languages. So I thought to myself, what is the bare minimum that I can learn in any particular language in order to get endorsed on Linked In. (audience laughs) And I thought to myself, well the one thing that I can learn. If I learn this, if I learn lambdas in
every single language then that should be enough to
get me endorsed on Linked In. So lambdas the thing I want to
learn in any target language. So what are cons, car, and cdr? Cons actually forms a
cell of a left and right, a left and right thing. So we just have two things here. Now the function car returns
the left side of the cell. So it'll give us back the left side. The function cdr will actually return the right side of the cell. Now can we do this only using lambdas? And the answer is yes we can and I've written it in three
languages here for you. This is R, Node, and Ruby. And I call it Node because
I don't know the difference between Node and Javascript. I actually posted this
function onto Twitter and I was like, "Oh why do I have to type return so much?" And then I got tons of
responses that are like, "why don't you use the S6 bro?" And I'm like I don't, (audience laughs) I don't know what that is. So. This is kind of a mouthful. And I need to move. But what it is is cons is a
function that returns a function and that function that it returns takes a function and calls that function. (laughs) (audience laughs) So it returns a function. Alright, don't think about
this too hard. Please. So car what it does. It's a function that takes a function and calls a function and passes a function. And the function that it
passes returns the left side. And actually the easiest way to see this is using R. So I've defined cons in R. And if we call cons, you'll see that it actually
returns two things. A function, so I like I said it returns a function, and it returns and environment. And the environment is
the thing that actually captures those two values. And you'll see car is just a function. And I can pass that, I can pass the return value of cons to car and recover my original value. Now here is the cdr implementation. And it actually just
returns the right side, it looks exactly the same as the car. We're just returning the right side rather than the left side. So this we can actually
form a tree of cells. So we can say like, okay we have a one and a two. Or we have a one on the left side. And the right side points at another cell. And then we can just keep doing this and end up with null on the very end. And we can actually
turn this into an array. So if I implement a function called each, we can iterate over that list. Now here's an example of usage. This is in Javascript. We can see we cons
together a bunch of stuff and we can actually loop over those and get our values back. So what's neat about this is it means well we don't need loops anymore. I don't need to know how your
language implements loops. I don't need to know how your
language implements arrays. If I implement my own hashing function, I don't need to know how your
language implements hashes cause I can just use a tree. So what about numbers? Well we can actually eliminate that too. I don't want to know how numbers work in your language either. And we can do that using a thing called church numeral encoding. And what that is is we define a function that calls something zero times. That's zero. We define a function that calls a function that you pass in once. That's one. And if we call the
function twice that's two. Or three times that's three. And that's the way that we encode these. And if you look at the very bottom there I can prove to you that
three is equivalent to three by passing in a function
that just increments by one. So we get three back from that. Now I'm lazy. I don't want to define
every single one of these. So I actually define a
function that's called add. Since we know that we just need to call that function the number of times for the number that it is. So we can define the function one. We define add. Then we can define two and three and et cetera, et cetera, et cetera. So I have all the numbers. And this is called church encoding. So I no longer need your numbers. I know how to do add. I can figure out how to do
subtract and multiply and divide. So I don't need your mathematics. I can even circumvent logic as well. So if implement, I can implement true and false in lambdas. And I can implement
conditionals with those. I'm not going to go into that because I do need to move along. So we don't need booleans. We don't need conditionals. I don't really need any of your language. I just need lambdas. So back to my crap data. Back to our original problem. I didn't want to rewrite my algorithm using one based arrays. Right? I wanted to rewrite the, I wanted to write it
using zero based arrays. And now that I can implement an array in any target language that I want to, I can make my arrays zero based. So what I did in R is I
implement cons, car, cdr. Then I implement each. Then I implement each with index. And now I have an array. And I implemented array indexing so I can say like at. I want to get an element
at a particular index and I'm going to make that index zero. And I can actually take
those vectors from R and turn them into con cells. And now I can actually
port that convolution code straight over to R using
zero based indexes. Yes. (audience laughs) (audience claps) Alright. Okay, okay, okay. So I'm pretty proud of myself. I feed in some test data. I got some test data there. I'm able to plot it. I can perform filters on it. I can define a filter, plot the data, the filter, and then do a convolution on the data and plot that as well. So I can see these are our crappy graphs. You can see I know how to use R. So I'm pretty proud of
myself at this point. I'm like yes, nice. It looks good. So I'm like okay let's take
our real data and input it. And I take my real data
that I'm trying to analyze. I input it and
unfortunately it says error. Evaluation nested too
deeply. Infinite recursion. It turns out that R does not
support tail call recursions. So this blows out the stack. And I just think I hate R. (audience laughs) So things I don't have time for: figuring out R. (audience laughs) Alright. Back to the crap data. So I have accomplished nothing (laughs). I want to share with you, I want to share with you
something that I've built that actually acquires this data. So I want to show you
where I got this data from and what I'm trying to, what I'm actually trying to
figure out with this data. So I want to show you
something that I've built. What I built was, I have this contraption right here. It's composed of a bathroom scale. So we have a bathroom scale. We have an msp430. This is a micro controller that is able to read off of the scale and give us the weight data
over a serial connection. And then I have a Raspberry
Pi plus a motion sensor. And the Raspberry Pi will
read that weight data over the serial connection. So I have it set up at home like this. So at the top there is, at the top is where I
have the Raspberry Pi mounted with the motion detector. And at the bottom there, that orange bar, is the scale. Now on top of the scale, I actually have what is
known as a litter box. And (audience laughs). When the cat goes in to the litter box, I can detect that. And the cat leaves. And I know that the cat has left. (audience laughs) So that is where this graph comes from. (audience laughs) (audience claps) I'm not done. Okay I'm not done. I'm not done. So on the left hand side, this is where the cat enters. And on the right hand side
that is where the cat leaves. And if we calculate the
difference between those two, we can figure out approximately
how much was left behind. (audience laughs) So I have calculated to
be about a hundred grams. (audience laughs) Alright so let's wrap this up. (audience laughs) I want to wrap this up
with a little kernel of knowledge for you to take home. (audience laughs) So that you don't feel like my entire talk was a waste of time. (laughs) So what I really want to let you know. What you should take home
from this conference today is that it's very hard to make shit scale. (audience laughs) (audience cheers and applause) (laughs) Thank you very much. It was an honor to be here. Thank you. (audience cheers and applauses)
