Just In Time (JIT) Compilers - Computerphile

we're going to be looking at uh just in time compilation which is a technique for making programming languages run faster so we all want faster programming languages because then we get more speed and there are various ways we can do it so we can statically compile things that's typically done for languages like C or we can just in time compile them which is often done for languages like Java or JavaScript and the difference between these two is that just in time compilation is looking at the program running and optimizing it after it's observed it running so just in time is really a terrible name because it should have happened before you you actually got to that stage but that's the name we've got and it's a technique that you will all have used today if you've used a browser you're using a JavaScript just-in-time compiler and they're sometimes considered a bit magical so I'm hoping we can look a little bit at the magic I'm digging the idea of magical computing so I mean there's no Wizards I don't have a beard uh and I kept the cape at home today actually to disambiguate something that people often confuse people often talk about compiled and interpreted languages and there's no such thing so for any given program language you can implement it as a compiler or an interpreter or a just-in-time compiler so people say C is a compiled language but you can and there are C interpreters and people say JavaScript is a an interpreted or just-in-time compiled language but you can write a static compiler for it as well so what do I mean in the sense by those things um so a static compiler reads A programming just looks at the code the program is written and then tries to convert it into machine code let's say an interpreter uh looks at the program and then it um doesn't convert into machine code it kind of executes it almost as is it probably converts into some other representation but it's a very simple way of implementing a programming language and a just-in-time compiler nearly always starts a program running an interpreter looks at it for a while says things like oh look you're calling that function a lot or there's a function that takes some parameters in and they're always integers always strings so I will now produce an optimized version of that functional part of the program based on that information I've observed and it's really that Dynamic analysis and conversion into machine code at runtime that makes just-in-time compilers very effective they can be faster than a static compiler because they've got more information so if you just look at a program at compile time you've just got the code you've written on screen you can can't fully analyze it as much as you want so you'll make certain assumptions and guesses but they may be incomplete or even Incorrect and you'll optimize based on that when you're actually running the thing you've got more information so you can make a much better quality optimization but you've had to watch the program and observe it so it started slow and then hopefully over time it's hopefully it's warmed up is the term and then it gets faster warming up turns out to be another kettle of fish it doesn't always quite work as we expect these things have some very surprising emergent Behavior but generally they do work and when they do work they can be very effective so this is things like uh it knows you're kind of again alluded to before it knows it's going to be calling this this function quite a bit so it keeps that nearby and things like that is that it can do those sorts of things maybe we could try a little simple example so if I log in so what I'll uh it's just a simple example so there's a load of these that I could use the Java virtual machine to adjust-in-time compiler the JavaScript VMS all of the ones V8 and chrome spider monkey and Firefox and those are all jits but I'll look at one for python because um that's one that I happen to know fairly well and let's just write a very silly little Python program so I can write this function let me make a little bit of a bigger font size for you and it takes two parameters in and I will just add those two parameters together thing is am I going to pass it integers strings I can do all of the above let's just check that I'm not lying because I do sometimes sometimes intentionally but mostly unintentionally but if I do hello world I have to have a space in there if I save that file and then right so it prints out five or hello world so you can see I'm calling the function in different ways so this is why it's hard to statically optimize that function because integers strings it can take in all sorts of things now the normal implementation of python which I'm using here with the Python 3 binary is an interpreter doesn't have a just-in-time compiler and we can see some obvious consequences of this let's put this in some sort of loop so I'll just put some very big number here that looks like a big number to me and just repeatedly call that function with some integers it doesn't really matter now if I run that and I'll just call it with the time function and I'm going to say this is my newish laptop that has from memory 6 fast cores and eight slower cores and the it's going to run randomly on those each time I do it so some of the performance numbers are not going to be quite as clear-cut as I would like so I'll try and explain it if I see that happening so I run that and it's taken that a tenth of a second to run and if I make that number a lot bigger so I make it an order of magnitude bigger this for Loop it now takes a bit longer and if I make it longer again we'll see depending on which style of core it's gone on I think it's yeah it's roughly gone on the two slower cores it's it's roughly as I make the loop Run 10 times longer the program is taking roughly 10 times longer to run so that's what we'd expect soon to interpreter now let's get rid of a couple of those zeros and what I can do instead is use a different implementation of python and this is one thing that we often confuse we say python when we mean the language and there's python the language and there's python the implementation I can use something called Pi pipe and I'm it has a jit let me turn the jit off and hope that I've not made it run too slow so we can see that running and I've still got a relatively large number so it's about the same speed as the normal version of python now we'll turn the jit on and it now runs in well that's a tenth of a second it has run two orders of magnitude faster and in fact I think we'll see if I've got this right let's add another couple of zeros there something very big it actually doesn't really matter how big I make that loop it's been able to observe it at runtime realize it can just optimize the whole thing away and that's the power of just in time compilation it's looked at my runtime values and made the thing very fast it's particularly effective on this sort of numeric code but it will often work well on things like strings and so on and of course there are some cases where it won't work it isn't as we said earlier magic but it is very effective in many cases so as you scale up um is that is there still a benefit to using it you know when you've got a million lines of code for instance good question um very dependent on your program in fact in some sense actually the scale of it is less important than the nature of your program there's a fundamental assumption here that programs tend to do the same thing over and over again so in this case Pi Pi is looking for loops and it's what's called a tracing just-in-time compiler so it looks at what the loop is doing and optimizes that there's also method base which just look at a function don't need to worry about the difference too much if your Loop or your function does the same thing repeatedly with only minor variations this will be very effective if you have a program which uh every time it goes around the loop does something completely different or every time you call the method then this will be less effective in some cases then it will even slow things down because the program will never appear to stabilize and that's really what your expecting programs do is that they typically when they start up they do some initialization that's all a bit random and then they tend to hit some sort of main part where they do the same thing over and over and over again and that's where jit compilation really comes to the fore and our process is doing a bit of that anyway you yeah so I think of our modern processes it's basically a just in time compiler I write machine code and okay I like the textual form and it looks like armor x86 or whatever that's not what the processor eventually executes it turns that into some other representation it does all sorts of clever optimizations if you ever want to see things uh like the store um the way a processor optimizes program reorders it like the reorder buffer they're scary at how clever they are and that's why they've got a lot faster even though the gigahertz part hasn't changed too much and there's been a little bit of knowledge transfer between the the processor jit world and the programming language world is this a new thing or how long have these kind of just in time components been around them they have been around in one form another for a while but they really Trace their modern lineage back to the 1980s in a language called self which has been largely forgotten a really interesting language that had a just-in-time compiler via a long sequence of events that ended up going to a company called sun and is then formed the basis and literally some of the code is still there in the Java virtual machine so the Java jit traces itself back to self V8 the JavaScript VM and chrome also traces this lineage back to the Java virtual machine hotspot and back to self so really that's been those have been the um the big movers and now you've got systems like Pi Pi and V8 and spider monkey that have um modernized the concept or spread it to more languages would probably be a better way of putting it and is this you know obviously it traces it through it's quite a long way back but is it only really being used now because machines have got that much faster yeah I think there's there's an element of that um because for you when I was a kid you could buy a new computer every 18 months and it was twice as fast and the the death of single core performance is a little exaggerated partly because the processes are now doing just-in-time compilation sorts of things but yeah we definitely are looking increasingly to programming languages to work faster and for many languages particularly but not only those that are Dynamic type like python or Java this is really the only effective technique and we and that's why you've seen increasing numbers of them being released for more and more languages despite the fact that they're really complicated and expensive to create you know these are not the sort of things you can knock out in an afternoon they take some big teams many years to create in most cases train a network to undo this process that's the idea and if we can do that then we can start with random noise a bit like I can and we can just iterate this process die a b c and d and I tell you that die a has a value of four how much did you learn about the data

Channel: Computerphile

Views: 260,808

Keywords: computers, computerphile, computer, science

Id: d7KHAVaX_Rs

Length: 10min 40sec (640 seconds)

Published: Mon Nov 28 2022