How Memory Usage Slows Down Your Programs

today we're going to talk about memory and how your use of memory can impact the speed the performance of your programs in c c plus plus pretty much any language out there [Music] hey welcome back everybody today i want to talk about performance specifically making your programs run faster and specifically i wanted to talk about memory and how memory and your use of memory can impact performance because most programmers don't typically think about this they don't typically think about how memory access times can actually impact your program performance we just think about the steps the actions the instructions that our programs are going to take and we get a rough idea maybe of how those work and maybe we think about algorithmic complexity of our different algorithms but we don't usually think about memory access but that stops today because today we're going to dig into this topic just a little bit but before we do a big thanks to all of you patrons that support this channel on patreon people like jorge vieira abby payton jones and dave mcdonald who helped me keep the lights on and the cameras rolling if you're new to the channel patreon is also how you can get access to all the source code from all my videos as well as access to my virtual office hours and that may be relevant because we are going to be making source code today so now with that said let's get back to memory so when you saw the title when you heard me introduce this most people when they think about memory impacting program performance you think about my program is using too much memory and that too much memory usage is impacting how it performs and maybe impacting other programs as well and this can happen maybe we'll talk about that in a future video but today i want to specifically just talk about how we access memory or the patterns with which we access memory and how that can impact performance in ways that maybe you haven't thought about so let's look at what i'm talking about in an example so this is a really simple program we're going to start out here now for this example i'm going to need a substantial amount of memory you know something fairly big it'll work with smaller amounts of memory too but the results won't be quite as dramatic so let's come up here first and let's make a couple of sizes what we're going to do is we're going to make a two-dimensional array um we'll just call it 2d array we talked about matrices in two-dimensional arrays in a previous video check that out if you haven't seen that or if any of this confuses you but we're gonna have a certain number of rows and a certain number of columns and let's just i'm gonna define that up here rows let's say we've got a hundred thousand rows and let's say struggle with writing define today um number of columns let's make 10 000. okay so if i'm doing my math correct uh 100 000 rows 10 000 columns that's gonna be what about four gigabytes of memory okay so that's a pretty big data structure notice that i made it global i didn't put it on the stack because i can guarantee you the default stack size on my machine is not four gigs so if i had made it local i would basically be getting a seg fault when i run it so now down in main let's actually put something in here so what i'm going to do is just i'm going to go through all the rows and columns and let's just say okay so rows and then let's also go through n j equals zero j is less than calls okay so we're going to go through each row and column basically go through each individual entry in this 2d array and then what we're going to do is set each of those to some random number okay i know rand isn't the best random number generator out there but it doesn't really matter for the example that we're looking at so i'm just gonna go with it and note that i'm only doing this because i need my program to actually be doing something otherwise the compiler might try to get smart and start optimizing things out and that would defeat the purpose of trying to measure how long things are taking okay so now we've got all these random numbers now what i'm going to do is i'm going to declare a n64 t a sum variable we'll start out as zero and then i'm basically going to do the same thing i did up here just going to copy and paste it down here and instead what we're going to do is we're going to just sum these up so we're going to say sum plus equals 2d array and we'll just remove the rand okay so what i'm doing basically is i've generated four gigabytes worth of random numbers then i'm going to go through and just sum them all up okay and then down here at the bottom then we'll just print it out we'll say sum equals lld the two l's are because this is a long long this is a very big number and then we're done okay so simple program example i just this is basically what i want to play with okay and i've called this program by row.c because i'm basically going through the array by rows first and then by column so we're going row by row and then hitting you know so we go down each row and then hit all the columns in that row and then we go to the next row now let's look i want to look at the alternative to this so if we just copy this whole thing i also have a file over here called by call and then what i'm going to do is just we're going to switch this up a little bit and instead of going by rows we're going to go by columns first and otherwise everything is going to stay the same okay so the summation is going to get the same result we're summing up the same random numbers we're just changing the order by which we actually go and so instead of going row by row we're going columns first and then rows so no big deal we're still doing exactly the same thing more or less we should still get the same result we'll make sure we run it to make sure but now let's take a look at what happens when we actually run them so again as usual i have a make file simple make file nothing fancy here except there is one thing that's a little different so i did make this timing target right here and all that's going to do is it's going to run these two programs that i'm going to build and then we're just gonna we're gonna use the time command to basically determine how long they take to run rather than just saying uh you know trying to ballpark it or use a stopwatch okay so let's come down here make sure everything compiles okay it looks like it does no problem and then let's run it with this timing command and let's just see it's going to take a little while and we'll wait for the first one to finish it's going to finish hopefully pretty soon okay so it finished took a few seconds not too bad now that's actually pretty fast for going through four gigabytes worth of integers and then the second one we're kind of waiting for it it seems like it's taking a little bit longer than the first one but we'll see when it happens to finish come on buddy you can do it um okay any time now okay so it did finish and it did take longer it took more than twice as long so what's going on well it really comes down to how we are accessing memory so not all memory accesses are created equal and you may or may not know this especially if you're a beginner but your computer actually has a lot of different kinds of memory in it it has registers which are a few fast and expensive all the way to disk or your solid state hard drive which is big and slow has tons of memory but it takes a long time to access and so when i access this big four gigabyte array each element in the array could be stored in cash in which case it's gonna be really fast or it could maybe not be in cash but in ram which is gonna be slower but maybe it's okay it is gonna be slower though and then maybe in some extreme cases some of that memory may have been pushed out swapped out to disk as we say and then in that case it's going to be really slow to bring in and access and the point is is that over here when i go through row by row what i'm doing is i'm accessing items in memory that are close to each other so these items are likely to fit into the same cache line meaning that if the first element on the line wasn't in the cache well so that was a little slow but then the next few are going to be in cache and that's going to be really really fast on the other hand when we go column by column every element we access is going to be far away from the previous one so the chance that any of our accesses are going to be loaded in the cache at any given time is actually really low and so our memory accesses are going to be slower significantly slower now this concept is called locality we talk about programs with good spatial locality meaning that they tend to access memory that is close to each other so each individual access tends to access areas in memory that are close to the previous accesses there's also temporal locality which deals with like accessing the same pieces of memory close together in time so there's there's close together in space and that's close together in time but both are basically just looking at how we access things in a way that tries to encourage elements in memory to be in cache to be in faster memory so we get higher performance now naturally there are going to be times where you just don't care and you have to access memory in a random way it just makes the most sense for your program and that's cool just know that if it's a lot of memory things are gonna be a bit slow and there's nothing wrong with that locality is just one factor to consider as a programmer and my point wasn't to tell you you always need to access things in a certain way but rather to just point this out so that you can keep it in mind as you're developing systems especially systems that have performance constraints that have to run really fast to make your programs run better if you do have control over the pattern with which you go through these arrays because even if you think i'm doing the exact same thing it can have an impact on how well things run now i can hear some of you saying what about compiler optimization and i'm hearing others of you out there saying well why what about c plus plus if you just use valerie or vector like any other respectable c plus programmer all of your problems would go away and of course that's not true that wouldn't solve all these issues and i do plan to address both of these issues in a video shortly but today this is all the time i have so i hope it's helpful we'll address both those topics in a future video very soon subscribe to the channel if you don't want to miss those videos like the video if you liked it and want to let the world know more about it of course share with your friends support the channel on patreon if you are so inclined and of course check out my courses if you want to spend a little more time together and i will see you next week

Channel: Jacob Sorber

Views: 4,613

Rating: 4.9816933 out of 5

Keywords: memory slows down programs, memory usage, memory performance, memory locality, memory program performance, memory slow programs, poor locality in programs, cache performance slows programs, memory, memory use in programs, memory usage makes programs slow, memory locality slow programs, slow program, why are my programs slow, Why is my program slow

Id: LRLlEzkCL4I

Length: 9min 17sec (557 seconds)

Published: Tue Oct 12 2021