Code overview - Thread Pool & Job System

hi my name's Trevor I'm the author of Lamanna engine a light 3d game engine featuring a software renderer which you just saw running today I'd like to talk about the engines threat Paul I run a custom thread pool that accepts a list of jobs and dispatches them to worker threads the job system supports the notion of job dependencies will examine the job system later but let's first look at the thread pool the broad idea is to have a number of worker threads contained in a system that has them fetched jobs offer job queue until that queue is empty and then wait or sleep we're sleeping implies relinquishing the threads back to the operating system in some fashion the emptying of the job queue signals a separate thread whose job it is to replenish the job queue after which it signals the worker threads that the queue is full and sleeps itself this is the classic producer-consumer model where the producer is a thread that fills the job queue in our case our master thread and the consumers are our worker threads contained in the thread pool the threads will need exclusive access to the job data when they read or write a job so we need the notion of a thread acquiring a lock on the job queue prior to accessing it and releasing that lock when done we want the time the thread holds the lock to be as short as possible to minimize contention we are operating in a Windows environment so we use critical sections as our exclusive locks and condition variables as our means of signalling between producer and consumer threads now let's look at our thread pool structure we set an upper limit on the number of threads in our pool but the actual number is discovered at runtime by querying the processor info information we have an array of thread handles condition variables are both the producer and consumer and a single critical section that get access to the job queue data let's look at the initialization of that thread pool data there we go we initialize our condition variables and critical sections do some initialization relating to the running of the job queue and loop over the worker threads and create them the parameters we pass in are an ID to identify the thread and a pointer to the thread pool structure the main worker thread function that is called is here the very dryly named main loop this is the function the worker threads executes as they are created so the worker threads into this wild loop which they will iterate over until broken out by this if statement which will be true only when the user has exited the program at this stage only note the following that we have these bookends for entering and leaving the critical section within which the worker thread can access the job queue data and within which it will both sleep and wake depending on whether these conditions are true the sleep is wrapped in a while loop to guard against spurious wakes when the worker threads are created and first into this loop the job queue will of course be empty so they will straightaway sleep to understand the remainder of the code we need to talk about the job system our job consists of the following elements we have a pointer to the function to be executed pointer to a structure containing parameters to that function a dependency counter indicating how many jobs this job must wait on before it can be run and a permission list indicating which jobs are dependent on this job's completion now jobs with dependencies are placed in a job pending queue jobs with no dependencies go in a job ready queue and when a worker thread finishes processing a job it adds it to the job complete queue the master thread will clear the job complete queue and decrement the dependency counters of the relevant jobs in the job pending queue so when a pending when a pending jobs dependency counter reaches zero it can be added to the jobs ready queue the queues are implemented as virtualized circular buffers note that jobs exist only explicitly in the jobs pending queue in the other queues thereof refer to VAR an ID that is simply an index into the pending of rain so let's return to our worker thread code we see a worker thread wakes in its critical section takes a job from the job ready queue exits the critical section and executes the job when it's done it re-enters the critical section adds the job to the finished job cube signals the producer thread that there is a finished job to process and either goes on to select another job or sleeps if there are no more jobs now let's look at the master thread code the process of adding jobs to the job system starts by setting the dependency counters which will be decremented as finished jobs are processed we then enter our job loop which will run until the jobs are completed we process the completed job queue decrementing the dependents encounters of pending jobs in the permission lists of each completed job we then fill the ready queue with any jobs that now have no dependencies and can be we then enter the critical section and add as many ready jobs as we can to the local job queue signal the thread pool consumers that there are available jobs and sleep the muster thread is signaled when there are finished jobs to process it wakes it in its critical section takes a copy of finished jobs exits the critical section and proceeds so using this job system I can characterize an entire game frame as a single large submission to the thread pool expressed as a network of interdependent jobs this certainly isn't to say that those dependencies are highly optimized in fact large sections of game logic still run serially due to the difficult nature of paralyzing such work but it is all still expressed using the same scheme rendering dynamic light map generation and kaliesha processes are all highly parallelized this source file contains all the jobs that make up the game frame it's a monolithic array of jobs with functions and parameters hard-coded broadly divided up into systems such as game logic collision rendering etc it's effective but a bit inflexible given it's all built out at compile time what I'd like going forward is worth dynamically creating jobs and adding them to the thread pool at runtime well that concludes our look at the thread pool and job system be sure to check out my code reviews of other game engine systems visit my website and follow me on Twitter thanks for watching

Channel: Trevor Nagel

Views: 1,045

Rating: 5 out of 5

Keywords:

Id: Df-6ws_EZno

Length: 8min 18sec (498 seconds)

Published: Wed Nov 28 2018