1. Maximum Likelihood Estimation Basics

to start off with maximum-likelihood we first need to think about our parameter theta so this is a parameter that's unknown and we're trying to figure stuff out about it so even though fada is unknown what we could do is go out into the world and collect some data so we would collect x1 through xn so n measurements and if they're in a pendant and identically distributed then we'd have a random sample so we have x1 through xn that's a random sample of size n from our PMF or PDF f of X given theta all right so now we've collected our data we need to think of that data as constants there's no more variability anymore now we're in math stat world where our data are collected and they're constant so we have this independent identically distributed set of data and now we're trying to figure out what is a good value for us to guess that theta would be a likely a good answer would be well what is the most likely value that theta could be given the data that we have observed so we need to define something called a likelihood function and this is just a function of our parameter given to the data that we've observed so we're holding constant X 1 through xn and now we're seeing if we plug in different values of theta what likelihood do we get out so maybe we could plot it and our likelihood function might look something like that so if we had a value of theta like here and our likelihood would be this so maybe that looks pretty high in comparison to this value of theta which gives a pretty low likelihood so the higher the likelihood is the more likely that parameter value is given the data that we have observed all right so now we have an idea of what that likelihood function is how do we actually get that likelihood function so our likelihood function is just the joint pmf or PDF of our random sample so we have x1 through xn given theta but we know that they're independent so we can split this up into the products of univariate okay so lots of time we can just write it like shift so it's just the product of each of these univariate PMF or PDFs okay so again we in math debt world are thinking of the data as fixed and we're looking at different very different values of theta but to get our joint PMF for PDF we write down the product of univariate PDF and of course these PDF are given theta plug in different values of X as a function of X with the value of theta whole how the constant okay so here's our likelihood function again the function of theta given the data held constant so if we look back to this little picture of our likelihood function a logical thing to do would be follow this uphill and try to find the most likely value of theta so we're wondering what is this value here of theta that maximizes the likelihood function so that value that Maximizer is called the maximum likelihood estimator and we write em le for short all right so our maximum likelihood estimator again is the value of theta that maximizes the likelihood function all right so if we have experiences calculus and if this likelihood function is pretty nice then we can just go ahead and use calculus to know take derivatives and find the Maximizer for the stuff that we're doing in this class it will be usually pretty much that simple as just using calculus there are cases where it is harder to find the Maximizer but will either save that for later or not talk about it at all okay so we're looking for this maximum likelihood estimator and that's again the Maximizer of our likelihood function I'd remember that a log is a monotone function so lots of times actually we'll be talking about the log likelihood and that's just the log of the likelihood function and in this class whenever we're talking about logs we always mean natural logs so we're using this log likelihood because it's a monotone function so if we're maximizing the log likelihood that's the same thing as maximizing the likelihood and it's just a lot easier to use log because remember if we're taking the log of a product that's just the sum of the log so it's a lot easier to take the derivative of a sum than is to take the derivative of a product like this so most of the times we want to write down the likelihood then take the log and then we can take our first derivative set that equal to zero find that Maximizer and then take the second derivative to make sure that we actually have a Maximizer not a minimizer

Channel: Professor Knudson

Views: 141,651

Rating: 4.7550111 out of 5

Keywords: statistics, mathematical statistics, statistical theory, maximum likelihood, likelihood, MLE, maximum likelihood estimation, point estimation

Id: 93fPFOf547Q

Length: 6min 32sec (392 seconds)

Published: Thu Aug 10 2017