The following content is
provided under a Creative Commons license. Your support will help MIT
OpenCourseWare continue to offer high quality educational
resources for free. To make a donation or view
additional materials from hundreds of MIT courses, visit
MIT OpenCourseWare at ocw.mit.edu. PROFESSOR: So last time
we started talking about random processes. A random process is a random
experiment that evolves over time. And conceptually, it's important
to realize that it's a single probabilistic
experiment that has many stages. Actually, it has an infinite
number of stages. And we discussed the simplest
random process there is, the Bernoulli process, which is
nothing but the sequence of Bernoulli trials-- an infinite sequence of
Bernoulli trials. For example, flipping a
coin over and over. Once we understand what's going
on with that process, then what we want is to move
into a continuous time version of the Bernoulli process. And this is what we will call
the Poisson process. And for the Poisson process,
we're going to do exactly the same things that we did for
the Bernoulli process. That is, talk about the number
of arrivals during a given time period, and talk also
about the time between consecutive arrivals, and
for the distribution of inter-arrival times. So let's start with a quick
review of what we discussed last time. First, a note about language. If you think of coin tosses,
we then talk about heads and tails. If you think of these as a
sequence of trials, you can talk about successes
and failures. The language that we will be
using will be more the language of arrivals. That is, if in a given slot you
have a success, you say that something arrived. If you have a failure,
nothing arrived. And that language is a little
more convenient and more natural, especially when we talk
about continuous time-- to talk about arrivals
instead of successes. But in any case, for the
Bernoulli process let's keep, for a little bit, the language
of successes. Whereas working in discrete
time, we have time slots. During each time slot,
we have an independent Bernoulli trial. There is probability p
of having a success. Different slots are independent
of each other. And this probability p is the
same for any given time slot. So for this process we will
discuss the one random variable of interest, which
is the following. If we have n time slots,
or n trials, how many arrivals will there be? Or how many successes
will there be? Well, this is just given
by the binomial PMF. Number of successes in n trials
is a random variable that has a binomial PMF, and
we know what this is. Then we talked about
inter-arrival times. The time until the first
arrival happens has a geometric distribution. And we have seen that
from some time ago. Now if you start thinking
about the time until k arrivals happen, and we denote
that by Yk, this is the time until the first arrival
happens. And then after the first arrival
happens, you have to wait some time until
the second arrival happens, and so on. And then the time from the
(k -1)th arrival, until arrival number k. The important thing to realize
here is that because the process has a memorylessness
property, once the first arrival comes, it's as if we're
starting from scratch and we will be flipping
our coins until the next arrival comes. So the time it will take until
the next arrival comes will also be a geometric
random variable. And because different slots
are independent, whatever happens after the first arrival
is independent from whatever happened before. So T1 and T2 will be independent
random variables. And similarly, all
the way up to Tk. So the time until the k-th
arrival is a sum of independent geometric random
variables, with the same parameter p. And we saw last time that we
can find the probability distribution of Yk. The probability that Yk takes
a value of t is equal to-- there's this combinatorial
factor here, and then you get p to the k, (1-p) to the (t-k),
and this formula is true for t equal to
k, k+1, and so on. And this distribution
has a name. It's called the Pascal PMF. So this is all there
is to know about the Bernoulli process. One important comment is to
realize what exactly this memorylessness property
is saying. So I discussed it a little
bit last time. Let me reiterate it. So we have a Bernoulli process,
which is a sequence of Bernoulli trials. And these are (0,1) random
variables that keep going on forever. So someone is watching this
movie of Bernoulli trials B_t. And at some point, they say
they think, or something interesting has happened,
why don't you come in and start watching? So at some time t, they
tell you to come in and start watching. So what you will see once
you come in will be this future trials. So actually what you will see
is a random process, whose first random variable is going
to be the first one that you see, B_(t +1). The second one is going
to be this, and so on. So this is the process that's
seen by the person who's asked to come in and start watching
at that time. And the claim is that this
process is itself a Bernoulli process, provided that the
person who calls you into the room does not look
into the future. The person who calls you into
the room decides to call you in only on the basis of what
they have seen so far. So for example, who calls you
into the room might have a rule that says, as soon as I see
a sequence of 3 heads, I ask the other person
to come in. So if they use that particular
rule, it means that when you're called in, the previous
3 were heads. But this doesn't give you any
information about the future. And so the future ones
will be just independent Bernoulli trials. If on the other hand, the person
who calls you in has seen the movie before and they
use a rule, such as, for example, I call you in just
before 3 heads show up for the first time. So the person calls you in based
on knowledge that these two would be three heads. If they have such foresight-- if they can look into
the future-- then X1, X2, X3, they're certain
to be three heads, so they do not correspond
to random independent Bernoulli trials. So to rephrase this, the
process is memoryless. It does not matter what has
happened in the past. And that's true even if you are
called into the room and start watching at a random time,
as long as that random time is determined in a causal
way on the basis of what has happened so far. So you are called into the room
in a causal manner, just based on what's happened
so far. What you're going to see
starting from that time will still be a sequence of
independent Bernoulli trials. And this is the argument that we
used here, essentially, to argue that this T2 is an
independent random variable from T1. So a person is watching the
movie, sees the first success. And on the basis of what
they have seen-- they have just seen the
first success-- they ask you to come in. You come in. What you're going to see is a
sequence of Bernoulli trials. And you wait this long until
the next success comes in. What you see is a Bernoulli
process, as if the process was just starting right now. And that convinces us that this
should be a geometric random variable of the same
kind as this one, as independent from what
happened before. All right. So this is pretty much all there
is to know about the Bernoulli process. Plus the two things that we
did at the end of the last lecture where we merge two
independent Bernoulli processes, we get a
Bernoulli process. If we have a Bernoulli process
and we split it by flipping a coin and sending things one way
or the other, then we get two separate Bernoulli
processes. And we see that all of
these carry over to the continuous time. And our task for today is
basically to work these continuous time variations. So the Poisson process is a
continuous time version of the Bernoulli process. Here's the motivation
for considering it a Bernoulli process. So you have that person whose
job is to sit outside the door of a bank. And they have this long sheet,
and for every one second slot, they mark an X if a person
came in, or they mark something else if no one came
in during that slot. Now the bank manager is a really
scientifically trained person and wants very
accurate results. So they tell you, don't use
one second slots, use milliseconds slots. So you have all those slots
and you keep filling if someone arrived or not
during that slot. Well then you come
up with an idea. Why use millisecond slots and
keep putting crosses or zero's into each slot? It's much simpler if I just
record the exact times when people came in. So time is continuous. I don't keep doing something
at every time slot. But instead of the time axis,
I mark the times at which customers arrive. So there's no real
need for slots. The only information that you
want is when did we have arrivals of people. And we want to now model a
process of this kind happening in continuous time, that has the
same flavor, however, as the Bernoulli process. So that's the model we
want to develop. OK. So what are the properties
that we're going to have? First, we're going to assume
that intervals over the same length behave probabilistically
in an identical fashion. So what does that mean? Think of an interval of
some given length. During the interval of that
length, there's going to be a random number of arrivals. And that random number of
arrivals is going to have a probability distribution. So that probability
distribution-- let's denote it by
this notation. We fix t, we fix the duration. So this is fixed. And we look at the
different k's. The probability of having 0
arrivals, the probability of 1 arrival, the probability of
2 arrivals, and so on. So this thing is essentially
a PMF. So it should have the property
that the sum over all k's of this P_(k, tau) should
be equal to 1. Now, hidden inside this notation
is an assumption of time homogeneity. That is, this probability
distribution for the number of arrivals only depends on the
length of the interval, but not the exact location of the
interval on the time axis. That is, if I take an interval
of length tau, and I ask about the number of arrivals
in this interval. And I take another interval of
length tau, and I ask about the number of arrivals
during that interval. Number of arrivals here, and
number of arrivals there have the same probability
distribution, which is denoted this way. So the statistical behavior of
arrivals here is the same as the statistical behavioral
of arrivals there. What's the relation with
the Bernoulli process? It's very much like
the assumption-- the Bernoulli process-- that in different slots,
we have the same probability of success. Every slot looks
probabilistically as any other slot. So similarly here, any interval
of length tau looks probabilistically as any other
interval of length tau. And the number of arrivals
during that interval is a random variable described
by these probabilities. Number of arrivals here is a
random variable described by these same probabilities. So that's our first
assumption. Then what else? In the Bernoulli process we
had the assumption that different time slots were
independent of each other. Here we do not have time slots,
but we can still think in a similar way and impose the
following assumption, that these joint time intervals are
statistically independent. What does that mean? Does a random number of arrivals
during this interval, and the random number of
arrivals during this interval, and the random number of arrivals during this interval-- so these are three different
random variables-- these three random variables are
independent of each other. How many arrivals we got here
is independent from how many arrivals we got there. So this is similar to saying
that different time slots were independent. That's what we did
in discrete time. The continuous time analog is
this independence assumption. So for example, in particular,
number of arrivals here is independent from the number
of arrivals there. So these are two basic
assumptions about the process. Now in order to write down a
formula, eventually, about this probability
distribution-- which is our next objective, we
would like to say something specific about this
distribution of number of arrivals-- we need to add a little more
structure into the problem. And we're going to make the
following assumption. If we look at the time interval
of length delta-- and delta now is supposed to
be a small number, so a picture like this-- during a very small time
interval, there is a probability that we get exactly
one arrival, which is lambda times delta. Delta is the length of the
interval and lambda is a proportionality factor, which
is sort of the intensity of the arrival process. Bigger lambda means that a
little interval is more likely to get an arrival. So there's a probability lambda times delta of 1 arrival. The remaining probability
goes to 0 arrivals. And when delta is small, the
probability of 2 arrivals can be approximated by 0. So this is a description
of what happens during a small, tiny slot. Now this is something that's
supposed to be true in some limiting sense, when delta
is very small. So the exact version of this
statement would be that this is an equality, plus order
of delta squared terms. So this is an approximate
equality. And what approximation means is
that in the limit of small deltas, the dominant terms-- the constant and the first order
term are given by this. Now when delta is very small,
second order terms in delta do not matter. They are small compared
to first order terms. So we ignore this. So you can either think in terms
of an exact relation, which is the probabilities are
given by this, plus delta squared terms. Or if you want to be a little
more loose, you just write here, as an approximate
equality. And the understanding is that
this equality holds-- approximately becomes more
and more correct as delta goes to 0. So another version of that
statement would be that if you take the limit as delta goes to
0, of p, the probability of having 1 arrival in an interval
of length delta, divided by delta, this
is equal to lambda. So that would be one version of
an exact statement of what we are assuming here. So this lambda, we call it the
arrival rate, or the intensity of the process. And clearly, if you double
lambda, then a little interval is likely -- you expect to get -- the probability of obtaining
an arrival during that interval has doubled. So in some sense we have twice
as intense arrival process. If you look at the number
of arrivals during delta interval, what is the expected
value of that random variable? Well with probability lambda
delta we get 1 arrival. And with the remaining probability, we get 0 arrivals. So it's just lambda
times delta. So expected number of arrivals
during a little interval is lambda times delta. So expected number of arrivals
is proportional to lambda, and that's again why we call lambda
the arrival rate. If you send delta to the
denominator in this equality, it tells you that lambda is
the expected number of arrivals per unit time. So the arrival rate is expected
number of arrivals per unit time. And again, that justifies why
we call lambda the intensity of this process. All right. So where are we now? For the Bernoulli process, the
number of arrivals during a given interval of length n had
the PMF that we knew it was the binomial PMF. What is the formula for the
corresponding PMF for the continuous time process? Somehow we would like to use
our assumptions and come up with the formula for
this quantity. So this tells us about the
distribution of number of arrivals during an interval
of some general length. We have made assumptions about
the number of arrivals during an interval of small length. An interval of big length is
composed of many intervals of small length, so maybe this
is the way to go. Take a big interval, and split
it into many intervals of small length. So we have here our time axis. And we have an interval
of length tau. And I'm going to split it into
lots of little intervals of length delta. So how many intervals are
we going to have? The number of intervals is going
to be the total time, divided by delta. Now what happens during each one
of these little intervals? As long as the intervals are
small, what you have is that during an interval, you're
going to have either 0 or 1 arrival. The probability of more than
1 arrival during a little interval is negligible. So with this picture, you have
essentially a Bernoulli process that consists
of so many trials. And during each one of those
trials, we have a probability of success, which is
lambda times delta. Different little intervals
here are independent of each other. That's one of our assumptions,
that these joint time intervals are independent. So approximately, what we have
is a Bernoulli process. We have independence. We have the number of
slots of interest. And during each one of the
slots we have a certain probability of success. So if we think of this as
another good approximation of the Poisson process-- with the approximation becoming
more and more accurate as delta goes to 0 -- what we should do would be to
take the formula for the PMF of number of arrivals in a
Bernoulli process, and then take the limit as
delta goes to 0. So in the Bernoulli process, the
probability of k arrivals is n choose k, and then
you have p to the k. Now in our case, we have here
lambda times delta, delta is tau over n. Delta is tau over n, so p is
lambda times tau divided by n. So here's our p -- Lambda tau over n -- to the power k, and then times
one minus this-- this is our one minus p-- to the power n-k. So this is the exact formula
for the Bernoulli process. For the Poisson process, what we
do is we take that formula and we let delta go to 0. As delta goes to 0, n
goes to infinity. So that's the limit
that we're taking. On the other hand, this
expression lambda times tau-- lambda times tau, what
is it going to be? Lambda times tau is equal
to n times p. n times p, is that
what I want? No, let's see. Lambda tau is np. Yeah. So lambda tau is np. All right. So we have this relation,
lambda tau equals np. These two numbers being equal
kind of makes sense. np is the expected number of successes
you're going to get in the Bernoulli process. Lambda tau-- since lambda is the arrival rate
and you have a total time of tau, lambda tau you can think
of it as the number of expected arrivals in the
Bernoulli process. We're doing a Bernoulli
approximation to the Poisson process. We take the formula for the
Bernoulli, and now take the limit as n goes to infinity. Now lambda tau over n is equal
to p, so it's clear what this term is going to give us. This is just p to the power k. It will actually take a little
more work than that. Now I'm not going to do the
algebra, but I'm just telling you that one can take the limit
in this formula here, as n goes to infinity. And that will give you another
formula, the final formula for the Poisson PMF. One thing to notice is that here
you have something like 1 minus a constant over
n, to the power n. And you may recall from calculus
a formula of this kind, that this converges
to e to the minus c. If you remember that formula
from calculus, then you will expect that here, in the limit,
you are going to get something like an e to
the minus lambda tau. So indeed, we will
get such a term. There is some work that needs
to be done to find the limit of this expression, times
that expression. The algebra is not hard,
it's in the text. Let's not spend more
time doing this. But let me just give you
the formula of what comes at the end. And the formula that comes at
the end is of this form. So what matters here is not so
much the specific algebra that you will do to go from this
formula to that one. It's kind of straightforward. What's important is the idea
that the Poisson process, by definition, can be approximated
by a Bernoulli process in which we have a very
large number of slots-- n goes to infinity. Whereas we have a very small
probability of success during each time slot. So a large number of slots,
but tiny probability of success during each slot. And we take the limit
as the slots become smaller and smaller. So with this approximation
we end up with this particular formula. And this is the so-called
Poisson PMF. Now this function P here -- has two arguments. The important thing to realize
is that when you think of this as a PMF, you fix t to tau. And for a fixed tau,
now this is a PMF. As I said before, the sum over
k has to be equal to 1. So for a given tau, these
probabilities add up to 1. The formula is moderately messy,
but not too messy. One can work with it without
too much pain. And what's the mean and
variance of this PMF? Well what's the expected
number of arrivals? If you think of this Bernoulli
analogy, we know that the expected number of arrivals
in the Bernoulli process is n times p. In the approximation that
we're using in these procedure, n times p is the
same as lambda tau. And that's why we get lambda tau
to be the expected number of arrivals. Here I'm using t
instead of tau. The expected number of
arrivals is lambda t. So if you double the time,
you expect to get twice as many arrivals. If you double the arrival rate,
you expect to get twice as many arrivals. How about the formula
for the variance? The variance of the Bernoulli
process is np, times one minus p. What does this go
to in the limit? In the limit that we're taking,
as delta goes to zero, then p also goes to zero. The probability of success in
any given slot goes to zero. So this term becomes
insignificant. So this becomes n times p, which
is again lambda t, or lambda tau. So the variance, instead of
having this more complicated formula of the variance is the
Bernoulli process, here it gets simplified and
it's lambda t. So interestingly, the variance
in the Poisson process is exactly the same as the
expected value. So you can look at this as
just some interesting coincidence. So now we're going to take
this formula and see how to use it. First we're going to do
a completely trivial, straightforward example. So 15 years ago when that
example was made, email was coming at a rate of five
messages per hour. I wish that was the
case today. And now emails that are coming
in, let's say during the day-- the arrival rates of emails
are probably different in different times of the day. But if you fix a time slot,
let's say 1:00 to 2:00 in the afternoon, there's probably
a constant rate. And email arrivals are
reasonably well modeled by a Poisson process. Speaking of modeling, it's
not just email arrivals. Whenever arrivals happen in a
completely random way, without any additional structure, the
Poisson process is a good model of these arrivals. So the times at which car
accidents will happen, that's a Poisson processes. If you have a very, very weak
light source that's shooting out photons, just one at a time,
the times at which these photons will go out is
well modeled again by a Poisson process. So it's completely random. Or if you have a radioactive
material where one atom at a time changes at random times. So it's a very slow
radioactive decay. The time at which these alpha
particles, or whatever we get emitted, again is going
to be described by a Poisson process. So if you have arrivals, or
emissions, that happen at completely random times, and
once in a while you get an arrival or an event, then the
Poisson process is a very good model for these events. So back to emails. Get them at a rate of five
messages per day, per hour. In 30 minutes this
is half an hour. So what we have is that
lambda t, total number of arrivals is-- the expected number
of arrivals is-- lambda is five, t is one-half,
if we talk about hours. So lambda t is two to the 0.5. The probability of no new
messages is the probability of zero, in time interval of length
t, which, in our case, is one-half. And then we look back into the
formula from the previous slide, and the probability of
zero arrivals is lambda t to the power zero, divided by zero
factorial, and then an e to the lambda t. And you plug in the numbers
that we have. Lambda t to the zero
power is one. Zero factorial is one. So we're left with e
to the minus 2.5. And that number is 0.08. Similarly, you can ask for the
probability that you get exactly one message
in half an hour. And that would be-- the
probability of one message in one-half an hour-- is going to be lambda t to the
first power, divided by 1 factorial, e to the minus
lambda t, which-- as we now get the extra lambda t
factor-- is going to be 2.5, e to the minus 2.5. And the numerical
answer is 0.20. So this is how you use the PMF
formula for the Poisson distribution that we had
in the previous slide. All right. So this was all about
the distribution of the number of arrivals. What else did we do last time? Last time we also talked about
the time it takes until the k-th arrival. OK. So let's try to figure out
something about this particular distribution. We can derive the distribution
of the time of the k-th arrival by using the
exact same argument as we did last time. So now the time of the
k-th arrival is a continuous random variable. So it has a PDF. Since we are in continuous
time, arrivals can happen at any time. So Yk is a continuous
random variable. But now let's think of
a time interval of length little delta. And use our usual interpretation
of PDFs. The PDF of a random variable
evaluated at a certain time times delta, this is the
probability that the Yk falls in this little interval. So as I've said before, this
is the best way of thinking about PDFs. PDFs give you probabilities
of little intervals. So now let's try to calculate
this probability. For the k-th arrival to happen
inside this little interval, we need two things. We need an arrival to happen in
this interval, and we need k minus one arrivals to happen
during that interval. OK. You'll tell me, but it's
possible that we might have the k minus one arrival happen
here, and the k-th arrival to happen here. In principle, that's possible. But in the limit, when we take
delta very small, the probability of having two
arrivals in the same little slot is negligible. So assuming that no two arrivals
can happen in the same mini slot, then for the
k-th one to happen here, we must have k minus one during
this interval. Now because we have assumed that
these joint intervals are independent of each other,
this breaks down into the probability that we have exactly
k minus one arrivals, during the interval from zero to
t, times the probability of exactly one arrival during that
little interval, which is lambda delta. We do have a formula for this
from the previous slide, which is lambda t, to the k minus 1,
over k minus one factorial, times e to minus lambda t. And then lambda times delta. Did I miss something? Yeah, OK. All right. And now you cancel this
delta with that delta. And that gives us a formula for
the PDF of the time until the k-th arrival. This PDF, of course, depends
on the number k. The first arrival is going
to happen somewhere in this range of time. So this is the PDF
that it has. The second arrival, of course,
is going to happen later. And the PDF is this. So it's more likely to happen
around these times. The third arrival has this PDF,
so it's more likely to happen around those times. And if you were to take
k equal to 100, you might get a PDF-- it's extremely unlikely that
the k-th arrival happens in the beginning, and it might
happen somewhere down there, far into the future. So depending on which particular
arrival we're talking about, it has a
different probability distribution. The time of the 100th arrival,
of course, is expected to be a lot larger than the time
of the first arrival. Incidentally, the time of the
first arrival has a PDF whose form is quite simple. If you let k equal to one here,
this term disappears. That term becomes a one. You're left with just lambda,
e to the minus lambda. And you recognize it, it's the
exponential distribution. So the time until the first
arrival in a Poisson process is an exponential
distribution. What was the time of the
first arrival in the Bernoulli process? It was a geometric
distribution. Well, not coincidentally, these
two look quite a bit like the other. A geometric distribution
has this kind of shape. The exponential distribution
has that kind of shape. The geometric is just a discrete
version of the exponential. In the Bernoulli case, we
are in discrete time. We have a PMF for the
time of the first arrival, which is geometric. In the Poisson case, what we
get is the limit of the geometric as you let those
lines become closer and closer, which gives you the
exponential distribution. Now the Poisson process shares
all the memorylessness properties of the Bernoulli
process. And the way one can argue is
just in terms of this picture. Since the Poisson process is
the limit of Bernoulli processes, whatever qualitative
processes you have in the Bernoulli process
remain valid for the Poisson process. In particular we have this
memorylessness property. You let the Poisson process run
for some time, and then you start watching it. What ever happened in
the past has no bearing about the future. Starting from right now, what's
going to happen in the future is described again by a
Poisson process, in the sense that during every little slot of
length delta, there's going to be a probability of lambda
delta of having an arrival. And that probably lambda
delta is the same-- is always lambda delta-- no matter what happened in
the past of the process. And in particular, we could use
this argument to say that the time until the k-th arrival
is the time that it takes for the first
arrival to happen. OK, let me do it for
k equal to two. And then after the first arrival
happens, you wait a certain amount of time until
the second arrival happens. Now once the first arrival
happened, that's in the past. You start watching. From now on you have mini slots
of length delta, each one having a probability of
success lambda delta. It's as if we started the
Poisson process from scratch. So starting from that time,
the time until the next arrival is going to be again an
exponential distribution, which doesn't care about what
happened in the past, how long it took you for the
first arrival. So these two random variables
are going to be independent and exponential, with the
same parameter lambda. So among other things, what we
have done here is we have essentially derived the PDF of
the sum of k independent exponentials. The time of the k-th arrival
is the sum of k inter-arrival times. The inter-arrival times are all
independent of each other because of memorylessness. And they all have the same
exponential distribution. And by the way, this
gives you a way to simulate the Poisson process. If you wanted to simulate it
on your computer, you would have one option to break time
into tiny, tiny slots. And for every tiny slot, use
your random number generator to decide whether there
was an arrival or not. To get it very accurate,
you would have to use tiny, tiny slots. So that would be a lot
of computation. The more clever way of
simulating the Poisson process is you use your random number
generator to generate a sample from an exponential distribution
and call that your first arrival time. Then go back to the random
number generator, generate another independent sample,
again from the same exponential distribution. That's the time between the
first and the second arrival, and you keep going that way. So as a sort of a
quick summary, this is the big picture. This table doesn't tell
you anything new. But it's good to have it as a
reference, and to look at it, and to make sure you understand
what all the different boxes are. Basically the Bernoulli process
runs in discrete time. The Poisson process runs
in continuous time. There's an analogy of arrival
rates, p per trial, or intensity per unit time. We did derive, or sketched the
derivation for the PMF of the number of arrivals. And the Poisson distribution,
which is the distribution that we get, this Pk of t. Pk and t is the limit of the
binomial when we take the limit in this particular way,
as delta goes to zero, and n goes to infinity. The geometric becomes an
exponential in the limit. And the distribution of the
time of the k-th arrival-- we had a closed form formula
last time for the Bernoulli process. We got the closed form
formula this time for the Poisson process. And we actually used exactly the
same argument to get these two closed form formulas. All right. So now let's talk about adding
or merging Poisson processes. And there's two statements
that we can make here. One has to do with adding
Poisson random variables, just random variables. There's another statement about adding Poisson processes. And the second is a bigger
statement than the first. But this is a warm up. Let's work with the
first statement. So the claim is that the sum of
independent Poisson random variables is Poisson. OK. So suppose that we have a
Poisson process with rate-- just for simplicity-- lambda one. And I take the interval
from zero to two. And that take then the interval
from two until five. The number of arrivals during
this interval-- let's call it n from
zero to two-- is going to be a Poisson
random variable, with parameter, or with mean, two. The number of arrivals during
this interval is n from time two until five. This is again a Poisson random
variable with mean equal to three, because the arrival rate
is 1 and the duration of the interval is three. These two random variables
are independent. They obey the Poisson
distribution that we derived before. If you add them, what you get
is the number of arrivals during the interval
from zero to five. Now what kind of distribution
does this random variable have? Well this is the number of
arrivals over an interval of a certain length in a
Poisson process. Therefore, this is also Poisson
with mean five. Because for the Poisson process
we know that this number of arrivals is Poisson,
this is Poisson, but also the number of overall arrivals
is also Poisson. This establishes that the sum
of a Poisson plus a Poisson random variable gives
us another Poisson random variable. So adding Poisson random
variables gives us a Poisson random variable. But now I'm going to make a more
general statement that it's not just number
of arrivals during a fixed time interval-- it's not just numbers of
arrivals for given time intervals-- but rather if you take two
different Poisson processes and add them up, the process
itself is Poisson in the sense that this process is going to
satisfy all the assumptions of a Poisson process. So the story is that you have
a red bulb that flashes at random times at the rate
of lambda one. It's a Poisson process. You have an independent process
where a green bulb flashes at random times. And you happen to be color
blind, so you just see when something is flashing. So these two are assumed to be
independent Poisson processes. What can we say about the
process that you observe? So in the processes that you
observe, if you take a typical time interval of length little
delta, what can happen during that little time interval? The red process may have
something flashing. So red flashes. Or the red does not. And for the other bulb, the
green bulb, there's two possibilities. The green one flashes. And the other possibility is
that the green does not. OK. So there's four possibilities
about what can happen during a little slot. The probability that the red one
flashes and the green one flashes, what is this
probability? It's lambda one delta that the
first one flashes, and lambda two delta that the
second one does. I'm multiplying probabilities
here because I'm making the assumption that the two
processes are independent. OK. Now the probability that
the red one flashes is lambda one delta. But the green one doesn't is
one, minus lambda two delta. Here the probability would be
that the red one does not, times the probability that
the green one does. And then here we have the
probability that none of them flash, which is whatever
is left. But it's one minus lambda
one delta, times one minus lambda two delta. Now we're thinking about
delta as small. So think of the case where delta
goes to zero, but in a way that we keep the
first order terms. We keep the delta terms, but
we throw away the delta squared terms. Delta squared terms are much
smaller than the delta terms when delta becomes small. If we do that-- if we only keep the order
of delta terms-- this term effectively
disappears. This is delta squared. So we make it zero. So the probability of having
simultaneously a red and a green flash during a little
interval is negligible. What do we get here? Lambda delta times
one survives, but this times that doesn't. So we can throw that away. So the approximation that we
get is lambda one delta. Similarly here, this
goes away. We're left with a lambda
two delta. And this is whatever remains,
whatever is left. So what do we have? That there is a probability of
seeing a flash, either a red or a green, which is
lambda one delta, plus lambda two delta. So if we take a little interval
of length delta here, it's going to see an arrival
with probability approximately lambda one, plus lambda
two, delta. So every slot in this merged
process has an arrival probability with a rate which
is the sum of the rates of these two processes. So this is one part
of the definition of the Poisson process. There's a few more things that
one would need to verify. Namely, that intervals of the
same length have the same probability distribution and
that different slots are independent of each other. This can be argued by starting
from here because different intervals in this process are
independent from each other. Different intervals here are
independent from each other. It's not hard to argue that
different intervals in the merged process will also be
independent of each other. So the conclusion that comes
at the end is that this process is a Poisson process,
with a total rate which is equal to the sum of the rate
of the two processes. And now if I tell you that an
arrival happened in the merged process at a certain time,
how likely is it that it came from here? How likely is it? We go to this picture. Given that an arrival
occurred-- which is the event that this
or that happened-- what is the probability that
it came from the first process, the red one? Well it's the probability
of this divided by the probability of this,
times that. Given that this event occurred,
you want to find the conditional probability
of that sub event. So we're asking the question,
out of the total probability of these two, what
fraction of that probability is assigned here? And this is lambda one
delta, after we ignore the other terms. This is lambda two delta. So that fraction is going to be
lambda one, over lambda one plus lambda two. What does this tell you? If lambda one and lambda two are
equal, given that I saw an arrival here, it's equally
likely to be red or green. But if the reds have a much
higher arrival rate, when I see an arrival here, it's
more likely this number will be large. So it's more likely to have
come from the red process. OK so we'll continue with
this story and do some applications next time.
