We have come back, we were discussing events.
So, we defined events as subsets of
omega, which are of interest. So, this is a very loose and informal. May, nobody would
think that, this is a mathematical definition. So, it is just some plain English kind of
a definition for now. So, we have been trying
to formalize this, what is this? That we actually mean by events. So, we have this
sample space omega, which contains all the possible elementary outcome of a random experiment.
And we are looking at subsets of this sample space. We said that all events are, in fact,
subsets of omega. But, not all subsets of omega are necessarily considered events. So,
this is not a point, you may fully appreciate this point. But, we are building towards
understanding this. So, one structure, we imposed for events yesterday was that, if
a was an interesting event, a complement should
be interesting. And similarly, if a and b are events, they
are interesting, then a union b should be interesting. And so this led to the structure
called algebra. So, an algebra is not is nothing
but, a collection of subsets of omega, which are closed and complementation and a finite .unions or also under finite intersections,
because of the Demorgan’s laws. So, we argued yesterday that, the structure of an algebra,
sometimes falls little bit short of being able to
describe a events of a day to day interest. So, we gave an example in terms of this, tossing
a coin, until you get a head. And we looked at the event; that is even number of
tosses. And that was not easy to describe in
terms of the algebra axes. So, it turns out, that to do an interesting probability theory,
to build a rich theory, you need a little more
than the structure of an algebra. So, all of
probability theory works with slightly stronger structure known as sigma algebra.
So, sigma algebra is nothing but, a collection of subsets again of omega, which are not
just closed in a finite unions. But closed under countable infinite unions, not necessarily
closed under orbiterly infinite unions, but countable infinite unions. So, that is what,
brings us to the concept of a sigma algebra. So, let me write down the definition of a
sigma algebra. A collection F of subsets of omega, it is called a sigma algebra is null
set is an F. 2, F A is an F. Then, A complement
is an F. Finally, so these two are same as in
an algebra. The last axiom of sigma algebra is different from axiom of the algebra.
. If A 1, A 2, dot, dot, dot is a countable
collection of subsets in F, then union A i in F. So,
that is the definition of the sigma algebra. So, this is scripted F, as I said, we denote
collection of subsets, collections of sets using scripted letters. So, we use scripted
F for a sigma algebra and these two are the same.
So, if the null set should be in F always. If A .is in F, then A is a complement should be
in F. And the only difference with an algebra is
that, if you have given any countable collection of sub sets in omega. The union of those
subsets, countable union of those subsets, must be in F.
So, there is one little point about say in a notational sense. So, this union, so if
you look at the right up, I had set theory bit. So,
this is really interpreted as union i belongs to n.
So, they had a little remark in the right up, we had uploaded on set theory. So, this
is simply. So, this is not like, you are not
unioning A 1, then A 2, then A 3 and so on. This
should be interpreted as a set of all elements contained in at least in one of the A i’s.
This should not be interpreted as some limit of some finite union or any such thing.
There is no such notion. So, this is better interpreted like this, it is union overall
A i’s; i belong to n, which means, this is the collection
of all little omegas contained in at least in
one of the A i’s. So, that is just an aside. So, these three constitute the axioms of the
sigma algebra. So, it turns out that, this structure is enough.
You need, as I mentioned that, the structure of an algebra falls a little bit short of
what we actually we need to build a very interesting theory of probability. It turns out as soon
as you impose closer under the countablely infinite unions. It is enough to develop a
very rich theory of probability. So, all of probability theory works with a sample space
and the sigma algebra of subsets define on the samples space.
So, you can also show by the ways. So, the exercise, you can also show that, if A i.
So, this is also are subsets in F. Then, you can
show that intersection also an F, using Demorgan’s law again, fine this is clear.
So, you apply this and that together, you will
get the fact that the sigma algebra is closed under countable intersections as well.
Yes. .
So, one thing, I want to make very clear here is that, see F naught both the algebra and
the sigma algebra, they I am not saying for example, that an algebra should only contain
a finitely immediate subsets. That is not, I am saying, it can have any number of sub
sets. But, it should be closed under finite unions.
Similarly, a sigma algebra can contain any .number of subsets. It can even be an unaccountability
infinity of subsets. But, it should be closed under countable unions. It can have
an even uncountable subsets in it. So, this F may have an uncountable infinity
of subsets of omega in it. But, it should be
closed under countable unions. That is all, we are imposing. I am not saying for example,
that the cardinality of the F is countable or any such thing. No, It should be closed
under countable unions, is that clear, everybody.
So, this is very easy to show, this is a very simple exercise.
So, one thing, you can show also is that, a sigma algebra. So, this is 1, exercise number
2, a sigma algebra is also an algebra. So, what we are saying is that, the sigma algebra
is strictly stronger structure than a...
. We have not said that, strictly stronger structure.
I am saying now that, every a sigma algebra is an algebra, which means that, if
you have closure under countably infinite unions, you will have closure finite unions,
why? So, let say, you can take A i’s beyond A n, you can take the all A i’s as null
sets. And you will have the closure under finite
unions. Just take after A n plus 1 onwards, you take as null sets. Then, you will get
closure under finite unions. So, this is very easy to show, this is very
simple exercise, you will show it. What turns out is, it is not very entirely trivial to
show that, there are algebras, which are not sigma
algebras. So, we will see an example in the first home work. But, the converse is not
true. The converse to this is not true, not every algebra necessarily a sigma algebra.
Every sigma algebra is necessarily in algebra. So, the converse is not true
and so in order to show that not every algebra is sigma
algebra; you just need to produce an example. We will see one such example in the home
work. Slightly conterminal, but you will see in the home work, you do the home work;
you will see is this everybody with me, any questions? So, another terminology subsets
in F are called F measurable sets. It is just another terminology.
So, omega is a sample space and you are collecting subsets of omega and you make a
sigma algebra F. And depending on what that sigma algebra is, elements of that F are .called F measurable sets. This is a another
terminology. So, omega is a sample space and your collecting subsets of omega and you make
a sigma algebra F. And depending on what that sigma algebra is elements of that
F are called F measurable sets. And there are subsets of omega, which may
not be in F. Those are not F measurable sets, those are just some subsets of omega. So,
there are some examples. So, if you have some trivial examples of sigma algebra, we can
give. Some non trivial examples, we will see later. So, the most trivial sigma algebra
in some in any sample space omega is that. So, it
is a collection of subsets of omega only containing null the set and the sample space
itself. This is a sigma algebra. It is a very, very
interval example. It is of no use in more situations, but it is a sigma algebra. This
is one such example. So, I am just trying to say
that, you can build many sigma algebra for a given sample space. And which sigma
algebra, you want to work with, again depends on, what you are interested in? So, not
only, is it your responsibility to build a sample space, based on the outcomes of the
random experiments. It is also your responsibility to decide,
what you are interested in and what subsets of
omega, you want to include in your F. So, another such example is, for example, just
say fee, some event A, some subset A, A complement
along omega. This is also sigma algebra; actually, there are also algebras,
because they are sigma algebras. So, here, what
I was saying, here there is other than fee and omega, which is always there in the sigma
algebra. There is one more event, one more subset A,
which is of interesting to me. So, I am including A, but which I have to include A,
I have to include A complement. So, this is another sigma algebra. This corresponds to
only one interesting event other than null set
A and it is compliment. At the other end of this spectrum is 2 power omega, what is 2
power omega? .
That of all possible sub sets of the sample space, it include all possible sub sets of
omega. So, if omega is natural numbers for example, we will include all possible subsets
of natural numbers. So, if omega is n, in this case, F will be 2 power n. So, you can
see .already, that 2 power n is an uncountable
collections of subsets. But, we are only imposing closure under countably infinite
unions, not necessarily uncountable unions. So, you are imposing only closing under countably
infinite unions. And there is everything in between, so this is completely
trivial. This sigma algebra has just one event, if you want two events A and B, you
will have A, B, A union B, A intersection B.
And all that complements and then omega that is how, you do it. And that the other ends
of this spectrum the very end of... So, this is the biggest sigma algebra. That you can
define on omega and it consists of all possible sub sets of omega, the sample space.
So, which one you choose, depends on what you are interested in again.
. It
is not true; an algebra is not sigma algebra, we will give an example. See, in these
cases, there are both algebras and sigma algebras, these very simple cases. But, there you
can construct an examples, where the algebra is not the sigma algebra. It is possible,
we will see in the home work. These are trivial
examples. So, now, that brings us to the question. Say, if I choose F is equal 2 power
omega, I can say that, all the subsets are interesting to me.
So, in some sense, if I can choose F equal to 2 power omega and include all the subsets
of omega as being interesting as being F measurable. Then, I can talk about all possible
subsets being interesting and eventually assigned probability to them. That is what, I
heading two words. The only issue is this, this is again an issue, you will not appreciate
now. When omega when the sample space is finite
or countable infinite. Then, you can actually afford to take F is equal to 2 power
omega, includes all the subsets of sigma algebra and still assigned probabilities to
them. So, for countable sample spaces, both finite and countable infinite samples spaces.
You can actually assign probabilities to all the subsets of sample space. It is possible
to do that, I we will get to all this and I am just
giving you a preview now, but if omega is uncountable like 0, 1 interval or of real
line or something like that. Then, the power set of
that uncountable set is too larger collection to
assign probabilities too. This is again something, you will not appreciate now. It is not
possible to always assigned probabilities to all possible subsets of let us say the
real .numbers uncountable sets. Therefore, this
problem arises; you cannot always take F equal 2 power omega.
Especially, when omegas are uncountable, you would have to settle for a sigma algebra,
which is smaller. This is something you will appreciate little more later. Otherwise, we
can see, if this problem never arose, we will just keep F as all subsets of omega. I want
to keep ideally, I do not want to through anything
out. But, all this machinery becomes necessary, only, because it is not possible
to do a interesting probability, a consistence probability theory for certain uncountable
sample spaces. . So, now another definition. So, omega F is
called a measureable space, again another terminology. So, in the beginning, we will
introduce a bunch of terminologies. So, what is a measurable space, it is a some set omega
endowed with a sigma algebra of subsets. The pare omega coma F is called a measureable
space. If F can be any sigma algebra, it does not have to be any one of these particular
one sets. It does not have to be any particular sigma
algebra on omega. As long as you endo omega with some sigma algebra F. This omega
F is called a measurable space. So, the one thing, that you should know about probability
theory is that, it is actually just a special case of measures theory. And the probabilities
are simply special cases of measures, which is where some people talk
of probability measure. .Have you heard that terminology, you do not
just say probability, but probability measure. It is a special case of mathematical
concept called a measure, which we will define very soon. So, let us define a measure
definition. So, a measure is a function mu from F to 0 infinity included. Such that,
mu of 5 is always 0 and number 2 is A 1, A 2,
dot, dot, dot is collection of this joint F measurable sets.
Then, they measure of the countable union of A i is equal to. So, that is the definition
of a measure. So, it is a mapping from, so you
are given some measureable space omega coma F and so this measure is the function
from F to 0 infinity; infinity included. So, it
can actually take values. So, it can take any real value or it can take the value plus
infinity. So, it is not just a real valued function.
it is an external real valued function. Say, it takes
value from 0 infinity included. So, what does it mean, you are assigning measures to
what, subsets, not all subsets of omega want to make this clear, only F measurable of
sets. So, measures are assigned not to all sets of omega. But, to those subsets, which
you have been decided to include in your sigma
algebra, which is a judgments called, you already made, let us say.
Depending on what you think is interesting to you, you have created some sigma algebra
F and you say omega F is a measurable space. Why, is it called a measurable, I am going
to put measures on this space, something you put, what is a measurable space, after all
something you put a measure on. That is why; it is called a measurable space. It is
actually find a value in simple definition. So, it takes as in put F measurable sets and
produce us a real number or plus infinity, positive real number or plus infinity. And
does not take negative value, that is all, it takes
all values from 0 to plus infinity. We must necessarily have, that the measure of the
null set is 0. Now, so remember this null set it
is in F, which is why we impose condition know, remember this. So, the null set is always
F measurable. And that is specific F measurable said fee
has measured 0 always. It is one of the axioms of measure. This second axiom of measure is
known as the countable additively axiom. This says that if I have a bunch of a countable
collection of disjoint F measurable sets, then the measure of the countable union is
equal to the some of the individual measures. .See, these A i’s are after all F measurable.
So, new of A i is well defined, some positive number.
And also, since A I’s are F measurable, the countable union is F measurable. So, mu
is also defined for the countable union. So,
and axiom imposes that, if you take disjoint A i,
so if I have some big sample space omega and I have disjoint events, disjoint F
measureable set. A disjoint means, they do not have each of these A i’s have null
intersection. So, A i intersection A j is null for all I
j, that is what disjoint means. So, if I take the
measure of the union of A i. So, I am only drawing the finite number of them obviously.
But, there are actually a countable infinite number of this A I’s. They all disjoint,
pare wise and if I take the measure of all these
guys, the union of all the guys is equal to the
mission of the measures of the each of them. Actually, very intuitive, come to the think
of measure as something that says how much is contained in that or something like that.
So, if they are disjoint, I would not be able to
add the measures. So, there are these two axioms.
. No, I am talking about the genetic measures;
a measure is a function from your sets. It is
functions the maps F measureable sets to 0 infinity. And you imposed two conditions.
One is that, the null set should have 0 measures. The other is that, if I have the disjoint
accountable union of disjoint F measurable sets, the measures can be added up. This
axiom if you note down, is called the countable additivity axiom. This is the most
important one; this is fairly a trivial one. This is most important property of a measure. .. Now, the triple, you just say the triple omega
F mu. So, now, we have defined some measure on this measurable space. So, we started
with omega and, then you endored with a sigma algebra F. So, this pair, you called
a measurable space, in anticipation, that you
going to put a measure on this space. Now, that you put a measure on this space, this
is called a measures space. Now, it has a measure
space. So, what is a measures space, it is a triple
consisting of a set, a sigma algebra on that of
subsets. And then a measure define on the F measureable sets. So, this property should
be satisfied good. So, far, we have actually defined a bunch of things. It is actually
a concept eventually fairly simple. Just that
the thing is very abstracted this point. You probably do not have the concrete examples.
I am deliberately keeping it abstract,, because once we develop a proper theory; we can
give number of examples and make several specifications. So, far logically everything is
good. So, now, remember that, see this fee is in this algebra always. Therefore, omega
is also in the sigma algebra. So, the whole space
is always F measurable, omega is always F measurable. So, omega must have a measure
associated with it,, because I cannot leave out any F measurable set.
So, mu of omega is well defined. So, if mu of omega, so what value can mu of omega
possibly can take, no, I mean, I have not said anything about it. So, mu map F
measurable sets to 0 infinity, infinity included. So, it could be that, this is finite. So,
this .is again something very intuitive. If it
so happens that, if your mu assigns only a finite
value to the entire space. Then, it is called a finite measure.
And similarly, if this is equal to infinity, plus infinity, then mu is called an infinite
measure. So, mu of the entire sample space of infinity. Then, you say the measure is
the infinite measure. If it something finite,
you say it is a, infinite measure. Finally, the case
that of most interest to us is mu of omega equal to 1, can take any real value, it can
be finite or infinite, if it is a finite, you
say is a finite measure. A finite measure in particular, if omega is
equal 1, then it is called a probability. So, in
that sense a probability measure is very special case of a measure. It is a finite measure
in particular with which as signs one as the measure to the entire sample space. It is
really all there is do it. In this sense, probably theory is the special case of the
measure theory.
So, this is not the course of measure theory, this is the course on probability theory.
So, we will not go on and on about, what do we
generic measures spaces, we will study probability measures in greater detail. So,
in particular, when mu is probability measure, when mu of omega is equal to 1, we will no
longer call it mu, we call it P and we will say omega F P. And we will not call it a measures
space; we will call it a probability space. So, because we are studying probability,
I want to write it down. So, that, there is no further confusion.
. .So, probability measure, so I already defined
what probability measure is, but I want to do it again, because it is so unfortunate.
A probability measure, it is denoted by P with
standard two lines, standard notation. P on omega as is a function key mapping, F
measurable sets to, what does say map to 0, 1. Now, that is function satisfies P of null
is equal to 0. P of omega is to 1 and, then if
A 1, A 2, dot, dot, dot are disjoint F measurable sets.
Then, probability of union i equals to 1 to infinity, A i is
equal to sum over i equals 1 to infinity, probability of A i. So, this is
just a repeat, what you have seen. So, probability measure is nothing but a special case of a
measure with this additional property. So, the
only the difference between a measure and a probability measure is that a probability
has this property.
An ordinary measure need not have this property. It is measure can be anything positive,
it can have a positive real number or plus infinity this. One thing, I like to point
out again. This is an important matter, it could
cause confusion. So, this I said, so if you have
this union, i equal to 1 infinity or a better notation opinion is union i belong to n, A
i. So, this is the set of all omegas containing at
least one of the A i’s. So, this union is not defined in terms of
some finite union going bigger and bigger or
anything like that. However, this summation, this is an infinite summation, infinite
summation is actually the limit of a finite summation. So, this is like limit n 10 into
infinity summation i equal to 1 to n, probability of A i. Now, if you write down an
infinite summation, the question that comes to your mind is, a converge, I have happily
written down, this summation, this summation, etcetera.
These are infinite summations. You have to mean to make in order for just to make
sense, you have to argue that, this is in fact, well defined. Why is this well defined,
see, when you write the summation like this, the
one thing, you do not want is, see, you do not mind, if it goes off to infinity. Then,
you call whole summation equal to infinity. So,
you do not want this summation to be something undefined.
If you write down I equal 1 to infinity minus 1 power i or something like that, for
example. It is something that keeps jumping; the summation is not defined at all. But,
such a situation is not arise here, why not? ..
Yes, so mu of A i is always non negative. So, if you really interpret this summation
is nothing but, limit n going to infinity, summation
i equals 1 to n, mu of A i. This is the definition of the infinite sum. So, what I
am saying is that, if you look at that sequence at
n, call it X n or something, it is a monotonically increase, monotonically non decrease
sequence in n. So, monotonically non decreasing sequence
has to either converge or go off to infinity, which also in some sense converge. What it
does not do is, oscillate to do any such things. So, when mu is not positive, we cannot
have this being well defined. Similarly, for probability measures, again, this is a
well defined summation. Except, what you are constrained here is that, after all what is
there on the left should also be at best one. You
can not the map value can never exceed one. So, you have to how the situational the summation
is not only well defined, it has to be at most 1, it cannot be bigger than one. And
finally, omega F P is called a probability space.
So, we have to the three steps. So, what is a probability space, you start off with a
random experiment and collect all it is possible elementary outcome and make your
sample space. And, then you collect the subsets of the sample
space, which you think are interesting, which you are interested in assigning probabilities
too. And you endow omega with a sigma algebra of subsets of omega. And finally,
so then omega F be a measurable space, then you finally, assign probability measures,
which satisfies these rules is axioms. So, these are called axioms of probability and
these two are called as axioms of measure. These three are axioms of probability.
So, you end up with a probability measure and this triple omega F P is called a
probability space. This is where everything begins with. So, this is the beginning of,
what we are going to build up on. So, this one thing I want to make clear. So, none of
this, what we have developed here, tells you, how to generate these F or how to assign
probabilities. It does tell you, how to do this, it is up to you.
But, if you were to do it, just tells you the rules, you have to satisfy. Within these
rules, you can do whatever you want, you can create
any F you want, satisfying those rules. .You can create any P you want, satisfying
these rules. But, how you do this, how exactly you do this is up to you, depends upon your
intension or depends on what you want to model what you want to capture.
So, it is just gives a frame work, it does not answers immediately. If you supply some
numbers to the theory, it will give you some other answers, that is all. It is your
responsibility to create F, to create P, etcetera, questions
. So, as I said it is a map from F to 0, 1.
So, it takes input F measurable sets. So, in this
specific case of probabilities, so now, after all the story, we are ready to give a précised
definition of what an event is. If omega F P, if we are looking at a probability space;
F measurable are called events, let us all just
do it. So, you go and put a sigma algebra in
omega, depending on what is interesting to you. Whatever, you include in your F is an
event; whatever you do not include in F is not an event as simple as that. So, let me
just write that down as well.
. So, if omega F P is a probability space. So,
F measurable sets are called events. So, now, we have a very précised and understanding
of, what events are. So, far, we have been giving some Wishy Washy definition that over
their subsets of sample space, which are have interest to you. Now, this is the correct
definition. So, in a general measures space, .you have omega F mu, some measure and the
elements of F are called F measureable subsets of omega.
And the specific of probability space, you do not say F measurable set, you say, it is
an event. So, there is difference between measurable
sets and events as far as probabilities spaces are concerned.
. This is an omega.
. That is not true. So, we have assigned to
probabilities those sets, that are in F. F is a
sigma algebra on omega, you assigned probabilities to only those subsets of omega,
which are in F. So, that F is always contain fee and omega and it may contain other
subsets of presume or believe. But, it is not necessarily contain all subsets of omega.
Unless, your F itself is 2 power omega, see, F is a collection of sets, it is not a subset
of omega, it is a collection of sets of omega.
So, unless F itself, F is two power omega, you will not be assigning probabilities to
all subsets of omega. What I was saying little
earlier was that, when omega is countable, finite countable infinite. You can actually
afford to take F as 2 power omega and you can
assign, it is actually possible to assign probabilities to all subsets of the sample
space. When, omega is countable.
But, when omega is uncountable, it turns out, if you take F equal to 2 power omega,
assigning probabilities consistently to all subsets of something like, the real line is
not always possible. This is not something that,
you will understand know, this is an impossible theorem, which is a non trivial
theorem. So, if you are working with an uncountable sample space omega, which is excel,
which is a real number or which is 0, 1 or something like that.
Then, you have to define a sigma algebra on omega, which is smaller than 2 power
omega. So, how you do that is interesting part. So, that we will we get later. So, far
any questions? ..
That is true. If omega is countable, let us say omega is n for example, we know that,
2 power omega is 2 power n. So, F will have
uncountable many subsets, but that is alright. I am saying, if omega itself is uncountable,
then it is too big 2 power omega is even bigger in some sets. So, if omega is 0, 1
for example, for r, 2 power r is a collection of all
subsets of r. That is a very huge collection of subsets and it is too difficult, it is
in many case, it is impossible to assign probabilities
to interesting measures on it. .
Yes, if omega countable infinite is fine, either omega is finite no problem, omega
accountably infinite is no problem. It does not matter that F has uncountable many
subsets, omega if uncountable, the sample space is uncountable, you get into trouble.
You have to be much more careful. So, probability theory is actually very easy, when
omega is countable. If it is either finite or countable infinite,
the probability is very easy. Because, you can
take F as 2 power omega and assign probabilities to all sets of sample space, which is
what you are use too. But, omegas uncountable, you get into trouble. That is something
you may not studied in elementary courses. So, I just want to make sure that, I done
with everything, I want to say here, yes, we have
defined probability space, event. So, F measurable is called event, any other
questions? .
They all are measures theories; measure theory is a very important branch of
mathematics and itself. And the only thing that is not included is this, mu of omega
can be anything positive or plus infinity. Probability
theory is a special case of measure theory, but also very interesting case. So,
very important special case, but measure theory in it is own, it is very important part of
mathematics. So, even things like length, area, volumes,
these are all examples of measures, they just. So, the concept of measure generalizes things
like length, area, volume, etcetera into orbiter measure space. It satisfies the things
are in property. That is the very important area of mathematics, is there anything else.
So, we will stop the lecture here. .
