We begin this lecture continuing our discussion
on the assignment problem. We have already formulated and solved the problem using the
Hungarian algorithm. We have also seen how Hungarian algorithm works and how it provides
the optimal solution to the problem. We have also shown that Hungarian algorithm is an
example of a dual algorithm like dual simplex. It finds a feasible solution to the dual of
the problem and once the primal becomes feasible the optimal solution is obtained. There are
still a couple of things to be seen with respect to the assignment problem. The first thing is unless otherwise stated;
the assignment problem is a minimization problem. Remember that the assignment problem has a
minimization objective of minimizing double sigma Cij Xij. There could be situations where
we have to solve a maximization assignment problem and what we do there is we convert
it into a minimization problem by simply multiplying the objective function coefficient with - 1. For example if we had a maximization assignment
problem, say with numbers such as this, we convert it into a minimization problem
by multiplying the objective function coefficients with -1 and then we can do row column subtraction
from this matrix and proceed as it is. Normally it is not recommended to do the row column
directly. It is always better to convert it to a minimization problem. Treat this as row
minimum and subtract from every element and so on.The assignment problem usually is a
square matrix, we have we have seen. This is a 4/4 example where 4 jobs have to be assigned
for 4 people unless otherwise stated. The assignment matrix is a square matrix but we
could have situations where we could have say for examples 5 jobs and they might have
to be allocated only to 4 people. For example we could have a 5th job which
might have something like 7, 6, 5 and 8. Now in situations such as this we need to convert
this into a square matrix by adding a dummy as we did in the transportation. Dummies will
have 0. We then proceed after converting this into a square matrix we proceed as we normally
do. This problem being a maximization problem, we go back and we need to write this. We do
not put a dummy here. This will become - 7, 6, 5 and 8 and then we could also put a dummy
here and converted it to a 5/5 problem. Sometimes we could have an assignment problem
such as 4 rows and 5 columns in which case we add an additional row which will be a dummy
row, make it into a 5/5 problem and then solve. The only thing that will happen is if we have
an unbalanced assignment where the number of rows and number of columns are different
then either all jobs cannot be assigned to all people or all people will not get a job.
For example if I have 5 jobs and 4 people and then we make it into a 5/5, now whichever
job is allocated to dummy is effectively not allocated at all. Similarly if there are 5
people and 4 jobs then in the final solution 1 person does not get any assignment. But
when we solve an assignment problem, we have to convert it into a square matrix and then
solve. There could be situations where a particular job may not be performed by a particular resource. For example if this is a minimization problem,
and say it is a 4/4 minimization problem and suppose we say that this job cannot be performed
by this person, and then we put a big M here. Big M as we defined M in the simplex once
again, M is large and positive when tending to infinity for a minimization problem. So
that when we make the allocations we will not have an allocation at all here. This M
+ constant = M, this M - any constant is = M. So this will never get 0. Therefore this can
never be a place for allocation. So this way we handle situations where a particular job
cannot be handled by a particular person. The last thing perhaps we need to look at
is this. When we formulated the assignment problem, this is the objective function. We
have constraints sigma Xij = 1; sigma Xij = 1, we defined it as Xij = 0, 1. So summed
over i for every j and summed over j for every i. This is the formulation of the assignment
problem that minimizes the total cost of assignment. Now these 2 constraints indicate that every
job goes to exactly 1 person and 1 person gets exactly 1 job. It is a square matrix
with n square decision variables. Now one more thing is 0, 1 is now relaxed
and this problem is written as greater than or equal to 0. It becomes a liner programming
problem and we solve this linear programming problem by simplex or any other liner programming
solver. Then we would get the solutions where these Xij's actually take either 1 or 0. We
will not have a situation where these Xij's will take fractional values. Xij will continue
to take 0 or 1, even if we relax the 0, 1 restriction and solve the resultant l 2. This
is very similar to what we saw in transportation problem. We also saw in the transportation
that we had ai's and bj's and we said that as long as the ai's and bj's are integers
we said this solution to the transportation problem would be an integer. Now here we have
1 which is an integer, so the solution to the assignment problem even if we solve it
as a LP would still be integer value which is 0 or a 1. However we do not solve this
using a liner programming because we have a much degenerated LP. There are n square
decision variables out of which only n variables are basic at the optimum. So we use different
method such as the Hungarian algorithm and solve the assignment problem. So with this
we come to the end of our treatment on the assignment problem and we look at the next
topic which is called dynamic programming. Now under dynamic programming we are going
to try and solve different examples to show to you the various terminologies that are
associated with DP. In dynamic programming, we divide the problem into several stages
and solve the problem for each stage. Now we go back and see the dynamic programming
is used when the given problem can be solved in stages. There the decisions are made in
a particular stage which will affect the decisions at the subsequent stages but they are dependent
on 1 another. We also called something called backward recursive approach in dynamic programming.
What we are going to do is we will illustrate the concepts of DP idea of the recursive approach
and the terminologies that we are using including the notation by using an example which is
called as Stage coach problem. We will solve the stage coach problem first. We explain the terminologies as well as the
general framework to solve all DP problems. Now stage coach problem is essentially the problem of finding the shortest distance
from a given source to a given destination which is nothing but the shortest path problem
so we take a stage coach problem like this. Now we have 10 nodes or 10 cities that a person
has. The person is right now at the start node or city node and wishes to go to the
destination node or city 10. This is the network that is connecting the nodes. This is a problem
where you can easily decompose the decision making into different stages and you we will
also realize that here you do not actually have connectivity between say, 1 and 5 or
3 and 8 and so on. You can easily decompose this into several stages and you find connectivity
from the nodes of1 stage to the next stage. It is a simple shortest path problem but we
are going to use in this problem to illustrate the concepts behind dynamic programming and
the terminologies that we are going to follow. Now the distances are also given here. So
we have 5, 5 and 6. With 5 is 4, 7 and 8, 8, 10 and 5, 4, 5 and
7, 6, 8, 9, 7, 5, 7, 8 and 9. We follow something called the backward recursive approach. In
order to reach 10, now this person has to either start from 8 or 9 and then the person
has to go to 10. So we are right now assuming that the person is here and we call this 1
more stage to go and the person is either at 8 or 9. Now the node or the place where
the person is is called the state of the system. So the state of the system is either at 8
or 9 and then given the state of the system, we try to find out the best decision that
the person has to make at that stage to go from 1 stage to the next stage and since we
are following a backward recursion we are going to say when we are here, we have 1 more
stage to go. When we are here we have 2 more stages to go and so on. So the first table
would look like this. We will have n = 1, one more stage to go.
Now s is called the state variable and then we are here with 1 more stage. To go the state
variable it is either at 8 or at 9. The person can be either at node 8 or at node 9. Then
the person has to make decision which we call X1, X1 as the decision that is made when you
have 1 more stage to go. Now whether the person is at 8 or the person is at 9 the decision
is very clear that is to go to 10 because there is no other destination. So in this
case the X1, the decision is to go to 10. Also the decision is to go to 10 and then
the value of the objective or the distance is the distance minimization problem. So the
distance that the person will cover is given by f1. f1 is the distance function. Now f1
is far from s. In this case from 8 to 10 the distance is 8 and from 9 to 10 the distance
is 9 and we have X1 star which is called the best decision which will be to go to 10 whether
you are going from 8 or from 9. Since we have only 1 alternative for X1, it
automatically becomes X1 star which is the best decision. So if the person is at 8, there
is only 1 choice which is going to 10. So that is best choice so the person would travel
a distance of 8 and go to 10 which is the best choice. If the person is at 9 then there
is still only 1 choice to go, 10 and the person will spend 9 or will travel 9 units and go
to X1 star which is 10. Now when we have n = 2 or 2 more stages to go which means, we
are say, somewhere here. Now this is 1 stage. This is another stage, so there are 2 more
stages to go. Now you will realize that there are more alternatives than 1. When the person
is here and he has 2 more stages to go, then he could either be in 5 or 6 or 7. So the
state variable s takes 3 values which is 5 or 6 or 7. Now when the person is in 5, the
person has to make a decision here so the decision is where to go next which are either
8 or 9. So now the X2 which is the decision that have to be made can take 2 values which
is 8 and 9. When the person is at 5 and decides to go
to 8, you incur a distance of 6 plus, already the best way from 8 to reach the destination
10 is already written in the previous table, so you add another 8 to this to repeat again.
The state could be 5, 6 or 7. If the person is at 5 and decides to go to 8, now to travel
from 5 to 8 you travel a distance of 6 and the best way to reach from this to the destination
is already seen in the previous table. So if the person is at 8 then the best decision
is to go to 10 to incur another 8. 6 + 8 = 14. Now if the person is at 5 and decides to go
to 9 there is a distance of 8 and the best way to go from 9 to destination is already
there in the previous table. So that will be 9 to destination, and an additional distance
of 9. So this becomes 8 + 9 = 17. Now this is the state variable which is at 5, the person
is at 5 and in order to go from 5 to the destination, the person either goes through 8 or 9 and
the total distance travelled from 5 to destination is already seen here. So the best thing the
person would do is to travel a distance of 14 and go via 8 because this is the minimum
distance. If the person is at 6 and decides to go to 8, there is 9 here and 8 to destination
is already seen, which is the same 8 which is here. So this becomes 9 + 8 = 17. If the
person is at 6 and decides to go to 9, it is 7 and 9 to destination is already an additional
9 which is the same 9 here. So 7 + 9 = 16 and the minimum of them is 16, so the total
value is 16 and the best decision now is to go via 9 and not via 8. Similarly if the person
is at 7 then you have 5 to go to 8 and from 8 to destination is another 8 which is 13
and 7 + another 9 which is 16. The smaller 1 is 13 and the decision is to go to 8. We
have now completed 2 of these stages. We have 2 more stages to go. The next the person would
be here to make a decision and then the person would come back to make a decision but then
since we have written a lot of numbers, let us define what we have actually written. What
we have here is this. When I have 1 more stage to go I define something called f1. I am in
a state s or s1, you could define it suitably. We call it as s and the person decides to
go to X1. Now the value of the function that we are going to have is dsX1. So in this table
the person is at 8 decides to go 10, so the value of the function is dsX1 which is distance
from 8 to 10 which is 8. Similarly from 9 to 10 which is 9. The best decision here would
be f1 star which is the best decision. For a given state s it is going to be the
best one so this will be the 1 that minimizes f1(s X1). In this case it will become much
more obvious and clear. Now we have 2 stages to go. So f2 I am in a state s, I decide to
go to city X2. Now that is dsX2 for example, if I am in 5 and decide to go to 8 it is d58
plus the best way to reach from 8, which is f1 star of 8. f1 start of 8 is another 8 which
is added here, so this will become + f1 star of X2. From s I go to X2 so from s to X2,
from 5 to 8, I incur a distance of 6 and then once I reach 8, the best way to reach the
destination from 8 is given by f1 star of 8. f star 1 of 8 is this 8. So we add 6 +
8 = 14. Now when I am in state s = 5, I have 2 choices. For X2 = 8 and 9, so I have evaluated
this function as 14 and 17, so the best way to go from 5 to destination would be f2 star
of s will be minimizes f2 of (sX2) and the corresponding X value is written as X2 star.
When I am in 5, now the best decision f2 star of 5, the best way to go from 5 to destination
f2 star of 5 is given by 14 and the corresponding X2 star is 8. So X2 star is the corresponding
value. So if I am at 5 the minimum distance to travel is 14 with the corresponding value
of 8 from 5 to 8, 6, 16 and next destination is 9 and if I am at 7 the best way to reach
is to travel to 13 and go to 8. Let us continue with the next 3 more stages.
I am here so I could be either in 2 or 3 or 4 now my state variable s can be in 2 or 3
or 4 my decision variable X3 can be to 5 or 6 or 7.
So if am in 2 and I decide to go 5 from 2 to 5 the distance is 4 + 5 to destination.
I already found i which I can get from the previous table, so 5 to destination f2 star
of 5 is 14. If I am in 2 and decide to go to 6 I have to travel 7 + 6, to destination
which is already here, minimum value of 16 and if I am in 2 and decide to go to 7 it
is 8 + from 7 to destination, it is another 13 so this is 4 + 14 = 18; 23 and 21, so the
best value is 18 and X3 star is 5. If I am in 3 and I decide to go to 5, it is
8. 5 to destination is this 14 which is written here. So you can directly copy this 14, 3
to 6 is 10, 6 to destination is 16 which is already written here as 16, so 10 + 16 and
2 to 7 is another 8. 3 to 7 is 5. So I have a 5 here and 7 to destination is 13 so this
is 22, 26 and 18 and minimum is 18 and the best decision is to go with 7. If I am at
4 then I incur 4 to 5 which is 4, 5 to destination is already here as 14. The same 14 is here.
4 to 6 is 5, 6 to destination is 16 which is here as well as here and 4 to 7 is 7, 7
to destination which is 13. The same 13 is here as well. So the values are 18, 16 + 5
= 21 and 20 so it is again 18 and the best decision is to go to 5. We also have written
down the equation. So f3, 3 more stages to go and at state s. I decide to go to X3. This
will be dsX3 plus, we go back to the previous stage and find out the best value. So f2 star
of X3 and f3 star for a given s will be to minimize f3 of X, X3 and X3 star is equal
to corresponding X3. We go to the last part n = 4, 4 more stages
to go which means we are here. The person is at
the beginning which is in city 1, so state variable has only 1 value. We are now here.
s is 1, now from 1, the person can go to either 2 or 3 or 4, so you would have X4 which can
take values 2 or 3 or 4. So 1 to 2 is 5, 2 to destination is already here, which is given
as f3 star of 2 which is another 18. 1 to 3 is another 5 and f3 star of 3 is again 18,
so 3 to destination is another 18. Now 1 to 4 is 6 and 4 to destination is also 18 so
f4 star of s is 23 and f4 star is 2 and 3. We have to write down the equations so we
write f4 of s, X4 is = dsX4 + f3 star of X4 indicating the define s and decides to go
to X4. I travel a distance from s to X4 + the shortest distance from the X4 to destination
is given in the previous table, so f3 star of X4. Now f4 star of s is to minimize f4
of + X4 and X4 star = corresponding value. Now we have solved the problem. Now the shortest
distance is 23 from 1 to 10 and there are 2 paths. So we start from 1 and you could
go to 2 from this or from 1 you could go to 3, you could go to 3. Now go back to this table and see if I am
in 2. The next destination is 5. If I am in 3, the next destination is 7, so 1 to 2, 2
to 5, 1 to 3, and 3 to 7; this table will tell you if you are in 5. The next destination
is 8. If you are in 7, the next destination is 8 and this table will tell you if you are
in 8. The next destination is 10, so there are 2 paths here which are the shortest paths
that have a total distance of 23. We also need to introduce some terminologies which
are consistent with all DP problems which we will do. We have to define 4 terms which
are called as stage, state, decision variable and criterion of effectiveness. These 4 parameters are common for every DP
problem and have to be defined for each problem. Now this problem stage is actually a stage
in the sense that we have defined the problem in such a way that the person goes from 1
stage to another. Each stage is characterized by a set of alternatives or cities that are
available. So here each 1 is a stage. So stage is actually the state of decision making,
where at every table you realize that the state is the place where the person is already
from where he or she has to make a decision. So state is
city at which the person presently is or you can say city in which the person is located.
Decision variable is the next destination where we have to go next and the criterion
of effectiveness is the minimization problem. So we need to minimize the total distance
travelled. So now we have completed this problem or solved this problem using DP. Now dynamic
program involves certain things. First thing is we solve the problem, stage wise so even
before we do all these things, the first thing we need to do is to write down the stage state
decision variable and criterion of effectiveness. May be for this example and perhaps the next
example we will try to write this in the end but then as we proceed with more examples
it is absolutely necessary that we define this first. In fact once the state variables
and decision variables are defined the problem becomes easy to solve here, since we are introducing
the terminologies we solve the problem and then later said that these 4 terms has to
be introduced for every problem. In fact first thing we need to do is to introduce these
4 terms and define them to this specific problem. Then we follow what is called a backward recursive
approach. We assumed that we have reached the destination and then we go backwards and
see that in order to reach 10 we need to go 1 stage back and you can start from 8 to 9
which is the 1st, 2nd, 3rd and 4th respectively. At each stage the calculations are shown in
the form of table here because the state variable and the decision variables take discrete values.
They are not continuous variables so here the state could be either 8 or 9 and decision
is to go to 10. Here state is discrete values 5, 6 and 7 and decision is 2, two values 8
or 9 and no other continuous variable. So we form our tabular approach. For every table
we have to define what it is in the table, we have to define the relationship. Now this
relationship is called recursive relationship. You could define a recursive relationship
for every table as we have done or you could write a general recursive relationship for
this problem which would be like this, consistent with our terminologies, fn, f is the function,
n more stages to go, I am at state s and have decided to go to Xn. Xn is the decision variable,
s is the state variable then I travel a distance of dsXn from s to Xn and then I go to the
previous table f( n - 1) star which is the best value of Xn. For example if I am in 5
and decide to go to 9 then somewhere we would have this. Here I am in 5; I decide I go 10
here so I have 2 more stages to go so s X2 is dsX2 + f1 star of. So the best value is
taken from the previous value so I have n more stages to go, I go to the previous table
which is n - 1 star of Xn and the best value would be fn star n more stages to go given
state s the best value is the 1 that minimizes fn of sXn and the value of Xn star is the
corresponding Xn for which this value is minimum. This kind completes the first example along
with that they will use few terminologies. We will now take few more examples, at least
a couple of more examples for the discrete case and then we would also look at some more
examples where the decision variables take continuous values. So let us look at the second
example here. The second example is what we call as the
reliability problem. Here we have assumed that we have equipment which is made up of
say 4 components. We call them A, B, C and D which are connected in series in order for
the equipment to function. Now each component has certain reliability in the series system.
The total reliability of the system is the product of the individual reliabilities. In
order to improve the reliability, we can also add another component in parallel or keep
them as a stand-by and so on. For example we could add another A here or use it in such
a way that the reliability of A together is increased and therefore the overall reliability
is increased. Now we have the data that tells us, for this problem that the data is like
this. Now for these 4 components the number of units
that we could have could be 1, 2, 3 and 4. Now this is component A, so we have 2 terms
reliability and cost reliability and cost C and D reliability and cost. 0.6, 100, 0.7,
130, 0.8, 150, 0.9, 170, 0.7, 80, 0.75, 100, 0.8, 120, 0.85, 140, 0.7, 0.75, 0.8, 0.80,
0.9, 0.85, for example if we have these 4 components here, I have only one of A then
I get the reliability of 0.6. I have to spend 100 Rs. If I decide to add another one of
A here then I would have to spend 130 and the total reliability of this A alone will
be 0.7 and so on. Similarly if I decide to put say, 2 of C, then I get a reliability
of 0.8 and the cost is going to be 80, obviously there is a restriction on the amount that
we have that we can spend. That is given by Rs 300. We assume that Rs 400 is available
as amount and we try to solve this problem. The problem now is how many units will have
A B C D such that we maximize the total reliability of this system and at the same time we have
a budget restriction or the money restriction Rs 400 with us. So as we start, we realize
that we first have to define the 4 important terms Stage, state, decision variable and
criterion of effectiveness. Now as far as this problem is concerned, if
we follow a backward recursive approach then when we have one more stage to go, we would
be somewhere here. If we have 2 more stages to go then will be here, 3 more will here,
4 more will be here. So as far as this problem is concerned, stage is each component because
we make a decision for every component as we follow a backward recursive. So stage is
component. Decision variable is easier to define in this than the state variable. So
we define this. We define this when we come back to this. Now decision variables are how
many units are we going to have for each one of these components, so decision variable
would be how many units? Criterion of effectiveness is the objective function with which we begin. The objective is to maximize the reliability
so criterion effectiveness is to maximize the reliability. What is state variable? State
variable in this example is the amount of money available for allocation, because the
decision is going to be based on the resource that we have and the resource in this case
is the money. If we have unlimited money then the decisions are obvious. Try to put the
maximum now because there is money restriction depending on how much money that is available
to make a decision in every stage. The decision variable changes, so state in this example
is the amount of money ava¬¬ilable or in general where ever there is a constraint the
resource availability represents the state of the system. The state can be directly linked
to resource availability. In this case, state variable is the amount of money which is available
for allocation to that particular component. So having defined these 4, let us go back
and try to solve this problem. Now when we have n is = 1, we have one more stage to go.
We call it as X1 or we call X4 as the number of units that we are going to have of D, X3
for C, X2 for B and X1 for A. So X4 is the variable which says how many units of D and
we have f4 of X4 or f1 star of X4 and then we have the corresponding X4 star. Now what
are the possible values that the state variable can take if I have 1 more thing to go? Now
one easy thing say is we can say the money that is available for allocation of D could
be anything from 0 to 4 hundred but we do not follow that we try to see what is the
bare minimum that we need and what is the maximum that we can have? now as far as this
is concerned that bare minimum money that we need is enough money to buy at least 1
of D which is at least 60, so we need to start with at least 60 otherwise we will not able
to buy even 1 of D and the equipment will not function. It does not make sense to look at state variables
of 0 to 59 for this because we cannot buy anything of D and the reliability will be
0. So it makes sense start with saying that we have at least 1 each of A B C D there which
will I have to buy and over and above that whatever money I have. I try to spend on buying
more of A B C D. So the minimum money that I need to have here is 60. The maximum money
that I can have for D is actually if I brought only 1 of A B and C, which means if I have
spent exactly 100 on A, 80 on B, 75 on C, which is 255. If I had spent then the balance
145 is the maximum that I could have for D. So D, the maximum that state variable will
have here is 145 because if I have anything more than 145 allocated for D which are available
for allocation to D which it means that I will not able to buy 1 of A, 1 of B, 1 of
C and therefore my reliability will be 0. So I make an assumption that I have at least
one of A B C and D therefore the resource that is available for D can vary between anything
from 60 to 145. Now if it is from 60 to say 70 or 69 then I can buy only one of this.
So X4 is 1 and the reliability is 0.8 and X4 star is 1. If I have between 60 and 69 Rs I can buy only
1 of D therefore my reliability is .8 for D and I buy exactly 1. So when I have 70 to
74, I can buy 2 of D that is what I will do. I will not buy 1 of D and keep the remaining
money un-utilized. The reason is by buying 2 of D, I can increase the reliability of
D and the overall reliability of the system. If I have more than 70, I will not consider
the possibility of buying only 1 of D and keeping the remaining money unspent. So we
try to buy as much as D as possible because the objective is to maximize reliability and
not to minimize the money spent. So the decisions are always made keeping in mind, the criterion
of effectiveness which is the objective function. So when state is from 70 to 74, X4 will be
2 and I can have a reliability of .85 and X4 star may be 2. Even though there are 2
alternatives here which is X4 = 1 and 2, the best value will be 2. Now similarly when I
have 75 to 79, i.e., enough money to buy 3 of D, X4 is 3, reliability is .88 and X4 star
is 3 and when I have anything greater than 80, 80 to 145, I will have 4 of this to get
.90, I have 4 of this. Now what is the recursive equation here? Recursive equation is f1 of
s, X1. I have s Rs available. I have s1 Rs available or s you could use it suitably.
If you use s1 here, you will have s1 here that is all and this will be f1 star of s
or s1. So we use s here. You could consistently use s, so I have 1 more stage to go, 1 decision
to make. That is D if I am left with s Rs and I decide to buy X4 units of D then this
will be reliability associated with X4. For example if I have 70 to 74, I can buy a maximum
of 2, so I go back and see that my reliability is 0.8. Now what is the best value given? f1 star
of s will be maximized R of X4 subject to the condition C of X4 should be less than
or equal to s. For example when I have state variable, suppose I have 72 Rs available here,
I can buy 2 s, as long as the money that I have for 2 which is 70 is less than the state
variable. So this constraint has to be satisfied by the best value and the best value X4 star
will be the corresponding value of X4. Now when we go back and try to
solve for this C and D put together which means, we have 2 more stages to go. So we
are at n = 2, 2 more stages to go. Now we have the state variable s and you could
have decision variables this is X3 = 1, 2, 3 and 4. There are 4 alternatives that are
possible then we have f2 star of X and we have X3 star. Now what is the minimum value
that s can take, when I am here? When I have 2 more stages to go, the minimum amount of
money that I definitely need should be sufficient to buy at least 1 of C and 1 of D. So this
would begin with 75 + 60 which is 135 and the maximum would be, if I had bought 1 of
A and 1 of B, so if I had bought 1 of A and 1 B and spent 180 then we had 400 so the maximum
I could have would be 220. Now let us complete this table. When I have exactly 135 here and I buy X3
= 1, so X3 = 1 gives me a reliability of 0.7 and I spend 75 on this, so the balance money
that I have out of the 135 is 60, 135 - 75 which is 60 which I carry to the next item.
So when I have 60, I get another reliability of .8 which I have. Now when I have 135, I
decide to put by 2 of C which means I spent 80, I have the balance 55 available for this.
I cannot buy anything. So I do not look at this alternative. So when I have 135 I have
only 1 alternative which gives me a reliability of .56 and X3 star is 1. When I have 136 now,
what we need to do is we need to look at all the state variable values from 135 to 220
which is roughly about 66 + 20 = 86 different values that we need to look at. So we do not
tabulate all 86 here. We try to bring them into the range like what we did here. So now
we look at 136 when we are at 136, the same thing will happen. For example I can buy only
1 of C and spend 75. I would have balance of 61 from which I go to this range and I
could have only 1 of this. So this will happen from 135 to139. I will
have only 1 possible that exists. I will be able to buy only 1 of C and get this. Now
when my state variable moves to 140, what will happen is I could have 2 alternatives.
I could buy 1 of C, spend this 75, and get 0.7. The balance would be spending 75 out
of 140, so the balance would be 65. So 65 would give me again .8, but I can do another
thing. I can spend 80 and buy 2 of C to get .8. I have already spent 80 so I carry the
remaining 60 to the next one, so at 60 I get another .8. So this becomes .64. Now these
two are not possible. Now the best decision here between these two is .64 and X = 2. Now
like this we have to build this table now. Wherever we find another alternate coming
in, we move to the next definition of the stage now. We show the entire table here. Now when I have from 140 to 144, my best decision
is .64. Now from 145 to 149 that is the next range and there I realize that I could get
into a third alternative which is here. Now this alternative is trying to buy 3 of C and
spending 85 and with the remaining 60 buying 1. So from 145 to 149, we have 3 alternatives
and the best value is .72 with X3 star = 3. Similarly I move again from 150 to 154. I
have 1 more alternative that comes in now when I am at 150 I could think of X3 = 4 which
means I could think of buying 4 of C spending 90 getting a reliability of .95 and with the
remaining 60, I buy 1 of this. Now that is given as the 4th option here which gives me
.95 from here and the remaining 60 would give me 0.8. So 0.95 into 0.8 gives me a 0.76.
This gives me 0.76 and the best alternative is still with .76 and 4. When I am in the range 155 to 159, all the
4 possible exists for buying C but then one of the values would change. One could see
that when you are in 151 - 159, if you decide to buy 1 here and spend 75 you get a balance
80. So you are moving into this range to get a reliability of .9. So you get .7 into .9
which is .63. This would be .8 into .88. If I buy 2 and spend 80 and then I have a 75.
So I move into this range. So I get .8 into .88 with is = .704. So all the 4 alternatives
exist but they have different reliabilities. So the best value would be .765 and this happens
when X3 is star = 3 which is shown here. Now again at 160 to 164, we need to calculate.
So at 160 to 164, we realize that the change actually happens here. 0.9 Into 0.88 and .95
into .85. So if I have 160 and I decide to buy 3, I have 85. 85 + 75, so I move into
the next range so I decide to buy 4. I have 90 giving me .95 into .88 which gives me .8075.
So when I am at 160 to 164 I have .8075 which comes as the maximum reliability and my decision
would be to buy all 4. Similarly when I am at 165 to 169, I again have all 4 possibilities
and the best value would be .836 with X3 star = 4 and I am from 170 to 220, I realize that
the 0.855 X star = 4. Now we have not shown the individual calculations in this table
but we have shown the individual calculations in the other table and one need to go back
and see that final particular range and all the 4 decisions are possible in this case.
Now depending on the decision you have to see the money that you have spent here and
then you carry the balance money to the previous table and find out the best result. We will
just right down the recursive relation for these 2 for the second one right now. So I
have 2 more stages to go. So if I have s Rs which is my state variable and decide to buy
X3 units of this, I will have R of X3 + or multiplied by f1 star of s - C of X3. For
example just to calculate say, some something like this if I have 160 with me and say I
decide to buy 4 here, then I have 160 for 4. I spend 90 and get a .95 here. I spend
90. I have a balance of 70. This 70 would give me another .85 which is from .8075. So
like this we actually complete this table. The most important thing here is that we have
s - C of X. I start with s, I spent some C which is the cost. The balance I carry to
the previous stage and put f1 star of s - C X3. So we have right now completed 2 stages.
There are still 2 more stages in this problem. We will look at the other 2 stages in the
next lecture.
