Tips for OT7 Worksheet PDF

Preview

Summary

Contains useful Maple tips for coding in Maple or in Maple TA...

Description

Markov Processes Before you attempt this worksheet, you should be familiar with the "Matrices" and "Systems of Linear Equations" worksheets. =========================================== Let's find the steady state solution for a Markov process. Consider the transition matrix T := < 0.03>

and initial state V0 := ;

Finding the steady state by iteration The next state V1 is found my multiplying V0 on the left by T. V1 := T.V0;

| | | | | | >;

The 10th and 11th states are V10 := (T^10).V0; V11 := T.V10;

and the 100th and 101th states are V100 := (T^100).V0; V101 := T.V100;

Notice that difference between V0 and V1 is quite large and the difference between V10 and V11 is comparitively small, but V100 and V101 only differ in the last decimal place. It seems that the more steps we take, the less the state changes from step to step. In fact, to 9 significant figures, V100 satisfies the equation T.V100 = V100 -- ie it is a steady state solution. Some times you might need to calculate V200 or even V1000 to find the steady state, but this approach will always work. Next, we'll find the steady state in the way that you have seen in lectures. Check that the steady state that we find at the end is the same as the one we found above. [Hint: the above method is probably the easiest way to answer the question in the test!] Finding the steady state by solving (T-1)V = 0 To find the steady state by hand you would solve the matrix equation (T-1)V=0. Let's try and do this with Maple - there are some pitfalls. We'll use the same transition matrix as above and use LinearSolve with(LinearAlgebra): LinearSolve(T-1, );

This is not right! What has gone wrong is that rounding errors have changed the system (T-1)V=0 from one with infinitely many solutions to one with a unique solution (the zero vector). To get around this problem, we can convert T to a matrix with exact rational

entries and try again. T := convert(T,rational);

SS := LinearSolve(T-1,);

This is all possible steady state solutions, but perhaps we only want the one that has entries summing to 1. Let's add up the entries in SS. add(SS[i], i=1..7);

We need this number to be 1. (The parameter is _t0[6], but this may vary, so modify the following if necessary.) solve(% = 1, _t0[6]);

5895866500 199040296531 evalf(subs(_t0[6]=%,SS));...