•States: S1, … SN
•Discrete time steps, 1, 2, …
•State at time t is q(t).
•Initial state, q(1).
pi(i) = P(q(1) = Si).
•P(q(t) = Sj | q(t-1)= Si, q(t-2)=Sk, … )
–= P(q(t) = Sj | q(t-1) = Si).
–This is what makes it Markov.
•Time independence:
–a(ij) = P(q(t) = Sj | q(t-1)
= Si).
•
•