Concept state transition

State transition

Taking an action $a_2$, the agent move from one state to another.

$$s_1\stackrel{a_2}{⟶}{s}_2$$

• if we take action $a_1$, then $s_1\stackrel{a_1}{⟶}{s}_1$

For forbidden area, at state $s_5$, if take action $a_2$, maybe 2 case here:

• case 1: accessible but with penalty, $s_5\stackrel{a_2}{⟶}{s}_6$
• case 2: inaccessible, $s_5\stackrel{a_2}{⟶}{s}_5$

Consider the first case later, which is more general and challenging.

We can use tabular representation to describe state transition

Use class dict in python maybe convenient to describe it, but this can only represent determinisitic cases.

• Intuition: At state $s_1$, if we choose action $a_2$, the next state is $s_2$
• Math: $$\begin{array}{rl}p(s_2\mid s_1,a_2)& =1\\ p(s_i\mid s_1,a_2)& =0,\forall i\ne 2\end{array}$$
• the case is determinisitic, the state transition could be stochastic
• also we can use dict to describe it in python