Skew-Fit: State-Covering Self-Supervised Reinforcement Learning

Try to find subgoals by maximizing the # mutual information:

$$ I(S;G) = H(S) - H(S|G) = H(G) - H(G|S) $$

that is trying to maximizing $H(S)$ and minimizing $H(S|G)$.

To minimizing $H(S|G)$, we train a policy by maximizing the reward: $$ r = log(G|S) $$ which is also the distance between $G$ and $S$. This paper use # RIG to deal with it.

To maximizing $H(S)$, they add a weight on each state to make the probability of less visited state become bigger.