According to the scientists, who published their work in Nature, experimenting with solving video games that require complex, hard exploration gives way to better learning algorithms. For instance, a slight change in the pattern of the clouds in the sky in a game environment is probably unimportant when exploring said game. Let’s say at first it goes left, it explores left, meaning that it gets this intrinsic reward for a while. It doesn’t quite finish exploring left, and at some point, the episode ends, and it starts anew in the starting room. But then, of course, you want your robot to run in the real world, right?