I study finite two player normal form games where player 2 (the ‘follower’) can observe the move of player 1 (the ‘leader’) by paying a small cost. I characterize the limit set of perfect equilibria of this game as the cost of information converges to zero and provide a simple algorithm for constructing it. Limit equilibria have the following properties: (a) both players choose pure strategies; (b) the follower plays a best response; (c) even though the set of limit equilibria always contains the Stackelberg equilibrium it can contain strategy profiles which are not even Nash equilibria of the normal form game. In fact, the follower will only purchase information in the Stackelberg equilibrium if the Stackelberg equilibrium is not a Nash equilibrium. Similar to Yariv and Solan (2004), the subgame perfect solution concept is therefore not robust to the introduction of small information costs. The Stackelberg equilibrium only reemerges as the unique limit equilibrium if we allow for the possibility that the leader is irrational. I test the theory experimentally using the classic Battle of the Sexes game with information acquisition. I find that subjects coordinate on the Stackelberg equilibrium and purchase information which can only be reconciled with theoretical predictions if it is common knowledge amongst subjects that the leader can be irrational.