Players cooperate in experiments more than game theory would predict. In order to explain this, we introduce the 'returns-based beliefs' approach: the expected returns of a particular strategy in proportion to the total expected returns of all strategies. Using a decision analytic solution concept, Luce's (1959) probabilistic choice model, and 'hyperpriors' for ambiguity in players' cooperability, our approach explains empirical observations in classic games such as the Prisoner's Dilemma. Testing the closeness of fit of our model on Selten and Chmura (2008) data for completely mixed 2x2 games shows that with loss aversion, returns-based beliefs explain the data better than other equilibrium concepts.