We aim to understand the role and evolution of beliefs in the indefinitely repeated prisoner's dilemma (IRPD). To do so, we elicit beliefs about the supergame strategies chosen by others. We find that heterogeneity in beliefs and changes in beliefs with experience are central to understanding behavior and learning in the IRPD. Beliefs strongly predict cooperation, initial beliefs match behavior quite well, most subjects choose strategies that perform well given their beliefs, and beliefs respond to experience while becoming more accurate over time. Finally, we uncover a novel mechanism whereby trusting subjects learn to cooperate through their interaction with experience.