We design a double-or-quits game to compare the speed of learning one’s specific ability with the speed of rising confidence as the task gets increasingly difficult. We find that people on average learn to be overconfident faster than they learn their true ability and we present a simple Bayesian model of confidence which integrates these facts. We show that limited discrimination of objective differences, myopia, and uncertainty about one’s true ability to perform a task in isolation can be responsible for large and robust confidence biases, namely the hard-easy effect, the Dunning-Kruger effect, conservative learning from experience and the overprecision phenomenon (without underprecision) if subjects act as Bayesian learners. Moreover, these biases are likely to persist since the Bayesian aggregation of past information consolidates the accumulation of errors and the perception of contrarian illusory signals generates conservatism and under-reaction to events. Taken together, these two features may explain why intuitive Bayesians make systematically wrong predictions of their own performance.