In this paper we present a counterfactual evaluation of the effect of ICT resources at school on student achievements conducted in Italy. In 2009 156 classes at 6th grade were endowed with additional resources earmarked for purchasing ICT equipments only. By selecting an equivalent number of classes in the same schools we are able to conduct an evaluation of the causal effect of ICT on student achievements, controlling for their initial level. Despite a significant financial investment (in the order of 1500 Euros per student over a three year period), results are very small: if we take the most encouraging results, the average improvement associated to the programme would be 3 test points, corresponding to 17% of a standard deviation. Even if it might be argued that our estimate represents a lower bound for the real effect, overall we conclude that the intervention has been far from being cost effective.