An increasing number of countries are introducing gender quotas in scientific committees. We analyze how a larger presence of female evaluators affects committee decision-making using information on 100,000 applications to associate and full professorships in all academic disciplines in two countries, Italy and Spain. These applications were assessed by 8,000 evaluators who were selected through a random draw. A larger number of women in evaluation committees does not increase either the quantity or the quality of female candidates who qualify. If anything, when evaluators' are not familiar with candidates' research area, gender-mixed committees tend to be less favorable towards female candidates than all-male committees, with the exception of evaluations to full professorships in Spain. Data from 300,000 individual voting reports suggests that men become less favorable towards female candidates as soon as a woman joins the committee.