Using an unbalanced panel of 23,007 academic records spanning from Spring 2019 to Spring 2022 representing one fourth of Queens College student population; and estimating event study analyses with individual fixed effects to control for time-invariant unobserved heterogeneity, we find unintended effects of the flexible grading policy (FGP), which allowed students to exercise the pass/fail option during the first academic year of the pandemic. Once the policy was no longer available, students who had used it underperformed relative to their own pre-pandemic performance relative to the change in performance of students who had never used the policy. FGP users earned 5.3% lower GPA in Spring 2021 and 4.7% lower GPA in Fall 2021 relative to Fall 2019 relative to the change observed among FGP non-users. This pattern is robust to sensitivity analysis and holds across tiers of the 2019 cumulative GPA distribution, as well as across various socio-demographic groups. Furthermore, these detrimental effects increased with the intensity of the policy use. Students' response to a survey rules out that these findings may be driven by pandemic-related health shocks, childcare disruptions, or challenges with online learning, financial aid, or job loss. We estimate that using the FGP is associated with a 16% lower likelihood of graduating and a 18% lower likelihood of graduating on-time by Spring 2022.