Firms in developing countries often avoid paying taxes by making informal payments to tax officials. These bribes may raise the cost of operating a business, and the price charged to consumers. To decrease these costs, we designed a feedback incentive scheme for business tax inspectors that rewards them according to the anonymous evaluation submitted by inspected firms. We show theoretically that feedback incentives decrease the equilibrium bribe amount, but make firms with more inelastic demand more attractive for inspectors. A tilted scheme that attaches higher weights to the evaluation of smaller firms limits the scope for targeting and decreases the bribe amount to a lesser extent. We evaluate both schemes in a field experiment in the Kyrgyz Republic and find evidence that is consistent with the model predictions. By decreasing bribes, our intervention reduces the average cost for firms and the price they charge to consumers. Since fewer firms substitute bribes for taxes, tax revenues increase. Our study highlights the role of firm heterogeneity and market structure in shaping the relationship between firms and tax inspectors, and provides clear evidence of pass-through of bribes to consumers.