Abstract: It is well known that the source-free Maxwell equations are invariant under electric-magnetic duality rotations, F -> F cos theta +*F sin theta. These transformations are indeed a symmetry of the theory in the Noether sense. The associated constant of motion is the difference in the intensity between self-dual and anti-self-dual components of the electromagnetic field or, equivalently, the difference between the right and left circularly polarized components. This conservation law holds even if the electromagnetic field interacts with an arbitrary classical gravitational background. After reexamining these results, we discuss whether this symmetry is maintained when the electromagnetic field is quantized. The answer is in the affirmative in the absence of gravity but not necessarily otherwise. As a consequence, the net polarization of the quantum electromagnetic field fails to be conserved in curved spacetimes. This is a quantum effect, and it can be understood as the generalization of the fermion chiral anomaly to fields of spin one.