Abstract: We disprove the widespread belief that higher order curvature theories of gravity in the metric-affine formalism are generally ghost-free. This is clarified by considering a sub-class of theories constructed only with the Ricci tensor and showing that the non-projectively invariant sector propagates ghost-like degrees of freedom. We also explain how these pathologies can be avoided either by imposing a projective symmetry or additional constraints in the gravity sector. Our results put forward that higher order curvature gravity theories generally remain pathological in the metric-affine (and hybrid) formalisms and highlight the key importance of the projective symmetry and/or additional constraints for their physical viability and, by extension, of general metric-affine theories.