Abstract: We consider the possibility that dark energy and baryons might scatter off each other. The type of interaction we consider leads to a pure momentum exchange, and does not affect the background evolution of the expansion history. We parametrize this interaction in an effective way at the level of Boltzmann equations. We compute the effect of dark energy-baryon scattering on cosmological observables, focusing on the cosmic microwave background (CMB) temperature anisotropy power spectrum and the matter power spectrum. Surprisingly, we find that even huge dark energy-baryon cross-sections sigma(xb) similar to O(b), which are generically excluded by non-cosmological probes such as collider searches or precision gravity tests, only leave an insignificant imprint on the observables considered. In the case of the CMB temperature power spectrum, the only imprint consists in a sub-per cent enhancement or depletion of power (depending whether or not the dark energy equation of state lies above or below -1) at very low multipoles, which is thus swamped by cosmic variance. These effects are explained in terms of differences in how gravitational potentials decay in the presence of a dark energy-baryon scattering, which ultimately lead to an increase or decrease in the late-time integrated Sachs-Wolfe power. Even smaller related effects are imprinted on the matter power spectrum. The imprints on the CMB are not expected to be degenerate with the effects due to altering the dark energy sound speed. We conclude that, while strongly appealing, the prospects for a direct detection of dark energy through cosmology do not seem feasible when considering realistic dark energy-baryon cross-sections. As a caveat, our results hold to linear order in perturbation theory.