Abstract: The understanding of stellar structure represents the crossroads of our theories of the nuclear force and the gravitational interaction under the most extreme conditions observably accessible. It provides a powerful probe of the strong field regime of General Relativity, and opens fruitful avenues for the exploration of new gravitational physics. The latter can be captured via modified theories of gravity, which modify the Einstein-Hilbert action of General Relativity and/or some of its principles. These theories typically change the Tolman-Oppenheimer-Volkoff equations of stellar's hydrostatic equilibrium, thus having a large impact on the astrophysical properties of the corresponding stars and opening a new window to constrain these theories with present and future observations of different types of stars. For relativistic stars, such as neutron stars, the uncertainty on the equation of state of matter at supranuclear densities intertwines with the new parameters coming from the modified gravity side, providing a whole new phenomenology for the typical predictions of stellar structure models, such as mass-radius relations, maximum masses, or moment of inertia. For non-relativistic stars, such as white, brown and red dwarfs, the weakening/strengthening of the gravitational force inside astrophysical bodies via the modified Newtonian (Poisson) equation may induce changes on the star's mass, radius, central density or luminosity, having an impact, for instance, in the Chandrasekhar's limit for white dwarfs, or in the minimum mass for stable hydrogen burning in high-mass brown dwarfs. This work aims to provide a broad overview of the main such results achieved in the recent literature for many such modified theories of gravity, by combining the results and constraints obtained from the analysis of relativistic and non-relativistic stars in different scenarios. Moreover, we will build a bridge between the efforts of the community working on different theories, formulations, types of stars, theoretical modelings, and observational aspects, highlighting some of the most promising opportunities in the field.