Abstract: We impose an S-3 symmetry on the quark fields under which two of three quarks transform like a doublet and the remaining one as singlet, and use a scalar sector with the same structure of SU(2) doublets. After gauge symmetry breaking, a Z(2) subgroup of the S-3 remains unbroken. We show that this unbroken subgroup can explain the approximate block structure of the CKM matrix. By allowing soft breaking of the S-3 symmetry in the scalar sector, we show that one can generate the small elements, of quadratic or higher order in the Wolfenstein parametrization of the CKM matrix. We also predict the existence of exotic new scalars, with unconventional decay properties, which can be used to test our model experimentally.