We explore the implications of the Standard Model effective field theory (SMEFT) with dimension-six terms involving the Higgs boson and third-generation fermion fields on the rate of Higgs boson production and decay into fermions, on the electric dipole moments (EDMs) of the electron, and on the baryon asymmetry of the Universe. We study the consequences of allowing these additional terms for each flavor separately and for combinations of two flavors. We find that a complex $\tau$ Yukawa coupling can account for the observed baryon asymmetry $Y_B^{\rm obs}$ within current LHC and EDM bounds. A complex $b$ ($t$) Yukawa coupling can account for $4\%$ ($2\%$) of $Y_B^{\rm obs}$, whereas a combination of the two can reach $12\%$. Combining $\tau$ with either $t$ or $b$ enlarges the viable parameter space owing to cancellations in the EDM and in either Higgs production times decay or the total Higgs width, respectively. Interestingly, in such a scenario there exists a region in parameter space where the SMEFT contributions to the electron EDM cancel and collider signal strengths are precisely SM-like, while producing sufficient baryon asymmetry. Measuring $CP$ violation in Higgs decays to $\tau$ leptons is the smoking gun for this scenario.