We report the discovery of an intermediate-mass transiting brown dwarf, TOI-503b, from the TESS mission. TOI-503b is the first brown dwarf discovered by TESS and orbits a metallic-line A-type star with a period of $P=3.6772 \pm 0.0001$ days. The light curve from TESS indicates that TOI-503b transits its host star in a grazing manner, which limits the precision with which we measure the brown dwarf's radius ($R_b = 1.34^{+0.26}_{-0.15} R_J$). We obtained high-resolution spectroscopic observations with the FIES, Ond\v{r}ejov, PARAS, Tautenburg, and TRES spectrographs and measured the mass of TOI-503b to be $M_b = 53.7 \pm 1.2 M_J$. The host star has a mass of $M_\star = 1.80 \pm 0.06 M_\odot$, a radius of $R_\star = 1.70 \pm 0.05 R_\odot$, an effective temperature of $T_{\rm eff} = 7650 \pm 160$K, and a relatively high metallicity of $0.61\pm 0.07$ dex. We used stellar isochrones to derive the age of the system to be $\sim$180 Myr, which places its age between that of RIK 72b (a $\sim$10 Myr old brown dwarf in the Upper Scorpius stellar association) and AD 3116b (a $\sim$600 Myr old brown dwarf in the Praesepe cluster). We argue that this brown dwarf formed in-situ, based on the young age of the system and the long circularization timescale for this brown dwarf around its host star. TOI-503b joins a growing number of known short-period, intermediate-mass brown dwarfs orbiting main sequence stars, and is the second such brown dwarf known to transit an A star, after HATS-70b. With the growth in the population in this regime, the driest region in the brown dwarf desert ($35-55 M_J \sin{i}$) is reforesting and its mass range shrinking.