The Galactic magnetar SGR1935+2154 has been reported to produce the first known example of a bright millisecond duration radio burst (FRB 200428) similar to the cosmological population of fast radio bursts (FRBs), bolstering the association of FRBs to active magnetars. The detection of a coincident bright X-ray burst has revealed the first observed multi-wavelength counterpart of a FRB. However, the search for similar emission at optical wavelengths has been hampered by the high inferred extinction on the line of sight. Here, we present results from the first search for second-timescale emission from the source at near-infrared wavelengths using the Palomar Gattini-IR observing system in J-band, made possible by a recently implemented detector read-out mode that allowed for short exposure times of 0.84 s with 99.9% observing efficiency. With a total observing time of 12 hours (47728 images) on source, we place median $3\,\sigma$ limits on the second-timescale emission of $< 20$ mJy (13.1 AB mag). We present non-detection limits from epochs of four simultaneous X-ray bursts detected by the Insight-{\it HXMT} and {\it NuSTAR} telescopes during our observing campaign. The limits translate to an extinction corrected fluence limit of $< 125$ Jy ms for an estimated extinction of $A_J = 2.0$ mag. These limits provide the most stringent constraints to date on the fluence of flares at frequencies of $\sim 10^{14}$ Hz, and constrain the ratio of the near-infrared (NIR) fluence to that of coincident X-ray bursts to $R_{\rm NIR} < 2.5 \times 10^{-2}$. Our observations were sensitive enough to easily detect a near-infrared counterpart of FRB 200428 if the NIR emission falls on the same power law as that observed across its radio to X-ray spectrum. The non-detection of NIR emission around the coincident X-ray bursts constrains the fluence index of the brightest burst to be steeper than $0.35$.