The NICER Collaboration recently reported the measurement of the mass and radius of a pulsar PSR J0030+0451. We here use this new measurement to constrain one of the higher-order nuclear matter parameters $K_{\mathrm{sym,0}}$. We further combine the tidal measurement of the binary neutron star merger GW170817 by LIGO/Virgo to derive a joint 1-$\sigma$ constraint as $K_{\mathrm{sym,0}} = -102^{+71}_{-72}$ MeV. We believe this is the most reliable bound on the parameter to date under the assumption that there is no new physics above the saturation density which impacts neutron star observations.