For space observatories, the glitches caused by high energy phonons created by the interaction of cosmic ray particles with the detector substrate lead to dead time during observation. Mitigating the impact of cosmic rays is therefore an important requirement for detectors to be used in future space missions. In order to investigate possible solutions, we carry out a systematic study by testing four large arrays of Microwave Kinetic Inductance Detectors (MKIDs), each consisting of $\sim$960 pixels and fabricated on monolithic 55 mm $\times$ 55 mm $\times$ 0.35 mm Si substrates. We compare the response to cosmic ray interactions in our laboratory for different detector arrays: A standard array with only the MKID array as reference; an array with a low $T_c$ superconducting film as phonon absorber on the opposite side of the substrate; and arrays with MKIDs on membranes. The idea is that the low $T_c$ layer down-converts the phonon energy to values below the pair breaking threshold of the MKIDs, and the membranes isolate the sensitive part of the MKIDs from phonons created in the substrate. We find that the dead time can be reduced up to a factor of 40 when compared to the reference array. Simulations show that the dead time can be reduced to below 1 % for the tested detector arrays when operated in a spacecraft in an L2 or a similar far-Earth orbit. The technique described here is also applicable and important for large superconducting qubit arrays for future quantum computers.