Most of planet formation models that incorporate planetesimal fragmentation consider a catastrophic impact energy threshold for basalts at a constant velocity of 3 km/s during all the process of the formation of the planets. However, as planets grow the relative velocities of the surrounding planetesimals increase from velocities of the order of m/s to a few km/s. In addition, beyond the ice line where giant planets are formed, planetesimals are expected to be composed roughly by 50 percentage of ices. We aim to study the role of planetesimal fragmentation on giant planet formation considering planetesimal catastrophic impact energy threshold as a function of the planetesimal relative velocities and compositions. We improve our model of planetesimal fragmentation incorporating a functional form of the catastrophic impact energy threshold with the planetesimal relative velocities and compositions. We also improve in our model the accretion of small fragments produced by the fragmentation of planetesimals during the collisional cascade considering specific pebble accretion rates. We find that a more accurate and realistic model for the calculation of the catastrophic impact energy threshold tends to slow down the formation of massive cores. Only for reduced grain opacity values at the envelope of the planet, the cross-over mass is achieved before the disk time-scale dissipation. While planetesimal fragmentation favors the quick formation of massive cores of 5-10 Earth masses the cross-over mass could be inhibited by planetesimal fragmentation. However, grain opacity reduction or pollution by the accreted planetesimals together with planetesimal fragmentation could explain the formation of giant planets with low-mass cores.