We have performed a systematic study of Gamma-Ray Bursts (GRBs), which have various values in the peak energy of the ${\nu}F_{\nu}$ spectrum of the prompt emission, $E_{{\rm peak}}$, observed by \textsl{Swift}/BAT and \textsl{Fermi}/GBM, investigating their prompt and X-ray afterglow emissions. We cataloged the long-lasting GRBs observed by the \textsl{Swift} between 2004 December and 2014 February in 3 categories according to the classification by \citet{2008ApJ...679..570S}: X-Ray Flashes (XRFs), X-Ray Rich GRBs (XRRs), and Classical GRBs (C-GRBs). We then derived $E^{{\rm obs}}_{{\rm peak}}$, as well as $E^{{\rm src}}_{{\rm peak}}$ if viable, of the \textsl{Swift} spectra of their prompt emission. We also analyzed their X-Ray afterglows and found the trend that the GRB events with a lower $E_{{\rm peak}}^{{\rm src}}$, i.e. softer GRBs, are fainter in the 0.3--10 keV X-ray luminosity and decay more slowly than harder GRBs. The intrinsic event rates of the XRFs, XRRs, and C-GRBs were calculated, using the \textsl{Swift}/BAT trigger algorithm. That of either of the XRRs and XRFs is larger than that of the C-GRBs. If we assume that the observational diversity of $E_{{\rm peak}}$ is explained with the off-axis model \citep{2002ApJ...571L..31Y,2004ApJ...607L..103Y}, these results yield the jet half-opening angle of $\Delta\theta\sim 0.3^\circ$, and the variance of the observing angles $\theta_{{\rm obs}} \lesssim0.6^{\circ}$. This implies that the tiny variance of the observing angles of $\lesssim0.6^{\circ}$ would be responsible for the $E_{{\rm peak}}$ diversity observed by \textsl{Swift}/BAT, which is unrealistic. Therefore, we conclude that the $E_{{\rm peak}}$ diversity is not explained with the off-axis model, but is likely to originate from some intrinsic properties of the jets.