In this work, we investigate the role of emission by dust at infrared wavelengths in the absorption of gamma radiation from distant extragalactic sources. We use an existing EBL model based on direct starlight emission at UV/visible and secondary radiation due to dust (PAHs (polycyclic aromatic hydrocarbons), small and large grains) at IR due to partial absorption of the stellar component. The relative contribution of each grain type to the total EBL energy density was determined from a combined fit to the Markarian 501 ($z\sim 0.034$) SED in flare state, where both the parameters of the intrinsic source spectrum and the dust fractions were allowed to vary. By separating the attenuation due to each EBL component, the importance of individual grain types to the opacity of the extragalactic medium for the TeV emission of a blazar like Markarian 501 could be better understood. Using a nested log-likelihood ratio test, we compared null hypotheses represented by effective 1- and 2-grain models against a 3-grain alternative scenario. When the temperatures of the grains are fixed a priori, the 1-grain scenario with only PAHs can be excluded at more than 5$\sigma$, irrespective of the curvature in the intrinsic spectrum. The 3-grain EBL model with the tuned fractions was finally used to fit the SEDs of a sample of extragalactic gamma-ray sources. Such a sample is still dominated by starlight attenuation, therefore, no statistically significant improvement in the quality of fits was observed when the tuned fractions are used to account for the EBL attenuation and the intrinsic spectrum parameters were allowed to vary during the fit. The potential of this kind of analysis when the next generation of IACTs, represented by the Cherenkov Telescope Array, starts observations is enormous. The newly discovered AGNs at a broad range of redshifts should break many of the degeneracies currently observed.