We analyze the influence of fractal structure of the interstellar matter (ISM) density on the parameter values for the radio surface brightness to diameter ($\Sigma-D$) relation for supernovae remnants (SNRs). We model a dense ISM as a molecular cloud with fractal density structure. SNRs are modelled as spheres of different radius scattered in the modelled ISM. The surface brightness of the SNRs is calculated from the simple relation $\Sigma \propto \rho^{0.5}D^{-3.5}$ and also from the parametrized more general form $\Sigma \propto \rho^{\eta}D^{-\beta_0}$. Our results demonstrate that empirical $\Sigma-D$ slopes that are steeper than the ones derived from theory, might be partly explained with the fractal structure of the ambient medium into which SNRs expand. The slope of the $\Sigma-D$ relation steepens if the density of the regions where SNRs are formed is higher. The simple geometrical effects combined with the fractal structure of the ISM can contribute to a steeper empirical $\Sigma-D$ slopes, especially for older remnants, and this is more pronounced if $\Sigma$ has a stronger dependence on ambient density.