We investigate in details the properties of the disk surrounding the 8 Myr old star HR 4796A, one of the few stars bringing precious clues to better understand the scenario which lead to planetary system formation. We propose a model able to reproduce all the available observations : the full spectral energy distribution from the mid-infrared to the millimeter wavelengths, resolved scattered light and thermal emission observations. We show that the circumstellar matter splits into two dust components : a cold annulus, peaked at 70 AU from the star, made of ISM-like grains (amorphous composition, porosity ~ 0.6) larger than 10 microns and a population of hot dust close to the star (at about 9 AU) made of comet-like grains (crystalline composition, porosity ~ 0.97). Both dust populations are highly collisional and the grain size distribution in the cold annulus is found to be cut-off by radiation pressure. At 70 AU, bodies as large as a few meters are required to fit the data leading to a minimum disk mass of a few Earth masses and to a gas to dust ratio less than 1. We discuss aftewards some implications on the disk structure and effects of larger bodies.