Context. There is no consensus on the amplitude of the historical solar forcing. The estimated magnitude of the total solar irradiance difference between Maunder minimum and present time ranges from 0.1 to 6 W/m2 making uncertain the simulation of the past and future climate. One reason for this disagreement is the applied evolution of the quiet Sun brightness in the solar irradiance reconstruction models. This work addresses the role of the quiet Sun model choice and updated solar magnetic activity proxies on the solar forcing reconstruction. Aims. We aim to establish a plausible range of the solar irradiance variability on decadal to millennial time scales. Methods. The spectral solar irradiance (SSI) is calculated as a weighted sum of the contributions from sunspot umbra/penumbra, fac- ulae and quiet Sun, which are pre-calculated with the spectral synthesis code NESSY. We introduce activity belts of the contributions from sunspots and faculae and a new structure model for the quietest state of the Sun. We assume that the brightness of the quiet Sun varies in time proportionally to the secular (22-year smoothed) variation of the solar modulation potential. Results. A new reconstruction of the TSI and SSI covering the period 6000 BCE - 2015 CE is presented. The model simulates solar irradiance variability during the satellite era well. The TSI change between the Maunder and recent minima ranges between 3.7 and 4.5 W/m2 depending on the applied solar modulation potential. The implementation of a new quietest Sun model reduces, by approximately a factor of two, the relative solar forcing compared to the largest previous estimation, while the application of updated solar modulation potential increases the forcing difference between Maunder minimum and the present by 25-40 %.