We discuss the origin of the optical variations in the Narrow line Seyfert 1 galaxy NGC 4051 and present the results of a cross-correlation study using X-ray and optical light curves spanning more than 12 years. The emission is highly variable in all wavebands, and the amplitude of the optical variations is found to be smaller than that of the X-rays, even after correcting for the contaminating host galaxy flux falling inside the photometric aperture. The optical power spectrum is best described by an unbroken power law model with slope $\alpha=1.4^{+0.6}_{-0.2}$ and displays lower variability power than the 2-10 keV X-rays on all time-scales probed. We find the light curves to be significantly correlated at an optical delay of $1.2^{+1.0}_{-0.3}$ days behind the X-rays. This time-scale is consistent with the light travel time to the optical emitting region of the accretion disc, suggesting that the optical variations are driven by X-ray reprocessing. We show, however, that a model whereby the optical variations arise from reprocessing by a flat accretion disc cannot account for all the optical variability. There is also a second significant peak in the cross-correlation function, at an optical delay of $39^{+2.7}_{-8.4}$ days. The lag is consistent with the dust sublimation radius in this source, suggesting that there is a measurable amount of optical flux coming from the dust torus. We discuss the origin of the additional optical flux in terms of reprocessing of X-rays and reflection of optical light by the dust.