In any imaging survey, measuring accurately the astronomical background light is crucial to obtain good photometry. This paper introduces BKGnet, a deep neural network to predict the background and its associated error. BKGnet has been developed for data from the Physics of the Accelerating Universe Survey (PAUS), an imaging survey using a 40 narrow-band filter camera (PAUCam). Images obtained with PAUCam are affected by scattered light: an optical effect consisting of light multiply that deposits energy in specific detector regions contaminating the science measurements. Fortunately, scattered light is not a random effect, but it can be predicted and corrected for. We have found that BKGnet background predictions are very robust to distorting effects, while still being statistically accurate. On average, the use of BKGnet improves the photometric flux measurements by 7% and up to 20% at the bright end. BKGnet also removes a systematic trend in the background error estimation with magnitude in the i-band that is present with the current PAU data management method. With BKGnet, we reduce the photometric redshift outlier rate