Upcoming large astronomical surveys are expected to capture an unprecedented number of strong gravitational lensing systems in the Universe. Deep learning is emerging as a promising practical tool in detection and quantification of these galaxy-scale image distortions. However, absence of large quantities of representative data from current astronomical surveys requires development of robust forward modeling of synthetic lensing images. Using a realistic and unbiased sample of the strong lenses created by using state-of-the-art extragalactic catalogs, we train a modular deep learning pipeline for uncertainty-quantified detection and modeling with intermediate image processing components for denoising and deblending the lensing systems. We demonstrate a higher degree of interpretability and controlled systematics due to domain-specific task modules that are trained with different stages of synthetic image generation. For lens detection and modeling, we obtain semantically meaningful latent spaces that separate classes and provide uncertainty estimates that explain the misclassified images and provide uncertainty bounds on the lens parameters. In addition, we obtain an improved performance---lens detection (classification) improved from 82% with the baseline to 94%, while the lens modeling (regression) accuracy improved by 25% over the baseline model.