The axisymmetric component of the large-scale solar magnetic fields has a pronounced poleward branch at higher latitudes. In order to clarify the origin of this branch we construct an axisymmetric model of the passive transport of the mean poloidal magnetic field in the convective zone, including meridional circulation, anisotropic diffusivity, turbulent pumping and density pumping. For realistic values of the transport coefficients we find that diffusivity is prevalent, and the latitudinal distribution of the field at the surface simply reflects the conditions at the bottom of the convective zone. Pumping effects concentrate the field to the bottom of the convective zone; a significant part of this pumping occurs in a shallow subsurface layer, normally not resolved in dynamo models. The phase delay of the surface poloidal field relative to the bottom poloidal field is found to be small. These results support the double dynamo wave models, may be compatible with some form of a mixed transport scenario, and exclude the passive transport theory for the origin of the polar branch.