We investigate the gravitational lens effect caused by a non-spherically distorted compact object. The non-spherical property of the gravitational potential is modeled by a quadrupole moment. Under the assumption that the quadrupole contribution is small, we solve perturbatively the lens equation and obtain the image positions and the amplification factors. We show that the separation angle of two major images is only slightly changed by the existence of the quadrupole contribution, whereas the difference of the amplification factors may be significantly modified. Our results indicate that even a tiny non-spherical distortion of the lens potential may cause significant amount of flux anomalies in the lensed images.