The surface brightness - color relationship (SBCR) is a poweful tool for determining the angular diameter of stars from photometry. It was for instance used to derive the distance of eclipsing binaries in the Large Magellanic Cloud (LMC), which led to its distance determination with an accuracy of 1%. We calibrate the SBCR for red giant stars in the 2.1 < V-K < 2.5 color range using homogeneous VEGA/CHARA interferometric data secured in the visible domain, and compare it to the relation based on infrared interferometric observations, which were used to derive the distance to the LMC. Observations of eight G-K giants were obtained with the VEGA/CHARA instrument. The derived limb-darkened angular diameters were combined with a homogeneous set of infrared magnitudes in order to constrain the SBCR. The average precision we obtain on the limb-darkened angular diameters of the eight stars in our sample is 2.4%. For the four stars in common observed by both VEGA/CHARA and PIONIER/VLTI, we find a 1 sigma agreement for the angular diameters. The SBCR we obtain in the visible has a dispersion of 0.04 magnitude and is consistent with the one derived in the infrared (0.018 magnitude). The consistency of the infrared and visible angular diameters and SBCR reinforces the result of 1% precision and accuracy recently achieved on the distance of the LMC using the eclipsing-binary technique. It also indicates that it is possible to combine interferometric observations at different wavelengths when the SBCR is calibrated.