We examine ~10 years of photometric data and find that the black hole X-ray binary V4641 Sgr has two optical states, passive and active, during X-ray quiescence. The passive state is dominated by ellipsoidal variations and is stable in the shape and variability of the light curve. The active state is brighter and more variable. Emission during the active state varies over the course of the orbital period and is redder than the companion star. These optical/infrared states last for weeks or months. V4641 Sgr spends approximately 85% of X-ray quiescence in the passive state and 15% in the active. We analyze passive colors and spectroscopy of V4641 Sgr and show that they are consistent with a reddened B9III star (with E(B-V) = 0.37 +/- 0.19) with little or no contribution from the accretion disk. We use X-ray observations with an updated ephemeris to place an upper limit on the duration of an X-ray eclipse of <8.3 deg in phase (~1.6 hours). High resolution spectroscopy yields a greatly improved measurement of the rotational velocity of the companion star of V_rot_sin(i) = 100.9 +/- 0.8 km s^-1. We fit ellipsoidal models to the passive state data and find an inclination angle of i = 72.3 +/- 4.1 deg, a mass ratio of Q = 2.2 +/- 0.2, and component masses for the system of M_BH = 6.4 +/- 0.6 M_sun and M_2 = 2.9 +/- 0.4 M_sun. Using these values we calculate an updated distance to V4641 Sgr of 6.2 +/- 0.7 kpc.