Despite its importance for understanding the nature of early stellar generations and for constraining Galactic bulge formation models, at present little is known about the metal-poor stellar content of the central Milky Way. This is a consequence of the great distances involved and intervening dust obscuration, which challenge optical studies. However, the Apache Point Observatory Galactic Evolution Experiment (APOGEE), a wide-area, multifiber, high-resolution spectroscopic survey within Sloan Digital Sky Survey III (SDSS-III), is exploring the chemistry of all Galactic stellar populations at infrared wavelengths, with particular emphasis on the disk and the bulge. An automated spectral analysis of data on 2,403 giant stars in twelve fields in the bulge obtained during APOGEE commissioning yielded five stars with low metallicity([Fe/H]$\le-1.7$), including two that are very metal-poor [Fe/H]$\sim-2.1$ by bulge standards. Luminosity-based distance estimates place the five stars within the outer bulge, where other 1,246 of the analyzed stars may reside. A manual reanalysis of the spectra verifies the low metallicities, and finds these stars to be enhanced in the $\alpha$-elements O, Mg, and Si without significant $\alpha$-pattern differences with other local halo or metal-weak thick-disk stars of similar metallicity, or even with other more metal-rich bulge stars. While neither the kinematics nor chemistry of these stars can yet definitively determine which, if any, are truly bulge members, rather than denizens of other populations co-located with the bulge, the newly-identified stars reveal that the chemistry of metal-poor stars in the central Galaxy resembles that of metal-weak thick-disk stars at similar metallicity.