We report the discovery of two ultra-faint stellar systems found in early data from the DECam Local Volume Exploration survey (DELVE). The first system, Centaurus I (DELVE J1238-4054), is identified as a resolved overdensity of old and metal-poor stars with a heliocentric distance of ${\rm D}_{\odot} = 116.3_{-0.6}^{+0.6}$ kpc, a half-light radius of $r_h = 2.3_{-0.3}^{+0.4}$ arcmin, an age of $\tau > 12.85$ Gyr, a metallicity of $Z = 0.0002_{-0.0002}^{+0.0001}$, and an absolute magnitude of $M_V = -5.55_{-0.11}^{+0.11}$ mag. This characterization is consistent with the population of ultra-faint satellites, and confirmation of this system would make Centaurus I one of the brightest recently discovered ultra-faint dwarf galaxies. Centaurus I is detected in Gaia DR2 with a clear and distinct proper motion signal, confirming that it is a real association of stars distinct from the Milky Way foreground; this is further supported by the clustering of blue horizontal branch stars near the centroid of the system. The second system, DELVE 1 (DELVE J1630-0058), is identified as a resolved overdensity of stars with a heliocentric distance of ${\rm D}_{\odot} = 19.0_{-0.6}^{+0.5} kpc$, a half-light radius of $r_h = 0.97_{-0.17}^{+0.24}$ arcmin, an age of $\tau = 12.5_{-0.7}^{+1.0}$ Gyr, a metallicity of $Z = 0.0005_{-0.0001}^{+0.0002}$, and an absolute magnitude of $M_V = -0.2_{-0.6}^{+0.8}$ mag, consistent with the known population of faint halo star clusters. Given the low number of probable member stars at magnitudes accessible with Gaia DR2, a proper motion signal for DELVE 1 is only marginally detected. We compare the spatial position and proper motion of both Centaurus I and DELVE 1 with simulations of the accreted satellite population of the Large Magellanic Cloud (LMC) and find that neither is likely to be associated with the LMC.