We observed diffuse cosmic and atmospheric gamma rays at balloon altitudes with the Sub-MeV gamma-ray Imaging Loaded-on-balloon Experiment I (SMILE-I) as the first step toward a future all-sky survey with a high sensitivity. SMILE-I employed an electron-tracking Compton camera comprised of a gaseous electron tracker as a Compton-scattering target and a scintillation camera as an absorber. The balloon carrying the SMILE-I detector was launched from the Sanriku Balloon Center of the Institute of Space and Astronomical Science/Japan Space Exploration Agency on September 1, 2006, and the flight lasted for 6.8 hr, including level flight for 4.1 hr at an altitude of 32-35 km. During the level flight, we successfully detected 420 downward gamma rays between 100 keV and 1 MeV at zenith angles below 60 degrees. To obtain the flux of diffuse cosmic gamma rays, we first simulated their scattering in the atmosphere using Geant4, and for gamma rays detected at an atmospheric depth of 7.0 g cm-2, we found that 50% and 21% of the gamma rays at energies of 150 keV and 1 MeV, respectively, were scattered in the atmosphere prior to reaching the detector. Moreover, by using Geant4 simulations and the QinetiQ atmospheric radiation model, we estimated that the detected events consisted of diffuse cosmic and atmospheric gamma rays (79%), secondary photons produced in the instrument through the interaction between cosmic rays and materials surrounding the detector (19%), and other particles (2%). The obtained growth curve was comparable to Ling's model, and the fluxes of diffuse cosmic and atmospheric gamma rays were consistent with the results of previous experiments. The expected detection sensitivity of a future SMILE experiment measuring gamma rays between 150 keV and 20 MeV was estimated from our SMILE-I results and was found to be ten times better than that of other experiments at around 1 MeV.