Advances in GNC, particularly from miniaturized control electronics, reaction-wheels and attitude determination sensors make it possible to design surface probes and small robots to perform surface exploration and science on low-gravity environments. These robots would use their reaction wheels to roll, hop and tumble over rugged surfaces. These robots could provide 'Google Streetview' quality images of off-world surfaces and perform some unique science using penetrometers. These systems can be powered by high-efficiency fuel cells that operate at 60-65 % and utilize hydrogen and oxygen electrolyzed from water. However, one of the major challenges that prevent these probes and robots from performing long duration surface exploration and science is thermal design and control. In the inner solar system, during the day time, there is often enough solar-insolation to keep these robots warm and power these devices, but during eclipse the temperatures falls well below storage temperature. We have developed a thermal control system that utilizes chemicals to store and dispense heat when needed. The system takes waste products, such as water from these robots and transfers them to a thermochemical storage system. These thermochemical storage systems when mixed with water (a waste product from a PEM fuel cell) releases heat. Under eclipse, the heat from the thermochemical storage system is released to keep the probe warm enough to survive. In sunlight, solar photovoltaics are used to electrolyze the water and reheat the thermochemical storage system to release the water. Our research has showed thermochemical storage systems are a feasible solution for use on surface probes and robots for applications on the Moon, Mars and asteroids.