High Time Resolution Astrophysics (HTRA) concerns itself with observations on short scales normally defined as being lower than the conventional read-out time of a CCD. As such it is concerned with condensed objects such as neutron stars, black holes and white dwarfs, surfaces with extreme magnetic reconnection phenomena, as well as with planetary scale objects through transits and occultations. HTRA is the only way to make a major step forward in our understanding of several important astrophysical and physical processes; these include the extreme gravity conditions around neutron stars and stable orbits around stellar mass black holes. Transits, involving fast timing, can give vital information on the size of, and satellites around exoplanets. In the realm of fundamental physics very interesting applications lie in the regime of ultra-high time resolution, where quantum-physical phenomena, currently studied in laboratory physics, may be explored. HTRA science covers the full gamut of observational optical/IR astronomy from asteroids to {\gamma}-rays bursts, contributing to four out of six of AstroNet's fundamental challenges described in their Science Vision for European Astronomy. Giving the European-Extremely Large Telescope (E-ELT) an HTRA capability is therefore importance. We suggest that there are three possibilities for HTRA and E-ELT. These are, firstly giving the E-ELT first light engineering camera an HTRA science capability. Secondly, to include a small HTRA instrument within another instrument. Finally, to have separate fibre feeds to a dedicated HTRA instrument. In this case a small number of fibres could be positioned and would provide a flexible and low cost means to have an HTRA capability. By the time of E-ELT first light, there should be a number of significant developments in fast detector arrays, in particular in the infra-red (IR) region.