Recent detections of merging black holes allow observational tests of the nature of these objects. In some proposed models, non-trivial structure at or near the black hole horizon could lead to echo signals in gravitational wave data. Recently, Abedi et al. claimed tentative evidence for repeating damped echo signals following the gravitational-wave signals of the binary black hole merger events recorded in the first observational period of the Advanced LIGO interferometers. We reanalyse the same data, addressing some of the shortcomings of their method using more background data and a modified procedure. We find a reduced statistical significance for the claims of evidence for echoes, calculating increased p-values for the null hypothesis of echo-free noise. The reduced significance is entirely consistent with noise, and so we conclude that the analysis of Abedi et al. does not provide any observational evidence for the existence of Planck-scale structure at black hole horizons.