The cold classical Kuiper belt objects have low inclinations and eccentricities and are the only Kuiper belt population suspected to have formed in situ. Compared with the dynamically excited populations, which exhibit a broad range of colours and a low binary fraction of ~10% cold classical Kuiper belt objects typically have red optical colours with ~30% of the population found in binary pairs; the origin of these differences remains unclear. We report the detection of a population of blue-coloured, tenuously bound binaries residing among the cold classical Kuiper belt objects. Here we show that widely separated binaries could have survived push-out into the cold classical region during the early phases of Neptune's migration. The blue binaries may be contaminants, originating at ~38 au, and could provide a unique probe of the formative conditions in a region now nearly devoid of objects. The idea that the blue objects, which are predominantly binary, are the products of push-out requires that the planetesimals formed entirely as multiples. Plausible formation routes include planetesimal formation via pebble accretion and subsequent binary production through dynamic friction and binary formation during the collapse of a cloud of solids.