The discoveries of the radio source Centaurus A and its optical counterpart NGC 5128 were important landmarks in the history of Australian astronomy. NGC 5128 was first observed in August 1826 by James Dunlop during a survey of southern objects at the Parramatta Observatory, west of the settlement at Sydney Cove. The observatory had been founded a few years earlier by Thomas Brisbane, the new governor of the British colony of New South Wales. Just over 120 years later, John Bolton, Gordon Stanley and Bruce Slee discovered the radio source Centaurus A at the Dover Heights field station in Sydney, operated by CSIRO's Radiophysics Laboratory (the forerunner of the Australia Telescope National Facility). This paper will describe this early historical work and summarise further studies of Centaurus A by other Radiophysics groups up to 1960.