Indigenous people tend to live in different parts of Australian towns and cities than the non-Indigenous population. This is due to a combination of historic and contemporary government policies, the agency of Indigenous people, and the constraints placed on residential location by the interaction of the housing and labour markets. This study traces the trajectory of Indigenous residential segregation in 60 Australian towns and cities, using census data from 1976 to 2016. Segregation is measured using the index of dissimilarity and the threshold method. Indigenous residential segregation has been declining steadily since 1976 nationally. However, there has been a great deal of variation in segregation trajectories among towns and cities. In Sydney and Melbourne, segregation remained relatively high over the study period. The level of segregation in 1976 appears to be related to the geographical remoteness of the town, with remote towns generally having lower levels of segregation in 1976. Segregation has been decreasing most rapidly in regional towns in New South Wales and Queensland. Finally, this study has found a long-run increase in the proportion of Indigenous residents living in highly Indigenous neighbourhoods, consistent with the increasingly close settlement of Indigenous people in Australian towns and cities. This trend is at odds with the apparent decrease in segregation found when segregation is measured using the index of dissimilarity. Detailed case studies may be required that examine how concrete historical geographies and policy legacies combine with contemporary housing markets to produce the configuration of segregation that we see today.