Age-related differences in colon and rectal cancer survival have been observed, even after accounting for differences in background mortality. To determine how stage, tumour site, and histology contribute to these differences, we extracted age-specific one-year relative survival ratio (RS) stratified by these factors. We used colon and rectal cancer cases diagnosed between 2012 and 2016 from 18 United States Surveillance Epidemiology and End Results cancer registries. For colon cancer, 1-year RS ranged from 87.8 % [95 % Confidence Interval: 87.5–88.2] in the 50–64-year-olds to 62.3 % [61.3–63.3] in 85–99-year-olds and for rectal cancer ranged from 92.3 % [91.8–92.7] to 65.0 % [62.3–67.5]. With respect to stage, absolute differences in RS between 50-64-year-olds and 75–84-year-olds increased with increasing stage (from 6 [5–7] %-points in localised disease to 27 [25–29] %-points in distant disease) and were the highest for cancers of unknown stage (> 28 %-points). Age-related differences in survival were smallest for persons with tumours in the right-sided colon (8 [7–9] %-points) and largest for tumours of the colon without tumour site further specified (25 [21–29] %-points). With respect to histology, differences ranged from 7.4 % to 10.6 %-points for cancers with one of the three primary histologies (adenocarcinoma, mucinous adenocarcinoma, signet ring cell carcinoma) and were several-fold higher (42 %-points) for those with unknown/other histology (< 6 % of cases). Because age-related differences in survival were observed for all histologies and tumour sites, RS differences are unlikely to be driven by differences in the distribution of these factors by age. Differences in stage distribution by age are likely to contribute toward age-related differences in survival. Within stage groups, age differences in survival could be explained by frailty and/or therapy. Future studies incorporating data on treatment and geriatric conditions including frailty and comorbidity would support further understanding of the age gap in colon and rectal cancer survival.