Experimental observations have shown that the transition characteristics of the boundary-layer flow over rotating cones depends on the cone half-angle. In particular, pairs of counter-rotating Görtler-type vortices are observed over cones with slender half-angles and co-rotating vortices are observed over broad cones. Garrett et al (2009 J. Fluid Mech. 622 209–32) have hypothesized the existence of a centrifugal instability mode over slender cones that is more dangerous than the types I (crossflow) and II (streamline curvature) modes which dominate over rotating disks and broad cones. Work is currently underway to clarify this alternative mode; however, a clear understanding of the growth rates of types I and II modes is crucial to the ultimate understanding of how the dominant mode changes with half-angle. In this paper, we demonstrate that the maximum growth rate for types I and II modes decreases with reduced half-angle, which clears the way for the dominance of the alternative instability mode. Furthermore, it is suggested that vortices travelling at 75% of the cone surface speed will be selected over smooth, clean rotating cones with half-angle such that the type I mode is dominant. Interestingly, this vortex speed has been experimentally observed by Kobayashi and Arai within the rotating-sphere boundary layer.