As an excellent artillery officer in the Virginia theater, Rosser drew the attention of Brig. Gen. J.E.B. Stuart, who recommended him for promotion to captain and then recruited him for the cavalry. Rosser joined the mounted branch in June 1862 and performed superbly as colonel of the Virginia 5th Cavalry. Although Stuart had acted as his patron, Rosser, eager for promotion to brigadier, blamed him in letters to his wife, Elizabeth “Betty” Winston Rosser, for not receiving advancement as soon as he thought he should. After Stuart died, he was succeeded by Maj. Gen. Wade Hampton. And again, Rosser — then in command of what he called the “Laurel Brigade” — blamed Hampton for his not being promoted to major general as quickly as he wanted.