On 22 August, Abu Bakr, the first caliph, died, having made Umar his successor. Umar's first move was to relieve Khalid from command and appoint Abu Ubaidah ibn al-Jarrah as the new commander-in-chief of the Islamic army. Khalid pledged his loyalty to the new Caliph and continued to serve as an ordinary commander under Abu Ubaidah. He is reported to have said, "If Abu Bakr is dead and Umar is Caliph, then we listen and obey."

Abu Ubaidah moved more slowly and steadily, which had a concomitant effect on military operations in Syria. Abu Ubaidah, being an admirer of Khalid, made him commander of the cavalry and relied heavily on his advice during the whole campaign.