Despite being in a potentially favorable situation, Hooker halted his brief offensive. His actions may have demonstrated his lack of confidence in handling the complex actions of such a large organization for the first time (he had been an effective and aggressive division and corps commander in previous battles), but he had also decided before beginning the campaign that he would fight the battle defensively, forcing Lee, with his small army, to attack his own, larger one. At the [First] Battle of Fredericksburg (December 13, 1862), the Union army had done the attacking and met with a bloody defeat.^[4]

Hooker knew Lee could not sustain such a defeat and keep an effective army in the field, so he ordered his men to withdraw back into the Wilderness and take a defensive position around Chancellorsville, daring Lee to attack him or retreat with superior forces at his back. He confused matters by issuing a second order to his subordinates to hold their positions until 5 p.m., but by the time it was received, most of the Union units had begun their rearward movements.

Hooker's subordinates were surprised and outraged by the change in plans. They saw that the position they were fighting for near the Zoan Church was relatively high ground and offered an opportunity for the infantry and artillery to deploy outside the constraints of the Wilderness. Meade exclaimed, "My God, if we can't hold the top of the hill, we certainly can't hold the bottom of it!" Viewing through the lens of hindsight, some of the participants and many modern historians judged that Hooker effectively lost the campaign on May 1. Stephen W. Sears observed, however, that Hooker's concern was based on more than personal timidity.^[4]