By Monday, May 4, six corps of the Army of the Potomac (I, II, III, V, XI, XII) had formed a strong defensive line in the shape of a horseshoe, with its right and left wings anchored on the river to protect the bridges at U. S. Ford and the apex at the Bullock Farm. A total of about 75,000 men were entrenched, and hour by hour they improved and strengthened their defenses in anticipation of an all out attack by Lee’s men that might be the most decisive battle of the war. Surely the men in blue recognized the unique position they were in—within stout breastworks on enemy soil with the enemy showing every sign of preparing to attack—and surely many still had thoughts of complete victory. Perhaps with one more great battle today, or perhaps tomorrow, we can shatter Lee’s army and end the war. (Click here for a battle map.)
But Maj. Gen. Hooker had other plans. Already, he had set army engineers to rapidly clearing a new road back to U. S. Ford, a road not for ready the ready resupply of his army, but a road to make the way withdrawal and escape easier. Late that night, and into the early hours of May 5, Hooker met with all of the corps commanders. Five of the six generals urged Hooker to stay and fight it out at Chancellorsville. Only Maj. Gen. Sickles thought retreat the better option. It was his corps that had seen the heaviest fighting on May 3.
It seems the meeting was but a formality. Hooker had already decided to withdraw to the north bank of the Rappahannock, citing as his justification his general mandate to protect Washington and not endanger the army. Orders were given, and late in the day on Tuesday, May 5, under the cover of heavy rain and darkness, units of the Army of the Potomac began to cross the pontoon bridges at U. S. Ford. Fittingly, Hooker went with the artillery and was among the first to cross. During the night, the rain-swollen Rappahannock rose quickly, threatening to sweep away the bridges. Engineers worked heroically in peril of their own lives to lengthen the bridges and anchor them on higher ground, and by early morning on May 6, the entire Army of the Potomac was north of the Rappahannock and filing back into the dreary winter camps around Falmouth and Stafford.
Author’s Commentary: In my opinion, Hooker lost one of the great opportunities of the war. Gen. Lee was indeed planning to attack the Union entrenchments at Chancellorsville, most likely on May 6, without Longstreet, and outnumbered by more than two-to-one. If not defeated outright, Lee would have suffered many casualties, perhaps extremely so, if Hooker had fought his army as he fought his corps at Antietam. I believe the Army of Northern Virginia would have been in such sad shape that offensive operations to the north would have been unthinkable and the Battle of Gettysburg never would have occurred.