Dix was born in Boscawen, New Hampshire. He joined the Artillery as a military cadet at the age of 14. He served in the United States Army, having attained the rank of captain.

In 1826, Dix married Catherine Morgan, the adopted daughter of Congressman John J. Morgan, who gave Dix a job overseeing his upstate New York land holdings in Cooperstown. Dix and his wife moved to Cooperstown in 1828, and he practiced law in addition to overseeing the land holdings. He was appointed Adjutant General of New York State by Governor Throop, he moved to Albany, New York, in 1830, and served as Secretary of State of New York from 1833 to 1839.

Dix was elected as a Democrat to the United States Senate to fill the vacancy caused by the resignation of Silas Wright, Jr., and served from 1845 to 1849. He was not a candidate for reelection, having become a candidate for Governor of the Van Buren wing (Barnburner / Free - Soil) of the Democratic Party. He was an unsuccessful candidate for Governor in 1848 losing to Hamilton Fish in a three way race.

In 1853 Dix was president of the Mississippi and Missouri Railroad. He was appointed postmaster of the City of New York and served from 1860 to 1861.

In addition to his military and public duties, Dix was the president of the Union Pacific from 1863 to 1868 during construction of the First Transcontinental Railroad. He was the figurehead for rail baron Thomas C. Durant, in both of his railroad presidencies.

Dix was appointed United States Secretary of the Treasury by President James Buchanan in 1861. At the outbreak of the Civil War, he sent a telegram to the Treasury agents in New Orleans ordering that: "If any one attempts to haul down the American flag, shoot him on the spot." Although the telegram was intercepted by Confederates, and was never delivered to the Treasury agents, the text found its way to the press, and Dix became one of the first heroes of the North during the Civil War. The saying is found on many Civil War tokens minted during the war, although the wording is slightly modified.

At the start of the American Civil War, Dix was appointed a major general in the New York Militia. He joined the Union Army as the highest ranking major general of volunteers during the war, effective May 16, 1861. In the summer of 1861, he commanded the Department of Maryland and the Department of Pennsylvania. His importance at the beginning of the Civil War was in arresting and thereby preventing the Maryland legislature from meeting. This prevented Maryland from seceding, and earned him President Lincoln's gratitude. That winter, he commanded a regional organization known as "Dix's Command" within Maj. Gen. George B. McClellan's Department of the Potomac. Dix commanded the Department of Virginia from June 1862 until July 1863, and the Department of the East from July 1863 until April 1865.

On July 22, 1862, Dix and Confederate Maj. Gen. D.H. Hill concluded an agreement for the general exchange of prisoners between the Union and Confederate armies. This agreement became known as the Dix - Hill Cartel.

Considered too old for field command, Dix's most distinguished contribution during the war was the suppression of the New York Draft Riots in July 1863. He was also active in the defense of Suffolk, which was part of his department. He served as the temporary chairman of the 1866 National Union Convention.

He was the United States Minister to France from 1866 to 1869.

He served as the Governor of New York as a Republican in his seventies, as one of the oldest governors of New York, from January 1873 to December 1874; he was an unsuccessful candidate for reelection in November 1874. He suffered another defeat when he ran for the mayor of New York City in 1876.

Dix died in New York City and was interred in the Trinity Church Cemetery.

Fort Dix, New Jersey, a United States Army post, is named for Dix, as is Dix, Illinois, and several revenue cutters, John A. Dix.