Roon was born at Pleushagen, near Kolberg (Kołobrzeg), in Pomerania. His family was of Flemish origin, and had settled in Pomerania. His father, an officer of the Prussian army, died in poverty during the French occupation, and the young Roon was brought up by his maternal grandmother in a country ravaged in the War of Liberation.

Roon entered the corps of cadets at Culm (Chełmno) in 1816, from where in 1818 he proceeded to the military school at Berlin, and in January 1821 received a commission in the 14th (3rd Pomeranian) regiment quartered at Stargard in Pomerania. In 1824 he went through the three years higher course of study at the General War School in Berlin (later called the Prussian Military Academy), where he improved his general education. In 1826 he was transferred to the 15th regiment at Minden. Also in 1826, he was appointed an instructor in the military cadet school at Berlin, where he devoted himself especially to the subject of military geography. He was a student of the noted geographer Carl Ritter who taught at the Berlin military school. In 1832, Roon published the well-known Principles of Physical, National and Political Geography, in three volumes (Grundlage der Erd-, Völker- und Staaten - Kunde), which gained him a great reputation, and of which over 40,000 copies were sold in a few years. This work was followed in 1834 by Elements of Geography (Anfangsgrunde der Erdkunde), in 1837 by Military Geography of Europe (Militärische Landerbeschreibung von Europa), and in 1839 by The Iberian Peninsula (Die Iberische Halbinsel).

In 1832, Roon rejoined his regiment, and was afterwards attached to the headquarters of General von Müffling's corps of observation at Krefeld, when he first became aware of the very inefficient state of the Prussian army. In 1833 he was appointed to the Topographical Bureau at Berlin, in 1835 he entered the General staff, and in the following year was promoted captain and became instructor and examiner in the military academy at Berlin. In 1842, after an illness of two years brought on by overwork, he was promoted to major and attached to the staff of the VII Corps, in which post he was again impressed with the inefficiency of the organization of the army, and occupied himself with schemes for its reform. Two years later, as tutor to Prince Friedrich Karl of Prussia, he attended him at Bonn University and in his European travels. In 1848 he was appointed chief of the staff of the VIII Corps at Koblenz. During the disturbances of that year he served under Prince William, later king and emperor, in the suppression of the insurrection at Baden, and distinguished himself by his energy and bravery, receiving the 3rd class of the order of the Red Eagle in recognition of his services. While attached to the Prince's staff at that time he broached to him the subject of his schemes of army reform. In 1850 came the revelation of defective organization and efficiency which led to the humiliating treaty of Olmütz. In the same year Roon was made a lieutenant - colonel, and in 1851 full colonel.

Promoted to major - general in 1856 and lieutenant - general in 1859, Roon had held several commands since 1850 and had been employed on important missions. Prince Wilhelm became regent in 1858, and in 1859 he appointed Roon a member of a commission to report on the reorganization of the army. During the Austro - Sardinian War he was charged with the mobilization of a division. At the end of 1859, although he was a junior lieutenant - general in the army, he succeeded Eduard von Bonin as war minister. Two years later, in 1861 the ministry of marine was also entrusted to him.

Supported by Edwin von Manteuffel and the new Prussian Chief of Staff Helmuth von Moltke, Roon drew up plans to adapt Scharnhorst's system to Prussia's altered circumstances. To attain this he proposed an increase in universal military service to three years, with new regiments raised and a reduced role for the reserve (Landwehr), whose role in the War of Liberation (1813) was still celebrated in nationalist myth. Roon's proposals for army reorganization met with strong opposition from the Prussian Landtag, which was dominated by the Progressive Party (i.e. liberals), who wanted parliamentary control over the military budget. It took years of political fighting and the strong support of the new Prime Minister Otto von Bismarck and Moltke, before he carried the day.

After the successful outcome of the Second Schleswig War of 1864, Roon went from being the most hated man in Prussia to a popular man in the country.

At the start of the Austro-Prussian War, Roon was promoted general of infantry. He was at the decisive victory at Königgrätz, under the command of Moltke. He received the Black Eagle at Nikolsburg on the road to Vienna. His army system was adopted after 1866 by the whole North German Confederation. In later years, his army system was copied throughout continental Europe.

During the Franco-Prussian War in 1870 - 71, Roon was in attendance on the (then) king Wilhelm. The war was a great victory for Prussia and Roon's contribution to success was considerable. He was created a Graf (count) at Versailles on January 19, 1871, just after Moltke. In December 1871, he succeeded Bismarck (who continued to be Imperial Chancellor) as Prime Minister of Prussia. Ill health compelled him to resign in the following year, handing the job back to Bismarck. Roon was promoted to field marshal on January 1, 1873.

Roon died in Berlin on February 23, 1879.

The armored cruiser SMS Roon, completed in 1906, was named for Albrecht von Roon.