He was born at Graz, the son of Charles II, Archduke of Austria, and Maria Anna of Bavaria. He was educated by the Jesuits and later frequented the University of Ingolstadt. After completing his studies in 1595, he acceded to his hereditary lands (where his older cousin, Archduke Maximilian III of Austria, had acted as regent between 1593 and 1595) and made a pilgrimage to Loreto and Rome. Shortly afterwards, he began to suppress non-Catholic faith in his territories.

With the Oñate treaty, Ferdinand obtained the support of the Spanish Habsburgs in the succession of his childless cousin Matthias, in exchange for concessions in Alsace and Italy. In 1617, he was elected King of Bohemia by the Bohemian diet, in 1618, King of Hungary by the Hungarian estates, and in 1619, Holy Roman Emperor.

His ultra catholicism caused immediate turmoil in his non-Catholic subjects, especially in Bohemia. He did not respect the religious liberties granted, that the Letter of Majesty conceded, signed by the previous emperor, Rudolph II, which had guaranteed the freedom of religion to the nobles and the inhabitants of the cities. Additionally, Ferdinand was an absolutist monarch and infringed several historical privileges of the nobles. Given the relatively great number of Protestants in the kingdom, including some of the nobles, the king's unpopularity soon caused the Bohemian Revolt. The Defenestration of Prague of 22 May 1618 is considered the first step of the Thirty Years' War.

In the following events he remained one of the staunchest backers of the Anti-Protestant Counter Reformation efforts as one of the heads of the German Catholic League. Ferdinand succeeded Matthias as Holy Roman Emperor in 1619. Supported by the Catholic League and the Kings of Spain and Poland, Ferdinand decided to reclaim his possession in Bohemia and to quench the rebels. On 8 November 1620 his troops, led by the Belgian general Tilly, smashed the rebels of Frederick V of Palatinate, who had been elected as rival King in 1618. After Frederick's flight to the Netherlands, Ferdinand ordered forced conversion to Catholicism in Bohemia and Austria, causing Protestantism there to nearly disappear in the following decades, and reduced the Diet's power.

In 1625, despite the subsidies received from Spain and the Pope, Ferdinand was in a bad financial situation. In order to muster an imperial army to continue the war, he applied to Albrecht von Wallenstein, one of the richest men in Bohemia: the latter accepted on condition that he could keep total control over the direction of the war, as well as over the booties taken during the operations. Wallenstein was able to recruit some 30,000 men (later expanded up to 100,000), with whom he was able to defeat the Protestants in Silesia, Anhalt and Denmark. In the wake of the overwhelming Catholic military successes, in 1629 Ferdinand issued the Edict of Restitution, by which all the land stripped to the Catholics after the Peace of Passau of 1552 would be returned.

His new ultra catholic demands caused the tottering Protestants to call in Gustavus II Adolphus, King of Sweden. Further, some of Ferdinand's Catholic allies started to complain about the excessive power gained by Wallenstein, as well as of the ruthless method he used to finance his huge army. Ferdinand replied by firing the Bohemian general in 1630. The lead of the war thenceforth was assigned to Tilly, who was however unable to stop the Swedish march from northern Germany towards Austria. Some historians directly blame Ferdinand for the large civilian loss of life in the Sack of Magdeburg in 1631: he had instructed Tilly to enforce the edict of Restitution upon Saxony, his orders causing the Belgian general to move the Catholic armies east, ultimately to Leipzig, where they suffered their first substantial defeat at First Breitenfeld. Tilly died in 1632. Wallenstein was recalled, being able to muster an army in only a week, and to expel the Swedish from Bohemia. In November 1632 the Catholics were defeated in the Battle of Lützen, but Gustavus Adolphus died. A period of minor operations followed, perhaps because of Wallenstein's ambiguous conduct, which ended with his assassination in 1634, perhaps ordered by Ferdinand himself.

Despite Wallenstein's fall, the imperial forces recaptured Regensburg and were victorious in the Battle of Nördlingen. The Swedish army was substantially weakened, and the fear that the Habsburgs' power could at that point become overwhelming in the empire triggered France, led by Louis XIII of France and Cardinal Richelieu, to enter the war on the Protestant side. (Louis's father Henry IV of France was once a Huguenot leader). In 1635 Ferdinand signed his last important act, the Peace of Prague, which however did not end the war.

He died in 1637, leaving to his son Ferdinand III an empire still entangled in a war and whose fortunes seemed to be increasingly fading away.

In 1600, Ferdinand married Maria Anna of Bavaria (1574 - 1616), daughter of Duke William V of Bavaria. They had seven children. In 1622, he married Eleonore of Mantua (Gonzaga) (1598 – 1655), the daughter of Duke Vincenzo I of Mantua and Eleonora de' Medici, at Innsbruck.