Born in Florence, Cosimo inherited both his wealth and his expertise in business from his father, Giovanni di Bicci de' Medici. In 1415 he accompanied the Antipope John XXIII at the council of Constance, and in the same year he was named Priore of the Republic. Later he acted frequently as ambassador, showing a prudence for which he became renowned.

His power over Florence stemmed from his wealth, which he used to control votes. As Florence was proud of its 'democracy', he pretended to have little political ambition, and did not often hold public office. Aeneas Sylvius, Bishop of Siena and later Pope Pius II, said: "Political questions are settled in [Cosimo's] house. The man he chooses holds office...He it is who decides peace and war...He is king in all but name." Quoted by C.Hibbert in The Rise and Fall of the House of Medici, 1974.

In 1433 Cosimo's power over Florence, which he exerted without occupying public office, began to look like a menace to the anti-Medici party, led by figures such as Palla Strozzi and Rinaldo degli Albizzi: in September of that year he was imprisoned, accused for the failure of the conquest of Lucca, but he managed to turn the jail term into one of exile. He went to Padua and then to Venice, taking his bank along with him. Prompted by his influence and his money, others followed him: within a year, the flight of capital from Florence was so great that the ban of exile had to be lifted. Cosimo returned a year later in 1434, to greatly influence the government of Florence (especially through the Pitti and Soderini families) and to lead by example for the rest of his long life. Cosimo's time in exile instilled in him the need to squash the factionalism that resulted in his exile in the first place. In order to do this, Cosimo, with the help of favourable priors in the Signoria, instigated a series of constitutional changes to secure his power through influence.

In the political sphere, Cosimo worked to create peace in Northern Italy through the creation of a balance of power between Florence, Naples, Venice and Milan during the wars in Lombardy, and discouraging outside powers (notably the French and the Holy Roman Empire) from interfering. In 1439 he was also instrumental in convincing pope Eugene IV to move the Ecumenical council of Ferrara to Florence. The arrival of figures from the Byzantine Empire, including Emperor John VIII Palaiologos himself, started the boom of culture and arts in the city.

Cosimo was also noted for his patronage of culture and the arts, liberally spending the family fortune (which his astute business sense considerably increased) to enrich Florence. According to Salviati's Zibaldone, Cosimo stated: "All those things have given me the greatest satisfaction and contentment because they are not only for the honor of God but are likewise for my own remembrance. For fifty years, I have done nothing else but earn money and spend money; and it became clear that spending money gives me greater pleasure than earning it." He also hired the young Michelozzo Michelozzi to create what is today perhaps the prototypical Florentine palazzo, the austere and magnificent Palazzo Medici. He was a patron and confidante of Fra Angelico, Fra Filippo Lippi, and Donatello, whose famed David and Judith Slaying Holofernes were Medici commissions. His patronage enabled the eccentric and bankrupt architect Brunelleschi to complete the dome of Santa Maria del Fiore, and the dome was perhaps his crowning achievement as sponsor.

In the realm of philosophy, Cosimo, influenced by the lectures of Gemistus Plethon, established a modern Platonic Academy in Florence. He appointed Marsilio Ficino as head of the Academy and commissioned Ficino's Latin translation of the complete works of Plato (the first ever complete translation). Through Ficino and others associated with the Academy, Cosimo had an inestimable effect on Renaissance intellectual life.

On his death in 1464 at Careggi, Cosimo was succeeded by his son Piero "the Gouty", father of Lorenzo the Magnificent or Il Magnifico. After his death the Signoria awarded him the title Pater Patriae, "Father of his Country", an honor once awarded to Cicero, and had it carved upon his tomb in the Church of San Lorenzo.

Cosimo married Contessina de' Bardi (the daughter of Giovanni, count of Vernio, and Emilia Pannocchieschi). They had two sons: Piero the Gouty and Giovanni de' Medici. Cosimo also had an illegitimate son by a Circassian slave; Carlo (c. 1428 - 1492), who became a prelate.