Born to a distinguished noble family at Carpi near Modena, where his uncle Alberto (c. 1475 - 1531) was lord of Carpi, Rodolfo was sent to study at the University of Padua and at Rome, where he took up a church career under Pope Clement VII, who made him bishop of Faenza in 1528. There Carpi hosted a synod in 1533. He attracted further notice in papal diplomacy and was established at Paris 1535 – 1537 as papal nuncio at the court of François I, where, assisted by his uncle Alberto, he presided over the peace between François and the Emperor Charles V, who was pleased enough to appoint him "protector of the Holy Roman Empire".

Pope Paul III made him cardinal on December 22, 1536, and sent him, in January 1540, as legate to the March of Ancona. Cardinal Carpi, as he now was, made his presence felt in the curia as a member of the Roman Inquisition and a defender of the new orders, the Capuchins and the Jesuits. His friend Pius III assigned him his choice of sees; he preferred the delightful see of Frascati to Faenza (1553 – 1555). Only the intractable resistance of his feudal superior, cardinal d'Este prevented his being made pope at the conclave of 1559.

His broader modern interest for historians centers on his collection of classical sculpture and other antiquities, which formed one of the prominent museums of Rome (Roma nihil possidet magnificentius, nihil admirabilius one guidebook remarked: "Rome possesses nothing more magnificent, nor to be more admired") and the Greek and Latin library, dispersed after his death, that brought scholars and humanists, not invariably good Catholics, to his palazzo in the Campo Marzio — the Campus Martius of Antiquity — and his suburban villa, on the site of the gardens of Sallust, on the flank of the Quirinal Hill. In the 1550s the Flemish medallist and epigrapher Antoine Morillon studied the Latin inscriptions in the Cardinal's gallery. Even the dry inventories furnish materials for the historian of taste (i.e. C. Franzoni et al., Gli inventari dell'eredità del cardinale Rodolfo Pio da Carpi Pisa, 2002, for the Musei Civici, Comune di Carpi.) The semi - public collections of princes and cardinals made Rome a museum city, memorialized by Ulisse Aldrovandi's guidebook Delle Statue antiche che per tutta Roma si veggono, 1556 (in international French, Les Antiquités de la cité de Rome, 1576). Aldrovandi praised the delights of the Carpi antiquities in their rustic suburban setting. Even after Cardinal Carpi's death, the collections drew sculptors and artists.

Among the antiquities that belonged to Cardinal Carpi: The bronze bust called the Capitoline Brutus that Pio da Carpi bequeathed to the City of Rome, now in the Capitoline Museums; The ecstatic marble head called the Dying Alexander, in the Uffizi Gallery, Florence by 1579; often copied in plaster, bronze and marble; Bronze and marbles bought from Duke Alfonso II d'Este some of which disappeared after they belonged to Rudolf II at Prague; A 5th century manuscript of the complete works of Virgil, called the Medici Virgil after it was purchased for the Laurentian Library, Florence.

Cardinal Carpi did not neglect the moderns; among his paintings: Madonna of the Divine Love, school of Raphael, which Vasari said had been commissioned by the cardinal's father, Leonello da Carpi; it passed to the Farnese and is now at the Capodimonte Museum, Naples; History of the Madonna of Loreto and Saint Jerome in his study by Lorenzo Lotto; Portrait of a Man, originally attributed to Sebastiano del Piombo; now thought to be by Francesco Salviati, at the Kunsthistorisches Museum, Vienna.

Cardinal Carpi is interred in Rome at Santa Trinità dei Monti, above the Spanish Steps, where there is a sepulchral monument erected to his memory by Pope Pius V.

The Lords of Carpi first made a position for themselves in the 14th century. From the house of Este they received the lordship of Carpi, and in 1518, through the influence of Pope Leo X they acquired the subsidiary fiefs of Meldola and Sassuolo, with which Rodolfo Pio da Carpi was invested. Many members of the family continued in the family tradition as condottieri: Alberto Pio obtained from the house of Savoy in 1450 the privilege of adding "di Savoia" to his name, as a reward for his military services. Others beside Cardinal Carpi made careers in diplomacy: Alberto Pio (1475 - 1531), who was French ambassador in Rome, won fame as a man of learning. Ascanio Pio (d. 1649) was a dramatic poet. Spain conferred the title of prince on the family, and one branch of the family is to this day established in Spain.