Baldi descended from a noble family from Urbino, Marche, where he was born. He pursued his studies at Padua, and is said to have spoken about sixteen languages during his lifetime, though according to Tiraboschi the inscription on his tomb limits the number to twelve.

The appearance of the plague at Padua forced him to return to his native city. Shortly afterwards he was called to act as tutor to Ferrante Gonzaga, from whom he received the rich abbey of Guastalla. The oldest biography of Nicolaus Copernicus was completed on 7 October 1588 by him. He held office as abbot for 25 years, and then returned once again to Urbino. In 1612 he was employed by the duke as his envoy to Venice. Baldi died at Urbino on 12 October 1617.

He is said to have written upwards of a hundred different works, the chief part of which have remained unpublished. His various works show his abilities as a theologian, mathematician, geographer, antiquary, historian and poet. The Cronica dei Matematici (published at Urbino in 1707) is an abridgment of a larger work on which he had written for twelve years, and was intended to contain the lives of more than two hundred mathematicians. His life has been written of by Affò, Mazzucchelli and others.

The painter Bernardino Baldi from Bologna is unrelated (active 1599, died 1615).