Little is known about his early life except that he studied in Correggio with Tuttovale Menon, a famous madrigalist who also worked in the Ferrara court; he also studied with Girolamo Donato, an organist. It is likely that he studied with Zarlino at St. Mark's in Venice. While in Venice he became close friends with Costanzo Porta, a friendship which was to endure for his entire life. On 21 October 1556, he was appointed organist at Brescia Cathedral ("Duomo Vecchio"), and his skill as an organist must have been impressive, because he became organist at St. Mark's, one of the most prestigious positions for an organist in Italy, in 1557; he was selected over a list of candidates that included Andrea Gabrieli. This was the first important event of his career, and he was considered to be the finest organist in Italy.

It is important to note that in St. Mark's there were two organs, and two separate organists were appointed to play them: in 1557 Merulo was appointed to the second, smaller organ, while Annibale Padovano remained at the post of first organist.

After Padovano's hurried departure from Venice in 1566, Merulo was appointed to the first organ, and Andrea Gabrieli became the second organist. He was appointed as ambassador of the Venetian Republic at the marriage of Franceso de’ Medici and Bianca Cappello in 1579 and wrote music of celebration for Henry III of France, who visited Venice in 1574.

In 1584, he suddenly left this position in Venice. The reasons for this are unclear, and somewhat surprising; in Venice he was well paid, and had a very good reputation; and St. Mark's was one of the most important places for an organist. However, in December 1584 his name appears in payment register of Farnesia Court of Parma.

In 1587 he was appointed as organist in Parma Cathedral and from 1591 also in Church of Santa Maria della Steccata. While here, he requested improvements to the organ, carried out by Costanzo Antegnati, the last heir of the great Brescian family of organ makers. We can deduce that Merulo used the Steccata's organ for his proofs of new composition, based on his Venetian experience, and continued to compose in this style.

He lived in Parma until his death. During this period, he made several trips in Venice and Rome, where he published his famous two volume Toccate per organo.

Merulo died in Parma on 4 May 1604 and was buried in Parma Cathedral near to the tomb of Cipriano de Rore. He left a daughter and his wife Amabilia Banzola.

Merulo is famous for his keyboard music. His Toccatas, in particular, are innovative; he was the first to contrast sections of contrapuntal writing with passage works; often he inserts sections which could be called ricercars into pieces which otherwise are labelled toccatas orcanzonas. (In the late 16th century, these terms are only approximately descriptive; different composers clearly had different ideas of what they meant). Often his keyboard pieces begin as though they are to be a transcription of vocal polyphony, but then gradually add embellishment and elaboration until they reach a climactic passage of considerable virtuosity. Sometimes, especially in his later music, he develops ornaments which acquire the status of a motif, which is then used developmentally; this anticipates a principal generative technique in the Baroque era. Often Merulo casually ignores the "rules" of voice - leading, giving the music an expressive intensity more associated with the late school of madrigalists than with keyboard music of the time. His keyboard music was hugely influential, and his ideas can be seen in the music of Sweelinck, Frescobaldi and others; because of the immense influence of Sweelinck as a teacher, much of the virtuoso keyboard technique of the north German organ school, culminating in Johann Sebastian Bach, can claim to be descended from the innovations of Merulo.

Even though the fame of his instrumental music has overshadowed much of his a cappella vocal output, Merulo was also a madrigalist. Since he was a member of what is known today as the Venetian School, he also wrote motets for double choir in the manner of Andrea and Giovanni Gabrieli. He published two books of Madrigali a 5 voices (1566 and 1604), one of Madrigali a 4 (1579) and a 3 (1580).

The famous essay of keyboard technique Il Transilvano (1593), by Girolamo Diruta, was dedicated to Merulo, indicating his status as one of the most important keyboard players of Renaissance.