Johann Michael Haydn was born in 1737 in the Austrian village of Rohrau, Austria near the Hungarian border. His father was Mathias Haydn, a wheelwright who also served as "Marktrichter", an office akin to village mayor. Haydn's mother Maria, née Koller, had previously worked as a cook in the palace of Count Harrach, the presiding aristocrat of Rohrau. Mathias was an enthusiastic folk musician, who during the journeyman period of his career had taught himself to play the harp, and he also made sure that his children learned to sing.

Michael's early professional career path was paved by his older brother Joseph, whose skillful singing had landed him a position as a boy soprano in the St. Stephen's Cathedral, Vienna where he worked as a chorister, under the direction of Georg Reutter. Other singers in that choir included Johann Georg Albrechtsberger and Franz Joseph Aumann, both composers with whom Haydn later traded manuscripts. The early 19th century author Albert Christoph Dies, reporting from Joseph's late-life reminiscences, says the following:

"Reutter was so captivated by [Joseph]'s talents that he declared to the father that even if he had twelve sons, he would take care of them all. The father saw himself freed of a great burden by this offer, consented to it, and some five years after dedicated Joseph's brother Michael and still later Johann to the musical muse. Both were taken on as choirboys, and, to Joseph's unending joy, both brothers were turned over to him to be trained."

The same source indicates that Michael was a brighter student than Joseph, and that (particularly when Joseph had grown enough to have trouble keeping his soprano voice), it was Michael's singing that was the more admired.

Shortly after he left the choir-school, Michael was appointed Kapellmeister at Nagyvárad (Großwardein, Oradea) and later, in 1762, at Salzburg. The latter office he held for forty-three years, during which time he wrote over 360 compositions for the church and much instrumental music.

On 17 August 1768 Haydn married the singer Maria Magdalena Lipp (1745 – 1827); they had a daughter, Aloisia Josefa in January 1770, but she died only a few days before her first birthday. Lipp was disliked by the women in Mozart's family. Still, Lipp had created the role of Barmherzigkeit (Divine Mercy) in Mozart's first musical play Die Schuldigkeit des ersten Gebots (1767), and later the role of Tamiri in Il re pastore (1775). Leopold Mozart criticized Haydn's alcoholism.

He was acquainted with Mozart, who had a high opinion of his work, and was the teacher of both Carl Maria von Weber and Anton Diabelli.

Michael remained close to Joseph all of his life. Joseph highly regarded his brother and felt that Michael's religious works were superior to his own. In 1802, when Michael was "offered lucrative and honourable positions" by "both Esterházy and the Grand Duke of Tuscany," he wrote to Joseph in Vienna asking for advice, though in the end he chose to stay in Salzburg. It has been hypothesized that Michael and Maria Magdalena named their daughter Josefa in honor of Michael's brother.

Michael Haydn died in Salzburg at the age of 68.

Michael Haydn never compiled a thematic catalog of his works, nor did he ever supervise the making of one. The earliest catalog was compiled in 1808 by Nikolaus Lang for 'Biographische Skizze'. In 1907 Lothar Perger compiled a catalogue of his orchestral works, the Perger - Verzeichnis, for 'Denkmäler der Tonkunst in Österreich', which is somewhat more reliable than Lang's catalog. Thus, some of Haydn's instrumental works are referred to by Perger numbers. And in 1915 Anton Maria Klafsky undertook a similar work regarding the sacred vocal music. In 1982, Charles H. Sherman, who has edited scores of many Haydn symphonies for Doblinger, published a chronological catalog of Haydn's symphonies, which some recording companies have adopted. Later, in 1991, Sherman joined forces with T. Donley Thomas to publish a chronological catalog of all Haydn's music, which used a single continuous range of numbers, as does Köchel's catalog of Mozart's music. Further important amendments to the Sherman/Thomas catalogue have been made by Dwight Blazin.

The task of cataloguing Haydn's music is simplified by the fact that he almost always put the date of completion on his manuscripts. Guesswork is necessary when the autograph manuscript of a given work did not survive to posterity.

Haydn's sacred choral works are generally regarded as his most important, including the Requiem pro defuncto Archiepiscopo Sigismundo (Requiem for the death of Archbishop Siegmund) in C minor, which greatly influenced the Requiem by Mozart, Missa Hispanica (which he exchanged for his diploma at Stockholm), a Mass in D minor, a Lauda Sion, and a set of graduals, forty-two of which are reprinted in Anton Diabelli's Ecclesiasticon. He was also a prolific composer of secular music, including forty symphonies and partitas, a number of concertiand chamber music including a string quintet in C major which was once thought to have been by his brother Joseph.

There was another case of posthumous mistaken identity involving Michael Haydn: for many years, the piece which is now known as Michael Haydn's Symphony No. 25 was thought to be Mozart's Symphony No. 37 and assigned K. 444. The confusion arose because an autograph was discovered which had the opening movement of the symphony in Mozart's hand, and the rest in somebody else's. It is now thought that Mozart had composed a new slow introduction for reasons unknown, but the rest of the work is known to be by Michael Haydn. The piece, which had been quite widely performed as a Mozart symphony, has been performed considerably less often since this discovery in 1907.

Indeed, several of Michael Haydn's works influenced Mozart. To give just two examples: the Te Deum "which Wolfgang was later to follow very closely in K. 141" and the finale of the Symphony No. 23 which influenced the finale of the G major Quartet, K. 387.