Le Tellier was born in Paris to a Parisian magistrate and his wife. He entered the public service and became maître des requêtes, (a higher level lawyer, or 'procureur') in 1631 for Louis XIII of France. In 1640 le Tellier was appointed Intendant of Justice for the French military stationed in Piedmont, Italy. In 1643, owing to his friendship with the head French minister Cardinal Jules Mazarin, he became Secretary of State for Military Affairs (known as 'Secretary of State for War' during that era), and was known as being an efficient administrator. He was active in the troubles associated with the aristocratic Fronde uprising, remaining loyal to Cardinal Mazarin and to the state.

In 1677 he was made Chancellor of France. One of his major contributions as chancellor included his transformation of the royal army into a considerably larger, more professional force that helped impose the absolute rule of Louis XIV, helping to ensure France's dominance of Europe.

Le Tellier, who despised Protestantism, was one of those who influenced Louis XIV to revoke the Edict of Nantes which had previously provided religious freedoms to them. He further encouraged the persecution of the Huguenots. He died a few days after the revocation had been signed.

Le Tellier also amassed great wealth during his life and left two sons, one being the famous statesman Louvois who also served France as Secretary of State for War, and who ultimately became one of the most powerful officials of the regime under his father's tutelage. Michel's other son Charles Maurice Le Tellier became the Archbishop of Reims.

Michel le Tellier's correspondence reside within the Bibliothéque Nationale in Paris.