He was born in Strasbourg, where he attended the gymnasium. He then studied at the Karlsruhe Institute of Technology, where Friedrich Walchner's lectures first attracted his interest to chemistry. After that, he attended the school of commerce in Leipzig, where he studied chemistry under Otto Linné Erdmann, who further developed his interest into a passion for questions of speculative chemistry.

Returning home in 1834, he entered his father’s white lead factory, but soon found that business was not to his liking, and after a sharp disagreement with his father in his 20th year enlisted in a cavalry regiment. In a few months military life became equally distasteful, and he purchased his discharge with the assistance of Justus von Liebig, with whom, after a short interval at Dresden, he went in 1836 to study at Giessen and work in Liebig's laboratory. His stay at Giessen lasted 18 months, and in 1837 he re-entered the factory. Again, however, he quarrelled with his father, and in 1838 went to Paris with introductions from Liebig.

In Paris, he attended Jean Baptiste Dumas’ lectures and worked with Auguste Cahours (1813 – 1891) on essential oils, especially cumin, in Michel Eugène Chevreul’s laboratory at the Jardin des Plantes, meanwhile earning a precarious living by teaching and making translations of some of Liebig’s writings. In 1841, by the influence of Dumas, he was charged with the duties of chemistry professor at the Montpellier faculty of sciences, becoming titular professor in 1844.

In 1842 he annoyed his friends in Paris by the matter and manner of a paper on the classification of organic compounds. Later, he published Précis de chimie organique (1844 – 1845). In 1845 he and his opinions were the subject of an attack by Liebig, unjustifiable in its personalities but not altogether surprising in view of his wayward disregard of his patron’s advice. The two were reconciled in 1850, but his faculty for disagreeing with his friends did not make it easier for him to get another appointment after resigning the chair at Montpellier in 1851, especially as he was unwilling to go into the provinces.

He obtained leave of absence from Montpellier in 1848 so he could pursue without interuption his special investigations, and from that year until 1855 resided in Paris. During that period, he established an École de chimie pratique ("School for practical chemistry") of which he had great hopes; but these were disappointed, and in 1855, after refusing the offer of a chair of chemistry at the new Zürich Polytechnic in 1854, he accepted the professorships of chemistry at the Faculty of Sciences and the École Polytechnique at Strassburg, where he died the following year, having just completed checking the proofs for Traité de chimie organique (4 vols., Paris, 1853 - 56), his magnum opus. This latter work embodies all his ideas and his discoveries.

Gerhardt is known for his work on reforming the notation for chemical formulas (1843 – 1846). He also worked on acid anhydrides, and synthesized acetylsalicylic acid, albeit in an unstable and impure form.

Gerhardt is usually linked with his contemporary, Auguste Laurent, with whom he shared a strong and influential interest in chemical combination.