Schmidt was born in Prenzlau, Province of Brandenburg. He was educated at Bonn and at Jena where he studied philology (historical linguistics) with the great Indo-Europeanist August Schleicher and specialized in Indo-European, especially Slavic, languages. He earned a doctorate in 1865 and worked from 1866 as a teacher at a gymnasium in Berlin.

In 1868 Schmidt was invited by the University of Bonn to take a position as professor of German and Slavic languages. In Bonn he published the work Die Verwandtschaftsverhältnisse der indogermanischen Sprachen ('The Relationships of the Indo-Germanic Languages', 1872), which presented his Wellentheorie ('wave theory'). According to this theory, new features of a language spread from a central point in continuously weakening concentric circles, similar to the waves created when a stone is thrown into a body of water. This should lead to convergence among dissimilar languages. The theory was directed against the doctrine of sound laws introduced by the Neogrammarians in 1870.

From 1873 to 1876 Schmidt was a professor of philology at the University of Graz in Austria. In 1876 he returned to Berlin, where he worked as a professor at Humboldt University. He died in Berlin at the age of fifty-seven.