Reinecke was born in Altona, Hamburg, Germany; until 1864 the town was under Danish rule. He studied with his father, Johann Peter Rudolph Reinecke, a music teacher. Carl began to compose at the age of seven, and his first public appearance as a pianist was when he was twelve years old.

At the age of 19, he undertook his first concert tour in 1843, through Denmark and Sweden. After a stay in Leipzig, where he studied under Felix Mendelssohn and Robert Schumann, Reinecke went on tour with Königslöw and Wasielewski (later Schumann's biographer), in North Germany and Denmark. In 1846, Reinecke was appointed Court Pianist for Christian VIII in Copenhagen. There he remained until 1848, when he resigned and went to Paris. Overall he wrote four concertos for his instrument (and many cadenzas for others' works, including a large set published as his Opus 87), as well as concertos for violin, cello, harp and flute. In the winter of 1850/51, Carl Schurz reports attending weekly “musical evenings” in Paris where Reinecke was in attendance.

In 1851, Reinecke became a professor at the Cologne Conservatory. In ensuing years he was appointed musical director at Barmen, and became the academic, musical director and conductor of the Singakademie at Breslau.

In 1860, Reinecke was appointed director of the Gewandhaus Orchestra concerts in Leipzig, and professor of composition and piano at the Conservatorium. He led the orchestra for more than three decades, until 1895. He conducted premieres such as the full seven - movement version of Brahms's German Requiem (1869). In 1865 the Gewandhaus Quartett premiered Brahms' piano quintet, and in 1892 his D major string quartet.

Reinecke is best known for his flute sonata "Undine", but he is also remembered as one of the most influential and versatile musicians of his time. He served as a teacher for 35 years, until his retirement in 1902. His students included Edvard Grieg, Basil Harwood, Christian Sinding, Leoš Janáček, Constanta Erbiceanu, Isaac Albéniz, August Max Fiedler, Walter Niemann, Johan Svendsen, Richard Franck, Felix Weingartner, Max Bruch, Mikalojus Konstantinas Čiurlionis and Felix Fox among many others.

After retirement from the conservatory, Reinecke devoted his time to composition, resulting in almost three hundred published works. He wrote several operas (none of which are performed today) including König Manfred. During this time, he frequently made concert tours to England and elsewhere. His pianoforte playing belonged to a school in which grace and neatness were characteristic, and at one time Reinecke was probably unrivalled as a Mozart player and an accompanist. At the age of 80, he recorded his playing on piano roll for the Welte - Mignon company, making him the earliest born pianist to have his playing preserved in any format. He died at 85 in Leipzig.