Born in Hartmannsdorf near Kirchberg in Saxony, Graupner received his first musical instruction from his uncle, an organist named Nicolaus Kuester. Graupner went to the University of Leipzig where he studied law (as did many composers of the time), and then completed his musical studies with Johann Kuhnau, the cantor of the Thomaskirche (St. Thomas Church).

In 1705 Graupner left Leipzig to play the harpsichord in the orchestra of the Hamburg Opera under the direction of Reinhard Keiser, alongside a young violinist named Handel. In addition to playing the harpsichord, Graupner composed six operas in Hamburg, some of them in collaboration with Keiser, a popular composer of operas in Germany.

In 1709 Graupner accepted a post at the court of Hesse - Darmstadt and in 1711 became the court orchestra’s Hofkapellmeister (court chapel master). Graupner spent the rest of his career at the court in Hesse - Darmstadt, where his primary responsibilities were to provide music for the court chapel. He wrote music for nearly half a century, from 1709 to 1754, when he became blind. He died six years later.

Graupner inadvertently played a key role in the history of music. Precarious finances in Darmstadt during the 1710s forced a reduction of musical life. The opera house was closed, and many court musicians' salaries were in arrears (including Graupner's). After many attempts to have his salary paid, and having several children and a wife to support, in 1723 Graupner applied for the Kantorship in Leipzig. Telemann had been the first choice for this position, but withdrew after securing a salary increase in Hamburg. Graupner's "audition" Magnificat, set in the style of his teacher and mentor Kuhnau, secured him the position. However, Graupner's patron (the Landgrave Ernst Ludwig of Hesse - Darmstadt) would not release him from his contract. Graupner's past due salary was paid in full, his salary was increased; and he would be kept on staff even if his Kapelle was dismissed. With such favorable terms, Graupner remained in Darmstadt, thus clearing the way for Bach to become the cantor in Leipzig.

After hearing that Bach was the choice for Leipzig, on May 4, 1723 Graupner graciously wrote to the city council in Leipzig assuring them that Bach "is a musician just as strong on the organ as he is expert in church works and capelle pieces" and a man who "will honestly and properly perform the functions entrusted to him."

Graupner was hardworking and prolific. There are about 2,000 surviving works in his catalog, including 113 sinfonias, 85 ouvertures (suites), 44 concertos, 8 operas, 1,418 religious and 24 secular cantatas, 66 sonatas and 40 harpsichord partitas. Nearly all of Graupner's manuscripts are housed in the ULB (Technical University Library) in Darmstadt, Germany.

Graupner wrote for exotic combinations of instruments, including the oboe d'amore, flute d'amore, and viola d'amore. Over half of his sinfonias require brass and timpani, with about 25 sinfonias requiring 3 to 4 timpani, one (sinfonia in G Major GWV 611) 5 timpani, and another, sinfonia in F Major (GWV 566), was composed for 6 timpani.

After he died, Graupner became an obscure composer for several reasons. His manuscripts became the source of a long legal battle between his heirs and the rulers of Hesse - Darmstadt. A final court decision denied the Graupner estate ownership of the music manuscripts. The heirs were unable to obtain permission to sell or publish his works, which remained inaccessible. Music styles had changed dramatically, reducing any interest in Graupner's music. On the positive side however, the Landgrave's seizure of Graupner's musical estate ensured its survival in toto. Fate was not so kind to J.S. Bach's musical legacy, for example. Another factor that contributed to Graupner's posthumous obscurity was that, unlike Bach, Graupner had very few pupils other than Johann Friedrich Fasch to carry on his musical legacy.

As critic David Vernier has summed up, Graupner is "one of those unfortunate victims of fate and circumstance - a contemporary of Bach, Handel, Telemann, etc., who has remained largely -- and unfairly - neglected."

Graupner's music is enjoying a revival, due in large part to the research efforts of many musicologists, performers, and conductors. Beginning in the early 20th century, research began with Willibald Nagel's study of Graupner's sinfonias. In the 1920s, Friedrich Noack published his research on Graupner's cantatas. Baerenreiter published several sinfonias and an ouverture in the 1950s. In the early 1980s, Myron Rosenblum edited four sinfonias for the massive Barry Brook project The Symphony, 1720 - 1840: A Comprehensive Collection of Full Scores, 60 vols. The year 1988 saw the publication of Oswald Bill's study of Graupner, with several articles by such Graupner experts as Peter Cahn (on the sinfonias), Joanna Cobb Biermann (musicians and salaries in Darmstadt), as well as source documents on court life in Darmstadt. Three dissertations were very important for Graupner research: H. Cutler Fall's study of the Passiontide cantatas, Rene Schmidt's study of the Christmas cantatas, and Vernon Wicker's study of solo bass cantatas. Christoph Grosspietsch published an extensive study of Graupner's ouvertures in 1994.

Despite all this research, there were relatively few recordings available to the general public. This changed in 1998, when Hermann Max conducted a CD of Graupner works on the CPO label. Montreal harpsichordist Geneviève Soly came across a Graupner manuscript in the Beinecke Library at Yale University in the year 2000 and started performing and recording his works.

Graupner was "always on the cutting edge for his time and very innovative in his ideas for harmony, notation, and the use of instruments," as Soly has noted. "You have to take into consideration his various styles in relation to the actual period and the ideas he was interested in developing at that moment. The size of the catalogue imposes added difficulties in this respect, because another composer might have written for just as long, but in one style only. Mozart comes to mind: although he composed over a shorter period, his style was always well defined."

In April 2005, a thematic catalog of Graupner's instrumental music (Oswald Bill and Christoph Grosspietsch, editors) was published by Carus Music. There are plans to catalog Graupner's vocal music.