He was born Siegfried Theodor Karg in Oberndorf am Neckar, Germany, the youngest of the twelve children of Johann Jacob Karg, a book dealer, and his wife Marie Auguste Karg, born Ehlert. The family finally settled in Leipzig in 1882, where Siegfried received his first musical training and private piano instruction. At a gathering of composers in Leipzig, he presented his first attempts at composition to the composer Emil Nikolaus von Reznicek, who arranged a three year tuition free scholarship at the Leipzig Conservatory. This enabled the young man to study with Salomon Jadassohn, Carl Reinecke, Alfred Reisenauer and Robert Teichmüller.

From August 1901 to September 1902 he worked as a piano teacher in Magdeburg. It was during this period that he changed his name to Sigfrid Karg - Elert, adding a variant of his mother's maiden name to his surname, and adopting the Swedish spelling of his first name.

Having returned to Leipzig, he started devoting himself to composition, primarily for the piano (encouraged by Edvard Grieg, whom he greatly admired); and in 1904 he met the Berlin publisher Carl Simon, who introduced him to the harmonium. From then on until his death he created one of the most significant and extensive catalogs of original works for this instrument. Encouraged by the organist Paul Homeyer, he reworked several of these harmonium compositions for organ, before composing his first original organ piece, 66 Chorale Improvisations, Op. 65 in 1909.

In 1910 Karg - Elert married Luise Kretzschmar (1890 – 1971); four years later their daughter Katharina was born.

After having served as a regimental musician during World War I, Karg - Elert was appointed instructor of music theory and composition at the Leipzig Conservatory in 1919.

The cultural climate in Germany in the 1920s and 1930s was very hostile to the internationally oriented, French influenced Karg - Elert; and although his works were admired outside Germany, especially in the U.K. (the Organ Music Society of London held a ten day festival in his honour in 1930) and in the U.S.A., in his home country his music was almost completely neglected. All this led to him accepting an invitation for an organ concert tour of America in the spring of 1932. The tour proved to be a disastrous mistake. He was suffering from the diabetes which would soon kill him, and his limited powers as an organist compared unfavourably to the virtuoso standard of organ performance (set by the likes of Marcel Dupré and Louis Vierne) to which American audiences had grown accustomed.

After his return to Leipzig, his health started deteriorating rapidly. He died in April 1933, only 55 years old. His grave is in the Südfriedhof in Leipzig. The popularity of his compositions declined for a period after World War II, before a successful revival in the late 1970s; today his works for the organ are frequently included in recitals.

Karg - Elert regarded himself as an outsider. Notable influences in his work include composers Johann Sebastian Bach (he often used the B.A.C.H. motif in Bach's honour), Edvard Grieg, Claude Debussy, Max Reger, Aleksandr Scriabin, and early Arnold Schoenberg. In general terms, his musical style can be characterised as being late romantic with impressionistic and expressionistic tendencies. His profound knowledge of music theory allowed him to stretch the limits of traditional harmony without losing tonal coherence.

His favourite instrument to compose for was the Kunstharmonium, a versatile French creation that allowed him the range of colors he preferred. He composed primarily for small ensembles or solo instruments like organ, piano and harmonium.