Among the compositions that his younger contemporary Felix Mendelssohn admired when they met in Edinburgh in the summer of 1829 was a G minor piano trio of 1826, in which stormy and sometimes fierce passages are mixed with Schubertian warmth. Another favourite of Mendelssohn's was his lively rondo of 1828. They became friends and Fanny Mendelssohn is cited as saying that she liked Thomson “best of all the Britons I know”. Thomson's C major trio is described as a fine work.

He studied in Germany with a letter of introduction to the Mendelssohn family, and his Drei Liederwere published in Leipzig in 1838. John Purser contends that we have to look to Schumann to find anything comparable to these songs published two years before Schumann composed any songs in his mature style. The first song, Keiner von den Schonheit Tochtern, is based on the poem There be none of beauty's daughters by Byron, set in German by Thomson to match the other songs.

Thomson became the first Reid Professor of Music at the University of Edinburgh in 1838. As a musicologist he edited the Vocal Melodies of Scotland, and he was one of the first conductors to provide his audience with a programme of his concerts giving a critical analysis of the works to be performed. His other compositions included a fine bagatelle for solo piano, a six part Glee With Whispering Winds, three operas, a flute concerto and a flute quartet, and concert arias. However his career was cut short by his death in 1841.

Subsequently his work was little performed, but there has been a recent revival of interest with his work being featured in John Purser's Scotland's Music series, and a group of people in Kelso organised a bicentennial festival in 2005. Scores of some of John Thomson's compositions, including the G minor piano trio and flute quartet have been made freely available on Sibelius Music's website. Some scores are also being added to the International Music Score Library Project.