Tomsky attempted to form a trade union at his factory in St. Petersburg resulting in his dismissal. His labour activities radicalized him politically and led him to become a socialist and join the Russian Social Democratic Labor Party in 1904 and eventually join the Bolshevik faction of the party.

Born in Kolpino near St. Petersburg, Tomsky moved to Estonia (then part of the Russian Empire) and was involved in the 1905 Revolution. He helped form the Revel Soviet of Workers' Deputies and the Revel Union of Metal Workers. Tomsky was arrested and deported to Siberia.

He escaped and returned to St. Petersburg where he became president of the Union of Engravers and Chromolithographers.

Tomsky was arrested in 1908 and then exiled to France, but returned to Russia in 1909 where he was again arrested for his political activities and sentenced to five years of hard labor. He was freed by the Provisional Government after the February Revolution in 1917 and moved to Moscow where he participated in the October Revolution. In 1918 he attended the Fourth All Russian Conference of Trade Unions (12 – 17 March), where he moved a resolution concerning the Relations between the Trade Unions and the Commissariat for Labor which stated that the October Revolution had changed "the meaning and character of state organs and significance of proletarian organs as well". It was elaborated that previously the old ministry of Labor had acted as arbitrator between Labor and Capital, whereas the new Commissariat was the champion of the economic policy of the working class.

In 1920, he became General Secretary of the Red International of Labor Unions. He was elected into the Central Committee in March 1919, to its Orgburo in 1921 and to the Central Committee of the Communist Party in April 1922. He became a full member of the Central Committee's Politburo on December 19, 1927.

Tomsky was an ally of Nikolai Bukharin and Alexey Rykov, who led the moderate (or right) wing of the Communist Party in the 1920s. Together, they were allied with Joseph Stalin's faction and helped him purge the Left Opposition - led by Leon Trotsky, Lev Kamenev, and Grigory Zinoviev - from the party during the struggle that followed Lenin's death in 1924.

In 1928 Stalin moved against his former allies, defeating Bukharin, Rykov and Tomsky at the April 1929 Plenary Meeting of the Central Committee and forcing Tomsky to resign from his position as leader of the trade union movement in May 1929. Tomsky was put in charge of the Soviet chemical industry, a position which he occupied until 1930. He was not re-elected to the Politburo after the 16th Communist Party Congress in July 1930, but remained a full member of the Central Committee until the next Congress in January 1934, when he was demoted to candidate (non - voting) member.

Tomsky headed the State Publishing House from May 1932 until August 1936, when he was accused of terrorist connections during the First Moscow Trial of Zinoviev and Kamenev. Rather than face arrest by the NKVD, Tomsky committed suicide by gunshot in his dacha in Bolshevo, near Moscow. He was posthumously accused of high treason and other crimes during the third (March 1938) show trial of Bukharin, Rykov and others. The Soviet government cleared Tomsky of all charges during perestroika in 1988.