Hartung was born in Leipzig, Germany, into an artistic family. He studied painters like Corinth and Nolde and also learned the basis of cubism and French painting. Studying both in Leipzig and Dresden, he reproduced the paintings of the masters, he then entered the Fine Arts academies of Dresden and Munich. To prevent succumbing to provincialism, he decided in 1926 that he would leave his native country. Consequently, after a bicycle trip through Italy, he moved to Paris.

He lived with Anna-Eva Bergmann and established himself in the French towns of Leucate, and then in the Spanish Balearic Islands, specifically Minorca, successively. He spent much time fishing. His first exhibition was held in 1931 in Dresden. His last bonds with Germany were broken when his father died in 1932. He was rejected from Nazi Germany on account of being a 'degenerate', because his painting style was associated with cubism - an art movement incompatible with Nazi Germany's ideals. In 1935 when he attempted to sell paintings while visiting Berlin, the police tried to arrest him. He was able to flee the country with the help of his friend Christian Zervos.

After returning to Paris as a refugee his wife left him, causing him to become depressive. His friends tried to help him with his financial difficulties, but his paintings were becoming more abstract and did not sell well. For the time being he could only afford a little shop where he could work at improving his technique.

In December 1939, he became a member of the French Foreign Legion. He was closely followed by the Gestapo and arrested for seven months by the French police. After they learned he was a painter, he was put in a red cell in order to wear off his vision. After being released he rejoined the Legion to fight in North Africa, losing a leg in a battle near Belfort. He earned French citizenship in 1945, and was awarded the Croix de Guerre.

After 1947 he became a more important French painter. That year he exhibited his works for the first time in Paris. In 1960 he was awarded the International Grand Prix for painting at the Venice Biennale.

Hartung was featured in the 1963 film documentary "School of Paris: (5 Artists at Work)" by American filmmaker Warren Forma.

His freewheeling abstract paintings set influential precedents for many younger American painters of the sixties, making him an important forerunner of American Lyrical Abstraction of the 1960s and 1970s.

He died in December 1989 in Antibes, France.