Hartmann was orphaned at a young age and grew up in St Petersburg in the house of his uncle, who was a well-known architect. He studied at the Academy of Fine Arts in St Petersburg and at first started working by illustrating books. He also worked as an architect and sketched, among other things, the monument to the thousandth anniversary of Russia in Novgorod, which was inaugurated in 1862. He made most of his water colors and pencil drawings on journeys abroad in the years 1864 to 1868. Together with Ivan Ropet, Hartmann was one of the first artists to include traditional Russian motifs in his work.

Since Vladimir Stasov had introduced him to the circle of Mily Balakirev in 1870, he had been a close friend of the composer Modest Mussorgsky. Following Hartmann's early death from an aneurysm at the age of only 39, an exhibition of over 400 of his paintings was displayed in the Academy of Fine Arts in St Petersburg, in February and March 1874. This inspired Mussorgsky to compose his suite Pictures at an Exhibition. Most of the works shown at the 1874 exhibition are now lost.