Born in Berlin, the son of a Jewish shopkeeper, he worked as a clown, a stable boy and a dog trainer before studying composition with Iwan Knorr (1891 - 92, Hoch Conservatory) and Engelbert Humperdinck (as private student) in Frankfurt. He later moved to Düsseldorf to study painting and art history. After a spell in Paris, he returned to Berlin in 1898 to study counterpoint with Xaver Scharwenka.

The performance of his composition Das trunkene Lied ("the drunken song") for chorus and orchestra brought Fried his first public success and led to his appointment in 1904 as the conductor of a Berlin choral society.

Fried first met Gustav Mahler in 1905. The meeting resulted in an invitation to conduct Mahler's "Resurrection" Symphony in Berlin in November 1905 (Otto Klemperer led the offstage band during this performance). The next year Fried introduced Russia to Mahler's music when he performed the same work in St Petersburg. From 1907 to 1910 he directed a choral society known as the Sternscher Gesangverein in Berlin. In 1913 Fried conducted the Berlin Philharmonic in the second performance of Mahler's Ninth Symphony.

In 1922 Fried went to the USSR as the first foreign conductor invited to perform after the Russian Revolution, and was greeted by Lenin on the station platform. In 1924 Fried made the first recording of any Mahler symphony, the Second, with the Berlin Staatskapelle in a performance that has been praised as "remarkably successful" and a "highly adventurous undertaking for an acoustic recording" which required "careful planning and experimentation". That same year Fried also made the first recording of any complete Bruckner symphony: his Seventh.

Driven away from Germany by the anti - Semitism of the Nazi regime, in 1934 Fried left Germany for the Georgian city of Tbilisi in the Soviet Union. He conducted the Tbilisi opera and later the Moscow Radio Symphony Orchestra, eventually becoming a Soviet citizen. He died in Moscow in 1941.