Born in Berlin in 1914, Stoph's father died the following year in World War I. In 1928, the young Stoph joined the Communist Youth League of Germany (Kommunistischer Jugendverband Deutschlands; KJVD) and, in 1931, he joined the Communist Party of Germany. He also served in the Wehrmacht from 1935 to 1937, and again during World War II from 1940 to 1945. He was awarded the Iron Cross 2nd Class and rose to the rank of Unteroffizier.

Following the establishment of the GDR, Stoph became a member of the central committee of the Socialist Unity Party of Germany (SED) and member of the Volkskammer in 1950. He served as interior minister from 1952 to 1955, and defense minister from 1956 to 1960. As such, he was awarded the rank of an Armeegeneral.

From 1964 to 1973, he was Chairman of the Council of Ministers (Ministerrat). After the death of Walter Ulbricht in 1973, Stoph became Chairman of the Council of State, and thus, head of state of the GDR. After Volkskammer elections in 1976, the state and party leadership structure was re-arranged, and Stoph once again became Chairman of the Council of Ministers.

As Prime Minister, Stoph began a series of negotiations with West German Chancellor Willy Brandt in 1970. It marked the first ever meeting between the leaders of East and West Germany.

Following the ousting of Erich Honecker in October 1989, Stoph and all 44 members of his cabinet resigned on 7 November in response to public pressure. On 8 November, the Central Committee of the SED nominated Hans Modrow as Stoph's replacement. Stoph was subsequently arrested for corruption in December 1989. He was later spared detention due to health reasons. In 1994, a court in Berlin decided that he should not get back his seized savings of 200,000 DM.

Stoph died in Berlin on 13 April 1999. He was buried in Wildau.