Pfilimlin was born in Roubaix in the département Nord. A lawyer and a member of the Christian democratic Popular Republican Movement (MRP), he was elected deputy of département Bas Rhin in 1945. With his personal roots in Alsace, Pflimlin numbered among his MRP party colleagues the Luxembourg born Robert Schuman, for both of whom relations with Germany played an important part in their political thinking.

He held some governmental offices during the Fourth Republic notably as Minister of Agriculture (1947 – 1949 and 1950 – 1951) and as Minister of Economy and Finance (1955 – 1956 and 1957 – 1958).

In 13 May 1958, the French National Assembly approved his nomination as Prime Minister. But the same day, riots took place in Algiers. The French generals in Algeria expected him to search for a negotiated solution with the Algerian nationalists and refused to recognize his cabinet. The crisis brought Charles de Gaulle to the head of the cabinet, on 1 June.

Minister of State until 1959, Minister of Cooperation in 1962, he resigned with the other MRP ministers in order to protest against the euro-scepticism of de Gaulle.

Pflimlin served as mayor of Strasbourg from 1959 to 1983.

He also was the President of the Parliamentary Assembly of the Council of Europe from 1963 to 1966 and President of the European Parliament from 1984 to 1987.

The Pierre Pflimlin bridge over the Rhine south of Strasbourg, connecting France to Germany, is named after him and was opened in 2002.