He was born in the commune of Montboudif, in the department of Cantal in central France. After his khâgne at Lycée Louis-le-Grand, where he befriended Senegalese poet and statesman Léopold Sedar Senghor, he graduated from the École Normale Supérieure with a degree of Agrégation in literature.

He first taught literature at a lycée until hired in 1953 by Guy de Rothschild to work at de Rothschild Frères. In 1956, he was appointed the bank's general manager, a position he held until 1962. Later, he was hired by Charles de Gaulle to manage the Anne de Gaulle Foundation for Down's Syndrome (de Gaulle's daughter Anne had Down's Syndrome).

He served as prime minister under de Gaulle after Michel Debré resigned, from 16 April 1962 to 21 July 1968, and to this day is the longest serving French prime minister under the Fifth Republic. His nomination was controversial because he was not a member of the National Assembly. In October 1962, he was defeated by a vote of non-confidence, but de Gaulle dissolved the National Assembly. The Gaullists won the legislative election and Pompidou was reappointed as Prime Minister. In 1964, he was faced with a miners' strike. He led the 1967 legislative campaign of the Union of Democrats for the Fifth Republic to a narrow victory. Pompidou was widely regarded as being responsible for the peaceful resolution of the student uprising of May 1968. His strategy was to break the coalition of students and workers by negotiating with the trade-unions and employers (Grenelle conference). Until this crisis, he was the Prime Minister of a quiet and prosperous France.

However, during the events of May 1968, disagreements arose between Pompidou and de Gaulle. Pompidou did not understand why the President did not inform him of his departure to Baden-Baden on 29 May. Their relationship, until then very good, would be strained from then on. Pompidou led and won the 1968 legislative campaign, then resigned. Nevertheless, in part due to his actions during the May 1968 crisis, he appeared as the natural successor to de Gaulle. Pompidou announced his candidature for the Presidency in January 1969. Some weeks later, his wife's name was mentioned in the Markovic scandal, thus appearing to confirm her husband's status as a cuckold. Pompidou was certain that de Gaulle's inner circle was responsible for this smear.

After the failure of the 1969 referendum, de Gaulle resigned and Pompidou was elected president of France, defeating in the second round by a wide margin the Centrist President of the Senate and Acting President Alain Poher. Though a Gaullist, Pompidou was more pragmatic than de Gaulle, notably allowing the United Kingdom to join the European Community in 1973. He embarked on an industrialisation plan and initiated the Ariane space project. He was sceptical about the "New Society" programme of his prime minister, Jacques Chaban-Delmas. In 1972, Chaban-Delmas was replaced by Pierre Messmer, a more conservative Gaullist.

While the left-wing opposition got organised in proposing a Common Programme before the 1973 legislative election, he widened out his "presidential majority" by including the Centrist pro-European parties.

While still in office, Pompidou died unexpectedly on 2 April 1974, 9 PM, from Waldenström macroglobulinemia.

Pompidou's wife Claude Pompidou lived more than 30 years longer than Pompidou. Pompidou had one foster son, Alain Pompidou, former president of the European Patent Office.

Pompidou's time in office was marked by a constant effort to modernise France's capital city. This can be seen through his construction of a modern art museum, the Centre Beaubourg (renamed Centre Pompidou after his death), on the edge of the Marais area of Paris. Other attempts at modernisation included tearing down the open air markets at Les Halles and replacing it with the shopping mall of the same name, building the Montparnasse Tower, and constructing an expressway on the right bank of the Seine.