Raab was born in a middle class Catholic family in Lower Austria. He attended a Catholic high school and enrolled at the Vienna University of Technology in 1911. He was drafted into the Austrian Army before graduation and fought on the Russian and Italian fronts of World War I. After the defeat of Central Powers, Raab returned to the university and engaged in politics. He married Harmine Haumer in 1923.

The death of his father and the beginning of his political career compelled Raab to drop out of the university in 1925. In 1927 he was elected to the Parliament of Austria (Christian Social Party, Lower Austria) for the first time. He was active in the Heimwehr, the paramilitary arm of right wing political forces, and was appointed Heimwehr chief for Lower Austria in 1928. In 1933 he joined the Fatherland Front, a right wing coalition led by Engelbert Dollfuss. During the austrofascist period of 1934 – 1938 Raab progressed through the ranks of the corporate state, and was appointed Minister of Commerce by Kurt von Schuschnigg just four weeks before the Anschluss. Raab also revealed his anti - semitism as a parliament member in 1931 when he famously branded the Socialist leader Otto Bauer an "insolent Jewish pig."

Raab was ousted after the Anschluss but, unlike many other political leaders, escaped death or imprisonment through the help of the Lower Austrian Nazi Gauleiter, whom he knew personally. He was never involved in the Austrian resistance but kept in touch with the old Christian Democrat elite.

In April 1945, Raab was made a member of Karl Renner's provisional government, formed in the Soviet occupation zone. Raab represented the forces of the past that were unacceptable to the Soviets and the left wing majority, and for a while was "relegated to the back seat". Raab co-founded the Austrian People's Party (ÖVP), which denounced the dark legacy of the 1930s, and assumed leadership of ÖVP parliamentary group after the legislative elections held in November 1945. He expanded his influence through presidency in the national Chamber of Commerce, the institution tasked with managing social partnership of the government, the political parties, the entrepreneurs and the employees' trade unions. He clearly favored a free market and minimal government regulation of the economy.

Raab succeeded Leopold Figl as the ÖVP party chairman in 1951 and as the elected Federal Chancellor of Austria in 1953. Despite clearly Western attitudes, Raab established excellent relations with post Stalin Soviet Union. In February 1955 Vyacheslav Molotov proposed resuming the talks on Austrian independence. On April 12, 1955 Raab arrived in Moscow for the negotiations that paved the way to the Austrian State Treaty concluded in Vienna on May 15. Austria declared neutrality, as did all individual Bundesländer.

The success of 1955 marked the peak of ÖVP influence. The party won 46% of the popular vote in the 1956 elections, Raab retained his seat as the Federal Chancellor. Despite criticism within the party, Raab strongly favored a tight coalition with Social Democrats. In 1957 he and trade union chief Johann Bohm co-founded the Joint Commission on Wages and Prices, the social partnership institution that became a cornerstone of Austrian corporatism.

In 1959 Raab suffered a heart attack. His own career and his party's influence declined. In 1961 he passed ÖVP leadership to Alfons Gorbach. In 1963 Raab lost the presidential elections to Adolf Schärf. His health rapidly deteriorated, and he died on January 8, 1964.