In the 1920s, Graziani commanded the Italian forces in Libya. He was responsible for pacifying the Senussi rebels. During this so-called "pacification", he was responsible for the construction of several concentration camps and labor camps, where tens of thousands of Libyan prisoners died, if not killed directly by hanging, like Omar Mukhtar, or bullets, then indirectly by starvation or disease. His deeds earned him the nickname "the Butcher of Fezzan" among the Arabs, but was called by the Italians the Pacifier of Libya (Pacificatore della Libia).

From 1926 to 1930, Graziani was the Vice Governor of Italian Cyrenaica in Libya. In 1930, he became Governor of Cyrenaica and held this position until 1934 when it was determined that he was needed elsewhere. In 1935, Graziani was made the Governor of Italian Somaliland.

From 1935 to 1936 during the Second Italo - Abyssinian War, Graziani was the commander of the southern front. His army invaded Ethiopia from Italian Somaliland and he commanded Italian forces in the Battle of Genale Doria and the Battle of the Ogaden. However, Graziani's efforts in the south were secondary to the main invasion launched from Eritrea by General Emilio De Bono and continued by Marshal of Italy Pietro Badoglio. It was Badoglio and not Graziani who entered Addis Ababa in triumph after his "March of the Iron Will". But it was Graziani who said: "The Duce will have Ethiopia, with or without the Ethiopians."

Addis Ababa fell to Badoglio on May 5, 1936. Graziani had wanted to reach Harar before Badoglio reached Addis Ababa, but failed to do so. Even so, on May 9, Graziani was awarded for his role as commander of the southern front with a promotion to the rank of Marshal of Italy. During his tour of an Ethiopian Orthodox church in Dire Dawa, Graziani fell into a pit covered by an ornate carpet, a trap that he believed had been set by the Ethiopian priests to injure or kill him. As a result he held Ethiopian clerics in deep suspicion.

After the war, Graziani was made Viceroy of Italian East Africa and Governor - General of Shewa/Addis Ababa. After an unsuccessful attempt to kill him by two Eritreans on 19 February 1937, Graziani ordered a bloody and indiscriminate reprisal upon the conquered country, later remembered by Ethiopians as Yekatit 12: thousands of civilian inhabitants of Addis Ababa were killed indiscriminately, another 1,469 were summarily executed by the end of the next month, and over one thousand Ethiopian notables were imprisoned then exiled from Ethiopia. He became known as "the Butcher of Ethiopia". Also in connection with the attempt on his life, Graziani authorized the massacre of the monks of the ancient monastery of Debre Libanos and the large number of pilgrims who had traveled there to celebrate the feast day of the founding saint of the monastery. Graziani's suspicion of the Ethiopian Orthodox clergy (and the fact that the wife of one of the assassins had briefly taken sanctuary at the monastery) had convinced him of the complicity of the monks in the attempt on his life.

From 1939 to 1941, Graziani was the Commander-in-Chief of the Italian Royal Army's General Staff.

At the start of World War II, Graziani was still the Commander-in-Chief of the Italian Royal Army's General Staff. After the death of Marshal Italo Balbo in a friendly fire incident on 28 June 1940, Graziani took his place as the Commander-in-Chief of Italian North Africa and as the Governor General of Libya.

Initially giving Graziani a deadline of 8 August, Italian dictator Benito Mussolini ordered Graziani to invade Egypt with the Tenth Army. Graziani expressed doubts about the ability of his largely un-mechanized force to defeat the British and put off the invasion for as long as he could. However, faced with demotion, Graziani ultimately followed orders and elements of the Tenth Army invaded Egypt on 9 September. The Italians made modest gains into Egypt and then prepared a series of fortified camps to defend their positions. In 1941, Graziani resigned his commission after the British counterattacked and the Tenth Army was completely defeated by them during Operation Compass. On 25 March 1941, Graziani was replaced by General Italo Gariboldi.

Graziani was the only Italian marshal to remain loyal to Mussolini after Dino Grandi's Grand Council of Fascism coup. He was appointed Minister of Defence of the Italian Social Republic and oversaw the mixed Italo - German LXXXXVII "Liguria" Army (Armee Ligurien) commanded by General Alfredo Guzzoni.

At the end of the war, Graziani spent a few days in San Vittore prison in Milan before being transferred to Allied control. He was brought back to Africa in Anglo - American custody, staying there until February 1946. Allied forces then felt the danger of assassination or lynching had passed, and returned him to Procida prison in Italy.

In 1950, a military tribunal sentenced Graziani to a further 19 years' jail for high treason, as punishment for his collaboration with the Nazis; but he was released after serving only a few months of the sentence. He was never prosecuted for specific war crimes. Unlike the Germans and Japanese, Italians were not subjected to prosecutions. In 1955 he died of natural causes.