In the 1920s, Graziani commanded the Italian forces in Libya. He was responsible for suppressing the Senussi rebellion. During this so-called "pacification", he was responsible for the construction of several concentration camps and labor camps, where thousands of Libyan prisoners died, some killed directly by hanging, like Omar Mukhtar, or were shot, but most indirectly by starvation or disease. His deeds earned him the nickname "the Butcher of Fezzan" among the Arabs, but was called by the Italians the Pacifier of Libya (Pacificatore della Libia).

From 1926 - 1930, Graziani was the Vice Governor of Italian Cyrenaica in Libya. In 1930, he became Governor of Cyrenaica and held this position until 1934, when it was determined that he was needed elsewhere. In 1935, Graziani was made the Governor of Italian Somaliland.

In 1935 - 1936, during the Second Italo - Abyssinian War, Graziani was the commander of the southern front. His army invaded Ethiopia from Italian Somaliland and he commanded Italian forces at the Battles of Genale Doria and the Ogaden. However, Graziani's efforts in the south were secondary to the main invasion launched from Eritrea by Generale Emilio De Bono and continued by Marshal of Italy Pietro Badoglio. It was Badoglio and not Graziani who entered Addis Ababa in triumph after his "March of the Iron Will". But it was Graziani who said: "The Duce will have Ethiopia, with or without the Ethiopians."

Addis Ababa fell to Badoglio on 5 May 1936. Graziani had wanted to reach Harar before Badoglio reached Addis Ababa, but failed to do so. Even so, on 9 May, Graziani was awarded for his role as commander of the southern front with a promotion to the rank of Marshal of Italy. During his tour of an Ethiopian Orthodox church in Dire Dawa, Graziani fell into a pit covered by an ornate carpet, a trap that he believed had been set by the Ethiopian priests to injure or kill him. As a result he held Ethiopian clerics in deep suspicion.

After the war, Graziani was made Viceroy of Italian East Africa and Governor - General of Shewa/Addis Ababa. After an unsuccessful assassination attempt by two Eritreans on 19 February 1937 (and after other murders of Italians in occupied Ethiopia), Graziani ordered a bloody and indiscriminate reprisal upon the conquered country, later remembered by Ethiopians as Yekatit 12: thousands of civilian inhabitants of Addis Ababa were killed indiscriminately, another 1,469 were summarily executed by the end of the next month, and over one thousand Ethiopian notables were imprisoned then exiled from Ethiopia. He became known as "the Butcher of Ethiopia". Also in connection with the attempt on his life, Graziani authorized the massacre of the monks of the ancient monastery of Debre Libanos and the large number of pilgrims who had traveled there to celebrate the feast day of the founding saint of the monastery. Graziani's suspicion of the Ethiopian Orthodox clergy (and the fact that the wife of one of the assassins had briefly taken sanctuary at the monastery) had convinced him of the complicity of the monks in the attempt on his life.

From 1939 - 1941, Graziani was the Commander - in - Chief of the Regio Esecito′s General Staff.

At the start of World War II, Graziani was still Commander - in - Chief of the Regio Esercito′s General Staff. After the death of Marshal Italo Balbo in a friendly fire incident on 28 June 1940, Graziani took his place as Commander - in - Chief of Italian North Africa and as the Governor General of Libya.

Initially giving Graziani a deadline of 8 August, Italian dictator Benito Mussolini ordered Graziani to invade Egypt with the 10th Army. Graziani expressed doubts about the ability of his largely un-mechanized force to defeat the British and put off the invasion for as long as he could.

However, faced with demotion, Graziani ultimately followed orders and elements of the 10th Army invaded Egypt on 9 September. The Italians made modest gains into Egypt and then prepared a series of fortified camps to defend their positions. In 1941, Graziani resigned his commission after the British counterattacked and the 10th Army was completely defeated by them during Operation Compass.

On 25 March 1941, Graziani was replaced by Generale Italo Gariboldi, and remained inactive for two years.

Graziani was the only Italian Marshal to remain loyal to Mussolini after Dino Grandi's Grand Council of Fascism coup.

He was appointed Minister of Defense of the Italian Social Republic by the "Duce" and oversaw the mixed Italo - German Army Group Liguria (Armee Ligurien) commanded by General Alfredo Guzzoni. He was able to obtain a defeat of the Allies in the "battle of Garfagnana" in December 1944.

At the end of the war, Graziani spent a few days in San Vittore prison in Milan before being transferred to Allied control. He was brought back to Africa in Anglo - American custody, staying there until February 1946. Allied forces then felt the danger of assassination or lynching had passed (many thousands of fascists were murdered in Italy in summer and fall 1945), and returned him to Procida prison in Italy.

In 1948, a military tribunal sentenced Graziani to a further 19 years in jail, as punishment for his collaboration with the Nazis; but he was released after serving only a few months of the sentence. He was never prosecuted for specific war crimes. Unlike the Germans and Japanese, Italians were not subjected to prosecutions by Allied tribunals.

Graziani in the early 1950s participated in some political activity with the neofascist Movimento Sociale Italiano, and became the "Honorary President" of this Italian party in 1953. In January 1955, at 72, he died of natural causes in Rome.

The League of Nations failed to prosecute Graziani and other Italian officers before World War II.

In 1943 the Allied Powers agreed to create a new body to replace the League: the United Nations. The "United Nations War Crimes Commission" was created to investigate war crimes. On March 4, 1948 charges against Graziani were presented to the United Nations War Crimes Commission. The commission was given evidence of the Italian policy of systematic terrorism and Graziani’s self - admitted intention to execute all Amharas authorities, and cited a telegram from Graziani to General Nasi, in which he had written, “Keep in mind also that I have already aimed at the total destruction of Abyssinian chiefs and notables and that this should be carried out completely in your territories.” The UN commission agreed that there was a prima facie case against eight Italians including Graziani.

The British Foreign Office consistently opposed Ethiopia’s inclusion in this Commission and the trial of Italian crimes committed during the 1935 – 36 invasion. Ethiopian efforts to bring Graziani to trial were frustrated by intransigence, by both Italy and Britain, and were finally abandoned, under pressure from the Foreign Office, whose support the Ethiopian Government considered essential for its claim to Eritrea.

However, in 1948 an Italian tribunal condemned Graziani to 19 years, but he served only four months because his lawyers demonstrated that he "received orders".