Voroshilov was appointed People's Commissar for Defence in 1934 and a Marshal of the Soviet Union in 1935. As a confirmed sycophant of Joseph Stalin, Voroshilov played a central role in Stalin's Great Purge of the 1930s, denouncing many of his own military colleagues and subordinates when asked to do so by Stalin. He went so far as to write personal letters to exiled former Soviet officers and diplomats such as Mikhail Ostrovsky to return voluntarily to the Soviet Union, reassuring them that they would not face retribution from authorities (they did). Voroshilov personally signed 185 documented execution lists, fourth among the Soviet leadership after Molotov, Stalin and Kaganovich.

During World War II, Voroshilov was a member of the State Defense Committee. Voroshilov commanded Soviet troops during the Winter War from November 1939 to January 1940, but, due to his poor planning and overall incompetence, the Red Army suffered about 185,000 casualties. When the leadership gathered at Stalin's dacha at Kuntsevo, Stalin shouted at Voroshilov who replied in kind, blaming the failure on Stalin for killing the Red Army's best generals in his purges. Voroshilov followed this by smashing a platter of roast suckling pig on the table. Nikita Khrushchev said it was the only time he ever witnessed such an outburst. Voroshilov still became the scapegoat for the initial failures in Finland. He was later replaced as Defence Commissar by Semyon Timoshenko. Voroshilov was then made Deputy Premier responsible for cultural matters.

Voroshilov initially argued that thousands of Polish army officers captured in September 1939 should be released but later signed the order for their execution. Vasili Blokhin of the NKVD personally shot 7000 of them in 28 nights using German Walther pistols.
After the German invasion of the Soviet Union in June 1941, Voroshilov was made commander of the short lived Northwestern Direction, controlling several fronts. Working alongside military commander Andrei Zhdanov as German advances threatened to cut off Leningrad he displayed considerable personal bravery, prancing around in defiance of heavy shelling at Ivanovskoye; at one point he rallied retreating troops and personally led a counter - attack against German tanks armed only with a pistol. Failing to prevent the Germans from surrounding Leningrad however, he was dismissed from that post and replaced by the far abler Georgy Zhukov on 8 September 1941.

In an embarrassing incident at the 1943 Tehran Conference, during a ceremony to receive the "Sword of Stalingrad" from Winston Churchill, he took the sword from Stalin but then allowed the sword to fall from its scabbard onto his toes in the presence of the Big Three wartime leaders.

In 1945-1947, he supervised the establishment of the communist regime in Hungary.

After Khruschev removed most of the old Stalinists, like Molotov and Malenkov from the party, Voroshilov's career began to fade. On 7 May 1960, the Supreme Soviet of the Soviet Union granted Voroshilov's request for retirement and elected Leonid Brezhnev chairman of the Presidium of the Supreme Council (the head of state). The Central Committee also relieved him of duties as a member of the Party Presidium (as the Politburo had been called since 1952) on 16 July 1960. In October 1961, his political defeat was complete at the 22nd party congress when he was excluded from election to the Central Committee. A curious story surrounds Voroshilov's last days as the head of state. During one dinner meeting with the Central Committee, every one else present ignored Voroshilov and gave him the cold shoulder. Their snubs made Voroshilov realize that all his colleagues had already decided to fire him, so he decided to preempt them and retire by his own will.

Following Khrushchev's fall from power, Soviet leader Brezhnev brought Voroshilov out of retirement into a figurehead political post. Voroshilov was again re-elected to the Central Committee in 1966. Voroshilov was awarded a second medal of Hero of the Soviet Union 1968. He died in 1969 in Moscow and was buried in the Kremlin Wall Necropolis. The KV series of tanks, used in World War II, was named after Voroshilov. Two towns were also named after him: Voroshilovgrad in Ukraine (now changed back to the historical Luhansk) and Voroshilov in the Soviet Far East (now renamed Ussuriysk after the Ussuri river), as well as the General Staff Academy in Moscow. Stavropol was called Voroshilovsk from 1935 to 1943.

Voroshilov was married to Ekaterina Davidovna, born Golda Gorbman, who came from a Jewish Ukrainian family from Mardarovka. She changed her name when she converted to Orthodox Christianity in order to be allowed to marry Voroshilov. They met while both exiled in Arkhangelsk, where Ekaterina was sent in 1906. While both serving on the Tsaritsyn Front in 1918, where Ekaterina was helping orphans, they adopted a four year old orphan boy who they named Petya. They also adopted the children of Mikhail Frunze, following his death in 1925. During Stalin's rule they lived in the Kremlin at the Horse Guards.