Vyshinsky was born into a Polish Catholic family in Odessa, who later moved to Baku. He became a Menshevik in 1903 and took an active part in the 1905 Russian Revolution at Baku, for which he was convicted and imprisoned in the Bailov prison. Here he first met Stalin: a fellow inmate with whom he engaged in ideological disputes. After graduating in Law at Kiev University, Vyshinsky became a practicing lawyer in Moscow. In 1917 he undersigned an order to arrest Vladimir Lenin, according to the decision of the Russian Provisional Government. In 1920, he joined the Bolsheviks.

He carried out administrative preparations for a "systematic" drive "against harvest wreckers and grain thieves."

In 1935 he became Prosecutor General of the USSR, the legal mastermind of Joseph Stalin's Great Purge. He is widely (and wrongly) cited for the principle that "confession of the accused is the queen of evidence" despite his monograph Theory of Judicial Proofs in Soviet Justice (which was awarded the Stalin Prize in 1947) stating directly the opposite. He first became a nationally known public figure as a result of the Semenchuk case of 1936. Konstantin Semenchuk was the head of the Glavsevmorput station on Wrangel Island; he was accused of oppressing and starving the local Eskimos and of ordering his subordinate, the sled driver Stepan Startsev, to murder Dr. Nikolai Vulfson, who had attempted to stand up to Semenchuk, on 27 December 1934 (though there were also rumors that Startsev had fallen in love with Vulfson's wife, Dr. Gita Feldman, and killed him out of jealousy). The case came to trial before the Supreme Court of the RSFSR in May 1936; both defendants, attacked by Vyshinsky as "human waste," were found guilty and shot, and "the most publicized result of the trial was the joy of the liberated Eskimos."

Three months later, Vyshinsky achieved international fame as the prosecutor at the Zinoviev - Kamenev trial, the first of the Moscow Trials during the Great Purge, lashing its defenseless victims with vituperative rhetoric:

Shoot these rabid dogs. Death to this gang who hide their ferocious teeth, their eagle claws, from the people! Down with that vulture Trotsky, from whose mouth a bloody venom drips, putrefying the great ideals of Marxism!... Down with these abject animals! Let's put an end once and for all to these miserable hybrids of foxes and pigs, these stinking corpses! Let's exterminate the mad dogs of capitalism, who want to tear to pieces the flower of our new Soviet nation! Let's push the bestial hatred they bear our leaders back down their own throats!

During the trials, Vyshinsky misappropriated the house and money of Leonid Serebryakov, one of the defendants of the infamous Moscow Trials, who was later executed.

In June, 1940, Vyshinsky was sent to the Republic of Latvia to supervise the establishment of a puppet government and incorporation of that country into the USSR, and later arranged for a communist regime to assume control of Romania in 1945. In 1953 he was among the chief figures accused by the U.S. Congress Kersten Committee, during its investigation of the Soviet occupation of the Baltic states.

He was responsible for the Soviet preparations for the trial of the major war criminals by the International Military Tribunal.

The positions he held included those of vice - premier (1939 – 1944), Deputy Minister of Foreign Affairs (1940 – 1949), Minister for Foreign Affairs (1949 – 1953), Academician of the Soviet Academy of Sciences from 1939, and permanent representative of the Soviet Union to the United Nations.

He died in 1954 while in New York and was buried near Red Square.

Vyshinsky was the director of the Soviet Academy of Sciences' Institute of State and Law. Until the period of destalinization, the Institute of State and Law was named in his honor. During his tenure as director of the ISL, Vyshinsky oversaw the publication of several important monographs on the general theory of state and law.

Vyshinsky married Kapa Mikhailova and had a daughter named Zinaida Andreyevna Vyshinskaya (born 1909).