When the Russian government began arresting Bolsheviks in 1906, Litvinov left the country and spent the next ten years an émigré and arms dealer for the party. Based in Paris he travelled throughout Europe, sometimes posing as a procurement officer from Ecuador, buying rifles in Belgium, Germany and the Austro - Hungarian empire. Despite some notable disasters, such as the wrecking of a gun running yacht on the Romanian coast, he had some success in smuggling these arms into Russia via Finland and the Black Sea.

In 1907 he attended the 5th Party Congress of the Russian Social Democratic Labour Party in London. Initially he had to rely on the charity of the Rowton Houses for accommodation in London. However, eventually the party arranged a rented house for him that he shared with Joseph Stalin, who had also been anxious to find more comfortable housing than the Rowton poor hostels.

In 1908 he was arrested under the name Meer Wallach by French police, while carrying twelve 500 ruble banknotes that were taken from a bank in Tiflis during the 1907 Tiflis bank robbery that took place on 26 June 1907. Litvinov was deported from France to England and lived in London, where he was active in the International Socialist Bureau. In early 1918, he was frequently reported in the British and American press as the foreign representative of the Bolsheviks in the UK, a claim given some substance by R.H. Bruce Lockhart, a British agent in Moscow at the time. In England he met and married Ivy Lowe, daughter of one of the most distinguished Jewish families in Britain. Miss Lowe’s ancestors emigrated from Hungary to England following the unsuccessful 1848 revolution. Her father, Walter Lowe, was a prominent writer and a close friend of H.G. Wells. They enjoyed frequent exchanges, Lowe espousing the Jewish point of view, and Wells a secular philosophy.

For a while Litvinov also lived in North Belfast in Northern Ireland.

After the October Revolution of 1917, Litvinov was appointed by Vladimir Lenin as the Soviet government's representative in Britain. His accreditation was never officially formalised, and his position as an unofficial diplomatic contact was analogous to that of Robert Lockhart. In 1918, Litvinov was arrested by the British government and held until exchanged for Lockhart, who had been imprisoned in Russia. The following year he published the English tract The Bolshevik Revolution: Its Rise and Meaning, distributed by the British Socialist Party.

Litvinov was then employed as the Soviet government's roaming ambassador. It was largely through his efforts that Britain agreed to end its economic blockade of the Soviet Union. Litvinov also negotiated several trade agreements with European countries. In February 1929 he concluded the Litvinov's Pact in Moscow, signed by the Soviet Union, Poland, Romania, Latvia and Estonia, in which those countries promised not to use force to settle their disputes (this was seen as an 'Eastern Kellogg - Briand Pact').

In 1930, Joseph Stalin appointed Litvinov as People's Commissar for Foreign Affairs. A firm believer in collective security, Litvinov worked very hard to form a closer relationship with France and Britain. In 1933 he successfully persuaded the United States to officially recognize the Soviet government. Franklin D. Roosevelt sent comedian Harpo Marx to the Soviet Union as a good-will ambassador, and Litvinov and Marx became friends and even performed a routine on stage together. Litvinov also actively facilitated the acceptance of the USSR into the League of Nations where he represented his country in 1934 — 1938.

After the Munich Agreement German media derided Litvinov about his Jewish ancestry, referring to him as "Finkelstein - Litvinov."

On 3 May 1939 Stalin replaced Litvinov with Vyacheslav Molotov. That night, NKVD troops surrounded the offices of the commissariat of foreign affairs. The phone at Litvinov's dacha was disconnected and, the following morning, Molotov, Georgii Malenkov, and Lavrenty Beria arrived at the commissariat to inform Litvinov of his dismissal. After Litvinov's dismissal, many of his aides were arrested and beaten, evidently in an attempt to extract compromising information.

The replacement of Litvinov with Molotov significantly increased Stalin's freedom to maneuver in foreign policy. The dismissal of Litvinov, whose Jewish ethnicity was viewed disfavorably by Nazi Germany, removed an obstacle to negotiations with Germany. Stalin immediately directed Molotov to "purge the ministry of Jews." Recalling Stalin’ s order, Molotov commented: `Thank God for these words! Jews formed an absolute majority in the leadership and among the ambassadors. It wasn’t good."

Given Litvinov's prior attempts to create an anti - fascist coalition, association with the doctrine of collective security with France and Britain, and pro-Western orientation by Kremlin standards, his dismissal indicated the existence of a Soviet option of rapprochement with Germany. Likewise, Molotov's appointment was a signal to Germany that the USSR was open to offers. The dismissal also signaled to France and Britain the existence of a potential negotiation option with Germany. One British official wrote that Litvinov's disappearance also meant the loss of an admirable technician or shock absorber, while Molotov's "modus operandi" was "more truly Bolshevik than diplomatic or cosmopolitan."

With regard to the signing of a German - Soviet nonaggression pact with secret protocols dividing eastern Europe three months later, Hitler remarked to military commanders that "Litvinov's replacement was decisive." A German official told the Soviet Ambassador that Hitler was also pleased that Litvinov's replacement, Molotov, was not Jewish. Hitler also wrote to Mussolini that Litvinov's dismissal demonstrated that Kremlin's readiness to alter relations with Berlin, which lead to "the most extensive nonaggression pact in existence." When Litvinov was later asked about the reasons for his dismissal, he asked, "Do you really think that I was the right person to sign a treaty with Hitler?"

Litvinov, like Churchill, had misgivings about Munich. Following the invasion of the U.S.S.R. on 22 June 1941, Litvinov said on a radio broadcast to Britain and the U.S., "We always realized the danger which a Hitler victory in the West could constitute for us," which one commentator described as, "in the tactful language which underlings must apply to dictators... tantamount to 'I told you so.'". With the Soviet Union embroiled in the Great Patriotic War, Joseph Stalin appointed Litvinov as Deputy Commissar of Foreign Affairs. Litvinov also served as Ambassador to the United States from 1941 to 1943 and significantly contributed to the lend lease agreement signed in 1941.

In 1945, he was mentioned by Halvdan Koht among seven candidates that were qualified for the Nobel Prize in Peace. However, he did not explicitly nominate any of them. The person actually nominated was Cordell Hull.

After Litvinov's death, his wife returned to live in Britain. His last words, directed at his wife, were "Englishwoman go home".

His grandson Pavel Litvinov is a Russian physicist, writer and a Soviet era dissident.