He was born in Vaihingen an der Enz, Kingdom of Württemberg, the son of minor Swabian nobility. He studied law in Tübingen and in Berlin. After graduating in 1892 he joined a local law firm in his home town. He joined the civil service in 1901 and worked for the Foreign Office in Berlin. In 1903 he was assigned to the embassy in London as Vice-Consul and from 1909 he was Legationsrat (legation counsel) at the embassy. In 1914 he was sent to the embassy in Constantinople.

On 30 May 1901 he married Marie Auguste Moser von Filseck (1875 - 1960) in Stuttgart. His son Konstantin was born in 1902, followed by his daughter Winifred in 1904.

During World War I he served as an officer with an infantry regiment until 1916 when he was badly wounded. In December 1914 he was awarded the Iron Cross. He returned to the diplomatic service in the Ottoman Empire. Towards the end of the war he headed the Württemberg government.

In 1919, Neurath returned to diplomacy, being assigned to the embassy in Copenhagen as Minister to Denmark. From 1921 until 1930 he was the ambassador to Rome; he was not overly impressed with Italian fascism. He was considered for a post in the new cabinet by Paul von Hindenburg in 1929. In 1930 he returned to head the embassy in London.

Neurath was recalled to Germany in 1932 and became Minister of Foreign Affairs under Franz von Papen in June. He continued to hold that position under Kurt von Schleicher and then under Adolf Hitler. During the early days of Hitler's rule, Neurath lent an aura of respectability to Hitler's expansionist foreign policy. In May 1933, the American chargé d'affaires reported that "Baron von Neurath has shown such a remarkable capacity for submitting to what in normal times could only be considered as affronts and indignities on the part of the Nazis, that it is still quite a possibility that the latter should be content to have him remain as a figurehead for some time yet". He was involved in the German withdrawal from the League of Nations in 1933, the negotiations of the Anglo-German Naval Accord (1935) and the remilitarization of the Rhineland. Neurath joined the Nazi Party in 1937 and in September of that year he was awarded an honorary rank of Obergruppenführer in the SS.

On 4 February 1938, Neurath was sacked as Foreign Minister. He felt his office was marginalised and was not in favor of Hitler's aggressive war plans, which were detailed in the Hossbach Memorandum of 5 November 1937. He was succeeded by Joachim von Ribbentrop, but he remained in government as a minister without portfolio. In March 1939, Neurath was appointed Protector (Reichsprotektor) of the Protectorate of Bohemia and Moravia, serving Hitler's personal representative in the protectorate. Soon after his arrival, he instituted harsh press censorship and banned political parties and trade unions. He ordered a harsh crack down on protesting students in October and November 1939 (1200 student protesters were sent to concentration camps and nine were executed). Draconian as these measures were, Hitler felt his rule was too lenient, and in September 1941 he was relieved of his day-to-day powers and replaced by Reinhard Heydrich. Neurath attempted to resign in 1941 but his resignation was not accepted until August 1943.

Late in the war, Neurath had contacts with the German resistance.

Neurath was tried at Nuremberg in 1946, where he was defended by Otto von Ludinghausen. The Allies accused him of "conspiracy to commit crimes against peace; planning, initiating and waging wars of aggression; war-crimes and crimes against humanity". He was found guilty by the Allied powers on all four counts and was sentenced to fifteen years imprisonment. He was held as a war criminal in Spandau Prison until 1954, when he was released due to ill health, having suffered a heart attack. He died in Enzweihingen in 1956.