Born in Kherson in the Russian Empire (today in Ukraine), Sharett emigrated to Ottoman controlled Palestine in 1906. In 1910 his family moved to Jaffa, and they became one of the founding families of Tel Aviv.

He graduated from the first class of the Herzliya Hebrew High School, even studying music at the Shulamit Conservatory. He then went off to Istanbul to study law at the University of Kushta (now Istanbul University), the same university Yitzhak Ben-Zvi and David Ben Gurion studied at. However, his time there was cut short due to the outbreak of World War I. He subsequently served as a 1st Lt., interpreter in the Turkish army.

After the war, he worked as an Arab affairs and land purchase agent for the Assembly of Representatives of the Yishuv. He also became a member of Ahdut Ha'Avoda, and later of Mapai.

In 1922 he went to the London School of Economics, and while there he actively edited the Workers of Zion. He then edited the Davar newspaper from 1925 until 1931.

In 1931, after returning to Palestine, he became the secretary of the Jewish Agency's political department. In 1933 he became its head, and he held that position until the formation of Israel in 1948.

Sharett was one of the signatories of Israel's Declaration of Independence. He was first elected to the Knesset in 1949, and served as Israel's first Minister of Foreign Affairs. In this role he established diplomatic relations with dozens of nations, and got Israel into the UN. He held this role until 1956.

In the debate on how to deal with the increasing infiltration of fedayeen across the borders in the years leading to the 1956 Suez Crisis, Sharett was skeptical of retaliatory operations.

Sharrett met with Pius XII in 1952 in an attempt to improve relations with the Holy See, although this was to no avail.

In December 1953 David Ben-Gurion retired from politics (temporarily as it turned out), and Sharett was elected to take his place. During his time as prime minister the Palestinian - Israeli conflict intensified and the Lavon Affair occurred. As a result David Ben-Gurion returned to the government as Defense Minister. At the next elections Ben Gurion replaced Sharett as head of the list and became prime minister.

After stepping down as Minister of Foreign Affairs, Sharett retired. During his retirement he became chairman of Am Oved publishing house, Chairman of Beit Berl College, and Chairman of the World Zionist Organization and the Jewish Agency. He died in 1965 in Jerusalem and was buried in Tel Aviv's Trumpeldor Cemetery.

Sharett wrote personal diaries, which were published posthumously. His son Yaakov founded an Institute for his heritage. Many cities have streets and neighborhoods named after him.

Since 1987, Sharett has appeared on the 20 NIS bills. The bill first featured Sharett, with the names of his books in small print, and with a small image of him presenting the Israeli flag to the United Nations in 1949. On the back of the bill, there was an image of the Herzliya Hebrew High School, from which he graduated. In 1998 the bill went through a graphic revision, the list of Sharett's books on the front side was replaced by part of Sharett's 1949 speech in the UN. The back side now features an image of Jewish Brigade volunteers, part of a speech by Sharett on the radio after visiting the Brigade in Italy, and the list of his books in small print.

In 2005, he was voted the 150th greatest Israeli of all time, in a poll by the Israeli news website Ynet to determine whom the general public considered the 200 Greatest Israelis.