Pupin was born in the village of Idvor (part of Kovačica) in Banat, Kingdom of Hungary, Austrian Empire, (today in Serbia).

After the sudden death of his father, Pupin emigrated to the United States of America in 1874. Pupin says, "I bless the stars that the immigration laws were different then than they are now ... My admission by a special favor of the examiners was a puzzle and a disappointment to me." After a short time as a farm laborer in Delaware, he spent the next few years in a series of menial jobs in New York City (most notably, the biscuit factory on Cortlandt Street in Manhattan), learning English and American ways; the library and lectures at Cooper Union were an important resource for him.

He entered Columbia College in 1879, where he became known as an exceptional athlete and scholar. A friend of Pupin's predicted that his physique would make him a splendid oarsman, and that Columbia would do anything for a good oarsman. A popular student, he was elected president of his class in his Junior year. He graduated with honors in 1883 and became an American citizen at the same time. He obtained his Ph.D. at the University of Berlin under Hermann von Helmholtz, and in 1889 he returned to Columbia University to become a lecturer of mathematical physics in the newly formed Department of Electrical Engineering. Pupin's research pioneered carrier wave detection and current analysis.

Pupin's 1899 patent for loading coils, archaically called "Pupin coils", followed closely on the pioneering work of the English physicist and mathematician Oliver Heaviside, which predates Pupin's patent by some seven years. The importance of the patent was made clear when the American rights to it were acquired by American Telephone & Telegraph (AT&T), making him wealthy. Although AT&T bought Pupin's patent, they made little use of it, as they already had their own development in hand led by George Campbell and had up to this point been challenging Pupin with Campbell's own patent. AT&T were afraid they would lose control of an invention which was immensely valuable due to its ability to greatly extend the range of long distance telephones.

Pupin was among the first to replicate Roentgen's production of x-rays in the United States. He in 1896 invented the method of placing a sheet of paper impregnated with fluorescent dyes next to the photographic plate, thereby permitting an exposure of only a few seconds, rather than that of an hour or more. He also carried out one of the first medically oriented studies of the utility of x-rays in the United States. Shortly afterward, in April 1896, he contracted pneumonia, and nearly died. His wife, who nursed him, also contracted it, and died. He never returned to his studies of x-rays.

In 1901, he became a professor, and, in 1931, a professor emeritus of Columbia University. Professor Pupin was a resident of New York, New York, and Norfolk, Connecticut, where he built a Serbian style manor house on his estate, Hemlock Hill Farm.

In 1911 Pupin became a consul of the Kingdom of Serbia in New York. In his speech to Congress on January 8, 1918, known as the Fourteen Points speech, U.S. president Woodrow Wilson, inspired by his conversations with Pupin, insisted on the restoration of Serbia and Montenegro, as well as autonomy for the peoples of the Austro - Hungarian monarchy.

Michael Pupin's autobiography, "From Immigrant to Inventor", won the Pulitzer Prize in 1924. He also wrote "The New Reformation" (1927) and "Romance of the Machine" (1930), as well as many technical papers. In his many popular writings, Pupin advanced the view that modern science supported and enhanced belief in God. Pupin was active with the Serb émigré societies in the USA, he was the first president and founder of the Serbian National Defense Council of America. In 1918, professor Pupin edited a book on Serbian monuments, under the title "Serbian Orthodox Church".

Pupin released about 70 tehnical articles and reviews and had 34 patents.

Mihajlo Pupin was faced on the old 50 millions Yugoslav dinars currency. In Belgrade in 1946 the institute Mihailo Pupin was founded. One smaller crater on the Moon, was named by Pupin, in his honor. A laboratory in Columbia University still bears his name. During 1931 Harold C. Urey discovered heavy water in Pupin's laboratory. Urey won the Nobel prize 1934. Pupin's students include Miliken, Langmuir, Armstrong. Two of them were awarded the Nobel prize.