Franck was born to Jacob Franck and Rebecca Nachum Drucker. Franck completed his Ph.D. in 1906 and received his venia legendi, or Habilitation, for physics in 1911, both at the University of Berlin, where he lectured and taught until 1918, having reached the position of extraordinarius professor. After World War I, in which he served and was awarded the Iron Cross 1st Class, Franck became the Head of the Physics Division of the Kaiser Wilhelm Gesellschaft for Physical Chemistry. In 1920, Franck became ordinarius professor of experimental physics and Director of the Second Institute for Experimental Physics at the University of Göttingen. While there he worked on quantum physics with Max Born, who was Director of the Institute of Theoretical Physics.

In 1925, Franck received the Nobel Prize in Physics, mostly for his work in 1912 - 1914, which included the Franck - Hertz experiment, an important confirmation of the Bohr model of the atom.

In 1933, after the Nazis came to power, Franck, being a Jew, decided to leave his post in Germany and continued his research in the United States, first at Johns Hopkins University in Baltimore and then, after a year in Denmark, in Chicago. It was there that he became involved in the Manhattan Project during World War II; he was Director of the Chemistry Division of the Metallurgical Laboratory at the University of Chicago. He was also the chairman of the Committee on Political and Social Problems regarding the atomic bomb; the committee consisted of himself and other scientists at the Met Lab, including Donald J. Hughes, J.J. Nickson, Eugene Rabinowitch, Glenn T. Seaborg, J.C. Stearns and Leó Szilárd. The committee is best known for the compilation of the Franck Report, finished on 11 June 1945, which recommended not to use the atomic bombs on the Japanese cities, based on the problems resulting from such a military application.

When Nazi Germany invaded Denmark in World War II, the Hungarian chemist George de Hevesy dissolved the gold Nobel Prizes of Max von Laue and James Franck in aqua regia to prevent the Nazis from stealing them. He placed the resulting solution on a shelf in his laboratory at the Niels Bohr Institute. After the war, he returned to find the solution undisturbed and precipitated the gold out of the acid. The Nobel Society then recast the Nobel Prizes using the original gold.

In 1946 Franck married Hertha Sponer, his former assistant in Göttingen. He died suddenly in 1964 while visiting Göttingen.