Ioffe was born into a middle class Jewish family in small town of Romny, Russian Empire (now in Sumy region, Ukraine). After graduating from Saint Petersburg State Institute of Technology in 1902, he spent two years as an assistant to Wilhelm Roentgen in his Munich laboratory. Ioffe completed his Ph.D. at Munich University in 1905. That year Ioffe stated he saw the names of two authors, Einstein – Marić, on the Annus Mirabilis papers when they were submitted. This seems to support the theory that the work was a co-authorship between Einstein and his wife.

After 1906, Ioffe worked in the Saint Petersburg (from 1924 Leningrad) Polytechnic Institute where he eventually became a professor. In 1911 he (independently of Millikan) determined the electron charge, using charged microparticles of metals balanced in electric field against gravity (published in 1913). In 1911 Ioffe converted to Lutheranism and married a non - Jewish woman. In 1913 he attained the title of Magister of Philosophy, in 1915 - Doctor of Physics. In 1918 he became a head of Physics and Technology division in State Institute of Roentgenology and Radiology. This division became the Leningrad Physico - Technical Institute (LPTI) in 1917.

In the early 1930s, there was a critical need in the Air Defense Forces of the Red Army for means of detecting invading aircraft. A number of research institutes were involved with radiolokatory (radio - location) techniques. The Russian Academy of Sciences called a conference in January 1934 to assess this technology. Ioffe organized this conference, then published a journal report, disclosing to researchers throughout the world the science and technology that would ultimately be called radar.

When the Soviet atomic bomb project began in 1942, Ioffe was asked to lead the technical effort, but refused the job on the grounds that he was too old. He saw great promise in the young Igor Kurchatov, and placed him in charge of the first nuclear laboratory. During Joseph Stalin's campaign against the so-called "rootless cosmopolitans" (Jews), in 1950 Ioffe was made redundant from his position of the Director of LPTI and from the Board of Directors. In 1952 - 1954 he headed the Laboratory of Semiconductors of the Academy of Sciences of the USSR, which in 1954 was reorganized into the Institute of Semiconductors. Following Ioffe's death, in 1960 the LPTI was renamed the Ioffe Physico - Technical Institute and remains one of Russia's leading research centers.

Ioffe's students include Aleksandr Aleksandrov, Pyotr Kapitsa, Isaak Kikoin, Igor Kurchatov, Yakov Frenkel, Nikolay Semyonov, Léon Theremin and Lev Artsimovich.

The crater Ioffe on the Moon is named after him. The Ioffe Physico - Technical Institute carries his name. The Russian oceanographic and Polar research vessel Akademik Ioffe is named after him.