The son of the senior town physician (Archiater) of Zürich, he received his education in that place, and, in 1692, went to the University of Altdorf near Nuremberg, being intended for the medical profession. Early in 1694 he obtained his degree of doctor in medicine at the University of Utrecht, and then returned to Altdorf, Germany to complete his mathematical studies. He went back to Zürich in 1696, and was made junior town physician (Poliater), with the promise of the professorship of mathematics; this he obtained in 1710, being promoted to the chair of physics, with the office of senior town physician, in January 1733, a few months before his death on June 23.

His published works (apart from numerous articles) were estimated at thirty-four in number. His historical writings are mostly still in manuscript. The more important of his published writings relate either to his scientific observations or to his journeys, in the course of which he collected materials for these scientific works.

In the former category is his self-published Beschreibung der Naturgeschichte des Schweitzerlandes (3 vols., Zürich, 1706–1708), the third volume containing an account in German of his journey of 1705. The first of the three parts deals with the Swiss mountains (summing up all that was then known about them, and serving as a link between Simmler's work of 1574 and Gruner's of 1760), the second with the Swiss rivers, lakes and mineral baths, and the third with Swiss meteorology and geology. Scheuchzer’s works, as issued in 1746 and in 1752, formed (with Tschudi's Chronicum Helveticum) one of the chief sources for Schiller's drama Wilhelm Tell (1804). In 1704 Scheuchzer was elected FRS; he published many scientific notes and papers in the Philosophical Transactions for 1706-1707, 1709 and 1727-1728.

In the second category are his Itinera alpina tria (written in 1702-1704), which was published in London in 1708, and dedicated to the Royal Society, while the plates illustrating it were executed at the expense of various fellows of the society, including the president, Sir Isaac Newton (whose imprimatur appears on the title-page), Sir Hans Sloane, Dean Aldrich, Humfrey Wanley, etc. The text is written in Latin, as is that of the definitive work describing his travels (with which is incorporated the 1708 volume) that appeared in 1723 at Leiden, in four quarto volumes, under the title of Itinera per Helvetiae alpinas regiones facta annis 1702-1711.

These journeys led Scheuchzer to almost every part of Switzerland, particularly its central and eastern districts. Apropos of his visit (1705) to the Rhône Glacier, he inserts a full account of the other Swiss glaciers, as far as they were then known, while in 1706, after mentioning certain wonders to be seen in the museum at Lucerne, he adds reports by men of good faith who had seen dragons in Switzerland. He doubts their existence, but illustrates the reports by fanciful representations of dragons, which have led some modern writers to depreciate his merits as a traveller and naturalist, for the belief in dragons was then widely spread.

In 1712 he published a map of Switzerland in four sheets (scale 1/290,000), of which the east portion (based on his personal observations) is far the most accurate, though the map as a whole was the best map of Switzerland till the end of the 18th century. At the end of his 1723 book he gives a full list of his writings from 1694 to 1721.

Scheuchzer is also known for his paleontological work. In his Lithographia Helvetica, he described fossils as "plays of nature" or alternately as leftovers from the biblical Flood. Most famously, he claimed that a fossilized skeleton found in a Baden quarry was the remains of a human who had perished in the deluge. This claim, which seemed to verify the claims of Christian scripture, was accepted for several decades after Scheuchzer's death, until 1811, when French naturalist Georges Cuvier re-examined the specimen and showed that it was actually a large prehistoric salamander.

Scheuchzerhorn (3462 m) and Scheuchzerjoch in the Bernese Alps are named after Johann Jakob Scheuchzer. Scheuchzeriaceae and Scheuchzeria palustris are named in his honor.