He moved to Haarlem in 1777 because the Haarlemmers had more taste in the sciences than anywhere else in the Netherlands. After his arrival in Haarlem he began to practise medicine, but devoted himself mainly to lecturing on physical subjects and creating instruments to demonstrate physical theory. In 1779 he was entrusted with the care of the Second society left to Haarlem by Pieter Teyler van der Hulst (1702-1778), which led under his direction to the foundation of the Teylers Museum. The Teyler legacy was split into three societies, one for religion, one for science, and one for the arts, known as the first, second, and third societies. The caretakers had to meet in Teyler's home weekly, and each society had 5 caretakers, so all of the gentlemen involved lived in Haarlem. In 1794 van Marum also became secretary of the Dutch Society of Science in that city, and under his management the society was advanced to the position of one of the most noted in Europe. Under his guidance the two societies slowly merged. His name is associated with the electriseermachine, the largest electricity demonstration machine built and at the time a crowd pleaser for the young Teylers museum. The demonstration model is still on display, as is a smaller version in the Museum Boerhaave of Leiden. His researches (especially in connection with electricity) were remarkable for their number and variety.

In 1784, when the Teylers Museum opened its new 'Oval Room', he began collecting for the Teyler library. van Marum concentrated his efforts on three aspects: 1) Greek and Latin authors, among them the church fathers, 2) Works of natural history including travelogues, and 3) natural history periodicals, including all publications of the Royal Society of London and all publications of the Dutch Society of Science, which Teyler had been a member of, but could not be on the board of, due to religious differences with the board. The criteria for purchase was always expense. If a Society member could afford to purchase it himself it was not worth adding to the collection. Any member could suggest purchases, however, which explains why the collection is filled with richly illustrated examples of contemporary publications. The most impressive of these are the large illustrated books of travellers. To view the collection, van Marum organized gentleman evenings in Pieter Teyler's library, a tradition that still exists. Though the public is allowed access during the day to the museum rooms, the private rooms of Pieter Teyler and the keepers of the collection are only open one day a year, on Monument Day. Because of this rich tradition, many noted scholars of Physics moved to Haarlem during the years, including Nobel prize winners Pieter Zeeman and Hendrik Lorentz.

Of van Marum's three categories, only the first was discontinued at the close of the 19th century. The collection of periodicals which has been expanded through exchange networks, contains uninterrupted series that are among the oldest in the world.

The museum created a new wing in 1996 to house a rotational display of van Marum's library collection, such as the works of John James Audubon in combination with contemporary stuffed birds of Naturalis.

He died in Haarlem.