Musil was the son of Alfred Edler von Musil (1846, Temesvár – 1924) and his wife Hermine Bergauer (1853, Linz – 1924), who lived together with an unrelated "uncle" Heinrich Reiter (born 1856), the houseguest in the Musil family. The elder Musil was an engineer. The family moved to Chomutov until October 1881, and in 1891 he was appointed to the chair of Mechanical Engineering at the German Technical University in Brno, and awarded a hereditary peerage in the Austro - Hungarian Empire shortly before it collapsed. He was a second cousin of Alois Musil, the orientalist. Hermine Bergauer was the daughter of a Bohemian German engineer, Franz (Xaver von) Bergauer (December 3, 1805, Horschowitz – October 11, 1886, Linz).

The young Musil was short in stature, but strong and skilled at wrestling, and by his early teens already more than his parents could handle. Accordingly they sent him to military boarding school at Eisenstadt (1892 – 1894) and then Hranice, at that time also known as Mährisch Weißkirchen, (1894 – 1897). These school experiences are reflected in his first novel, Die Verwirrungen des Zöglings Törless (The Confusions of Young Törless).

After graduating as a cadet, Musil briefly studied at a military college in Vienna during the fall of 1897, but then switched to engineering, joining his father's department at Brno. During his college career he studied engineering by day, but at night read literature and philosophy, and went to the theater and art exhibits. Nietzsche, Dostoyevsky, Ralph Waldo Emerson, and Ernst Mach were particular interests of his college years. Musil finished his studies in three years, then in 1902 – 1903 served as an unpaid assistant to Professor Julius Carl von Bach, in Stuttgart. During this time he began work on Young Törless.

Even then, however, Musil was growing tired of engineering and what he perceived as the limited worldview of engineers, and rather than settle into an engineering career, he launched a new round of doctoral studies (1903 – 1908) in psychology and philosophy at the University of Berlin under the renowned Professor Carl Stumpf. In 1905, Musil had met Martha Marcovaldi (née Heinemann, January 21, 1874 – November 6, 1949) who was in subsequent years to become his wife. She had already been widowed and remarried, with two children, and was seven years older than Musil. In the midst of these studies his first novel, Young Törless, was published in 1906.

In 1909, Musil completed his doctorate and was offered a position by Professor Alexius Meinong, at the University of Graz, which he turned down to concentrate on literature. Over the next two years, he wrote and published two stories ("The Temptation of Quiet Veronica" and "The Perfecting of a Love") collected in Vereinigungen (Unions) published in 1911. During this same year, Martha's divorce was completed and Musil married her. Until this time, Musil had been supported by his family, but he now found employment first as a librarian in the Technical University of Vienna, and then in an editorial role with the Berlin Literary Journal, during which time he worked on a play entitled Die Schwärmer (The Enthusiasts), which was eventually published in 1921.

When World War I began, Musil joined the Army, stationed first in Tirol, and then away from danger at Austria's Supreme Army Command in Bolzano. In 1916 Musil visited Prague and met Franz Kafka, whose work he held in high esteem, as he did the work of Bohemian poet Rainer Maria Rilke. At the memorial service for Rilke in Berlin, Musil remarked that Rilke was "undervalued" for most of his life, and by the time of his death, he had turned into "a delicate, well matured liqueur suitable for grown up ladies", but that his work is "too demanding" to be "considered relaxing". After the war's end, and the collapse of the Austro - Hungarian empire, Musil returned to his literary career in Vienna. He published a collection of short stories, Drei Frauen (Three Women), in 1924, and then in 1930 and 1932 the first two volumes of his masterpiece, Der Mann ohne Eigenschaften (The Man Without Qualities). The novel deals with the moral and intellectual decline of the Austro - Hungarian empire through the eyes of the book's protagonist Ulrich, an ex-mathematician who has failed to engage with the world around him in a manner that would allow him to possess 'qualities'. It is set in Vienna on the eve of World War I.

The Man Without Qualities brought Musil only mediocre commercial success. Though he was nominated for the Nobel Prize, he felt he did not receive the recognition he deserved. He sometimes expressed annoyance at the success of more famous colleagues like Thomas Mannor Hermann Broch, who admired his work deeply, and, moved by his material poverty, tried to shield him against daily worries and encouraged him to further his literary work, even though Musil was initially critical of Mann.

In the early 1920s Musil lived mostly in Berlin. In Vienna Musil was a frequent visitor to Eugenie Schwarzwald's salon (the model for Diotima in The Man Without Qualities). In 1932, the Robert Musil Society was founded in Berlin on the initiative of Thomas Mann. The same year Thomas Mann was asked to name an eminent contemporary novel and he cited exclusively The Man Without Qualities. In 1936, Musil had his first stroke.

The last years of Musil's life were dominated by Nazism and World War II: the Nazis banned his books. He saw early Nazism first hand while living in Berlin from 1931 to 1933. In 1938, when Austria became a part of the Third Reich, Musil and his Jewish wife Martha left for exile in Switzerland, where he died on April 15, 1942. Martha wrote to Franz Theodor Csokor that taking off his clothes in the bathroom, maybe when doing gymnastics or just making a hefty movement, he had been hit by a stroke and, when she found him a few minutes later, did not look dead at all but so alive with some mockery and astonishment on his face. He was 61. Only eight people were present at his cremation. Martha cast his ashes into the woods of Mont Salève. From the galleys Musil had retrieved from the printer and had been reworking until his very last day, she published Part 3 of The Man Without Qualities in 1943; she died in Rome in 1949.

After his death Musil's work was almost forgotten in German speaking countries. His writings began to reappear during the early 1950s. The first translation of The Man Without Qualities in English was published by Ernst Kaiser and Eithne Wilkins in 1953, 1954 and 1960. An updated translation by Sophie Wilkins and Burton Pike, containing extensive selections from unpublished drafts, appeared in 1995.