Kubin was born in Bohemia in the town of Litoměřice, which was then part of the Austro-Hungarian Empire. From 1892 to 1896, he was apprenticed to the landscape photographer Alois Beer, although he learned little. In 1896, he attempted suicide on his mother's grave, and a short stint in the Austrian army the following year ended with a nervous breakdown. In 1898, Kubin began a period of artistic study at a private academy run by the painter Ludwig Schmitt-Reutte, before enrolling at the Munich Academy in 1899, without finishing his studies there. In Munich, Kubin discovered the works of Odilon Redon, Edvard Munch, James Ensor,  Henry de Groux and Félicien Rops. He was profoundly affected by the prints of Max Klinger, and later recounted: "Here a new art was thrown open to me, which offered free play for the imaginative expression of every conceivable world of feeling. Before putting the engravings away I swore that I would dedicate my life to the creation of similar works". The aquatint technique used by Klinger and Goya influenced the style of his works of this period, which are mainly ink and wash drawings of fantastical, often macabre subjects. Kubin produced a small number of oil paintings in the years between 1902 and 1910, but thereafter his output consisted of pen and ink drawings, watercolors, and lithographs. In 1911, he became associated with the Blaue Reiter group, and exhibited with them in the Galerie Der Sturm exhibition in Berlin in 1913. After that time, he lost contact with the artistic avant-garde.

He is considered an important representative of Symbolism and Expressionism, noted for dark, spectral, symbolic fantasies (often assembled into thematic series of drawings). Like Oskar Kokoschka and Albert Paris Gütersloh, Kubin had both artistic and literary talent. He illustrated works by Edgar Allan Poe, E.T.A. Hoffmann, Fyodor Dostoevsky and others. He was also the author of several books, the best known being his novel Die Andere Seite (The Other Side) (1909), an apocalyptic fantasy set in an oppressive imaginary land which has an atmosphere of claustrophobic absurdity reminiscent of the writings of Franz Kafka.

His literary works also include: The Looking Box, 1925; Of the Desk of a Draughtsman, 1939; Adventure of an Indication Feather/Spring, 1941; Sober Balladen, 1949; Evening-red, 1950; Fantasies in the Boehmerwald, 1951; Daemons and Night Faces, 1959 (autobiography).

From 1906 until his death, he lived a withdrawn life in a small castle on a twelfth century estate in Zwickledt, Upper Austria. In 1938, at the Anschluss of Austria and Nazi Germany, his work was declared entartete Kunst ('degenerate art'), but he managed to go on working during World War II. Kubin was awarded the Great Austrian State Prize in 1951, and the Austrian Decoration for Science and Art in 1957.