He was born in Okrouhlice near Německý Brod in Bohemia (present day Czech Republic). He studied privately in Prague and then attended the UMPRUM there for 2 years starting in 1907, before being expelled. He first visited France in 1907, returning to Paris and Brittany frequently until 1939, but maintaining close links to his homeland.

After the war he became an associate professor at Palacký University of Olomouc from 1947 to 1950. Later he maintained private studios in Prague and Okrouhlice. He grew increasingly recognized on a national and international level in the 1950s and 1960s, and was honoured with the title of a National Artist in 1965. He died in Prague on October 12, 1977.

Zrzavý was a key member of the Czech, and more broadly European, modernism movement of the early part of the 20th century. Although he is regarded as a symbolist, he was heavily influenced by European Medieval art. Throughout his life he was also inspired by spectacular landscapes, both abroad (France, Italy, and Greece) as well as in his native country (Vodňany, Okrouhlice, Prague). He reworked many of his themes multiple times. He was admired by and written about by one of the founders of the Czech artistic movement called Poetism, Karel Teige.

His œuvre dedicated to the Czech National Gallery is on display in the town of Telč and in Prague.