Rousseau was of Walloon heritage and descended from a line of stonemasons. He began carving stone at age 11, working at the site of the Law Courts of Brussels, designed by architect Joseph Poelaert. He later apprenticed under sculptor George Hanstout, and took classes at the Académie Royale des Beaux-Arts in Brussels.

A winning competition entry for the Prix Godecharle in 1890 gave Rousseau the chance to travel in England, Italy, and two full years in France, after which he returned to the atelier of Belgian sculptor and teacher Charles van der Stappen for another two years, 1887 through 1889. Rousseau himself served as professor of sculpture at the Académie Royale des Beaux-Arts from 1901 through 1919, and as director from 1919 through 1922 (succeeding van der Stappen in that role) and then again from 1931 through 1935.