Rauch was born at Arolsen in the Principality of Waldeck. His father was employed at the court of Prince Frederick II of Hesse, and in 1790 the lad was apprenticed to the court sculptor of Arolsen, Friedrich Valentin. In 1795, he became assistant to Johann Christian Ruhl, the court sculptor of Kassel. On the death of his father in 1797, he moved to Berlin where he was appointed groom of the chamber in the king's household. He abandoned sculpture temporarily, but his new position provided a wider field for improvement, and he soon used the opportunity and practised his art in spare hours. He came under the influence of Johann Gottfried Schadow.

In 1802, he exhibited his “Sleeping Endymion.” Queen Louisa of Prussia, surprising him one day in the act of modeling her features in wax, sent him to study at the Academy of Art. Not long afterwards, in 1804, Count Sandrecky gave Rauch the means to complete his education at Rome, where Wilhelm von Humboldt, Antonio Canova and Bertel Thorvaldsen befriended him. He also executed his life size bust of Queen Louise in marble, and among his other early works were busts of the poet Zacharias Werner, Count Wenyerski and the painter Raphael Mengs, the latter executed on a commission from Ludwig I of Bavaria. Other works were bas-reliefs of “Hippolytus and Phaedra,” “Mars and Venus wounded by Diomede,” and a “Child praying.” He remained in Rome for six years.

In 1811, Rauch was commissioned to execute a monument for Queen Louisa of Prussia. The statue, representing the queen in a sleeping posture, was placed in a mausoleum in the grounds of Charlottenburg, and procured great fame and a European reputation for the artist. A similar statue of the Queen, even more successful, was placed in the Sanssouci Park at Potsdam. The erection of nearly all public statues came to be entrusted to him. There were, among others, Bülow, Yorck and Scharnhorst at Berlin, Blücher at Breslau, Maximilian at Munich, Francke at Halle, Dürer at Nuremberg, Luther at Wittenberg, and Grand Duke Paul Friedrich at Schwerin.

By 1824, he had executed 70 busts in marble of which 20 were of colossal size. His colossal bronze statues of Blücher are 13 feet in height, and he also executed the greater part of the 12 statues in iron which compose the national monument at Kreuzberg, near Berlin. One of his finest works is the group “Faith, Hope and Charity,” which he presented to his native town, Arolsen.

At length, in 1830, Rauch began, along with the architect Karl Friedrich Schinkel, the models for a colossal equestrian monument at Berlin to honor King Frederick II of Prussia (Frederick the Great). This work was inaugurated with great pomp in May 1851, and is regarded as one of the masterpieces of modern sculpture, the crowning achievement of Rauch's work as a portrait and historic sculptor. Princes decorated Rauch with honors and the academies of Europe enrolled him among their members. A statue of Immanuel Kant for Königsberg and a statue of Albrecht Thaer for Berlin occupied his attention during some of his last years; and he had just finished a model of Moses praying between Aaron and Hur when he was attacked by his last illness.