After the early death of his father Johann Baptist, a poor painter, Ary's mother Cornelia, herself a painter and daughter of landscapist Arie Lamme, took him to Paris and placed him in the studio of Pierre - Narcisse Guérin. When Scheffer left Guérin's studio, Romanticism had come into vogue in France, with such painters as Xavier Sigalon, Eugène Delacroix and Théodore Géricault. Scheffer did not show much affinity with their work and developed his own style, which has been called "frigidly classical".

Scheffer often painted subjects from literature, especially the works of Dante, Byron and Goethe. Two versions of "Dante and Beatrice" have been preserved at Wolverhampton Art Gallery, United Kingdom, and Museum of Fine Arts, Boston, USA. Particularly highly praised was his "Francesca da Rimini" painted in 1836. Ary Sheffer's popular Faust themed paintings include "Margaret at her Wheel"; "Faust Doubting"; "Margaret at the Sabbat"; "Margaret Leaving Church"; "The Garden Walk", and "Margaret at the Well". In 1836, he painted two pictures of Goethe's character "Mignon".

He now turned to religious subjects: "Christus Consolator" (1836) was followed by "Christus Remunerator", "The Shepherds Led by the Star" (1837), "The Magi Laying Down their Crowns", "Christ in the Garden of Olives", "Christ bearing his Cross", "Christ Interred" (1845), and "St Augustine and Monica" (1846).

The "Christus Consolator" painting was lost for 70 years, was discovered in a janitor's closet in Gethsemane Lutheran Church in Dassel, Minnesota in 2007. It is restored and is on display at the Minneapolis Institute of Arts.

Scheffer was also an accomplished portrait painter. His subjects included composers Frédéric Chopin and Franz Liszt, the Marquis de la Fayette, Pierre - Jean de Béranger, Alphonse de Lamartine, Charles Dickens, Duchess de Broglie, Talleyrand and Queen Marie Amélie.

After 1846, he ceased to exhibit. His strong ties with the royal family caused him to fall out of favour when, in 1848, the Second Republic came into being. Shut up in his studio, he produced many paintings that were only exhibited after his death, which took place at Argenteuil on the 15th of June 1858. He is buried in the Cimetière de Montmartre.

Works first exhibited posthumously include "Sorrows of the Earth", and the "Angel Announcing the Resurrection", which he had left unfinished. By the time of his death, his reputation was damaged: though his paintings were praised for their charm and facility, they were condemned for poor use of color and vapid sentiment.

Scheffer was married to the widow of General Baudrand. His brother Henry, born at the Hague on 27 September 1798, was also a prolific painter. Scheffer was made commander of the Legion of Honour in 1848, that is, after he had wholly withdrawn from the Salon.