Hart was born in Paisley, Scotland, and was taken to America in early youth by his family. He was apprenticed to a carriage painter at Albany, New York, and his first artistic experience was in decorating the panels of coaches with landscapes. He also spent time as a portrait painter. He returned to Scotland, probably in the early or mid 1840s, where he studied for three years.

By the time he returned to America, Hart had shifted his energy to landscape painting. He exhibited his first work at the National Academy of Design in 1848, became a full member in 1858, and continued to show his paintings there regularly through the mid 1870s. He also exhibited at the Brooklyn Art Association and at major exhibitions around the country. Hart was a member of the American Watercolor Society, and was its president from 1870 to 1873.

Like most of the major American landscape artists of the time, Hart settled in New York City, where he opened a studio in the Tenth Street Studio building in 1858. His mature landscape style embraced the mannerism of the late Hudson River School by emphasizing light and atmosphere. He became particularly adept at depicting angled sunlight and foreground shadow; the best examples of this are: Seashore Morning (1866) in the Metropolitan Museum of Art, New York; After the Storm (1860s) in the Museum of Fine Arts, Boston; The Last Gleam (1865) in the High Museum of Art in Atlanta, Georgia; Sunset in the Valley (1870) in a private collection, featured on pp. 82 – 83 of All That is Glorious Around Us: Paintings from the Hudson River School by John Driscoll; and A Quiet Nook (1885) in the Smithsonian American Art Museum, Washington, D.C.

As strong as Hart's technical abilities were, he is also known for his prolific and occasionally formulaic paintings of cows. Cattle were a popular motif in Hudson River School art, and nearly every artist included them in at least some of their landscapes as diminutive symbols of man's harmonious relationship with nature. Some artists, including William and James Hart along with Thomas Bigelow Craig, made a specialty of cow portraits. These paintings, which were very popular with late 19th century American collectors, typically featured several cattle grazing or watering in the foreground or middle distance with the landscape playing a supporting role as a bucolic backdrop.

The Albany Institute of History & Art has in its collection over 400 sketches, water colors and sketch books which were retained en masse from the artist's studio after his death, by the family of the subsequent donor. Since each piece is signed, dated, and annotated with the location of its subject, many previously unsigned and unattributed paintings are now being associated with the artist. The museum was preparing an exhibition of this material.

Hart died at Mount Vernon, New York, on June 17, 1894. His daughter Jessie Hart White was the mother of E.B. White.

He first exhibited his paintings the following year at the Pennsylvania Academy of Fine Arts, after which he sailed to Europe, first joining a colony of American painters who were studying in Düsseldorf, then traveling up the Rhine into Switzerland and Italy. In late 1857 he settled in Rome, and in the following months made numerous excursions to draw the landscape around Rome and on Capri.

In 1858 Haseltine returned to Philadelphia, and by late 1859 was installed in the Tenth Street Studio Building in New York City, then a central point for American landscape painters; also in the building were Frederic Edwin Church, Albert Bierstadt, and Worthington Whittredge, the latter two having befriended Haseltine in Europe. Though many of his paintings from this time derived from his European sketches, Haseltine also began to paint the oceanside of New England, especially favoring the rockbound coasts of Narragansett, Rhode Island, Nahant, Massachusetts, and Mount Desert Island, Maine. The precision with which he painted these landscapes won critical praise, and Haseltine was elected an Associate of the National Academy of Design in 1860, and a full Academician in 1861.

In 1864 Haseltine's wife died in childbirth. He spent some time training his nephew, Howard Russell Butler, but he moved after he married Helen Marshall in 1866. Initially the family considered settling in Paris, but in 1867 they moved to Rome, which would for most of Haseltine's subsequent years serve as his home and point of departure from which to produce views of the European landscape. While his paintings of Capri and Sicily would prove popular with visiting American tourists, Haseltine also traveled and drew in France, Holland, Belgium and the Netherlands, summering in Bavaria and the Tyrol in the 1880s and 1890s. In his later years he also returned periodically to the United States, making a final trip to the west in 1899.

Haseltine died of pneumonia in Rome in 1900. He is buried at the American cemetery right next to Episcopalian Church, of which he was one of the founding members. His son Herbert Chevalier Haseltine became one of the most celebrated sculptors of animals, especially horses. Some of his most famous works were done for clients such as Barbara Hutton, Cary Grant and Guy de Rothschild.