Alexander Milne Calder was born in Aberdeen, Scotland, the son of a tombstone carver. He began his career in Scotland, working for sculptor John Rhind, the father of sculptor J. Massey Rhind while attending the Royal Academy in Edinburgh. He moved to London and worked on the Albert Memorial. Calder immigrated to the United States in 1868 and settled in Philadelphia, where he studied with Joseph A. Bailly, and took classes (as would his son Alexander Stirling Calder) with Thomas Eakins at the Pennsylvania Academy of Fine Arts.

In 1873 he was hired by architect John McArthur, Jr. to produce models for the architectural sculpture of Philadelphia City Hall. The commission involved more than 250 pieces in marble and bronze, and took Calder 20 year to complete. In 1875 he won the competition for the colossal bronze statue of William Penn that was to crown its tower.

He is buried in West Laurel Hill Cemetery in Bala Cynwyd, Pennsylvania.