As a young boy, Jackson worked as an office boy for a lithograph company, after his father abandoned the family of six children. It was at this company that Jackson began his art training. In the evenings, he took classes at Montreal's Le Monument National.

In 1905, he worked his way to Europe on cattle boat, returning by the same means and travelling on to Chicago. In Chicago, he joined a commercial art firm and took courses at the Art Institute of Chicago. He saved his earnings and, by 1907, was able to visit France to study Impressionism. In France, he decided to become a professional painter, and studied at Paris' Académie Julian under J.P. Laurens. He also visited Italy and the Netherlands. Jackson painted in an Impressionist style.

Jackson returned to Canada, settling in Sweetsburg, Quebec, where he began painting works such as The Edge of Maple Wood. He held his first single artist exhibition at the Montreal Art Gallery with Randolph Hewton in 1913. Unable to make ends meet and discouraged by the Canadian art scene, he considered moving to the United States. However, he received a letter from J.E.H. MacDonald and changed his mind. MacDonald inquired about The Edge of Maple Wood, which he had seen at a Toronto art show. MacDonald said that Toronto artist Lawren Harris wanted to purchase the painting, if Jackson still owned it. Harris purchased the painting and Jackson struck up correspondence with the Toronto artists, often debating Canadian art. Jackson soon began long visits to Toronto.

In his visits, A.Y. Jackson often joined the painters who would one day be known as the Group of Seven on major trips to Algonquin Park, Georgian Bay, Algoma and the North Shore. Like the other Group painters, Jackson embraced landscape themes and sought to develop a bold style. An avid outdoorsman, Jackson became good friends with Tom Thomson, and the duo often fished and sketched.

In 1910, Harris convinced Jackson to spend the summer painting in Georgian Bay. A local doctor offered use of his cottage, a studio and paid expenses.

Jackson enlisted in the Canadian Army's 60th battalion in 1915. Private Jackson was wounded at the Battle of Sanctuary Wood in June 1916, soon after he reached the front. While recovering from his injuries, he came to the attention of Lord Beaverbrook. He was transferred to the Canadian War Records branch as an artist. He would create important pictures of events connected with the war. He later worked for the Canadian War Memorials as an official war artist from 1917 to 1919.

Afterwards, Jackson returned to Toronto, often making painting expeditions to the lower St Lawrence, the Arctic, and British Columbia.

In 1919 Jackson and six painter colleagues formed the Group of Seven. These artists were considered bold, because the Canadian wilderness had previously been considered too rugged and wild to be painted. Although his name is conventionally associated with this group, he would also remain something of a loner throughout his life.

In 1925, he taught at the Ontario College of Art (OCA), in Toronto; this was the only year that he missed his annual spring trip to Quebec.

In 1933, Jackson helped found the Canadian Group of Painters. Several members of the Group of Seven later became members including Lawren Harris, A.J. Casson, Arthur Lismer and Franklin Carmichael.

He moved to the Ottawa region in 1955, settling in Manotick.

In his later years, he was often accompanied on his painting trips into the Ottawa Valley region, the Gatineau Hills, the Lievre River Valley and Ripond with friend, painter and former student, Ralph Wallace Burton, and fellow painters Maurice Haycock and Stuart D. Helmsley. One such venture almost ended in disaster: "... in the 1950s, when Ralph and A.Y. were painting on the banks of the Ottawa River at Deux Rivieres, a bullet ricoheted off a rock where Jackson was sitting."

In 1958 he published A Painter's Country, his autobiography, dedicated to the memory of J.E.H. MacDonald, "who visualized a Canadian school of painting and devoted his life to the realization of it".

In 1964, Jackson submitted his own design during the Great Flag Debate, a similar design to the Pearson Pennant.

In 1965, Jackson had a serious stroke that put an end to his painting career. He recuperated at the home of friend and painter Ralph Wallace Burton and later moved to the McMichael Conservation Estate in Kleinburg, Ontario. Jackson died in 1974, over the Easter holiday in a nursing home in Toronto. He is buried on the grounds of the McMichael Gallery.

In 1967 he was appointed a Companion of the Order of Canada.

A.Y. Jackson Secondary School in Toronto was named after him. He attended the opening of the school in 1970. A.Y. Jackson Secondary School in Ottawa is also named after him and opened in 1976.

In 1970, the Royal Canadian Academy awarded him its medal for lifetime achievement.

The A.Y. Jackson Lookout on Highway 144 in Sudbury overlooks the waterfall depicted in Jackson's 1953 painting Spring on the Onaping River. The Ottawa River Institute has established an A.Y. Jackson Trail in his honour.