Born in St. Andrews, Lower Canada (now Saint-André-d'Argenteuil, Quebec) to Rev. Joseph Abbott (an Anglican missionary) and Harriet Bradford, he became Canada's first native-born prime minister. Abbott married Mary Bethune (1823 – 1898), a relative of Dr. Norman Bethune, in 1849. The couple had five sons and four daughters, many of whom died without descendants. Their eldest surviving son, William Abbott, married the daughter of Colonel John Hamilton Gray, a Father of Confederation and Premier of Prince Edward Island. The direct descendants of Abbott and Hamilton Gray include John Kimble Hamilton ("Kim") Abbott, a political commentator and lobbyist and a WWII Royal Canadian Airforce pilot in the infamous "Demon Squadron". Abbott was also the great-grandfather of Canadian actor Christopher Plummer and the first cousin (once removed) of Maude Abbott, one of Canada's earliest female medical graduates and an expert on congenital heart disease.

Abbott was a successful Montreal corporate lawyer and businessman and a practising Freemason. In 1849, he signed the Montreal Annexation Manifesto calling for Canada to join the United States, an action which later in life, he regretted as a youthful error. He eventually joined the Loyal Orange Lodge of British North America, well known as a pro-British organization. He was involved in the promotion of several railway projects, including the Canadian Pacific Railway (of which he served as President). He worked to incorporate and arrange financing for the first Canadian Pacific Railway syndicate. As legal advisor to its main financier, Sir Hugh Allan, Abbott was the recipient of the infamous telegram from Prime Minister Sir John A. Macdonald during the 1872 Canadian federal election campaign which read "I must have another ten thousand; will be the last time of calling; do not fail me; answer today." This telegram was stolen from Abbott's office and published, breaking the 1873 Pacific Scandal which brought down Macdonald's government. Abbott was subsequently a key organizer of a second syndicate which eventually completed the construction of Canada's first transcontinental railway in 1885, serving as its solicitor from 1880 to 1887 and as a director from 1885 to 1891.

He received a Bachelor of Civil Law from McGill College (now McGill University) located in Montreal in 1854, and a Doctor of Civil Law (DCL) in 1867. Most of his legal practice was in corporate law; however, his most celebrated court case was the defence of, first fourteen, then upon release and recapture, four of those fourteen Confederate agents who had raided St. Albans, Vermont, from Canadian soil during the American Civil War. Abbott successfully argued that the Confederates were belligerents rather than criminals and therefore should not be extradited. The episode brought Canadian - American tensions close to armed conflict. Abbott was widely viewed as the most successful lawyer in Canada for many years, as measured by professional income. He began lecturing in commercial and criminal law at McGill in 1853, and in 1855 he became a professor and dean of its Faculty of Law, where Sir Wilfrid Laurier, future prime minister of Canada, was among his students. He continued in this position until 1880. Upon his retirement, McGill named him emeritus professor, and in 1881 appointed him to its Board of Governors.

Abbott first ran for Canada's Legislative Assembly in 1857 in the Argenteuil district, northwest of Montreal. Defeated, he challenged the election results on the grounds of voting list irregularities and was eventually awarded the seat in 1860. He served as solicitor general for Lower Canada (Quebec) until 1863. He reluctantly supported Canada's confederation, fearing the reduction of the political power of Lower Canada's English speaking minority. His proposal to protect the electoral borders of 12 English Quebec constituencies was eventually incorporated into the British North America Act of 1867.

Abbott was elected to the House of Commons in 1867 as member for Argenteuil. He was removed from his seat by petition in 1874 following his involvement in the Pacific Scandal. He narrowly lost the 1878 election, then won in February 1880, only to have his victory declared void because of bribery allegations. He was, however, subsequently elected in a by-election in August 1881. In 1887, Macdonald appointed him to the Senate. He served as Leader of the Government in the Senate from May 12, 1887 to October 30, 1893 (including his term as Prime Minister) and as Minister without Portfolio in Macdonald's cabinet. He also served two one-year terms as mayor of Montreal from 1887 to 1889.

When Prime Minister Macdonald died in office, Abbott supported John Thompson to succeed him, but reluctantly accepted the plea of the divided Conservative party that he should lead the government. He was one of just two Canadian Prime Ministers, the other being Mackenzie Bowell, to have held the office while serving in the Senate rather than the House of Commons.

In his seventeen months in office, Abbott worked on revitalizing the government and the party. Despite the scandals exposed during his term, he dealt with the backlog of government business awaiting him after Macdonald's death. Reform of the civil service, revisions of the criminal code and a reciprocity treaty with the U.S. were just a few of the issues initiated by Abbott. During his term, there were 52 by-elections, 42 of which were won by the Conservatives, increasing their majority by 13 seats — evidence of Abbott's effectiveness as prime minister. One year into his time as prime minister, Abbott attempted to turn the office over to Thompson, but this was rejected due to anti-Catholic sentiment in the Tory caucus. Suffering from the early stages of cancer of the brain, Abbott's health failed in 1892 and he retired to private life, whereupon Thompson finally became Prime Minister. Abbott died less than a year later at the age of 72.

Sir John Abbott is buried in the Mount Royal Cemetery, Montreal, Quebec.

John Abbott College in Sainte-Anne-de-Bellevue, Quebec, near Abbott's 300-acre (1.2 km²) country estate (Boisbriant), is named after him.

His "most memorable" political comment is "I hate politics". The full quote was "I hate politics and what are considered their appropriate measures. I hate notoriety, public meetings, public speeches, caucuses and everything that I know of which is apparently the necessary incident of politics — except doing public work to the best of my ability."

In their 1999 look at the Canadian Prime Ministers through Jean Chrétien, J.L. Granatstein and Norman Hillmer included a survey of Canadian historians ranking the Premiers. Abbott's term of service was considered below par and he was ranked #17 out of 20 (up to then).