Ayub Khan was born on May 14, 1907, in Abbottabad, British India, in the village of Rehana near the Haripur District of North-West Frontier Province. He was a non-Pashto-speaking Pashtun (Pathan) of the Tareen tribe. He was the first child of the second wife of Mir Dad Khan Tareen, who was a Risaldar-Major (the senior most non-commissioned rank) in Hodson's Horse, a cavalry regiment of the pre-independence Indian Army. For his basic education, he was enrolled in a school in Sarai Saleh, which was about four miles from his village and used to go to school on a mule's back. Later he was moved to a school in Haripur, where he started living with his grandmother. He enrolled at Aligarh Muslim University in 1922, but did not complete his studies there, as he was accepted into the Royal Military Academy Sandhurst.

Khan’s son Gohar Ayub Khan was Pakistan’s Foreign Minister in the Nawaz Sharif government. Gohar’s son and Ayub’s grandson Omar Ayub Khan was Pakistan’s Minister of State for Finance. Gohar Ayub Khan and Omar Ayub Khan are politicians of Hazara.

Ayub Khan did well at Sandhurst and was given an officer's commission in the British Indian Army on 2 February, 1928 and then joined the 1st Battalion of the 14th Punjab Regiment (Sherdils), later known as 5th Punjab Regiment. During World War II, he served as a captain and later as a major on the Burma front. Following the war, he joined the fledgling Pakistani Army as the 10th ranking senior officer (his Pakistan Army number was 10). He was promoted to Brigadier and commanded a brigade in Waziristan and then in 1948 was sent with the local rank of Major General to East Pakistan (present day Bangladesh) as General Officer Commanding of 14th Infantry division responsible for the whole East Wing of Pakistan, for which non-combatant service he was awarded the Hilal-i-Jurat (HJ). He returned to West Pakistan in November 1949 as Adjutant General of the Army and then was briefly Deputy Commander-in-Chief.

Ayub Khan was made Commander-in-Chief of the Pakistan Army on January 17, 1951, succeeding General Sir Douglas Gracey, thus becoming the first native Pakistani general to hold that position. Therefore, he superseded two of his seniors, Maj Gen Muhammed Akbar Khan and Maj Gen N.A.M. Raza. Ayub Khan was promoted to C-in-C only due to the death of Maj Gen Iftikhar Khan, who was nominated as the first native C-in-C, but unfortunately died in an air-crash enroute to his C-in-C training in the UK. Iskandar Mirza, Secretary of Defence, was instrumental in Ayub's promotion, commencing a relationship in which Mirza became Governor General of the Dominion of Pakistan and later President of Pakistan, when it became a republic on March 23, 1956. The events surrounding his appointment set the precedent for a Pakistani general being promoted out of turn, ostensibly because he was the least ambitious of the Generals and the most loyal. It was only 3 months before the end of his tenure as Commander-in-Chief that Ayub Khan deposed his mentor, Iskandar Mirza, Pakistan's President, in a military coup, after Mirza had declared Martial Law and made Ayub Martial Law commander.

He would later go on to serve in the second cabinet (1954) of Muhammad Ali Bogra as Defence Minister, and when Iskander Mirza declared martial law on October 7, 1958, Ayub Khan was made its chief martial law administrator. Both Nawab Amir Mohammad Khan and Sandhurst trained General Wajid Ali Khan Burki were instrumental in Ayub Khan's Rise to power, until today the three families retain adjoining houses in Islamabad. This would be the first of many instances in the history of Pakistan of the military becoming directly involved in politics. As a result of his having control of the Pakistan Army, Ayub deposed Mirza on October 27 in a bloodless coup, sending Generals Wajid Burki, Azam, and Sheikh in the middle of the night to pack Mirza off to exile in England. This was actually welcomed in Pakistan, since the nation had experienced a very unstable political climate since independence. In 1960, he held an indirect referendum of his term in power. Functioning as a kind of electoral college, close to 80,000 recently elected village councilmen were allowed to vote yes or no to the question: "Have you confidence in the President, Field Marshal Mohammed Ayub Khan?" Winning 95.6% of the vote, he used the confirmation as impetus to formalise his new system. Ayub moved to have a constitution created, and this was completed in 1961. A fairly secular person by nature, Ayub Khan's constitution reflected his personal views of politicians and the use of religion in politics.

In 1962, he pushed through a new constitution that while it did give due respect to Islam, it did not declare Islam the state religion of the country. It also provided for election of the President by 80,000 (later raised to 120,000) Basic Democrats—men who could theoretically make their own choice but who were essentially under his control. He justified this as analogous to the Electoral College in the United States and cited Thomas Jefferson as his inspiration. The government "guided" the press and, while Ayub permitted a National Assembly, it had only limited powers. Ayub Khan introduced the Muslim Family Laws through an Ordinance on March 2, 1961 under which unmitigated polygamy was abolished, consent of the current wife was made mandatory for a second marriage, brakes were also placed on the practice of instant divorce where men would divorce women by saying "I divorce you" three times. The Arbitration Councils set up under the law in the urban and rural areas were to deal with cases of (a) grant of sanction to a person to contract a second marriage during the subsistence of a marriage; (b) reconciliation of a dispute between a husband and a wife; (c) grant maintenance to the wife and children.

In 1964, Ayub confident in his apparent popularity and seeing deep divisions within the political opposition, called for Presidential elections. He was however taken by surprise when despite a brief disagreement between the five main opposition parties ( a preference for a former close associate of Ayub Khan General Azam Khan as candidate was dropped), the joint opposition agreed on supporting the respected and popular Fatima Jinnah, the sister of the founder of Pakistan Muhammad Ali Jinnah. Despite Jinnah's considerable popularity and public disaffection with Ayub's government, Ayub won with 64% of the vote in a bitterly contested election on January 2, 1965. The election did not conform to international standards and journalists. It is widely held, as subsequent historians and analysts almost uniformly say, that the elections were rigged in favour of Ayub Khan.

As President, Ayub Khan allied Pakistan with the global U.S. military alliance against the Soviet Union. This in turn led to major economic aid from the U.S. and European nations, and the industrial sector of Pakistan grew very rapidly, improving the economy, but the consequences of cartelization included increased inequality in the distribution of wealth. It was under Ayub Khan that the capital was moved from Karachi to Rawalpindi, in anticipation of the construction of a new capital: Islamabad. In 1960, Khan's government signed the Indus Waters Treaty with archrival India to resolve disputes regarding the sharing of the waters of the six rivers in the Punjab Doab that flow between the two countries. Khan's administration also built a major network of irrigation canals, high-water dams and thermal and hydroelectric power stations.

Despite the Indus Waters Treaty, Ayub maintained icy relations with India. He established close political and military ties with Communist China, exploiting its differences with Soviet Russia and its 1962 war with India. To this day, China remains a strong economic, political and military ally of Pakistan.

The turning point in his rule was the Indo-Pakistani War of 1965, Despite many repulsed Indian attacks, Pakistan was on the verge of losing the war which adversely affected Pakistan's then rapidly developing economy and it ended in a settlement reached by Ayub at Tashkent, called the Tashkent Declaration. The settlement was perceived negatively by many Pakistanis and led Zulfiqar Ali Bhutto to resign his post and take up opposition to Khan. According to Morrice James, "For them [Pakistanis] Ayub had betrayed the nation and had inexcusably lost face before the Indians." The war also increased opposition in East Pakistan [Now Bangladesh] where the Awami League headed by Sheikh Mujibur Rahman sought more autonomy for the province.

General Ayub Khan, who had assumed office of the commander in chief in 1951, supported Governor General Ghulam Muhammad when he dismissed the first constituent assembly on the grounds "The constituent assembly being power hungry and having a tendency of being corrupt." Moulvi Tamizuddin, the first speaker of the assembly, challenged the dismissal (he had to take a rickshaw, wear a burka and go through Sindh court backdoor to seek justice for a nation). Sindh court accepted the appeal but the Federal Court dismissed the Sindh court judgment as the "Doctrine of necessity". Later on the decision has been the basis of all autocratic adjustments in Pakistan.

These were the years when Pakistan allowed the US to establish a USAF communications monitoring facility near Peshawar at Badaber and use its air space and air bases to conduct high-altitude spy-flights over the USSR. Due to this, and the soon-to-follow U2 incident led Pakistan into an open hostility with the USSR.

Pakistani civilian nuclear programme started in 1956 under the government of Prime Minister of Pakistan, Huseyn Shaheed Suhrawardy. When Ayub Khan imposed martial law in Pakistan, Pakistani Civilian Nuclear Programme was freezed until 1972. On December 11, 1965, President Ayub Khan had a brief meeting with Pakistani nuclear engineer Mr. Munir Ahmad Khan at the Dorchester Hotel in London. The meeting was arranged by Foreign Minister of Pakistan, Mr. Zulfikar Ali Bhutto. In this meeting Munir Ahmad Khan told President Ayub Khan that Pakistan must acquire the necessary facilities that would give the country a nuclear deterrent capability, which were available free of safeguards and at an affordable cost. Munir Ahmad Khan also told President Ayub Khan that there were no restrictions on nuclear technology, that it was freely available, and that India and Israel were moving forward in deploying it. Munir Ahmad Khan estimated the cost of nuclear technology at that time. Because things were less expensive, it was not more than 150 million dollars. President Ayub Khan listened to him very patiently, but at the end of the meeting, Ayub Khan was remained unconvinced. Ayub Khan clearly refused Munir Ahmad Khan's offer and said that Pakistan was too poor to spend that much money. Moreover, if we ever need the bomb, we will buy it off the shelf.

President Ayub Khan, who was very close to Dr. Abdus Salam, established Pakistan's National Space Agency, Space and Upper Atmosphere Research Commission (SUPARCO) on September 16, 1961. Ayub Khan also appointed Dr. Abdus Salam as its head. It was during Ayub Khan's administration when National Aeronautics and Space Administration began training of Pakistani scientists and engineers in NASA's headquarters. President Ayub Khan was eager to make Pakistan a space power, so he appointed a noted aeronautical engineer and military scientist, Air Mar. Gen. W. J. M. Turowicz as Pakistan's Rocket Program head. Gen. W. J. M. Turowicz' efforts led Pakistan to develop ballistic missile series on its own. General W. J. M. Turowicz had led a series of Rehbar Sounding Rockets fired from Pakistani soil. However, after Ayub Khan's removal from office the Space Programme was frozen for more than 2 decades.

In 1969, he opened up negotiations with the opposition alliance, except for Maulana Bhashani and Zulfiqar Ali Bhutto. However, under increasing pressure from Bhutto and Bhashani, who were allegedly encouraged to continue the agitation by elements within the Army, and in violation of his own constitution which required him to transfer power to the speaker of the assembly, Ayub turned over control of Pakistan to Commander in Chief General Yahya Khan on 25 March 1969. He was the President's most loyal lieutenant, and was promoted over seven more senior generals in 1966 to the army's top post.

Ayub Khan stressed birth control in a country that has the seventh largest population in the world: 115 million. He dismissed criticism with the comment that if there was no family planning, the time would surely come when "Pakistanis eat Pakistanis." Government corruption and nepotism, in addition to an environment of repression of free speech and political freedoms, increased unrest. Criticisms of his sons and family's personal wealth increased, especially his son's actions after his father's election in the allegedly rigged 1965 Presidential elections against Fatima Jinnah is a subject of criticism by many writers. Gohar Ayub also faced criticisms during that time on questions of family corruption and cronyism through his business links with his father-in-law retired Lt. General Habibullah Khan Khattak. One Western commentator in 1969 estimated Gohar Ayub's personal wealth at the time at $4 million dollars, while his family's wealth was put in the range of $10–$20 million dollars. Ayub began to lose both power and popularity. On one occasion, while visiting East Pakistan, there was a failed attempt to assassinate him, though this was not reported in the press of the day. Ayub was persuaded by underlings to award himself the Nishan-e-Pakistan, Pakistan's highest civil award, on the grounds that to award it to other heads of state he should have it himself and also promoted himself to the rank of Field Marshal. He was to be Pakistan's second Field Marshal, if the first is regarded as Field Marshal Sir Claude Auchinleck (1884-1981), supreme commander of military forces in India and Pakistan in the lead-up to independence in 1947. Aggravating an already bad situation, with increasing economic disparity in the country under his rule, hoarding and manipulation by major sugar manufacturers resulted in the controlled price of 1 kg sugar to be increased by 1 rupee and the whole population took to the streets. As Ayub's popularity plummeted, he decided to give up rule.

In 1971 when war broke out, Ayub Khan was in West Pakistan and did not comment on the events of the war. He died in 1974.