Gokhale became a member of the Indian National Congress in 1889, as a protégé of social reformer Mahadev Govind Ranade. Along with other contemporary leaders like Bal Gangadhar Tilak, Dadabhai Naoroji, Bipin Chandra Pal, Lala Lajpat Rai and Annie Besant, Gokhale fought for decades to obtain greater political representation and power over public affairs for common Indians. He was moderate in his views and attitudes, and sought to petition the British authorities by cultivating a process of dialogue and discussion which would yield greater British respect for Indian rights. Gokhale had visited Ireland and had arranged for an Irish nationalist, Alfred Webb, to serve as President of the Indian National Congress in 1894. The following year, Gokhale became the Congress’s joint secretary along with Tilak. In many ways, Tilak and Gokhale’s early careers paralleled – both were Chitpavan Brahmin (though unlike Gokhale, Tilak was wealthy), both attended Elphinstone College, both became mathematics professors, and both were important members of the Deccan Education Society. When both became active in the Congress, however, the divergence of their views concerning how best to improve the lives of Indians became increasingly apparent.

Gokhale’s first major confrontation with Tilak centered around one of his pet projects, the Age of Consent Bill introduced by the British Imperial Government, in 1891-2. Gokhale and his fellow liberal reformers, wishing to purge what they saw as superstitions and abuses from their native Hinduism, wished through the Consent Bill to curb child marriage abuses. Though the Bill was not extreme, only raising the age of consent from ten to twelve, Tilak took issue with it; he did not object per se to the idea of moving towards the elimination of child marriage, but rather to the idea of British interference with Hindu tradition. For Tilak, such reform movements were not to be sought after under imperial rule when they would be enforced by the British, but rather after independence was achieved when Indians would enforce it on themselves. Despite Tilak’s opposition, however, Gokhale and the reformers won the day and the bill became law in the Bombay Presidency. In 1905, Gokhale became president of the Indian National Congress. Gokhale used his now considerable influence to undermine his longtime rival, Tilak, refusing to support Tilak as candidate for president of the Congress in 1906. By now, Congress was split: Gokhale and Tilak were the respective leaders of the moderates and the "extremists" (the latter now known by the more politically correct term, 'aggressive nationalists') in the Congress. Tilak was an advocate of civil agitation and direct revolution to overthrow the British Empire, whereas Gokhale was a moderate reformist. As a result, the Congress Party split into two wings and was largely robbed of its effectiveness for a decade. The two sides would later patch up in 1916 after Gokhale died.

In 1905, when Gokhale was elected president of the Indian National Congress and was at the height of his political power, he founded the Servants of India Society to specifically further one of the causes dearest to his heart: the expansion of Indian education. For Gokhale, true political change in India would only be possible when a new generation of Indians became educated as to their civil and patriotic duty to their country and to each other. Believing existing educational institutions and the Indian Civil Service did not do enough to provide Indians with opportunities to gain this political education, Gokhale hoped the Servants of India Society would fill this need. In his preamble to the SIS’s constitution, Gokhale wrote that “The Servants of India Society will train men prepared to devote their lives to the cause of country in a religious spirit, and will seek to promote, by all constitutional means, the national interests of the Indian people.” The Society took up the cause of promoting Indian education in earnest, and among its many projects organized mobile libraries, founded schools, and provided night classes for factory workers. Although the Society lost much of its vigor following Gokhale’s death, it still exists to this day, though its membership is small.

Gokhale, though an earlier leader of the Indian nationalist movement, was not primarily concerned with independence but rather with social reform; he believed such reform would be best achieved by working within existing British government institutions, a position which earned him the enmity of more aggressive nationalists such as Tilak. Undeterred by such opposition, Gokhale would work directly with the British throughout his political career in order to further his reform goals.

In 1899, Gokhale was elected to the Bombay Legislative Council. He was elected to the Council of India of Governor-General of India on 22 May 1903 as non-officiating member represeting Bombay Province. He later served to Imperial Legislative Council after its expansion in 1909. He there obtained a reputation as extremely knowledgeable and contributed significantly to the annual budget debates. Gokhale developed so great a reputation among the British that he was invited to London to meet with secretary of state Lord John Morley, with whom he established a rapport. Gokhale would help during visit to shape the Morley-Minto Reforms introduced in1909. Gokhale was appointed a CIE (Companion of the Order of the Indian Empire) in the 1904 New Year's Honours List, a formal recognition by the Empire of his service.

Gokhale was famously a mentor to Mahatma Gandhi in his formative years. In 1912, Gokhale visited South Africa at Gandhi's invitation. As a young barrister, Gandhi returned from his struggles against the Empire in South Africa and received personal guidance from Gokhale, including a knowledge and understanding of India and the issues confronting common Indians. By 1920, Gandhi emerged as the leader of the Indian Independence Movement. In his autobiography, Gandhi calls Gokhale his mentor and guide. Gandhi also recognised Gokhale as an admirable leader and master politician, describing him as 'pure as crystal, gentle as a lamb, brave as a lion and chivalrous to a fault and the most perfect man in the political field'. Despite his deep respect for Gokhale, however, Gandhi would reject Gokhale's faith in western institutions as a means of achieving political reform and ultimately chose not to become a member of Gokhale's Servants of India Society. Gokhale was also the role model and mentor of Mohammed Ali Jinnah, the future founder of Pakistan, who in 1912, aspired to become the "Muslim Gokhale". Gokhale famously praised Jinnah as an "ambassador of Hindu-Muslim Unity."

The Gokhale Institute of Politics and Economics (GIPE), commonly known as Gokhale Institute, is one of the oldest research and training institutes in Economics in India. It is located on BMCC Road in the Deccan Gymkhana area of Pune, Maharashtra. The Institute was founded with an endowment offered to the Servants of India Society by Shri R R Kale. The Servants of India Society are the trustees of the Institute.

Gokhale continued to be politically active through the last years of his life. This included extensive travelling abroad: in addition to his 1908 trip to England, he also visited South Africa in 1912, where his protégé Gandhi was working to improve conditions for the Indian minority living there. Meanwhile, he continued to be involved in the Servants of India Society, the Congress, and the Legislative Council while constantly advocating the advancement of Indian education. All these stresses took their toll, however, and Gokhale died in Feb 19 1915 at forty-nine years of age.

Gokhale's impact on the course of the Indian nationalist movement was considerable. Through his close relationship with the highest levels of British imperial government, Gokhale forced India's colonial masters to recognize the capabilities of a new generation of educated Indians and to include them more than ever before in the governing process. Gokhale’s firm belief in the need for a spiritualization of politics, social amelioration and universal education deeply inspired the next great man on the Indian political stage, Mohandas K. Gandhi; his ultimate faith in western political institutions and classical liberalism, though rejected by Gandhi, would come to ultimate fruition in the Westminster model of government adopted by an independent India in 1950.