Astbury was the fourth child of seven, born in Longton, Stoke-on-Trent. His father, William Edwin Astbury, was a potter and provided comfortably for his family. Astbury also had a younger brother, Norman, with whom he shared a love of music. Astbury might well have become a potter but, luckily, won a scholarship to Longton High School, where his interests were shaped by the Headmaster and second master, both chemists. After becoming Head boy and winning the Duke of Sutherland's Gold Medal, Astbury won the only local scholarship available and went up to Jesus College, Cambridge.

After two terms at Cambridge, his studies were interrupted by service during the First World War. A poor medical rating following appendectomy resulted in his posting in 1917 to Cork, Ireland, with the Royal Army Medical Corps. He later returned to Cambridge and finished his last year with a specialization in physics.

After graduating from Cambridge, Astbury worked with William Bragg, first at University College London and then, in 1923, at the Davy-Faraday Laboratory at the Royal Institution in London. Fellow students included many eminent scientists, including Kathleen Lonsdale and J.D. Bernal and others. Astbury showed great enthusiasm for his studies and published papers in classic crystallography, such as the structure of tartaric acid.

In 1928, Astbury was given a lectureship at the University of Leeds where he studied the properties of fibrous substances such as keratin and collagen with funding from the textile industry. (Wool is made of keratin.) These substances did not produce sharp patterns of spots like crystals, but the patterns provided physical limits on any proposed structures.

In the early 1930s, Astbury showed that there were drastic changes in the diffraction of moist wool or hair fibers as they are stretched significantly (100%). The data suggested that the unstretched fibers had a coiled molecular structure with a characteristic repeat of 5.1 Å (=0.51 nm). Astbury proposed that (1) the unstretched protein molecules formed a helix (which he called the α-form); and (2) the stretching caused the helix to uncoil, forming an extended state (which he called the β-form). Although incorrect in their details, Astbury's models were correct in essence and correspond to modern elements of secondary structure, the α-helix and the β-strand (Astbury's nomenclature was kept), which were developed twenty years later by Linus Pauling and Robert Corey in 1951. Hans Neurath was the first to show that Astbury's models could not be correct in detail, because they involved clashes of atoms. Interestingly, Neurath's paper and Astbury's data inspired H.S. Taylor (1941, 1942) and Maurice Huggins (1943) to propose models of keratin that are very close to the modern α-helix.

In 1931, Astbury was also the first to propose that mainchain-mainchain hydrogen bonds (i.e., hydrogen bonds between the backbone amide groups) contributed to stabilizing protein structures. His initial insight was taken up enthusiastically by several researchers, including Linus Pauling.

Astbury's worked moved on to X-ray studies of many proteins (including myosin, epidermin and fibrin) and he was able to deduce from the diffraction patterns that the molecules of these substances were coiled and folded.

In 1937 Torbjörn Caspersson of Sweden sent him well prepared samples of DNA from calf thymus. The fact that DNA produced a diffraction pattern indicated that it also had a regular structure and it might be feasible to deduce it. Astbury reported that DNA's structure repeated every 2.7 nanometres and that the bases lay flat, stacked, 0.34 nanometres apart. At a symposium in 1938 at Cold Spring Harbor, Astbury pointed out that the 0.34 nanometre spacing was the same as amino acids in polypeptide chains. (The currently accepted value for the spacing of the bases in B-form of DNA is 0.332 nm.)

In 1946 Astbury presented a paper at a symposium in Cambridge in which he said: "Biosynthesis is supremely a question of fitting molecules or parts of molecules against another, and one of the great biological developments of our time is the realisation that probably the most fundamental interaction of all is that between the proteins and the nucleic acids." He also said that the spacing between the nucleotides and the spacing of amino acids in proteins "was not an arithmetical accident".

Astbury's was unable to propose the correct structure of DNA from his rudimentary data. However in 1952 Linus Pauling used Astbury's insufficient data to propose a structure for DNA, which was also incorrect. Nevetheless Astbury's insights led directly to the work of Maurice Wilkins and Rosalind Franklin and from there to the structure of DNA devised by Francis Crick and James D. Watson in 1953.

In later life he was given many awards and honorary degrees.

Astbury was known for his unfailing cheerfulness, idealism, imagination and enthusiasm. He foresaw correctly the tremendous impact of molecular biology and transmitted his vision to his students, "his euphoric evangelizing zeal transforming laboratory routine into a great adventure". Astbury's enthusiasm may also account for an occasional lack of scientific caution observable in his work; Astbury could make speculative interpretations sound plausible. Astbury was an excellent writer and lecturer; his works are characterized by remarkable clarity and an easy-going, natural manner. He also enjoyed music, playing both piano and violin.

Astbury met Frances Gould when he was stationed in Cork, Ireland with the Royal Army Medical Corps during World War I. They married in 1922 and had a son Bill and a daughter.