James was born Elmore Brooks in the old Richland community in Holmes County, Mississippi (not to be confused with two other locations of the same name in Mississippi). He was the illegitimate son of 15 year old Leola Brooks, a field hand. His father was probably Joe Willie "Frost" James, who moved in with Leola, and so Elmore took this as his name. His parents adopted an orphaned boy at some point named Robert Holston.

Elmore began making music at the age of 12 using a simple one - string instrument ('diddley bow' or 'jitterbug') strung up on a shack wall. As a teen he was playing at local dances under the names Cleanhead and Joe Willie James. His first marriage was to Minnie Mae, circa 1942 (whom he apparently never divorced). He subsequently married twice, to Georgianna Crump in 1947 and to a woman called Janice circa 1954. (Another reported marriage of Elmore to a Josephine Harris has been found to be a mistaken record of a different Elmore James.)

Becoming a well known musician in those days, with the rewards of prestige, good free food, illicit free liquor, women's favors, the promise of escape from the hard agricultural work, etc., must have been as attractive to Elmore as it was to the other musicians of that time and earlier, such as the 'second' Sonny Boy Williamson, with whom he played and Robert Johnson with whom he also possibly played. Although Robert Johnson died in 1938, James (like many other musicians) was strongly influenced by him, and by Kokomo Arnold and Tampa Red. Elmore recorded several of Tampa's songs, and even inherited from his band two of his famous 'Broomdusters', 'Little' Johnny Jones (piano) and Odie Payne (drums). There is a dispute as to whether Robert Johnson or Elmore wrote James' trademark song, "Dust My Broom".

James's demise may have been hastened by his lifelong taste for, and manufacture of, moonshine whiskey, to which he was introduced at an early age. Alcohol definitely killed his band mates / friends Willie Love and Johnny Jones at an early age. His regular rhythm guitarist Homesick James maintaining his longevity was due to his not partaking of the heavy drinking sessions after — and often during — gigs, a refusal that was unpopular with the rest of the band. James was also reportedly an extremely fast driver who also loved hunting with guns and dogs in Mississippi.

During World War II James joined the United States Navy, was promoted to coxswain and took part in the invasion of Guam against the Japanese. Upon his discharge, Elmore returned to central Mississippi and eventually settled in Canton with his adopted brother Robert Holston; it was at this time he learned that he had a serious heart condition. Working in Robert's electrical shop, he devised his unique electric sound, using parts from the shop and an unusual placement of two D'Armond pickups. He began recording with Trumpet Records in nearby Jackson in January 1951, first as sideman to the second Sonny Boy Williamson and also to their mutual friend Wille Love and possibly others, then debuting as a session leader in August with "Dust My Broom". It was a surprise R&B hit in 1952 and turned James into a star. He then broke his recording contract with Trumpet Records to sign up with the Bihari Brothers through Ike Turner (who played guitar and piano on a couple of his early Bihari recordings). His "I Believe" was another hit a year later. During the 1950s he recorded for the Bihari brothers' Flair Records, Meteor Records and Modern Records labels, as well as for Chess Records and Mel London's Chief Records (his "It Hurts Me Too" was later a hit when he re-recorded it for Enjoy Records). His backing musicians were known as the Broomdusters. In 1959 he began recording for Bobby Robinson's Fire Records label. These include "The Sky Is Crying" (credited to Elmo James and His Broomdusters), "My Bleeding Heart", "Stranger Blues", "Look on Yonder Wall", "Done Somebody Wrong", and "Shake Your Moneymaker", all of which are among the most famous of blues recordings.

James died of his third heart attack in Chicago in 1963, just prior to a tour of Europe with that year's 'American Folk Blues Festival.' He was buried in the Newport Baptist Church Cemetery in Ebenezer, Mississippi.

James played a wide variety of "blues" (which often crossed over into other styles of music) similar to that of Muddy Waters, Howlin' Wolf and some of B.B. King's work, but distinguished by his guitar's vastly more powerful sound (arguably only equaled in technical ability by King, although in a different style) unbelievably coming from a modified, hollow body traditional acoustic guitar, which sounded like the more modern solid body guitars. He most often played using a slide, but on several recordings he plays without. His voice and style was as instantly recognizable as King's, Muddy's and Wolf's and until he fell foul of the Chicago union, he and his 'Broomdusters' were as popular in the Chicago clubs as any of these musician's bands. James could be reportedly 'difficult' (drinking on the job, not paying out cash, abandoning musicians, double booking etc.) Due to his early death, just before the 1960s "blues boom", and the silence of other famous blues performers, the then current music writers only being interested in 'the stars' not caring to interview his ex-band members, immediate family, children, friends etc. little is known about him. There are no known photos of James performing, apart from those taken (some at the following occasion, and some at a packed club with stylishly dressed couples dancing closely) by George Adins, and no other detailed descriptions or any live recordings.

Muddy Waters took the Belgian blues fan (George Adins) to see James play in Chicago in 1959, Adins recalled,

Elmore will always remain the most exciting, dramatic blues singer and guitarist that I've ever had a chance to see perform in the flesh. On our way we listened to him on the radio as Big Bill Hill ... was broadcasting direct from that place. I was burning to see Elmore James and before we even pushed open the door of the club, we could hear Elmore's violent guitar sound. Although the place was overcrowded, we managed to find a seat close to the bandstand and the blues came falling down on me as it had never done before. Watching Elmore sing and play, backed by a solid blues band (Homesick James, J.T. Brown, Boyd Atkins and Sam Cassell) made me feel real fine. Wearing thick glasses, Elmore's face always had an expressive and dramatic look, especially when he was real gone on the slow blues. Singing with a strong and rough voice, he really didn't need a mike. On such slow blues as "I'm Worried - "Make My Dreams Come True" - "It Hurts Me", his voice reached a climax and created a tension that was unmistakably the down and out blues. Notwithstanding that raw voice, Elmore sang his blues with a particular feeling, an emotion and depth that showed his country background. His singing was... fed, reinforced by his own guitar accompaniment which was as rough, violent and expressive as was his voice. Using the bottleneck technique most of the time, Elmore really let his guitar sound as I had never heard a guitar sound before. You just couldn't sit still! You had to move...

George also witnessed James at 'Alex Club' in West Side Chicago where...

...he always played for a dance audience and he made the people jump. "Bobby's Rock" was at that time one of the favorite numbers with the crowd and Elmore used to play [it] for fifteen minutes and more. You just couldn't stand that hysteric sound coming down on you. The place was rocking, swinging!

The nearest we have to a recording of a 'live' set by James is his last recorded session by Bobby Robinson, in New York City in 1963 shortly before James death. This session features several takes of "Hand In Hand" which was abandoned and James then played a 'live' set.

His best known song is the blues standard "Dust My Broom" (also known as "Dust My Blues"). The song gave its name to James's band, The Broomdusters. The song's opening slide guitar riff is one of the best known sounds in all of blues. It is essentially the same riff that appeared in the recording of the same song by Robert Johnson, but James played the riff with electric slide guitar. It was even transformed into a doo-wop chorus on Jesse Stone's "Down in the Alley", recorded by The Clovers and Elvis Presley. Stone transcribed the riff as: "Changety changety changety changety chang chang!"

Most electric slide guitar players will admit to the massive, if not, total influence of James' style. He was also a major influence on such successful 'Blues' guitarists as 'Homesick James' (who was a member of Elmore's band 'The Broomdusters' and featured on many of his recordings) John Littlejohn, Hound Dog Taylor, J.B. Hutto and many others. He also influenced many rock guitarists such as The Rolling Stones' Brian Jones and Fleetwood Mac's Jeremy Spencer. John Mayall included the song, "Mr. James," on his 1969 "Looking Back" album as a dedication to James. His songs "Done Somebody Wrong" and "One Way Out" were often covered by The Allman Brothers Band, who cited him as a major influence. James was also covered by blues - rock band Stevie Ray Vaughan and Double Trouble many times in concert. The most famous of these covers is one that came by an indirect route - James' fellow bluesman Albert King recorded a cover of "The Sky Is Crying", and Stevie Ray Vaughan copied King's version of the song. That song was also covered by George Thorogood on his second album, Move It On Over and by Eric Clapton on his album There's One in Every Crowd.

The most famous guitarist who admired Elmore James was Jimi Hendrix. Early in his career Jimi styled himself variously as 'Maurice James' and subsequently as 'Jimmy James.' This, according to former bandmate and recording partner Lonnie Youngblood, was a tribute to Elmore James. There is a photo of Hendrix (that can be seen in the sleeve of his :blues album) in London wearing his iconic military jacket and holding Elmore James's UK LP The Best Of Elmore James. (Hendrix was frequently photographed throughout his performing career holding LP covers of musicians that influenced him.) He performed James' "Bleeding Heart" during the Experience's Royal Albert Hall concert in 1969, and also with the Band of Gypsys at their New Year's concerts at the Fillmore East in 1969 / 70 as well as recording two different versions of it in the studio. Buddy Miles also sang lyrics along with Jimi playing some motifs from James' "The Sky Is Crying" during his famous jam session the Newport Pop Festival on June 22, 1969 (the video is now available in collector's circles) as well as quoting several lyrics and motifs from Elmore's catalog throughout his career.

James is mentioned in The Beatles' song "For You Blue": while John Lennon plays the slide guitar (James' trademark), George Harrison says, "Elmore James got nothin' on this, baby." Other artists influenced by Elmore James include Frank Zappa and Jeffrey Evans of the band 68 Comeback.

The Grateful Dead, John Primer (Blue Steel CD), Billy Gibbons and Eric Clapton are other notable artists to have recorded Elmore James covers. Clapton also recorded a song with Jimmy Page called "Tribute to Elmore" that first appeared on the 1968 compilation album Blues Anytime Vol. 1. On the 1974 record Second Album, Roy Buchanan included an instrumental song he wrote titled "Tribute to Elmore James," which begins with James' classic slide guitar riff, and uses his soloing style throughout.

James's older cousin "Homesick" James Williamson, a regular companion from an early age, played with James in the Broomdusters from 1957 on. He was also a successful blues man who played electric slide guitar in James's style. He continued to record and tour until his death in 2006.

James's music & style still today has immense influence upon today's modern blues icons, including the young Derek Trucks of the Allman Brothers and The Derek Trucks Band, who has attributed James as his most predominant influence, and inspiration.