His stay in Venice is undocumented, but the influence on his style is clear. He probably worked as an assistant to Johann Rottenhammer, some of whose drawings he owned. Rottenhammer was a German who had been living in Italy for some years, and was the first German painter to specialize in cabinet paintings. Uffenbach had specialized in large altarpieces, and although Elsheimer's earliest small paintings on copper seem to date from before he arrived in Italy, Rottenhammer's influence is clear on his mature work.

Elsheimer is believed to have produced some significant works in Venice, such as The Baptism of Christ (National Gallery, London) and The Holy Family (Gemäldegalerie, Berlin) which show the influence of the Venetian painters Tintoretto and Paolo Veronese, as well as Rottenhammer.

In early 1600, Elsheimer arrived in Rome and quickly made friends with contacts of Rottenhammer, notably Giovanni Faber, a Papal doctor, botanist and collector originally from Bamberg. He was Curator of the Vatican Botanical Garden, and a member of the Accademia dei Lincei, a small intellectual coterie founded in 1603, and mainly concerned with the natural sciences. Another friend of Rottenhammer was the Flemish landscape painter Paul Bril, already established in Rome, who was (with Faber) a witness at Elsheimer's marriage, painted a picture together with him (now Chatsworth House), and was owed money by him at his death. Like Faber, Bril was a long-term resident in Rome who had converted from Lutheranism to Catholicism, as Elsheimer did later.

Both Faber and Bril knew Rubens, who was in Rome in 1601, and who became another friend, later reproaching Elsheimer for not producing more work. He knew David Teniers the Elder, recently Rubens' pupil, and there is evidence that they lodged together. In 1604 Karel van Mander, a Dutchman recently returned from Rome, published his Schilder-Boeck which praised Elsheimer's work, and described him as slow-working and making few drawings. He also spent much time in churches, studying the works of the masters. Other writers mention his exceptional visual memory, his melancholy and his kind-heartedness. In a letter after his death, Rubens wrote: "he had no equal in small figures, landscapes, and in many other subjects. ...one could have expected things from him that one has never seen before and never will see."

In 1606, Elsheimer married Carola Antonia Stuarda da Francoforte (i.e. Stuart of Frankfurt - she was of Scottish ancestry and a fellow Frankfurter), and in 1609 they had a son. The son was not mentioned in a census a year later, possibly (Klessman says optimistically) because he had been put out to a wet-nurse. She was the recent widow of the artist Nicolas de Breul (born in Verdun) and after Elsheimer's death remarried an Italian artist, Ascanio Quercia, within a year of his death. Elsheimer converted to Catholicism by 1608 (possibly 1606). He was admitted to the Accademia di San Luca, the Roman painters' Guild in 1606, giving them a self-portrait (his only portrait, and only painting on canvas) now in the Uffizi. In spite of his fame and talents, he appears to have both lived and died in difficult financial circumstances.

Elsheimer's painting of Tobias and the Angel (1602 – 1603) (the "small" Tobias - now Frankfurt) was especially well received because of its new conception of landscape. This picture was engraved by Count Hendrick Goudt and as a result was published across Europe. However, his association with Goudt, who lodged and trained with him for several years, was difficult. Elsheimer seems to have borrowed money from Goudt, which according to one account resulted in his brief incarceration in Debtor's prison. After Elsheimer's early death in 1610 in Rome, Goudt owned several of his pictures. Goudt made seven engravings of Elsheimer's paintings, which were crucial in spreading his influence, as very few of his paintings were viewable even by artists; as cabinet paintings they were mostly kept in small and very private rooms.

Elsheimer had a definite preference for choosing rare or original subjects, both for his mythological and religious paintings. Jupiter & Mercury in the house of Philemon & Baucis, (c1608, now Dresden) is based on an episode in Ovid, and had never been painted before. The Mocking of Ceres (US private collection, copy Prado), Apollo and Coronis (Liverpool), and Il Contento (Edinburgh) were equally new. Some of his religious scenes were more conventional, but his selection of the moment to depict, as in St Lawrence prepared for Martyrdom (London), is often unusual.

His perfectionism, and an apparent tendency to depression, resulted in a small total output, despite the small size of all his pictures. In all about forty paintings are now generally agreed to be by him. He made a few etchings, not very successfully. However, his work was highly regarded by other artists and a few important collectors for its quality. He had a clear and direct influence on other Northern artists who were in Rome such as Paul Bril, Jan Pynas, Leonaert Bramer and Pieter Lastman, later Rembrandt's master, who was probably in Rome by 1605. Rembrandt's first dated work is a Stoning of St Stephen which appears to be a response to Elsheimer's painting of the subject, now in Edinburgh. Some works by Italian artists, such as the six pictures from Ovid by Carlo Saraceni now in the Museo di Capodimonte, Naples, also show Elsheimer's clear influence. Rubens, who owned at least four of his works, knew Elsheimer in Rome, and praised him highly in a letter after his death.

In a wider sense, he was influential in three respects. Firstly his night scenes were highly original. His lighting effects in general were very subtle, and very different from those of Caravaggio. He often uses as many as five different sources of light, and graduates the light relatively gently, with the less well-lit parts of the composition often containing important parts of it.

Secondly, his combination of poetic landscape with large foreground figures gives the landscape a prominence that had rarely been seen since the Early Renaissance. His landscapes do not always feature an extensive view; often the lushness of the vegetation closes it off. They are more realistic, but no less poetic, than those of Bril or Jan Brueghel, and play a part in the formation of those of Poussin and Claude. His treatment of large figures with a landscape backdrop looks forward, through Rubens and van Dyck, to the English portrait in the eighteenth century. Soon after his death he became very popular with English collectors, notably King Charles I of England, the Earl of Arundel, and the George Villiers, 1st Duke of Buckingham, and over half his paintings have been in English collections at some time (nearly one third are still in the UK).

Thirdly, his integration of Italian styles with the German tradition he was trained in is perhaps more effective than that of any Northern painter since Dürer (with the exception of his friend Rubens). His compositions tend to underplay the drama of the events they depict (in noticeable contrast to those of Rubens), but often show the start of moments of transformation. His figures are relatively short and stocky, and reflect little of classical ideals. Their poses and gestures are unflamboyant, and their facial expressions resemble those in Early Netherlandish painting rather than the Bella Figura of most Italian Renaissance work.

The largest collection of his work is in Frankfurt. The Alte Pinakothek, Munich has two of his finest night-scene paintings, and Berlin, Bonn, Dresden and Hamburg have paintings. The National Gallery, London has three paintings with others in the National Gallery of Scotland, Edinburgh, Apsley House, Windsor Castle, Petworth House, the Wellcome Library and Liverpool. The only work on public display outside Europe is a tiny but beautiful one in the Kimbell Art Museum, Fort Worth. In 2006 an exhibition at the Städel, Frankfurt, then Edinburgh, & Dulwich Gallery in London reunited almost all his oeuvre. There are drawings, especially in the Louvre and Edinburgh.