Born in Plau am See, Elvers was born the son of a merchant. He went to school first in Plau and then to secondary school at Waren by Müritz. During his Wehrmachts years in the Second World War he was captured by soviet soldiers. After his return he studied with Walter Gerstenberg at the University of Rostock from 1946–1948. He then pursued doctoral studies at the Free University of Berlin where he received his doctorate in 1953. His dissertation, Untersuchungen zu den Tempi in Mozarts Instrumentalmusik', was on tempi in the instrumental music of Mozart.
After completing his education, Elvers worked as music dealer for the Merseburger Verlag in Berlin. In 1965 he left the post to become director of the Felix Mendelssohn archives at the Berlin State Library's Staatsbibliothek Preussischer Kulturbesitz. He was promoted to director of the music department at the Berlin State Library in 1967; a postition he held until his retirement in 1988.
During his years in the music department of the Berlin State Library,Elvers was - partly together with Hans-Günter Klein - responsible for a number of catalogues that are now considered standard reference works, including Ludwig van Beethoven (1970), Felix Mendelssohn (1972), Fanny Mendelssohn (1972 and 1983), and Johann Sebastian Bach (1985). During this time he also worked as a music critic and served as the editor of the Musikbibliographische Arbeiten from 1973 through 1996. He also prepared several volumes of an edition of Felix Mendelssohn Bartholdy's letters, which is now published by the Kassel-based Bärenreiter-Verlag.
Elvers remained an enthusiastic collector throughout his life. He built up one of the most important collections of autograph letters, manuscripts and artifacts of Felix Mendelssohn Bartholdy and his family, which he donated to the Stadtgeschichtliches Museum Leipzig in 2009.
Elvers died in Berlin at the age of 87.
- Musikdrucker, Musikalienhändler und Musikverleger in Berlin 1750 bis 1850. Eine Übersicht, in Festschrift Walter Gerstenberg zum 60. Geburtstag, ed. by Georg von Dadelsen, Wolfenbüttel 1964, pp. 37–44
- Die Berliner Musikverlage im 19. Jahrhundert, in Bericht über den internationalen musikwissenschaftlichen Kongreß Berlin 1974, Kassel 1980, pp. 379–380
- Berliner Musikverleger, in Studien zur Musikgeschichte Berlins im frühen 19. Jahrhundert, ed. by Carl Dahlhaus, Regensburg 1980
- Mendelssohn-Studien, published for the Mendelssohn-Gesellschaft by Cécile Lowenthal-Hensel, Rudolf Elvers, Hans-Günter Klein, Christoph Schulte und Roland Schmidt-Hensel, Berlin, later Hannover 1972ff.
- Hans Heinrich Eggebrecht, revised by Jutta Pumpe (2001). "Elvers, Rudolf". Grove Music Online. Oxford Music Online. Oxford University Press. doi:10.1093/gmo/9781561592630.article.08756.
- Musikdrucker, Musikalienhändler und Musikverleger in Berlin 1750 bis 1850. Eine Übersicht on WorldCat
- Die Berliner Musikverlage im 19. Jahrhundert on WorldCat
- Datierte Verlagsnummern Berliner Musikverleger II on WorldCat