In contrast to Emerson and Poe, who receive just one mention apiece in A la recherche, James McNeill Whistler appears by name more often than Flaubert, Manet, Mallarmé, Debussy, Zola, the Goncourt brothers, Gautier, Degas, or Monet. Only Baudelaire is referred to more frequently. Whistler is mentioned more than Mozart, Dante, Watteau, Shakespeare, Clemenceau, or Napoleon; while among painters only Vermeer and Rembrandt merit more attention. And, of course, Elstir, Proust's fictional artist, to whom Whistler contributed both a near–anagram of his name and a number of personal and artistic traits.

Proust was interested in Whistler's art from as early as 1891, when he was just shy of twenty and Whistler, at fifty–six, remained at the height of his powers. Between November 1890 and the following September, Proust was involved in editing a magazine called Le Mensuel (The Monthly) to which he contributed a number of articles. Most, as with “Impressions of the Salons,” which appeared in May 1891 and contained some mention of Whistler, appeared under a nom de plume, in this case Fusain (an artist's charcoal crayon used for drawing). From the beginning, then, Whistler is associated with Proust's fictionalizing of himself. Writing and art, fusain and stylo, are interchangeable.

Proust's admiration for Whistler surely played some part in the friendship that sprang up in 1895 at Beg–Meil on the Brittany coast, where along with his friend, the Venezuelan–born composer Reynaldo Hahn, he met the American painter, Alexander Harrison. While Beg–Meil was later to form the basis for Balbec, Harrison is a vital member of the artistic family–tree that was to culminate in Elstir. As with Marcel's first sight of Elstir while dining with Saint–Loup at Rivebelle, so Proust and Hahn dropped Harrison a note telling him how much they admired his work and asking that he join them for dinner. Unlike Elstir, Harrison accepted. What is more, Harrison's later enjoinder that the friends visit the Pointe de Penmarch because it was “a sort of mixture of Holland, the West Indies and Florida” (Carter 2000, 1988) became transposed in A la recherche to the fictional Carquethuit that Elstir paints and which puts him in mind of “certain aspects de la Floride” [somewhere in Florida] (671; II:433).