This paper reviews what we know on the ExtragalacticBackground (EB)and on the nature of its contributing galaxies in the infrared (IR) and sub-millimeter (sub-mm).I first present the EB observationnal constraints, their main cosmologicalimplications,and results from deep IR and sub-mm surveys.Then, using a phenomelogical model of IR galaxy evolution,I show how these data constrain the evolution of IR galaxies,in particular the luminosity densityand the luminosity function.And finally, I briefly review the detection of the IR EB fluctuations and theirpotential implications.