Geoffroi de Charny's year of birth is unknown. However, a terminus ad quem has been provided by the chronicler Jean de Joinville, his maternal grandfather, who instituted a memorial for the anniversary of his daughter Marguerite de Joinville (Geoffroi's mother), in late 1306. Because the infant Geoffroi was Marguerite's last child, and would not appear in the historical record for nearly three decades, it is possible that she died giving birth to him.

Geoffroi's father Jean de Charny, lord of the Charny castle – now long demolished – that commanded the river Armançon south of Troyes, was a leading vassal of the dukes of Burgundy. Being a cadet branch of the lords of nearby Mont-Saint-Jean, whose heraldry comprised gules, trois escutcheons d’or, the Charny family had to be content with escutcheons of silver. Their forebear Hugues I de Mont-Saint-Jean, who had held the dynasty's lordship at the turn of the twelfth and thirteenth centuries, was the very first lay signatory to the founding of the abbey of Cîteaux, from which sprang the Cistercian monastic order. Thereafter successive generations of the Mont-Saint-Jeans and Charnys regularly supported the Cistercians and their related military arm the Knights Templar, both with funding and with manpower. The Templar Draper (also named Geoffroi) de Charny, famously burnt at the stake along with the Order's Grand Master Jacques de Molay in 1314, may well have belonged to the same branch of the family. Support for the Cistercian and Templar orders was notably shared by the families with whom the Mont-Saint-Jeans and Charnys inter-married, the expected reward being burial in a favoured Cistercian abbey.

If as seems likely the young Geoffroi spent almost his whole childhood without his natural mother, a further handicap of his birth was that he was the youngest of three brothers. As required by Salic law his eldest brother, Dreux de Charny, inherited the main Charny lordship and castle, in addition to which an advantageous marriage brought Dreux two lucrative Morea baronies, Nivelet and Vostitza, all of which passed to Dreux's daughters on his untimely death c. 1325. Jean II de Charny, the second eldest brother, seems to have inherited the lesser Charny fiefs of Lirey and Savoisy, which passed to his widow Jeanne de Frolois following his early death in the 1320s. Geoffroi, as the third brother, seems to have been expected to fend for himself.