Rousset's patron was first-cousin to François I of France. His father, Philippe de Savoie (1490–1534) was Count of Genevois and brother to the Queen Mother, Louise de Savoie (1476–1531). Despite being the King's uncle, Philippe resided at the court of the King's rival, Emperor Charles V (1500–1558). François attracted Philippe back to France in 1528 by contracting his marriage to Charlotte d'Orléans-Longueville (1512–1549) and dowering him with 60,000 livres; he promised him another 100,000 livres, giving him the Duchy of Nemours (which had passed to the Queen Mother in 1524) as collateral until such time as that sum could be paid.

After Philippe's death in 1534, similar terms were extended to his son Jacques, then an infant under his mother's tutelage. The Royal purse never could come up with the 100,000 livres, so this arrangement continued until 1570 under François' successors, Henri II and Charles IX. In 1566, Jacques married Anne d'Este (1531–1607), daughter of Renée de France and widow of François de Lorraine (1519–1563), the 2nd Duc de Guise. After returning to France from Italy in 1559, Renée de France pressed her claims to her late mother's vast estates in Brittany, which had been terminated without her knowledge when she married in 1528. A resolution of sorts was achieved in 1570, whereby Renée de France would be granted ‘usage’ of the Duchy of Nemours during her lifetime; after her death, the Duchy would be owned outright by her daughter Anne.