Theoretical insights dominate the literature examining the incentive compatibility of payment mechanisms. Despite their elegance, theoretical insights are rarely empirically validated. We fill this gap by empirically exploring the effects of frequently used payment mechanisms using a collective sample of over 3000 participants across two experiments. In Experiment 1, we obtained offer prices to sell a card, systematically varying between-subjects the way subjects received payments over repeated rounds, by either paying for all decisions (and various modifications) or just one, as well as making the payments certain, probabilistic, or purely hypothetical. While we find that the magnitude of the induced value and the range of the prices used to draw a random price significantly affect misbidding behavior, neither the payment mechanism nor the certainty of payment affected misbidding. In Experiment 2, we replaced the BDM mechanism with a Second Price-Auction and found similar results, albeit lower rates of misbidding behavior. Overall, our empirical exercise shows that theoretically relevant elements do not produce empirical differences, while design choices that are theoretically irrelevant produce empirical differences. As such, payment mechanism design considerations should carefully consider the choice architecture in addition to incentive compatibility.