We employ an appropriate change of measure technique to offer a general result connecting a general form of the Gerber–Shiu function with the distribution of the deficit at ruin under the new (exponentially tilted) measure. Exploiting this result, we extract closed-form formulae for special forms of the Gerber–Shiu function assuming two cases of bivariate distributions that describe the dependence structure between claim sizes and inter-claim times. More specifically, initially, we employ the Downton–Moran bivariate exponential distribution, and we offer explicit formulae for cases of the Gerber–Shiu functions that include the time and the number of claims until ruin. In addition, we derive a closed formula for the defective discounted joint density of the number of claims until ruin, the deficit at ruin, and the time until ruin. The same is achieved for the joint density of the number of claims and the deficit at ruin. We further generalize these results by assuming that the inter-claim times and the claim sizes follow a Kibble–Moran bivariate Erlang distribution. Finally, we offer numerical examples in order to illustrate our main results.