Classical methods such as ordinary least squares (OLS) were designed for datasets with just a handful of predictors. However, the trend over the last 30 years or so has been towards larger datasets, in particular where the number of predictors may be large even compared with the number of observations --- so-called high-dimensional data. In such settings, it may be advantageous to consider penalised estimators that reduce variance, at the expense of introducing some bias. We study both ridge regression and Lasso estimators, with the latter having the attractive property that it performs simultaneous variable selection and parameter estimation. The Lasso can be computed efficiently using coordinate descent, and we present finite-sample guarantees on its prediction, estimation and variable selection performance. We also consider extensions such as the square-root Lasso, the debiased Lasso and complementary pairs stability selection.