This chapter probes the intricate relationship between cause and effect. It navigates through Wiener–Granger causality and introduces transfer entropy, providing the tools to dissect causal relationships in data-driven models.

Within this chapter, the focus is on delving into univariate probability distributions. It dissects the anatomy of the normal distribution, explores characteristic functions, and analyzes stable distributions. This chapter establishes a comprehensive understanding of various probability distributions, setting the stage for more advanced discussions.

This chapter delves into methods for comparing probability estimates and assessing the goodness of models. Tools and methodologies such as null models, p-values, and Bayesian model selections are introduced and discussed. From regression to classification, this chapter illuminates model evaluation via likelihood and model selection techniques.

Unveiling the intricate world of networks and graphs, this chapter unearths the significance of adjacency matrices, centrality, and propagation. From paths to higher-order networks, readers gain insights into employing network structures to model complexities inherent in diverse systems.

Focusing on networks as potent representations of complex, real systems, this chapter unearths network construction, information filtering, and higher-order networks. Readers are empowered to capture the essence of intricate relationships within complex systems by using network representations.

The concluding chapter reflects on the scientific method, automated model construction, and the evolving landscape of modeling. It contemplates the balance between complexity and interpretability, providing insights into the future of data-driven modeling.

This chapter is dedicated to nonparametric estimation methods, dissecting the sample mean, moments, and probability mass functions. It delves into the convergence laws for the sample means, enabling the construction of probability distributions from empirical data.

Centered on the fusion of probabilistic modeling and network representations, this chapter introduces an information-theoretic lens for network learning. It illuminates probability decomposition, clique tree inference structures, and network representation learning, offering insights into probabilistic modeling of intricate relationships in complex systems.

The intricate dynamics of dependence and correlation are explored in this chapter. The discussion spans from linear to nonlinear correlations, encompassing regression and information-theoretic measures. This chapter lays the groundwork for deciphering the complex interplay between variables.

Delving into the fundamentals of machine learning, this chapter dissects supervised, unsupervised, and reinforcement learning approaches. The chapter discusses training, validation, and testing models, furnishing readers with the requisite skills for harnessing machine learning in data-driven modeling.

Expanding on estimation techniques, this chapter focuses on parametric approaches such as maximum likelihood estimation and Bayesian parameter estimation. It unveils methods to fit parametric distributions to empirical data, providing complementary modeling tools to the nonparametric ones discussed in the previous chapter.

Delving into multivariate probabilities, this chapter explores both nonparametric and parametric estimation techniques. From Pearson’s covariance matrix to multivariate normal distribution, readers gain insights into modeling relationships among multiple variables.