“What if?” in Probabilistic Logic Programming

A ProbLog program is a logic program with facts that only hold with a specified probability. In this contribution we extend this ProbLog language by the ability to answer “What if” queries. Intuitively, a ProbLog program defines a distribution by solving a system of equations in terms of mutually independent predefined Boolean random variables. In the theory of causality, Judea Pearl proposes a counterfactual reasoning for such systems of equations. Based on Pearl’s calculus, we provide a procedure for processing these counter-factual queries on ProbLog programs, together with a proof of correctness and a full implementation. Using the latter, we provide insights into the influence of different parameters on the scalability of inference. Finally, we also show that our approach is consistent with CP-logic, i.e. with the causal semantics for logic programs with annotated with disjunctions.


Introduction
Humans show the remarkable skill to reason in terms of counterfactuals.This means we reason about how events would unfold under different circumstances without actually experiencing all these different realities.For instance we make judgements like: "If I had taken a bus earlier, I would have arrived on time."without actually experiencing the alternative reality in which we took the bus earlier.As this capability lies at the basis of making sense of the past, planning courses of actions, making emotional and social judgments as well as adapting our behaviour, one also wants an artificial intelligence to reason counterfactually (Hoeck, 2015).
Here, we focus on the counterfactual reasoning with the semantics provided by Pearl (2000).Our aim is to establish this kind of reasoning in the ProbLog language of De Raedt et al. (2007).
To illustrate this issue we introduce a version of the sprinkler example from Pearl (2000), §1.4.
It is spring or summer, written szn spr sum, with a probability of π 1 := 0.5.Consider a road, which passes along a field with a sprinkler on it.In spring or summer, the sprinkler is on, written sprinkler, with probability π 2 := 0.7.Moreover, it rains, denoted by rain, with probability π 3 := 0.1 in spring or summer and with probability π 4 := 0.6 in fall or winter.If it rains or the sprinkler is on, the pavement of the road gets wet, denoted by wet.When the pavement is wet, the road is slippery, denoted by slippery.Under the usual reading of ProbLog programs one would model the situation above with the following program P: 0.5::u1.0.7::u2.0.1::u3.0.6::u4.szn_spr_sum :-u1.sprinkler :-szn_spr_sum, u2.rain :-szn_spr_sum, u3.
To construct a semantics for the program P we generate mutually independent Boolean random variables u1-u4 with π(ui) = π i for all 1 ≤ i ≤ 4. The meaning of the program P is then given by the following system of equations: szn spr sum := u1 rain := (szn spr sum ∧ u3) ∨ (¬szn spr sum ∧ u4) sprinkler := szn spr sum ∧ u2 wet := (rain ∨ sprinkler) slippery := wet (1) Finally, assume we observe that the sprinkler is on and that the road is slippery.What is the probability of the road being slippery if the sprinkler were switched off?
Since we observe that the sprinkler is on, we conclude that it is spring or summer.However, if the sprinkler is off, the only possibility for the road to be slippery is given by rain.Hence, we obtain a probability of 0.1 for the road to be slippery if the sprinkler were off.
In this work, we automate this kind of reasoning.However, to the best of our knowledge, current probabilistic logic programming systems cannot evaluate counterfactual queries.While we may ask what the probability of slippery is if we switch the sprinkler off and observe some evidence, we obtain a zero probability for sprinkler after switching the sprinkler off, which renders the corresponding conditional probability meaningless.To circumvent this problem, we adapt the twin-network method of Balke and Pearl (1994) from causal models to probabilistic logic programming, with a proof of correctness.Notably, this reduces counterfactual reasoning to marginal inference over a modified program.Hence, we can immediately make use of the established efficient inference engines to accomplish our goal.
We also check that our approach is consistent with the counterfactual reasoning for logic programs with annotated disjunctions or LPAD-programs (Vennekens et al., 2004), which was presented by Vennekens et al. (2010).In this way, we fill the gap of showing that the causal reasoning for LPAD-programs of Vennekens et al. (2009) is indeed consistent with Pearl's theory of causality and we establish the expressive equivalence of ProbLog and LPAD regarding counterfactual reasoning.
Apart from our theoretical contributions, we provide a full implementation by making use of the aspmc library (Eiter et al., 2021).Additionally, we investigate the scalability of the two main approaches used for efficient inference, with respect to program size and structural complexity, as well as the influence of evidence and interventions on performance.

Preliminaries
Here, we recall the theory of counterfactual reasoning from Pearl (2000) before we introduce the ProbLog language of De Raedt et al. (2007) in which we would like to process counterfactual queries.

Pearl's Formal Theory of Counterfactual Reasoning
The starting point of a formal theory of counterfactual reasoning is the introduction of a model that is capable of answering the intended queries.To this aim we recall the definition of a functional causal model from Pearl (2000), §1.4.1 and §7 respectively: Definition 1 (Causal Model) A functional causal model or causal model M on a set of variables V is a system of equations, which consists of one equation of the form X := f X (pa(X), error(X)) for each variable X ∈ V. Here, the parents pa(X) ⊆ V of X form a subset of the set of variables V, the error term error(X) of X is a tuple of random variables and f X is a function defining X in terms of the parents pa(X) and the error term error(X) of X.
Fortunately, causal models do not only support queries about conditional and unconditional probabilities but also queries about the effect of external interventions.Assume we are given a subset of variables X := {X 1 , ..., X k } ⊆ V together with a vector of possible values x := (x 1 , ..., x k ) for the variables in X.In order to model the effect of setting the variables in X to the values specified by x, we simply replace the equations for X i in M by X i := x i for all 1 ≤ i ≤ k.
To guarantee that the causal models M and M do(X:=x) yield well-defined distributions π M ( ) and π M ( | do(X := x)) we explicitly assert that the systems of equations M and M do(X:=x) have a unique solution for every tuple e of possible values for the error terms error(X), X ∈ V and for every intervention X := x.

Example 1
The system of equations (1) from Section 1 forms a (functional) causal model on the set of variables V := {szn spr sum, rain, sprinkler, wet, slippery} if we define error(szn spr sum) := u1, error(sprinkler) := u2 and error(rain) := (u3, u4).To predict the effect of switching the sprinkler on we simply replace the equation for sprinkler by sprinkler := T rue.
Finally, let E, X ⊆ V be two subset of our set of variables V. Now suppose we observe the evidence that E = e and ask ourselves what would have been happened if we had set X := x.Note that in general X = x and E = e contradict each other.In this case, we talk about a counterfactual query.

Example 2
Reconsider the query π(slippery|slippery, sprinkler, do(¬sprinkler)) in the introduction, i.e. in the causal model (1) we observe the sprinkler to be on and the road to be slippery while asking for the probability of the road to be slippery if the sprinkler were off.This is a counterfactual query as our evidence {sprinkler, slippery} contradicts our intervention do(¬sprinkler).
To answer this query based on a causal model M on V we proceed in three steps: In the abduction step we adjust the distribution of our error terms by replacing the distribution π M (error(V )) with the conditional distribution π M (error(V )|E = e) for all variables V ∈ V. Next, in the action step we intervene in the resulting model according to X := x.Finally, we are able to compute the desired probabilities π M ( |E = e, do(X := x)) from the modified model in the prediction step (Pearl, 2000, §1.4.4).For an illustration of the treatment of counterfactuals we refer to the introduction.
To avoid storing the joint distribution π M (error(V )|E = e) for V ∈ V Balke and Pearl (1994) developed the twin network method.They first copy the set of variables V to a set V * .Further, they build a new causal model M K on the variables V ∪ V * by setting for every V ∈ V ∪ V * , where pa(X) * := {X * |X ∈ pa(X)}.Further, they intervene according to X * := x to obtain the model M K,do(X * :=x) .Finally, one expects that In Example 8 we demonstrate the twin network method for the ProbLog program P and the counterfactual query of the introduction.

The ProbLog Language
We proceed by recalling the ProbLog language from De Raedt et al. (2007).As the semantics of non-ground ProbLog programs is usually defined by grounding, we will restrict ourselves to the propositional case, i.e. we construct our programs from a propositional alphabet P: Definition 2 (propositional alphabet) A propositional alphabet P is a finite set of propositions together with a subset E(P) ⊆ P of external propositions.Further, we call I(P) = P \ E(P) the set of internal propositions.
From propositional alphabets we build literals, clauses, and random facts, where random facts are used to specify the probabilities in our model.To proceed, let us fix a propositional alphabet P.
Definition 3 (Literal, Clause and Random Fact) A literal l is an expression p or ¬p for a proposition p ∈ P. We call l a positive literal if it is of the form p and a negative literal if it is of the form ¬p. A clause LC is an expression of the form h ← b 1 , ..., b n , where head(LC) := h ∈ I(P) is an internal proposition and where body(LC) := {b 1 , ..., b n } is a finite set of literals.A random fact RF is an expression of the form π(RF ) :: u(RF ), where u(RF ) ∈ E(P) is an external proposition and where π(RF ) ∈ [0, 1] is the probability of u(RF ).

Example 4
In Example 3 we have that szn spr sum is a positive literal, whereas ¬szn spr sum is a negative literal.Further, rain ← ¬szn spr sum, u4 is a clause and 0.6 :: u4 is a random fact.
Next, we give the definition of logic programs and ProbLog programs: Definition 4 (Logic Program and ProbLog Program) A logic program is a finite set of clauses.Further, a ProbLog program P is given by a logic program LP(P) and a set Facts(P), which consists of a unique random fact for every external proposition.We call LP(P) the underlying logic program of P.
To reflect the closed world assumption we omit random facts of the form 0 :: u in the set Facts(P).

Example 5
The program P from the introduction is a ProbLog program.We obtain the corresponding underlying logic program LP(P) by erasing all random facts of the form :: ui from P.
For a set of propositions Whether a formula ϕ is satisfied by a Q-structure M, written M |= ϕ, is defined as usual in propositional logic.As the semantics of a logic program P with stratified negation we take the assignment E → M(E, P) that relates each E-structure E with the minimal model M(E, P) of the program P ∪ E.

Counterfactual Reasoning: Intervening and Observing Simultaneously
We return to the objective of this paper, establishing Pearl's treatment of counterfactual queries in ProbLog.As a first step, we introduce a new semantics for ProbLog programs in terms of causal models.
Definition 5 (FCM-semantics) For a ProbLog program P the functional causal models semantics or FCM-semantics is the system of equations that is given by , where u(RF ) FCM are mutually independent Boolean random variables for every random fact RF ∈ Facts(P) that are distributed according to π u(RF Here, an empty disjunction evaluates to F alse and an empty conjunction evaluates to T rue.Further, we say that P has unique supported models if FCM(P) is a causal model, i.e. if it posses a unique solution for every E-structure E and every possible intervention X := x.In this case, the superscript FCM indicates that the expressions are interpreted according to the FCM-semantics as random variables rather than predicate symbols.It will be omitted if the context is clear.For a Problog program P with unique supported models the causal model FCM(P) determines a unique joint distribution π FCM P on P. Finally, for a P-formula ϕ we define the probability to be true by

Example 6
As intended in the introduction, the causal model ( 1) yields the FCM-semantics of the program P. Now let us calculate the probability π FCM P (sprinkler) that the sprinkler is on.
As desired, we obtain that the FCM-semantics consistently generalizes the distribution semantics of Poole (1993) and Sato (1995).
Theorem 1 (Rückschloß and Weitkämper ( 2022)) Let P be a ProbLog program with unique supported models.The FCM-semantics defines a joint distribution π FCM P on P, which coincides with the distribution semantics π dist P .□ As intended, our new semantics transfers the query types of functional causal models to the framework of ProbLog.Let P be a ProbLog program with unique supported models.First, we discuss the treatment of external interventions.
Let ϕ be a P-formula and let X ⊆ I(P) be a subset of internal propositions together with a truth value assignment x.Assume we would like to calculate the probability π F CM P (ϕ| do(X := x)) of ϕ being true after setting the random variables in X FCM to the truth values specified by x.In this case, the Definition 1 and Definition 5 yield the following algorithm: Procedure 1 (Treatment of External Interventions) We build a modified program P do(X:=x) by erasing for every proposition h ∈ X each clause LC ∈ LP(P) with head(LC) = h and adding the fact h ← to LP(P) if h x = T rue.
Finally, we query the program P do(X:=x) for the probability of ϕ to obtain the desired probability π FCM P (ϕ| do(X := x)).
From the construction of the program P do(X:=x) in Procedure 1 we derive the following classification of programs with unique supported models.
Proposition 2 (Characterization of Programs with Unique Supported Models) A ProbLog program P has unique supported models if and only if for every E-structure E and for every truth value assignment x on a subset of internal propositions X ⊆ I(P) there exists a unique model M E, LP P do(X:=x) of the logic program LP P do(X:=x) ∪ E. In particular, the program P has unique supported model if its underlying logic program LP(P) is acyclic.□

Example 7
As the underlying logic program of the ProbLog program P in the introduction is acyclic we obtain from Proposition 2 that it is a ProbLog program with unique supported models i.e. its FCM-semantics is well-defined.However, we do not only want to either observe or intervene.We also want to observe and intervene simultaneously.
Let E ⊆ I(P) be another subset of internal propositions together with a truth value assignment e.Now suppose we observe the evidence E FCM = e and we ask ourselves what is the probability π FCM P (ϕ|E = e, do(X := x)) of the formula ϕ to hold if we had set X FCM := x.Note that again we explicitly allow e and x to contradict each other.The twin network method of Balke and Pearl (1994) yields the following procedure to answer those queries in ProbLog: Procedure 2 (Treatment of Counterfactuals) First, we define two propositional alphabets P e to handle the evidence and P i to handle the interventions.In particular, we set E(P e ) = E(P i ) = E(P) and I(P e/i ) := p e/i : p ∈ I(P) with I(P e ) ∩ I(P i ) = ∅.In this way, we obtain maps e/i : P → P e/i , p → p e/i , p ∈ I(P) p, else that easily generalize to literals, clauses, programs etc. Further, we define the counterfactual semantics of P by P K := P e ∪ P i .Next, we intervene in P K according to do(X i := x) and obtain the program P K,do(X i :=x) of Procedure 1. Finally, we obtain the desired probability π F CM P (ϕ|E = e, do(X := x)) by querying the program P K,do(X i :=x) for the conditional probability π(ϕ i |E e = e).
Note that we use the string __ to refer to the superscript e/i.
In the Appendix, we prove the following result, stating that a ProbLog program P yields the same answers to counterfactual queries, denoted π FCM P ( | ), as the causal model FCM(P), denoted π FCM(P) ( | ).
Theorem 3 (Correctness of our Treatment of Counterfactuals) Our treatment of counterfactual queries in Procedure 2 is correct i.e. in the situation of Procedure 2 we obtain that π F CM P (ϕ|E = e, do(X := x)) = π FCM(P) (ϕ|E = e, do(X := x)).
4 Relation to CP-logic Vennekens et al. (2009) establishes CP-logic as a causal semantics for the LPAD-programs of Vennekens et al. (2004).Further, recall Riguzzi (2020), §2.4 to see that each LPAD-program P can be translated to a ProbLog program Prob(P) such that the distribution semantics is preserved.Analogously, we can read each ProbLog program P as an LPAD-Program LPAD(P) with the same distribution semantics as P.
As CP-logic yields a causal semantics it allows us to answer queries about the effect of external interventions.More generally, Vennekens et al. (2010) even introduce a counterfactual reasoning on the basis of CP-logic.However, to our knowledge this treatment of counterfactuals is neither implemented nor shown to be consistent with the formal theory of causality in Pearl (2000).
Further, it is a priori unclear whether the expressive equivalence of LPAD and ProbLog programs persists for counterfactual queries.In the Appendix, we compare the treatment of counterfactuals under CP-logic and under the FCM-semantics.This yields the following results.
Theorem 4 (Consistency with CP-Logic -Part 1) Let P be a propositional LPAD-program such that every selection yields a logic program with unique supported models.Further, let X and E be subsets of propositions with truth value assignments, given by the vectors x and e respectively.Finally, we fix a formula ϕ and denote by π CP/F CM Prob(P)/P (ϕ|E = e, do(X := x)) the probability that ϕ is true, given that we observe E = e while we had set X := x under CP-logic and the FCM-semantics respectively.In this case, we obtain π CP P (ϕ|E = e, do(X := x)) = π F CM Prob(P) (ϕ|E = e, do(X := x)).
Theorem 5 (Consistency with CP-Logic -Part 2) If we reconsider the situation of Theorem 4 and assume that P is a ProbLog program with unique supported models, we obtain π CP LPAD(P) (ϕ|E = e, do(X := x)) = π F CM P (ϕ|E = e, do(X := x)).

Remark 1
We can also apply Procedure 2 to programs with stratified negation.In this case the proofs of Theorems 4 and 5 do not need to be modified in order to yield the same statement.However, recalling Definition 1, we see that there is no theory of counterfactual reasoning for those programs.Hence, to us it is not clear how to interpret the results of Procedure 2 for programs that do not possess unique supported models.
In Theorem 4 and 5, we show that under the translations Prob( ) and LPAD( ) CP-logic for LPAD-programs is equivalent to our FCM-semantics, which itself by Theorem 3 is consistent with the formal theory of Pearl's causality.In this way, we fill the gap of showing that the causal reasoning provided for CP-logic is actually correct.Further, Theorem 4 and 5 show that the translations Prob( ) and LPAD( ) of Riguzzi (2020), §2.4 do not only respect the distribution semantics but are also equivalent for more general causal queries.

Practical Evaluation
We have seen that we can solve counterfactual queries by performing marginal inference over a rewritten probabilistic logic program with evidence.Most of the existing solvers for marginal inference, including ProbLog (Fierens et al., 2015), aspmc (Eiter et al., 2021), and PITA (Riguzzi and Swift, 2011), can handle probabilistic queries with evidence in one way or another.Therefore, our theoretical results also immediately enable the use of these tools for efficient evaluation in practice.

Knowledge Compilation for Evaluation
The currently most successful strategies for marginal inference make use of Knowledge Compilation (KC).They compile the logical theory underlying a probabilistic logic program into a so called tractable circuit representation, such as binary decision diagrams (BDD), sentential decision diagrams (SDD) (Darwiche, 2011) or smooth deterministic decomposable negation normal forms (sd-DNNF).While the resulting circuits may be much larger (up to exponentially in the worst case) than the original program, they come with the benefit that marginal inference for the original program is possible in polynomial time in their size (Darwiche and Marquis, 2002).
When using KC, we can perform compilation either bottom-up or top-down.In bottom-up KC, we compile SDDs representing the truth of internal atoms in terms of only the truth of the external atoms.After combining the SDDs for the queries with the SDDs for the evidence, we can perform marginal inference on the results (Fierens et al., 2015).
For top-down KC we introduce auxiliary variables for internal atoms, translate the program into a CNF and compile an sd-DNNF for the whole theory.Again, we can perform marginal inference on the result (Eiter et al., 2021).
Implementation As the basis of our implementation, we make use of the solver library aspmc.It supports parsing, conversion to CNF and top-down KC including a KC-version of SHARPSAT1 based on the work of Korhonen and Järvisalo (2021).Additionally, we added (i) the program transformation that introduces the duplicate atoms for the evidence part and the query part, and (ii) allowed for counterfactual queries based on it.
Furthermore, to obtain empirical results for bottom-up KC, we use PySDD2 , which is a python wrapper around the SDD library of Choi and Darwiche (2013).This is also the library that ProbLog uses for bottom-up KC to SDDs.

Empirical Evaluation
Here, we consider the scaling of evaluating counterfactual queries by using our translation to marginal inference.This can depend on (i) the number of atoms and rules in the program, (ii) the complexity of the program structure, and (iii) the number and type of interventions and evidence.
We investigate the influence of these parameters on both the bottom-up and top-down KC.Although top-down KC as in aspmc can be faster (Eiter et al., 2021) on usual marginal queries, results for bottom-up KC are relevant nevertheless since it is heavily used in ProbLog and PITA.
Furthermore, it is a priori not clear that the performance of these approaches on usual instances of marginal inference translates to the marginal queries obtained by our translation.Namely, they exhibit a lot of symmetries as we essentially duplicate the program as a first step of the translation.Thus, the scaling of both approaches and a comparison thereof is of interest.

Questions and Hypotheses
The first question we consider addresses the scalability of the bottom-up and top-down approaches in terms of the size of the program and the complexity of the program structure.
Q1. Size and Structure: What size and complexity of counterfactual query instances can be solved with bottom-up or top-down compilation?
Here, we expect similar scaling as for marginal inference, since evaluating one query is equivalent to performing marginal inference once.While we duplicate the atoms that occur in the instance, thus increasing the hardness, we can also make use of the evidence, which can decrease the hardness, since we can discard models that do not satisfy the evidence.
Since top-down compilation outperformed bottom-up compilation on marginal inference instances in related work (Eiter et al., 2021), we expect that the top-down approach scales better than the bottom-up approach.
Second, we are interested in the influence that the number of intervention and evidence atoms has, in addition to whether it is a positive or negative intervention/evidence atom.

Q2. Evidence and Interventions: How does the number and type of evidence and intervention atoms influence the performance?
We expect that evidence and interventions can lead to simplifications for the program.However, it is not clear whether this is the case in general, whether it only depends on the number of evidence/intervention atoms, and whether there is a difference between negative and positive evidence/intervention atoms.

Setup
We describe how we aim to answer the questions posed in the previous subsection.
Here, d(X) refers to the number of outgoing arcs of X in G, and s_1(X), ..., s_d(X) refer to its direct descendants.We obtain the final program by replacing the variables X, Y with constants corresponding to the vertices of G.
This program models that we reach (denoted by r(.)) the starting vertex s and, at each vertex v that we reach, decide uniformly at random which outgoing arc we include in our path (denoted by p(.,.)).If we include the arc (v, w), then we reach the vertex w.However, we only include an outgoing arc, if we do not get trapped (denoted by trap(.)) at v.This allows us to pose counterfactual queries regarding the probability of reaching the goal vertex g by computing for some positive or negative evidence of reaching v 1 , . . ., v n and some positive or negative interventions on reaching v ′ 1 , . . ., v ′ m .In order to obtain instances of varying sizes and difficulties, we generated acyclic digraphs with a controlled size and treewidth.Broadly speaking, treewidth has been identified as an important parameter related to the hardness of marginal inference (Eiter et al., 2021;Korhonen and Järvisalo, 2021) since it bounds the structural hardness of programs, by giving a limit on the dependencies between atoms.
Using two parameters n, k ∈ N, we generated programs of size linear in n and k and treewidth min(k, n) as follows.We first generated a random tree of size n using networkx.As a tree it has treewidth 1.To obtain treewidth min(k, n), we added k vertices with incoming arcs from each of the n original vertices in the tree. 3Finally, we added one vertex as the goal vertex, with incoming arcs from each of the k vertices.As the start we use the root of the tree.
Benchmark Platform All our solvers ran on a cluster consisting of 12 nodes.Each node of the cluster is equipped with two Intel Xeon E5-2650 CPUs, where each of these 12 physical cores runs at 2.2 GHz clock speed and has access to 256 GB shared RAM.Results are gathered on Ubuntu 16.04.1 LTS powered on Kernel 4.4.0-139 with hyperthreading disabled using version 3.7.6 of Python3.

Compared Configurations
We compare the two different configurations of our solver WHATIF (version 1.0.0,published at github.com/raki123/counterfactuals).Namely, bottom-up compilation with PySDD and top-down compilation with SHARPSAT.Only the compilation and the following evaluation step differ between the two configurations, the rest stays unchanged.
Comparisons For both questions, we ran both configurations of our solver using a memory limit of 8GB and a time limit of 1800 seconds.If either limit was reached, we assigned the instance a time of 1800 seconds.

Q1. Size and Structure
For the comparison of scalability with respect to size and structure, we generated one instance for each combination of n = 20, 30, . . ., 230 and k = 1, 2, . . ., 25.We then randomly chose an evidence literal from the internal literals (¬) r(v).If possible, we further chose another such evidence literal consistent with the previous evidence.For the interventions we chose two internal literals (¬) r(v) uniformly at random.

Q2. Evidence and Interventions
For Q2, we chose a medium size (n = 100) and medium structural hardness (k = 15) and generated different combinations of evidence and interventions randomly on the same instance.Here, for each e, i ∈ {−5, . . ., 0, . . ., 5} we consistently chose |e| evidence atoms that were positive, if e > 0, and negative, otherwise.Analogously we chose |i| positive/negative intervention atoms.

Results and Discussion
We discuss the results (also available at github.com/raki123/counterfactuals/tree/finalresults) of the two experimental evaluations.

Q1. Size & Structure
The scalability results for size and structure are shown in Figure 1.
In Figure 1b, we see the overall comparison of bottom-up and top-down compilation.Here, we see that top-down compilation using SHARPSAT solves significantly more instances than bottom-up compilation with PYSDD.This aligns with similar results for usual marginal inference (Eiter et al., 2021).Thus, it seems like top-down compilation scales better overall.
In Figure 1a, we see that the average runtime depends on both the size and the width for either KC approach.This is especially visible in the subplots on top (resp.right) of the main plot containing the average runtime depending on the size (resp.width).While there is still a lot  of variation in the main plots between patches of similar widths and sizes, the increase in the average runtime with respect to both width and size is rather smooth.
As expected, given the number of instances solved overall, top-down KC scales better to larger instances than bottom-up KC with respect to both size and structure.Interestingly however, for bottom-up KC the width seems to be of higher importance than for top-down KC.This can be observed especially in the average plots on top and to the right of the main plot again, where the change with respect to width is much more rapid for bottom-up KC than for top-down KC.For bottom-up KC the average runtime goes from ∼500s to ∼1800s within the range of widths between 1 and 16, whereas for top-down KC it stays below ∼1500s until width 28.For the change with respect to size on the other hand, both bottom-up and top-down KC change rather slowly, although the runtime for bottom-up KC is generally higher.

Q2. Number & Type of Evidence/Intervention
The results for the effect of the number and types of evidence and intervention atoms are shown in Figure 2.
Here, for both bottom-up and top-down KC, we see that most instances are either solvable rather easily (i.e., within 500 seconds) or not solvable within the time limit of 1800 seconds.Furthermore, in both cases negative interventions, i.e., interventions that make an atom false, have a tendency to decrease the runtime, whereas positive interventions, i.e., interventions that make an atom true, can even increase the runtime compared to a complete lack of interventions.
However, in contrast to the results for Q1, we observe significantly different behavior for bottom-up and top-down KC.While positive evidence can vastly decrease the runtime for topdown compilation such that queries can be evaluated within 200 seconds, even in the presence of positive interventions, there is no observable difference between negative and positive evidence for bottom-up KC.Additionally, top-down KC seems to have a much easier time exploiting evidence and interventions to decrease the runtime.
We suspect that the differences stem from the fact that top-down KC can make use of the restricted search space caused by evidence and negative interventions much better than bottom-up compilation.Especially for evidence this makes sense: additional evidence atoms in bottom-up  compilation lead to more SDDs that need to be compiled; however, they are only effectively used to restrict the search space when they are conjoined with the SDD for the query in the last step.
On the other hand, top-down KC can simplify the given propositional theory before compilation, which can lead to a much smaller theory to start with and thus a much lower runtime.
The question why only negative interventions seem to lead to a decreased runtime for either strategy and why the effect of positive evidence is much stronger than that of negative evidence for top-down KC is harder to explain.
On the specific benchmark instances that we consider, negative interventions only remove rules, since all rule bodies mention r(x) positively.On the other hand, positive interventions only remove the rules that entail them, but make the rules that depend on them easier to apply.
As for the stronger effect of positive evidence, it may be that there are fewer situations in which we derive an atom than there are situations in which we do not derive it.This would in turn mean that the restriction that an atom was true is stronger and can lead to more simplification.This seems reasonable on our benchmark instances, since there are many more paths through the generated networks that avoid a given vertex, than there are paths that use it.
Overall, this suggests that evidence is beneficial for the performance of top-down KC.Presumably, the performance benefit is less tied to the number and type of evidence atoms itself and more tied to the strength of the restriction caused by the evidence.For bottom-up KC, evidence seems to have more of a negative effect, if any.
While in our investigation interventions caused a positive or negative effect depending on whether they were negative or positive respectively, it is likely that in general their effect depends less on whether they are positive or negative.Instead, we assume that interventions that decrease the number of rules that can be applied are beneficial for performance, whereas those that make additional rules applicable (by removing an atom from the body) can degrade the performance.

Conclusion
The main result in this contribution is the treatment of counterfactual queries for ProbLog programs with unique supported models given by Procedure 2 together with the proof of its cor-rectness in Theorem 3. We also provide an implementation of Procedure 2 that allows us to investigate the scalability of counterfactual reasoning in Section 6.This investigation reveals that typical approaches for marginal inference can scale to programs of moderate sizes, especially if they are not too complicated structurally.Additionally, we see that evidence typically makes inference easier but only for top-down KC, whereas interventions can make inference easier for both approaches but interestingly also lead to harder problems.Finally, Theorem 4 and 5 show that our approach to counterfactual reasoning is consistent with CP-logic for LPAD-programs.Note that this consistency result is valid for arbitrary programs with stratified negation.However, there is no theory for counterfactual reasoning in these programs.In our opinion, interpreting the results of Procedure 2 for more general programs yields an interesting direction for future work.
Finally, we associate to each P-formula ϕ the probability We call π dist P the distribution semantics of the LPAD-program P.
Note that each LPAD-program P can be translated into a ProbLog program Prob(P) that yields the same distribution semantics.
Definition 7 (Riguzzi (2020), §2.4)Let P be a LPAD-program in P and choose for every LPAD-clause RC ∈ P and for every natural number 1 ≤ i ≤ l(RC) distinct propositions h RC i , u i (RC) ̸ ∈ P. The ProbLog transformation Prob(P) of the LPAD-program P is the ProbLog program that is given by the logic program LP(Prob(P)), which constists of the clauses for every LPAD-clause RC ∈ P and for every 1 ≤ i ≤ l(RC) as well as the random facts Indeed, we obtain the following result.
Theorem 6 (Riguzzi (2020), §2.4)Let P be a LPAD-program.In this case, we obtain for every selection σ of P a set of possible worlds E(σ), which consists of all possible worlds E such that ¬u i (RC) holds unless σ(RC) ̸ =⊥ or i > σ(RC) and such that u σ(RC) (RC) holds for every RC ∈ P with σ(RC) ̸ =⊥.We obtain that P σ yields the same answer to every P-formula as the logic programs LP(Prob(P)) ∪ E for every E ∈ E(σ) and that π(E(σ)) = π(σ).Further, the distribution semantics π dist P of P and the distribution semantics π dist Prob(P) of Prob(P) yield the same joint distribution on P. □ Finally, each ProbLog program can be read as an LPAD-program as follows.
Definition 8 (Riguzzi (2020), §2.4)For a ProbLog program P the LPAD-transformation LPAD(P) is the LPAD-program that consists of one clause of the form u(RC) : π(RF ) ← for every random fact π(RF ) :: u(RF ) of P and a clause of the form head(LC) : 1 ← body(LC) for every logic clause LC ∈ LP(P).In this case, every selection σ of LPAD(P) of probability not zero corresponds to a unique possible world E(σ), in which u(RC) is true if and only if σ(RC) ̸ =⊥.
Again, we obtain that the LPAD-transformation respects the distribution semantics.

Lemma 8
Choose a proposition X ∈ P together with a truth value x. i) In the situation of Theorem 6, for every possible world E ∈ E(σ) the logic programs P σ,do(X:=x) and LP(Prob(P) do(X:=x) ) ∪ E yield the same answer to every P-formula.ii) In the situation of Theorem 7, for every selection σ of LPAD(P), the logic programs LPAD(P) σ,do(X:=x) and LP P do(X:=x) ∪ E(σ) yield the same answer to every Pformula.

Proof
We only give a proof of i) since ii) is proven analogously.Form Theorem 6 we obtain that the programs P σ and LP(Prob(P)) ∪ E yield the same answer to every P-formula ϕ.As logic programs are modular this behaviour doesn't change if in both programs we erase all clause with X in the head.Finally, we also do not disturb the desired behaviour if we eventually add the fact X ← to both programs.
The intention of CP-logic is now to introduce a causal semantics for LPAD-programs.The target object of this semantics is given by P-processes, which are themselves a generalization of Shafer's probability trees.
Definition 9 (Probabilistic P-process) A P-process T is given by a tuple (T, I), where: i) T is a directed tree, in which each edge is labeled with a probability such that for all non-leaf nodes n the probabilities of the edges leaving n sum up to one.ii) I is a map that assigns to each node n of T an Herbrand interpretation I(n) in P.
Next, we associate to each node n of T the probability π T (n), which is given by the product of the probabilities of all edges that we pass on the unique path from the root ⊥ of T to n.This yields a distribution π T on the Herbrand interpretations I of P by setting Further, we connect LPAD-programs to P-processes.To this aim we fix a LPAD-program P and proceed to the following definition.
Definition 10 (Hypothetical Derivation Sequences, Firing, Execution Model) A hypothetical derivation sequence of a node n in a P-process T := (T, I) is a sequence of three-valued interpretations (ν i ) 0≤i≤n that satisfy the following properties: i) ν 0 assigns F alse to all atoms not in I(n) ii) For each i > 0 there exists a clause RC ∈ P and a 1 ≤ j ≤ l(RC) with body(RC) νi ̸ = F alse, with h νi+1 j = U ndef ined and with ν i (p) = ν i+1 (p) for all other proposition p ∈ P Such a sequence is called terminal if it cannot be extended.As it turn out each terminal hypothetical derivation sequence in n has the same limit ν n , which we call the potential in n.
Let RC ∈ P be a LPAD-clause.We say that RC fires in a node n of T if for each 1 ≤ i ≤ l(RC) there exists a child n i of n such that I(n i ) = I(n) ∪ {h i (RC)} and such that each edge (n, n i ) is labeled with π i (RC).Moreover, there exists a child n l(RC)+1 of n with I(n l(RC)+1 ) = I(n).
Further, we say that T is an execution model of P, written T |= P if there exists a mapping E from the non-leaf nodes of T to P such that: i) I(⊥) = ∅ for the root ⊥ of T ii) In each non-leaf node n a LPAD-clause E(n) ∈ R E (n) fires with I(n) |= body(E(n)).iii) For each leaf l of T there exists no LPAD-clauses RC ∈ R E (l) with I(l) |= body(RC).iv) For every node n of T we find body(E(n)) νn ̸ = U ndef ined, where ν n is the potential in n.
Here, R E (n) denotes the set of all rules RC ∈ P, for which there exists no ancestor a of n with E(a) = RC.
It turns out that every execution model T |= P gives rise to the same probability distribution π CP P := π T , which coincides with the distribution semantics π dist P .In particular, we obtain the following result.
Lemma 9 (Vennekens et al. (2009), §A.2) Let l be a leaf node in an execution model T of the LPAD-program P. In this case, there exists a unique path ρ from the root ⊥ of T to l. Define the selection σ(l) by setting σ(l)(RC) := i ∈ N if and only if there exists a node n j along ρ with E(n j ) = RC and I(n j+1 ) := I(n j )∪{h i (RC)}.Otherwise, we set σ(l)(RC) :=⊥.In this way, we obtain that P σ(l) |= I(l).On the other hand, we find for each selection σ of P a leaf l of T with σ(l) = σ.□ Finally, we recall the treatment of counterfactuals in CP-logic from Vennekens et al. (2010).
Procedure 3 (Treatment of Counterfactuals in CP-logic) Let X, E ⊆ I(P) be subsets of internal propositions.Further, let x and e be truth value assignments for the propositions in X and E, respectively.Finally, we fix a P-formula ϕ.We calculate the probability π CP P (ϕ|E = e, do(X := x)) of ϕ being true if we observe E = e while we had set X := x in two steps: 1.) Choose an execution model T of P. 2.) For every leaf l of T we intervene in the logic program P σ(l) according to X := x to obtain the logic program P σ(l),do(X:=x) from Procedure 1. Further, we set These are exactly the possible worlds that make the query ϕ true after intervention while the observation E = e is true before intervening.Hence, we can consult the proof of Theorem 3 to see that (B1) computes the same value as Procedure 2.
Proof of Theorem 5 By Theorem 7, Lemma 8 and Lemma 9 the right-hand side of (B1) for LPAD(P) is the sum of the conditional probabilities π(E|E = e) of all possible worlds E of P such that M(E, LP(P do(X:=x) )) |= ϕ and M(E, LP(P) |= (E = e).
These are exactly the possible worlds that make the query ϕ true after intervention while the observation E = e is true before intervening.Hence, we can consult the proof of Theorem 3 to see that (B1) computes the same value as Procedure 2.
plots showing the average runtime using bottom-up (right) and top-down (left) compilation.The x-axis denotes the size n and the y-axis denoted the width k.For each square in the main plots the color of the square denotes the average runtime of all instances in the covered range.The extra plot on the top (resp.right side) denote the average for the size range (resp.width range) over all widths (resp.sizes).performance of the top-down (denoted as sharpsat) and bottom-up compilation (denoted as pysdd) on counterfactual queries regarding graph traversal.The x-axis denotes the number of solved instances and the y-axis denotes the runtime in seconds.

Fig. 2 :
Fig. 2: Two plots showing the runtime using bottom-up (right) and top-down (left) compilation with varying evidence and intervention.The x-axis denotes the signed number of interventions, i.e., −n corresponds to n negative interventions and n corresponds to n positive interventions.The y-axis denotes the signed number of evidence atoms using analogous logic.For each square in the main plots the color of the square denotes the runtime of the instance with those parameters.The extra plot on the top (resp.right side) denote the average for the number and type of evidences (resp.interventions) over all interventions (resp.evidences).