3.1 Answer set programming

We give a succinct account of all the concepts in ASP that are relevant to this paper, adapted from Law et al. (Reference Law, Russo and Broda2019). For a thorough description, please refer to Lifschitz (Reference Lifschitz2019).

An answer set program consists of a set of rules. We typically deal with normal rules, which are made up of atoms in the following manner:

\begin{equation*}\mathtt {h:\!-\, b_1,\ldots ,b_m,not\ b_{m+1},\ldots not\ b_n}.\end{equation*}

where the atom $\mathtt {h}$ represents the head and $\mathtt {b_1,\ldots ,b_m,not\ b_{m+1},not\ b_n}$ make up the body of the rule. The symbol $\mathtt {not}$ is called negation as failure and is true if its corresponding atom cannot be derived from the rules in the program.

A solution of an answer set program $P$ is a set of ground atoms $I$ , that is atoms that do not contain variables, and is defined in terms of the reduct of $P$ . $I$ is a subset of the Herbrand base $HB_P$ , which is the set of ground atoms that can be formed from the predicates and constants in $P$ . Given such an interpretation $I\subseteq HB_P$ , the reduct $P^I$ is constructed as follows: First, the grounding of $P$ is constructed. Second, all rules are removed from the grounding that contain an atom of the form $\mathtt {not\ b}$ , where $b\in I$ . Third, all remaining negation as failure atoms are removed. This results in a definite program, and $I$ is a model if it makes every rule in it true. If $I$ is the smallest possible model, it is minimal. $I$ is a solution of $P$ if it is a minimal model of the reduct $P^I$ and is called an answer set or stable model of $P$ . A program might have multiple answer sets.

The language provides many constructs to facilitate modeling, two of which are relevant for NeurASP. A choice rule has the form

\begin{equation*}\mathtt {1\{h_1,\ldots ,h_k\}1:\!-\, b_1,\ldots ,b_m,not\ b_{m+1},\ldots ,not\ b_n}.\end{equation*}

Whenever the body holds, exactly one of the head atoms has to hold. Since there is a choice between head atoms, each decision can create an additional answer set. A weak constraint provides an ordering of answer sets and has the form

\begin{equation*}\mathtt {:\sim b_1,\ldots ,b_m,not\ b_{m+1},\ldots ,not\ b_n}.[w@l,t_1,\ldots ,t_o]\end{equation*}

$\mathtt {w}$ and $\mathtt {l}$ specify the weight and optional priority level of each constraint, while $t_1,\ldots ,t_o$ are identifiers. Optimal answer sets minimize the sum of the weights of all weak constraints whose bodies are true, starting at the highest priority level.

3.2 NeurASP

We provide an overview of the structure of NeurASP and its learning mechanism. For a detailed explanation, please refer to the original paper from Yang et al. (Reference Yang, Ishay and Lee2020).

Fig. 1.

The inference procedure of NeurASP exemplified with an MNIST addition example.

Fig. 1. Long description

A diagram of the inference procedure of NeurASP with an MNIST addition example. The diagram shows the process of combining neural networks and answer set programming. Inputs are processed by a neural network, which predicts latent concepts. These concepts are then fed into a hard-coded answer set program that generates a downstream output. The diagram includes two input images, labeled as 1 and 3, which are processed by separate neural networks. The outputs of these networks are represented as sequences of black and white bars. These outputs are then fed into neural atoms, which contain rules for digits. The neural atoms are connected to an answer set program that combines the digits from the neural atoms to produce a result. The result is then processed by Clingo to generate the downstream output, labeled as result(4).

NeurASP combines neural networks with ASP in a cascading fashion: A neural network takes in data and predicts latent concepts, which in turn are integrated into an answer set program and processed by a solver to create a downstream label.

We illustrate the workings of NeurASP with an example task: In MNIST addition, each datapoint consists of two MNIST (Deng Reference Deng2012) images depicting handwritten digits and a downstream label representing their sum. Crucially, no latent labels of the digits themselves are available, only of their sum.

The inference procedure of NeurASP is shown in Figure 1. First, the framework runs each image through a neural network, yielding prediction vectors. Each entry in the vector represents a symbolic concept, known as a neural atom, and is encoded as a weak constraint. Entries with higher probabilities are given lower weights and are therefore preferred in the minimization task. We omit the identifiers for the sake of conciseness. The weak constraints are added to the hard-coded answer set program, which in this case calculates the sum of the two neural atoms. The answer set solver Clingo (Gebser et al. Reference Gebser, Kaminski, Kaufmann and Schaub2017) then computes the optimal answer set of the program and the result is extracted from it.

While inference is straightforward, training the neural network to predict the correct neural atoms is challenging, as no latent labels are provided. Instead, the learning signal has to be propagated from the downstream label through the non-differentiable answer set program to the neural network. NeurASP solves this challenge by utilizing semantic loss, which rewards the neural network for all latent predictions that lead to a correct downstream prediction (Xu et al. Reference Xu, Zhang, Friedman, Liang and den Broeck2017). The authors adapt semantic loss to ASP by computing all possible answer sets that yield a correct downstream prediction and training the neural network to predict concepts that are present in these answer sets. This process involves three major steps:

1. 1. Answer set computation: Given a downstream label, all possible answer sets are computed that predict that label. Each answer set contains a unique combination of neural atoms.

2. 2. Probability calculation: Each answer set is given a probability based on the neural network confidences of neural atoms within it.

3. 3. Gradient calculation: The gradient of the semantic loss with regard to the neural network outputs is computed and propagated through all parameters.

Before we explore each step in more detail, we introduce the necessary notation. Let $\Pi ^{asp}$ denote the hard-coded ASP program and $\Pi ^{nn}$ be the set of neural atoms. A neural atom $c$ can take on a value $v\in \{v_1,\ldots ,v_n\}$ . The probability $P(c=v)$ is given by the corresponding entry of the output vector of a neural network $N$ that has processed input $x$ . $N$ has parameters $\boldsymbol{\theta }$ that are learnable. The downstream label is an observation $O$ and a set of neural atom values form an interpretation $I$ .

3.2.1 Answer set computation

The program $\Pi$ is composed of $\Pi ^{asp}$ , the downstream label $O$ and the neural atoms $\Pi ^{nn}$ . $O$ is encoded as a constraint, ensuring that each answer set must prove $O$ . Each neural atom $c_i$ is encoded as a choice rule $\mathtt {1\{c(i,v_1);\ldots ;c(i,v_n)\}1}$ , which instructs Clingo to choose exactly one value for the atom per answer set.

Fig. 2.

The learning procedure of NeurASP exemplified with an MNIST addition example.

Figure 2 illustrates the answer set computation for an example in MNIST addition. In this case, Clingo computes five valid answer sets with neural atoms that sum to 4: $\{(0,4),(1,3),(2,2),(3,1),(4,0)\}$ .

3.2.2 Probability calculation

The probability of a set of neural atom values $I$ is determined by its neural network confidences:

(1) \begin{equation} P(I)=\prod _{c=v\in I}P(c=v) \end{equation}

Answer sets that contain neural atom values with higher confidences thus attain higher probabilities. Figure 2 illustrates a probability calculation with the green arrows leading from the neural prediction to the answer set. Since the neural network confidences are highest for the values 1 and 3, this particular answer set will have the highest probability. The probability of an observation $O$ is the sum of the probabilities of all interpretations that model $O$ :

\begin{equation*}P(O)=\sum _{I\models O}P(I)\end{equation*}

3.2.3 Gradient calculation

The learning goal of NeurASP corresponds to finding neural network parameters that maximize the log likelihood of all observations in the dataset. The gradient of this goal with regard to neural parameters $\boldsymbol{\theta }$ is given by:

\begin{equation*}\sum _O\frac {\partial \log (P(O))}{\partial \mathbf{p}}\times \frac {\partial \mathbf{p}}{\partial \boldsymbol{\theta }}\end{equation*}

where $\mathbf{p}$ represents the output vector of the neural network for the example relevant to $O$ . The second part of the multiplication is processed automatically by backpropagation frameworks. The first part is calculated explicitly for each entry of the vector, where $p_v$ denotes the vth entry of $\mathbf{p}$ .

(2) \begin{equation} \frac {\partial \log (P(O))}{\partial p_v}=\frac {\sum \limits _{\substack {I:I\models O\\I\models c=v}}{\frac {P(I)}{P(c=v)}-\sum \limits _{\substack {I,v':I\models O\\I\models c=v',v\neq v'}}{\frac {P(I)}{P(c=v')}}}}{\sum \limits _{I:I\models O}{P(I)}} \end{equation}

Intuitively, each time an answer set includes the current prediction, the gradient is increased (left-hand side) and if it includes another value for this atom, it is decreased (right-hand side). Using gradient ascent, this calculation encourages predictions that agree with more answer sets. Figure 2 illustrates the two-step gradient calculation with the blue arrows leading from the answer sets to neural predictions and into the network.

4.1 Vectorization

NeurASP spends a considerable amount of time on calculating answer set probabilities (Equation 1) and gradients (Equation 2). These calculations are slow due to the use of nested loops. We provide an illustrative example for the probability calculation in Figure 3a. The original NeurASP code iterates through each interpretation $I$ in a list comprehension, calling prob_of_interpretation. That function consists of two for-loops. The outer loop iterates through the entries $c=v$ of the answer set, while the inner loop iterates through all neural atoms. The if-statement checks whether the current atom is in the interpretation $I$ , which is a list of all atoms in the answer set, including non-neural atoms. If the check is true, the neural net prediction is multiplied with the running probability.

Fig. 3.

NeurASP probability calculation function.

Our improved implementation replaces the loops with Pytorch tensor operations, as shown in Figure 3b. To begin with, all interpretations are stored as a tensor of numbers, rather than strings, and only include neural atoms. The function processes all interpretations at once, rather than looping through each $I$ individually. Similarly, all vectors $\textbf {p}$ are stacked into a single 2D tensor. The probability function then retrieves the probability value of each atom from this tensor using indexing and multiplies it with each answer set using the Pytorch $\texttt {prod}$ function.

Similarly, the gradient calculation for Equation (2) uses four nested loops in the original code. A list comprehension calls the gradient function for each input $x$ and its corresponding prediction vector $\mathbf{p}$ . The gradient function first iterates through each entry $p_v$ . For each entry, a loop iterates through all answer sets $I$ for that example. For each answer set, a loop iterates through all possible atoms until the atom $c=v$ is reached that is within the answer set. At this point, its probability is either added or subtracted to the running sum.

Our improved code again replaces all loops with Pytorch operations. In short, it weights all answer set entries at once by their network confidences. Then, it multiplies each entry using a tensor of 1/-1 entries depending on whether their value is present in corresponding the answer set. Last, the entries are summed together using Pytorch’s $\texttt {sum}$ function to arrive at the gradients.

4.2 Answer set caching

Answer sets are computed using the third-party library Clingo, leaving little room for efficiency improvements. Instead, we avoid as many Clingo calls as possible by making use of caching. The original NeurASP implementation already has a version of caching built in, but it is suboptimal. Given a dataset, NeurASP calculates the answer sets of each example once and then stores them. After the first epoch, the answer sets for the same example can be retrieved from storage. However, this still results in a Clingo call of every dataset entry, and most datasets include tens of thousands of examples.

We make use of the observation that the answer sets of an example only depend on its downstream label, not its inputs, as Figure 2 illustrates. Thus, we only have to calculate answer sets once per unique label, rather than once per example. This leads to significantly fewer Clingo calls and substantially reduces the size of the answer set cache. In the case of MNIST addition, the dataset contains 30,000 entries, but only 19 unique labels (all possible two-digit sums, i.e. the numbers 0 to 18). This allows us to skip 29,981 out of 30,000 Clingo calls! We also store the answer sets in a file, so that we can reuse the cache for subsequent experiments with the same dataset.

4.3 Other improvements

We have introduced further improvements to the NeurASP code that enhance the experience of running experiments on larger datasets.

• • Batch processing: The original code requires two separate lists for inputs and observations. This setup requires you to load all input data into the list first, which is very memory-intensive. Moreover, it processes each entry one at a time, which is slow. We have refactored the code to support batched data from data loaders, which can load data on the fly and run it through the neural network in batches.

• • Validation sets: We include the ability to use validation sets for hyperparameter tuning.

• • Seeding: To ensure reproducible results, you can provide seeds to fix the random number generators.

• • Logging: Instead of just printing accuracy results, we have added more extensive logging capabilities. The code calculates latent and downstream accuracies every specified step and stores them in JSON files. Model parameters with the best accuracy are saved in a file as well.

• • Unit tests: We have written unit tests for every function that we changed, ensuring that the new implementation is functionally correct.

Overall, these additions modernize the code and make it usable for extensive experimentation with large datasets.

6.1 MNIST addition and member

Both datasets use MNIST images as their inputs, which are 28x28 grayscale images of handwritten digits (Deng Reference Deng2012). In MNIST addition, the reasoning task consists of calculating the sum of the two input digits. In Member, a symbolic number is given with a list of MNIST images. The reasoning task consists of checking whether the symbolic number is contained in this list. The downstream label is 1 if the digit is contained in the list and 0 otherwise. Member can be configured by changing the number of images in the list, we investigate lengths of 3 and 5.

6.2 Card arithmetic

In Card arithmetic, the input consists of real photos of playing cards with a resolution of 174 × 274 pixels, which we have taken from Cunnington et al. (Reference Cunnington, Law, Lobo and Russo2023). There are 52 latent concepts, one for every combination of rank (2–10, Jack, Queen, King, Ace) and suit (Diamonds, Clubs, Spades, Hearts). Both the resolution and number of concepts represent a significant increase in challenge compared to the MNIST (28 × 28, 10 concepts) or CIFAR10 (32 × 32, 10 concepts) (Krizhevsky and Hinton Reference Krizhevsky and Hinton2009) datasets commonly used in the literature.

The reasoning component can be freely configured to create problems with different levels of complexity. For this paper, we have created two tasks with three levels of difficulty each. Both tasks assign each rank a value from 2–14 and calculate the downstream result by summing up card values:

The Card sum task assigns the values 0, 13, 26, and 39 to the suits and computes the card_value as the sum of its rank and suit value. The Card prodsum task assigns the values 1, 2, 3, and 4 to the suits and computes the card_value as the product of its rank and suit value.

6.3 Dataset complexities

Table 1 provides an overview of the complexity of each task in terms of different properties. Both Member and Card tasks reach tens of thousands of answer sets, but they differ in the number of labels and concepts. Member always has 20 distinct downstream labels, one for each combination of label (true or false) and symbolic input (0 to 9). The Card tasks include distinct labels for each possible result of the arithmetic operation, and their number increases with the number of inputs. While MNIST digits include ten concepts, there are 52 different playing cards.

Table 1.

Complexity of different tasks in terms of the average number of answer sets, the number of distinct downstream labels, the number of input images per example and the number of latent concepts per input

7.1 Benchmark speed tests

The actual running time of NeurASP depends on more factors than just probability and gradient calculations, such as data loading, processing and answer set computations. This section summarizes the execution times of the original and improved NeurASP, as well as SLASH, on real datasets. Each task was run for the number of epochs necessary to achieve at least 90% accuracy, emulating a real training run-through. For the hardest task, we had to extrapolate the times for the original NeurASP and SLASH, as they would have taken days or weeks.

Table 2 presents the results of our speed tests. Our implementation exhibits substantially lower execution times than both the original NeurASP and SLASH. The harder the tasks become, the larger the gap grows. The time breakdown illustrates which enhancements save the most time.

Table 2.

Average execution times (seconds). Bold values indicate the fastest time. ^*Times with an asterisk were extrapolated as follows: We ran the framework for three epochs and summed together the first epoch statistics with the average of the second two epochs multiplied by the number necessary to match the final number of epochs. $^{\dagger }$ For times with a dagger, both NeurASP and SLASH took multiple hours to process a few hundred datapoints. Therefore, no meaningful extrapolation of individual times was possible

Table 2. Long description

The table presents the results of speed tests for different algorithms across various tasks. It includes columns for Task, Algorithm, Total Time, and Time breakdown in seconds for Model, Prob, and Grad. The tasks listed are MNIST addition, Member 3, Member 5, Card sum 2, Card sum 3, and Card sum 4. The algorithms compared are NeurASP original, SLASH, and NeurASP improved. Each row provides the total time and time breakdown for each algorithm on each task. Notable trends include significantly lower execution times for the NeurASP improved algorithm compared to NeurASP original and SLASH, with the gap increasing for harder tasks. The table also includes specific notes on extrapolated times and instances where meaningful extrapolation was not possible.

The answer set caching results in the largest overall speed up. NeurASP calls Clingo 30,000 times for MNIST addition and 10,000 times for Card sum and Member before it makes use of its cache in epochs 2 and 3. SLASH does not cache answer sets at all, resulting in 90,000 and 30,000 calls respectively. It does employ the SAME method (Skryagin et al. Reference Skryagin, Ochs, Dhami and Kersting2024) to reduce the size of choice rules as training goes on. While this speeds up answer set computations, it is still slower than caching. In contrast, our improved code only calls Clingo once per unique label, that is, 19, 20, and 101-182 times, leading to vast improvements in model computation speed.

Probability and gradient functions also take up much less time thanks to vectorization. Only for the easiest task, MNIST addition, is the overhead of loading data into tensors slower than NeurASP. The larger the dataset is, the more substantial the time improvements get.

We have shown that our improvements do not just exist in isolation, but significantly speed up training on real datasets. For the largest tasks, runtimes reduce from months to hours, making them feasible to learn.

Table 3.

Test accuracies for various tasks. Values indicate downstream and latent accuracy averages and standard deviations over five runs. We compare our improved NeurASP implementation and Embed2Sym. T/O indicates a time-out after 24 hours

7.2 Performance tests

The improved implementation opens up the possibility of running NeurASP on tasks it previously timed out on. In particular, NeurASP was reported to time out on Member 5 (Aspis et al. Reference Aspis, Broda, Lobo and Russo2022) and the Card arithmetic tasks are even more challenging. We compare the performance of our improved NeurASP implementation with the neurosymbolic framework Embed2Sym. The experimental setup and instructions on how to run them can be found in the supplementary material.

The test accuracies are shown in Table 3. Notably, NeurASP now matches accuracies with Embed2Sym in MNIST addition and Member, even though it timed out in the original Embed2Sym paper. Moreover, with the Card arithmetic tasks, the opposite occurs: NeurASP is able to finish running, while Embed2Sym times out. The reason is that Embed2Sym clusters the latent space of the neural network and assigns symbolic labels with an ASP optimization task. The answer set program finds assignments of clusters to latent concepts by computing whether the assignments optimize the downstream labels. This is manageable for 10 clusters, as is the case in the MNIST-based tasks. However, there are 52 clusters with playing cards, so the optimization task has up to $52!$ solutions, timing out after 24 hours. Embed2Sym scales better in terms of input numbers, as it was able to solve Member 20, which NeurASP cannot solve due to millions of possible answer sets. However, we have shown that it scales worse than NeurASP in the number of latent concepts.

Analyzing the performance of NeurASP on Card arithmetic, it becomes apparent that the accuracy decreases as the number of cards increase. For Card sum 4, each label yields more than 40k answer sets on average, yet NeurASP still manages to achieve a latent accuracy of over 80%. Interestingly, the standard deviation of results becomes quite large as the input size increases, reaching levels of 9–10% for the 4-card tasks. This is much higher than we would expect for fully neural models and showcases the volatility of semantic loss. The network can minimize the semantic loss of a single example by matching any of its answer sets. This leads to a large number of suboptimal local minima, since the neural network can achieve significant loss decreases by fitting to answer sets that do not match the true latent labels.

7.2.1 Reasoning shortcuts

Networks trained on Card prodsum exhibit signs of reasoning shortcuts, where they learn to model the downstream task with incorrect latent labels (Marconato et al. Reference Marconato, Teso, Vergari and Passerini2023). As Table 3 shows, NeurASP achieves higher downstream accuracies than latent accuracies for all input sizes of Card prodsum. The network is able to predict the correct end result with incorrect latent labels. This is possible because different cards are assigned the same values: For example, the 3 of Clubs has the value $3\times 2=6$ , which is the same as the 2 of Spades with $2\times 3=6$ . The network either learns to assign these labels arbitrarily, or collapses two card labels with the same value into one. Such reasoning shortcuts do not occur with Card sum, as each suit/rank combination has a unique value.

Fig. 5.

Differences between downstream and latent validation accuracies for card prodsum 2 over 5 runs.

Figure 5 illustrates the reasoning shortcuts for Card prodsum 2. Even though the downstream accuracies all converge to the same value, the latent accuracies settle at vastly different values. For each run, the network learns different assignments to latent concepts and collapses a different number of them. Card prodsum is therefore a good dataset for investigating reasoning shortcuts and possible mitigations in the future.