THE IMPORTANCE OF AGROFORESTRY

We agree with Sileshi and Akinnifesi (Reference Sileshi and Akinnifesi2019) about the emerging importance of agroforestry for addressing food security in Africa, and how growing recognition of this has been built on evidence, much of which they have generated. It is because of this that we are concerned about how research results are presented to policy makers and farmers. If only mean yield increases are communicated, without information about variation around these means, then an overly simplistic impression that all farmers can expect to achieve the mean increase, or something close to this, is implied. In fact, individual farmers will experience very different results depending on a range of contextual factors that determine performance of any agroforestry practise in their specific circumstances. The aim of our paper was to show this with an example and to show how more nuanced understanding of effects of trees on crop yield can be generated by understanding how contextual factors affect performance, giving rise to more useful messages. We firmly believe that this is necessary for agroforestry to reach its potential, and it should not be seen as an argument against agroforestry.

ORIGINS AND NATURE OF THE DATA

The points that Sileshi and Akinnifesi raise about the origins and nature of the data are curious. Sileshi was the nodal co-ordinator for Southern Africa at ICRAF, and line manager of Joyce Njolma, when the data were collected but left ICRAF before the analyses and interpretation of the two year dataset were done and the paper was written. The data were explicitly collected to evaluate performance of agroforestry options promoted by the Agroforestry for Food Security Program (AFSP) that Sileshi refers to, but through a supplementary activity and funding for impact evaluation. Coe and Sinclair discussed the design with Sileshi, and Njoloma was responsible for the bulk of field work. Contrary to the assertion by Sileshi, the data collection was never envisaged as a baseline and could not have been since it was done some years after the start of the interventions. As is clearly stated in our paper, the comparison presented is of yield of maize in plots with and without trees, on farms where the agroforestry interventions had been adopted for a period of at least four years and does not involve a temporal comparison with a baseline. The nature and limitations of such a comparison are fully discussed in the paper. However, regardless of the original intent when collecting the data, what is important is whether the analysis and interpretation that we presented are valid and insightful, which we address in the detailed response to Sileshi and Akinnifesi's points below.

HOW MUCH CONTROL IS DESIRABLE?

Sileshi and Akinnifesi point out that what farmers did on their plots was beyond the control of researchers. This is precisely what makes it possible for us to detect the variation in response to using trees for soil fertility that happens in the real world, outside the controlled conditions of researcher's experiments whether they be on station or on farm. We do not agree with their assertion that this form of analysis is not correct. It is certainly different from what is possible with more controlled experimental data but not less valid. We clearly stated in our paper what analysis had been done and discussed its merits and limitations. What is consistent is the use of trees for enhancing soil fertility in maize plots, a recommendation made in the promotion of agroforestry in Malawi. The myriad ways in which that practice and its context can vary is the main difference between what researchers can see in conventional trials and what farmers experience. When a study is not a randomised controlled experiment, as in this case, it is perhaps misleading to discuss ‘treatments’ and ‘controls’. However, we do take the farmer's plot without trees as representing what that farmer experiences without trees, compared with what they experience on their plot with trees. This difference is a measure of the difference the trees make as experienced by that farmer. There are many factors that influence that, and the paper is about quantifying, interpreting and explaining that variation. Of course, the validity of this comparison as a measure of the effect of incorporating trees on maize yield depends on the extent to which the plots were comparable before the intervention – that is, the extent to which crop performance on the no-tree plot is a good estimate of the counterfactual, which was not and cannot be observed. This is a common problem in many observational studies.

This focus on what farmers experience is indeed distinct from the focus typical in researcher's reductionist approaches. The extreme is the classical randomised trial in which an attempt is made to control at a constant level every factor except the one being examined as a treatment. This is, of course, a very sound design for some research purposes but does not necessarily generate good predictions of responses under the variable conditions and management that farmers use. An alternative design, that would have given data with a more obvious direct interpretation as a ‘tree effect’, would have been to encourage each farmer who adopted trees on one plot to keep a comparable plot, managed in the same way, without trees. This could have generated information for both farmers and researchers on the impact of trees. The variation in contexts (such as in soils, on-farm niche and crop management) across farms would have revealed context × tree effect interactions. This design was suggested at the start of the second phase of the AFSP project, but the implementers chose not to use it. That design, and the options for varying levels of control, is the subject of a paper in this issue (Vanlauwe et al., Reference Vanlauwe, Coe and Giller2019), and it also represents a distinct change from traditional agronomic research practise.

However, the idea that the two plots should be managed in the same way is itself problematic, for two reasons. First, the presence of trees may well change how farmers behave, so that they manage crops differently in plots with trees than in those without. For example, planting and harvesting dates might need modifying as trees influence soil moisture regimes, weeding strategy may have to change as trees influence weed populations, and so on. Secondly, the trees and their impact on cropping will lead to different choices by farmers regarding allocation of resources. When considered part of a system, rather than in isolation, the ‘tree effect’ is indeed a response to many different factors beyond simple biophysical interactions with everything else held constant. This gross effect of trees on maize yield is what has been assessed in our paper, though other impacts of the trees have not been assessed.

Given all that, we do acknowledge that deciding ‘what to keep constant’ is difficult. In this study, the paired plots measured were said by farmers to be ‘comparable’ but exactly what that means was not defined or investigated. It would have been helpful to ask farmers their opinion on reasons for differences in maize performance on the pair of plots (Tiwari et al., Reference Tiwari, Brook and Sinclair2004). Their observations might have provided insights that could then be investigated with data collected specifically to test their general validity.

SYNERGISTIC OR ANTAGONISTIC EFFECTS OF NITROGEN FIXING TREES AND FERTILISER USE BY FARMERS?

Sileshi and Akinnifesi identify fertiliser use as an important factor that influences maize yield and varies amongst plots, both within and between farms. We did take that into account in the analyses to the extent allowed by the data. As explained in the paper, the only credible data available on fertiliser use was whether or not it had been used on the plot that season. Rates of use reported by farmers could not be relied on, as many were unrealistically high. This is a common problem when trying to assess resource use from farmer recall. The results are summarised in Table 3 of the paper which, as suggested to be desirable by Sileshi and Akinnifesi, disaggregates farms that have different fertiliser regimes on different plots. Our point regarding variation is reinforced if we take the results in Table 3 and look more deeply. Of the nine farmers with gliricidia that had no fertiliser on either the sole or tree plot, six differences were positive and three negative. The sample size is too small to infer much but the fact that there is important variation is clear. Of the 51 farms that used fertiliser on both plots, 35 had positive and 16 negative differences between yield on tree and sole plots.

The evidence for a negative interaction between gliricidia and fertiliser use on crop yield is also presented in Table 3. The average difference between the + and – tree plots is higher when no fertiliser is applied to either (1.31 t ha⁻¹) than when it is applied to both (0.64 t ha⁻¹). This is a negative interaction. The difference between the two differences, −0.67, is the same as the difference in mean effect of fertiliser with and without trees. We are aware of and refer to alternative conclusions from others about synergistic effects, and that is why we were keen to point out that in this context of farmers experience the evidence points to the opposite. The inference is necessarily based on means across farms as no farms had all four combinations of +/− trees and +/− fertiliser.

DIFFERENT DIMENSIONS OF RISK

Sileshi and Akinnifesi (Reference Sileshi and Akinnifesi2019) correctly point out that farmers face numerous risks, these being ecological, social or economic in origin, but are presumptuous in suggesting that their conception of risk is correct whereas that presented in our paper is not. In our paper, we were considering the specific risk of an individual farmer in Malawi following a simple message such as ‘intercropping with gliricidia will increase maize yield’. The data we present show that sometimes it will and sometimes it will not increase yields, variation that represents risk to farmers acting on the recommendation. That risk can be reduced if we understand more about what determines variable performance amongst farmers, and hence for which farmers this message is most likely to be appropriate. Note that we do not equate this risk with adoption potential, nor are we looking at the effect of trees on the inherent riskiness of farming that Sileshi and Akinnifesi (Reference Sileshi and Akinnifesi2019) highlight in their response. This is a different issue and we concur with them that trees often mitigate risks associated with farming through diversification. As we point out in the paper, and Sileshi and Akinnifesi echo in their response, there are many benefits (and costs) to adoption of agroforestry beyond any impact on maize yield, and it was not the intention of our paper to assess those, nor is there information on them in the data that were available to us. This does not detract from the importance of understanding impacts of trees on staple crop yield, as is clearly explained in the paper. This follows precedent, including influential meta-analyses presented elsewhere by Sileshi and Akinnifesi themselves (see for example, Sileshi et al., Reference Sileshi, Akinnifesi, Ajayi and Place2008). The analyses that we present are trying to identify the factors that predict the variation in crop yields experienced by different farmers adopting different agroforestry practises. As we show in our paper, not many factors could be identified in the available data, symptomatic of a general tendency for agronomic data to be collected without sufficient contextual information to elucidate the reasons for crop yields to increase in some circumstances and not others (Sileshi et al., Reference Sileshi, Akinnifesi, Ajayi and Place2008). A richer data set that could have been generated by hypothesising those factors and measuring them might have revealed more. The central point we are making is that this variation (that represents risk to farmers adopting an innovation based on mean yield increases) should be assessed and included as part of the message to extension workers and farmers. In our paper, we suggest how it can be reduced by acknowledging what uncertainty exists about how contextual factors condition performance and then systematically evaluating this as scaling out progresses, through the use of planned comparisons.

WHAT COMPARISONS ARE VALID?

Sileshi and Akinnifesi (Reference Sileshi and Akinnifesi2019) criticise our comparison of present results with previous published data on effects of trees on crop yield, on the basis that the way in which yield differences were calculated were different, but that, as explained above, is the whole basis of our comparison and of our discussion of the observed differences. Similarly, contradiction of earlier results on risk is not in itself a problem, if it can be explained. Kamanga et al. (Reference Kamanga, Waddington, Robertson and Giller2010) and Sirrine et al. (Reference Sirrine, Shennan, Snapp, Kanyama-Phiri, Kamanga and Sirrine2010) were considering different risks estimated in a different way than those we present, so it is not surprising that their results are different.

Finally, Sileshi and Akinnifesi (Reference Sileshi and Akinnifesi2019) state that we ‘. . . claim that agroforestry promotion was biased towards more fertile than average fields and more industrious than average farmers’. We do not. We stated that ‘This difference [in sole maize yields of farmers in the sample with national averages] is commensurate with a bias towards more fertile than average fields and more industrious than average farmers having taken part in the agroforestry promotion’. This is simply a possible explanation for why the yields in this study are higher than those often quoted for Malawi. It is not a comment on the process of promoting agroforestry. However, it would be unusual if these agroforestry technologies were neutral with regard to the type of farm and farmer interested in trying them. It is not usual for farmers participating in this type of project and trying new technologies to represent a cross-section of farmers within the communities concerned.