1. Introduction

The study of the CR Yamabe problem began with the celebrated works of Jerison and Lee [Reference Jerison and Lee26–Reference Jerison and Lee29]. The prototypical nonlinear partial differential equation in this problem is

\[ \mathscr L u = u^{\frac{Q+2}{Q-2}}, \]

where $\mathscr L$ indicates the negative sum of squares of the left-invariant vector fields generating the horizontal space in the Heisenberg group $\mathbb {H}^n$ with real dimension $2n+1$, whereas $Q = 2n+2$ denotes the so-called homogeneous dimension associated with the non-isotropic group dilations. (In this paper, we always use the group law dictated by the Baker–Campbell–Hausdorff formula. When the Lie group is $\mathbb {H}^n$, or more in general a group of Heisenberg type, this choice obviously affects the expression of the horizontal Laplacian.) In the present paper, we are interested in the following nonlocal version of the above equation:

(1.1)\begin{equation} \mathscr L_s u = u^{\frac{Q+2s}{Q-2s}}, \end{equation}

where the fractional parameter $s\in (0,\,1)$, and $\mathscr L_s$ denotes a certain pseudodifferential operator which arises in conformal CR geometry. As an application of our main result we derive sharp decay estimates of nonnegative solutions of (1.1).

The operator $\mathscr L_s$ in (1.1) was first introduced in [Reference Branson, Fontana and Morpurgo2] via the spectral formula:

(1.2)\begin{equation} \mathscr L_s = 2^s |T|^s \frac{\Gamma(-\frac 12\mathscr L |T|^{{-}1} + \frac{1+s}2)}{\Gamma(-\frac 12 \mathscr L |T|^{{-}1} + \frac{1-s}2)}, \end{equation}

where $\Gamma (x) = \int _0^\infty t^{x-1} {\rm e}^{-t} {\rm d}t$ denotes Euler gamma function. In (1.2) we have let $T = \partial _\sigma$, where for a point $g\in \mathbb {H}^n$ we have indicated with $g = (z,\,\sigma )$ its logarithmic coordinates. More in general, in a group of Heisenberg type $\mathbb {G}$, with logarithmic coordinates $g= (z,\,\sigma )\in \mathbb {G}$, where $z$ denotes the horizontal variable and $\sigma$ the vertical one, the pseudodifferential operator $\mathscr L_s$ is defined by the following generalization of (1.2):

(1.3)\begin{equation} \mathscr L_s = 2^s (-\Delta_\sigma)^{s/2} \frac{\Gamma(-\frac 12\mathscr L (-\Delta_\sigma)^{{-}1/2} + \frac{1+s}2)}{\Gamma(-\frac 12 \mathscr L (-\Delta_\sigma)^{{-}1/2} + \frac{1-s}2)}, \end{equation}

where $-\Delta _\sigma$ is the positive Laplacian in the centre of the group, see [Reference Roncal and Thangavelu37]. Formulas (1.2) and (1.3) should be seen as the counterpart of the well-known spectral representation $\widehat {(-\Delta )^s u} = (2\pi |\xi |)^{2s} \hat u$, where we have denoted by $\hat f$ the Fourier transform of a function $f$, see [Reference Stein38, Chap. 5]. An important fact, first proved for $\mathbb {H}^n$ in [Reference Frank, del Mar González, Monticelli and Tan12] using hyperbolic scattering, and subsequently generalized to any group of Heisenberg type in [Reference Roncal and Thangavelu37] using non-commutative harmonic analysis, is the following Dirichlet-to-Neumann characterization of $\mathscr L_s$:

\[ - \underset{y\to 0^+}{\lim} y^{1-2s} \partial_y U((z,\sigma),y) = 2^{1-2s} \frac{\Gamma(1-s)}{\Gamma(s)}\mathscr L_s u(z,\sigma), \]

where $U((z,\,\sigma ),\,y)$ is the solution to a certain extension problem from conformal CR geometry very different from that of Caffarelli–Silvestre in [Reference Caffarelli and Silvestre5]. Yet another fundamental fact, proved in [Reference Roncal and Thangavelu36, Proposition 4.1] and [Reference Roncal and Thangavelu37, Theorem 1.2] for $0< s<1/2$, is the following remarkable Riesz type representation:

(1.4)\begin{equation} \alpha(m,k,s)\ \mathscr L_s u(g)=\int_{\mathbb{G}} \frac{u(g)-u(h)}{|h|^{Q+2s}}{\rm d}h, \end{equation}

where with $g = (z,\,\sigma )$, we have denoted by $|g| = |(z,\,\sigma )| = (|z|^4 + 16 |\sigma |^2)^{1/4}$ the non-isotropic gauge in a group of Heisenberg type $\mathbb {G}$. Using the heat equation approach in [Reference Garofalo and Tralli17, Reference Garofalo and Tralli18], formula (1.4) can be extended to cover the whole range $0< s<1$. In (1.4) the number $\alpha (m,\,k,\,s)>0$ denotes an explicit constant depending on $s$ and the dimensions $m$ and $k$ of the horizontal and vertical layers of the Lie algebra of $\mathbb {G}$. While by (1.2), (1.3), and the classical formula $\Gamma (x+1) = x \Gamma (x)$, it is formally almost obvious that in the limit as $s\nearrow 1$ the operator $\mathscr L_s$ tends to the negative of the horizontal Laplacian $\mathscr L$, we emphasize that, contrarily to an unfortunate misconception, when $\mathbb {G} = \mathbb {H}^n$, or more in general it is of Heisenberg type, for no $s\in (0,\,1)$ does the standard fractional power

(1.5)\begin{equation} \mathscr L^s u(g) \overset{def}= (-\mathscr L)^s u(g) ={-} \frac{s}{\Gamma(1-s)} \int_0^\infty \frac{1}{t^{1+s}} \left(P_t u(g) - u(g)\right) {\rm d}t \end{equation}

coincide with the pseudodifferential operator defined by the left-hand side of (1.4) (in (1.5) we have denoted by $P_t = {\rm e}^{-t\mathscr L}$ the heat semigroup constructed in [Reference Folland10]). Unlike their classical predecessors $(-\Delta )^s$, in the purely non-Abelian setting of $\mathbb {H}^n$ the pseudodifferential operators $\mathscr L^s$ in (1.5) are not CR conformally invariant, nor they have any special geometric meaning, while the operators $\mathscr L_s$ are CR conformally invariant. For these reasons, we will refer to the operator $\mathscr L_s$ as the geometric (or conformal) fractional sub-Laplacian, even in the general setting of groups of Heisenberg type, see [Reference Frank, del Mar González, Monticelli and Tan12, Section 8.3] for relevant remarks in the remaining non-Abelian groups of Iwasawa type. Furthermore, it is not true that the fundamental solution $\mathscr E^{(s)}(z,\,\sigma )$ of $\mathscr L^s$ is a multiple of $|(z,\,\sigma )|^{2s-Q}$, see [Reference Garofalo and Tralli17, Theor. 5.1]. What is instead true, as proven originally by Cowling and Haagerup [Reference Cowling and Haagerup6], see also [Reference Roncal and Thangavelu36, (3.10)], and with a completely different approach based on heat equation techniques in [Reference Garofalo and Tralli17, Theor. 1.2] (the reader should also see in this respect the works [Reference Garofalo and Tralli18] and [Reference Garofalo and Tralli19]), is that the fundamental solution of the conformal fractional sub-Laplacian $\mathscr L_s$ in (1.3) is given by

(1.6)\begin{equation} \mathscr E_{(s)}(z,\sigma) = \frac{C_{(s)}(m,k)}{|(z,\sigma)|^{Q-2s}}, \end{equation}

where

\[ C_{(s)}(m,k) = \frac{2^{\frac m2 + 2k-3s-1} \Gamma(\frac 12(\frac m2+1-s)) \Gamma(\frac 12(\frac m2 + k -s))}{\pi^{\frac{m+k+1}2} \Gamma(s)}. \]

It is worth emphasizing here that, when $s\to 1$, one recovers from (1.6) the famous formula for the fundamental solution of $-\mathscr L$, first found by Folland in [Reference Folland9] in $\mathbb {H}^n$, and subsequently generalized by Kaplan in [Reference Kaplan30] to groups of Heisenberg type. Before proceeding, we pause to notice that from the stochastic completeness and left-invariance of $P_t$, in any Carnot group $\mathbb {G}$ one tautologically obtains from (1.5)

(1.7)\begin{equation} \mathscr L^s u(g) = \frac{1}{2}\int_{\mathbb{G}} \frac{2u(g)-u(gh)-u(gh^{{-}1})}{||h||_{(s)}^{Q+2s}}{\rm d}h, \end{equation}

where for $g\in \mathbb {G}$ we have defined

(1.8)\begin{equation} \frac{1}{||g||_{(s)}^{Q+2s}} \overset{def}= \frac{2s}{\Gamma(1-s)} \int_0^\infty \frac{1}{t^{1+s}} p(g,t) {\rm d}t, \end{equation}

with $p(g,\,t)$ the positive heat kernel of $-\mathscr L$. While in the Abelian case $\mathbb {G} = \mathbb {R}^n$, with Euclidean norm $|\cdot |$, an elementary explicit calculation in (1.8), based on the knowledge that $p(x,\,t) = (4\pi t)^{-\frac n2} {\rm e}^{-\frac {|x|^2}{4t}}$, gives

\[ \frac{1}{||x||_{(s)}^{n+2s}} = \frac{s 2^{2s+1} \Gamma(\frac{n+2s}2)}{\pi^{\frac n2}\Gamma(1-s)} \frac{1}{|x|^{n+2s}}, \]

and one recovers from (1.5) Riesz’ classical representation, when $\mathbb {G}$ is a non-Abelian Carnot group it is not true that the right-hand side of (1.8) defines a function of the gauge $|g| = |(z,\,\sigma )| = (|z|^4 + 16 |\sigma |^2)^{1/4}$. In fact, in any (non-Abelian) group of Heisenberg type the following explicit expression of (1.8) was computed in [Reference Garofalo and Tralli17, Theorem 5.1] (to obtain it, one should change $s$ into $-s$ in that result, see [Reference Garofalo and Tralli17, Remark 5.2])

(1.9)\begin{align} \frac{1}{||g||_{(s)}^{Q+2s}} & = \frac{s 2^{k+ 2s}\Gamma(\frac m2 + k + s)}{\pi^{\frac{m+k}2}\Gamma(1-s)\Gamma(\frac k2)} \frac{1}{ |z|^{2(\frac m2 + k + s)}}\notag\\ & \quad \times \int_0^1 (\tanh^{{-}1} \sqrt y)^{{-}s-1} \left(1-y\right)^{\frac m4-1} y^{\frac 12(k + s-1)}\notag\\ & \quad\times F\left(\frac{1}{2}\left(\frac m2 + k + s\right),\frac 12\left(\frac m2 + k+1+s\right);\frac k2;- \frac{16|\sigma|^2}{|z|^4} y\right) \ {\rm d}y, \end{align}

where we have denoted by $F(a,\,b;c;z)$ the Gauss hypergeometric series. Formula (1.9) proves in particular that the function defined by (1.8) is not a function of the gauge $N(z,\,\sigma ) = (|z|^4 + 16 |\sigma |^2)^{1/4}$ (although it does have the expected cylindrical symmetry since it depends on $|z|^4$ and $|\sigma |^2$). If we substitute (1.9) in (1.8), and then (1.8) in (1.7), by comparing with formula (1.4), we conclude that $\mathscr L^s \not = \mathscr L_s$ for every $0< s<1$.

Formulas (1.4) and (1.6) motivated the results in the present work. As we have mentioned, we are interested in optimal decay estimates for nonnegative subsolutions of (1.1). In this respect, [Reference Roncal and Thangavelu36, Theorem 3.1] and [Reference Roncal and Thangavelu37, Theorem 3.7] gave the explicit form of a solution to the fractional Yamabe equation on group of Heisenberg type as a consequence of the intertwining properties of $\mathscr L_s$ for $0 < s < n + 1$, see also [Reference Garofalo and Tralli18] for a different approach to intertwining based on the heat equation. In the notation of [Reference Garofalo and Tralli18, Corollary 3.3], the result is that if $\mathbb {G}$ is of Heisenberg type, and $0< s<1$, then for every $(z,\,\sigma )\in \mathbb {G}$, and $y>0$ one has the following intertwining identity:

(1.10)\begin{align} & \mathscr L_s\left(((|z|^2+y^2)^2+16|\sigma|^2)^{-\frac{m + 2k - 2s}4}\right)=\frac{\Gamma\left(\frac{m+2+2s}{4}\right)\Gamma\left(\frac{m+2k+2s}{4}\right)}{\Gamma\left(\frac{m+2-2s}{4}\right)\Gamma\left(\frac{m+2k-2s}{4}\right)} \\ & \quad\times\ (4y)^{2s}((|z|^2+y^2)^2+16|\sigma|^2)^{-\frac{m + 2k + 2s}4}. \nonumber \end{align}

Here, it might be worth clarifying for the reader that the parameter $y$ appearing in (1.10) is precisely the ‘extension’ variable in the parabolic counterpart of the conformal version of the extension problem discovered in [Reference Frank, del Mar González, Monticelli and Tan12]. An immediate consequence of (1.10) is that, for any real positive number $y>0$, the function

(1.11)\begin{align} u_y(z,\sigma)= \left(\frac{\Gamma\left(\frac{m+2+2s}{4}\right)\Gamma\left(\frac{m+2k+2s}{4}\right)}{\Gamma\left(\frac{m+2-2s}{4}\right)\Gamma\left(\frac{m+2k-2s}{4}\right)}\right)^{\frac{m + 2k - 2s}{4s}}\left(\frac{16y^2}{(|z|^2+y^2)^2+16|\sigma|^2}\right)^{\frac{m + 2k - 2s}4} \end{align}

is a positive solution of the nonlinear equation (1.1). In this sense, we might say that functions (1.11) represent the counterpart of the so-called ‘bubbles’ from conformal geometry. We note that in the particular setting of the Heisenberg group $\mathbb {H}^n$ (which corresponds to the case $m = 2n$ and $k=1$) the function appearing in the left-hand side of (1.10) defines, up to group translations, the unique extremal of the Hardy–Littlewood–Sobolev inequalities obtained by Frank and Lieb in [Reference Frank and Lieb14]. (We emphasize that letting $s\nearrow 1$ one recovers from (1.11) the functions that, in the local case $s=1$, were shown to be the unique positive solutions of the CR Yamabe equation respectively in [Reference Jerison and Lee27], for the Heisenberg group $\mathbb {H}^n$, and [Reference Ivanov, Minchev and Vassilev24], for the quaternionic Heisenberg group. See also the important cited work [Reference Frank and Lieb14], and [Reference Garofalo and Vassilev20, Theor. 1.1] and [Reference Garofalo and Vassilev21] for partial results in groups of Heisenberg type.)

Whether in a group of Heisenberg type $\mathbb {G}$ all nonnegative solutions of (1.1) are, up to left-translations, given by (1.11) presently remains a challenging open question. A first step in such problem is understanding the optimal decay of nonnegative solutions to (1.1). Keeping in mind that the number $m+2k$ in (1.11) represents the homogeneous dimension $Q$ of the group $\mathbb {G}$, by setting the scaling factor $y=1$, we see that there exists a universal constant $C>0$ such that

\[ u_1(z,\sigma)\le \frac{C}{|(z,\sigma)|^{Q-2s}}. \]

It is thus natural to guess that the optimal decay of all nonnegative solutions to (1.1) should be dictated by (1.6), i.e. by the fundamental solution of $\mathscr L_s$. In theorem 1.2 we prove that this guess is correct.

To facilitate the exposition of the ideas and underline the general character of our approach, in this paper we have chosen to work in the setting of homogeneous Lie groups $\mathbb {G}$ with dilations $\{\delta _\lambda \}_{\lambda >0}$, as in the seminal monograph of Folland and Stein [Reference Folland and Stein11]. We emphasize that such groups encompass the stratified, nilpotent Lie groups in [Reference Stein39], [Reference Folland10], and [Reference Folland and Stein11] (but they are a strictly larger class). In particular, our results include Lie groups of Iwasawa type for which (1.1) becomes significant in the case of pseudo-conformal CR and quaternionic contact geometry. We shall assume throughout that $|\cdot |$ is a fixed homogeneous norm in $\mathbb {G}$, i.e. $g\mapsto |g|$ is a continuous function on $\mathbb {G}$ which is $C^\infty$ smooth on $\mathbb {G}\setminus \{e\}$, where $e$ is the group identity, $|g|=0$ if and only if $g=e$, and for all $g\in \mathbb {G}$ we have

(1.12)\begin{equation} (i) \ |g^{{-}1}|=|g|; \quad (ii)\ |\delta_\lambda g|=\lambda|g|. \end{equation}

Finally, we shall assume that the fixed norm satisfies the triangle inequality:

(1.13)\begin{equation} |g\cdot h|\leq |g|+|h|, \qquad g,h\in \mathbb{G}. \end{equation}

We stress that, according to [Reference Hebisch and Sikora23], any homogeneous group allows a norm which satisfies the triangle inequality. (It is well-known that in a group of Heisenberg type the anisotropic gauge $|g| = |(z,\,\sigma )| = (|z|^4 + 16 |\sigma |^2)^{1/4}$ satisfies (i)–(iii), see [Reference Cygan7].) We shall denote with

\[ B_R(g)\equiv B(g,R)=\{h\, \mid\, |g^{{-}1}\circ h|< R\} \]

the resulting open balls with centre $g$ and radius $R$.

For $1\le p < \infty$ and $0< s<1$ we consider the Banach space $\mathcal {D}^{s,p}(\mathbb {G})$ defined as the closure of the space of functions $u\in \mathcal {C}^\infty _0 (\mathbb {G})$ with respect to the norm:

(1.14)\begin{equation} ||u||_{\mathcal{D}^{s,p}(\mathbb{G})}= [u]_{s,p} = \left(\int_{\mathbb{G}}\int_{\mathbb{G}}\frac{|u(g)-u(h)|^p}{|g^{{-}1}\cdot h|^{Q+ps}} {\rm d}g {\rm d}h\right)^{1/p} < \infty. \end{equation}

We are particularly interested in the case $p=2$. In this case, the Euler–Lagrange equation of (1.14) involves the following left-invariant nonlocal operator, initially defined on functions $u\in C^\infty _0(\mathbb {G})$

(1.15)\begin{align} \mathcal{L}_s u(g)=\frac{1}{2}\int_{\mathbb{G}} \frac{2u(g)-u(gh)-u(gh^{{-}1})}{|h|^{Q+2s}}{\rm d}h =\lim_{\varepsilon\rightarrow 0}\int_{\mathbb{G}\setminus B(g,\varepsilon)} \frac{u(g)-u(h)}{|g^{{-}1}\cdot h|^{Q+2s}}{\rm d}h, \end{align}

see [Reference Garofalo, Loiudice and Vassilev16] for a general construction of the fractional operator $\mathcal {L}_s$ on the Dirichlet space $\mathcal {D}^{s,2}(\mathbb {G})$ and relevant Sobolev-type embedding results. In (1.15), and hereafter in this work, the number $Q>0$ represents the homogeneous dimension of $\mathbb {G}$ associated with the group dilations $\{\delta _\lambda \}_{\lambda >0}$. It is clear from (1.4) that, when $\mathbb {G}$ is of Heisenberg type, the nonlocal operator $\mathcal {L}_s$ defined using the Koranyi gauge is just a multiple of $\mathscr L_s$ in (1.3), and this provides strong enough motivation to work with (1.15). A second motivation comes from [Reference Garofalo, Loiudice and Vassilev16, Theor. 1.2], in which we prove that, if $X_1,\,\ldots,\,X_m$ are the left-invariant vector fields of homogeneity one with associated coordinates $x_j$, and the fixed homogeneous norm $|g|$ is a spherically symmetric function of the coordinates $(x_1,\,\dots,\,x_m)$, then for a function $u\in C^\infty _0(\mathbb {G})$ we have the identities:

(1.16)\begin{equation} \lim_{s\rightarrow 0^+} \frac{2s}{\sigma_Q}\mathcal{L}_s u(g)={-}u(g) \quad\text{and}\quad \lim_{s\rightarrow 1^-} \frac{4m(1-s)}{\tau_m}\mathcal{L}_s u(g)={-}\sum_{i=1}^m X_i^2u(g), \end{equation}

where $\sigma _Q,\, \tau _m>0$ are suitable universal constants. Throughout the paper, for $0< s<1$ we let

(1.17)\begin{equation} 2^*(s)\overset{def}=\frac{2Q}{{Q-2s}} \qquad \text{and}\qquad (2^*(s))'=\frac{2Q}{{Q+2s}}, \end{equation}

so that $2^*(s)$, which is the Sobolev exponent associated with the fractional Sobolev inequality [Reference Garofalo, Loiudice and Vassilev16, Theorem 1.2]:

(1.18)\begin{equation} \left ( \int_{\mathbb{G}} |u|^{2^*(s)}(g){{\rm d}g}\right)^{1/2^*(s)}\leq S \left(\int_{\mathbb{G}}\int_{\mathbb{G}} \frac{|u(g)-u(h)|^2}{| h^{{-}1}\cdot g|^{Q+2s}}\, {{\rm d}g}{\rm d}h \right )^{1/2}, \end{equation}

and $(2^*(s))'$ is its Hölder conjugate. In addition to the fractional Sobolev exponent $2^*(s)$, the following exponents will be used:

(1.19)\begin{equation} r\overset{def}=\frac{2^*(s)}{2}=\frac{Q}{Q-2s}\qquad\text{and}\qquad r'=\frac{r}{r-1} = \frac{Q}{2s}. \end{equation}

With all this being said, we are ready to state our results. The first one concerns the nonlocal Schrödinger type equation (1.20). For the notion of subsolution to such equation, see (2.6).

Theorem 1.1 Let $\mathbb {G}$ be a homogeneous group. Let $u\in \mathcal {D}^{s,2}(\mathbb {G})$ be a nonnegative subsolution to the equation:

(1.20)\begin{equation} \mathcal{L}_s u=Vu. \end{equation}

Suppose the following conditions hold true:

1. (i) for some $t_0>r'= \frac {Q}{2s}$ we have $V\in L^{r'}(\mathbb {G})\cap L^{t_0}(\mathbb {G})$;

2. (ii) there exist $\bar R_0$ and $K_0$ so that for $R>\bar R_0$ we have

   (1.21)\begin{equation} \int_{\{|g|>R\}} |V(g)|^{t_0}{{\rm d}g}\leq \frac{K_0}{R^{2st_0-Q}}. \end{equation}

Then there exists a constant $C>0$, depending on $Q$, $s$, and $K_0$, such that for all $g_0\in \mathbb {G}$ with $|g_0|=2R_0\geq 4\bar R_0$, we have for $0< R\leq R_0$

(1.22)\begin{equation} \sup_{B(g_0,R/2)} u\ \leq\ C {\unicode{x2A0F}}_{B(g_0,R)} u + {C}T(u; g_0,R/2), \end{equation}

where the ‘tail’ is given by

(1.23)\begin{equation} T(u;g_0,R)=R^{2s}\int_{\{| g_0^{{-}1}\cdot h|>R\}}\frac{u(h)}{|g_0^{{-}1}\cdot h|^{Q+2s}}{\rm d}h. \end{equation}

We note that the potential $V$ in (1.20) is not assumed to be radial (i.e. a function of the norm $|\cdot |$), or controlled by a power of $u$. Hypothesis (1.21) goes back to the work [Reference Bando, Kasue and Nakajima1], see also [Reference Vassilev42] where a similar assumption was used in the case of Schrödinger type equations modelled on the equations for the extremals to Hardy–Sobolev inequalities with polyradial symmetry. For other results about the Schrödinger equation see [Reference Frank, Lenzmann and Silvestre13]. The ‘tail’ in (1.23) appeared in [Reference Palatucci and Piccinini35] in the setting of the Heisenberg group $\mathbb {H}^n$.

Our second result is the following theorem in which we establish the sharp asymptotic decay of weak nonnegative subsolutions to the fractional Yamabe type equation (1.1). The result applies to weak solutions of $\mathcal {L}_s u=|u|^{2^*(s)-2} u$, since then $|u|$ is a weak subsolution of the Yamabe type equation.

Theorem 1.2 Let $\mathbb {G}$ be a homogeneous group of homogeneous dimension $Q$ and $0< s<1$. If $u\in \mathcal {D}^{s,2}(\mathbb {G})$ is a nonnegative subsolution to the nonlocal Yamabe type equation

(1.24)\begin{equation} \mathcal{L}_s u=u^{\frac{Q+2s}{Q-2s}}, \end{equation}

then $|\cdot |^{{Q-2s}}\,u\in L^\infty (\mathbb {G})$.

We mention that in [Reference Brasco, Mosconi and Squassina3, Theor. 1.1] the authors established, in the setting of $\mathbb {R}^n$, the sharp asymptotic behaviour of the spherically symmetric extremals for the fractional $L^p$ Sobolev inequality, i.e. for the radial nonnegative solutions in $\mathbb {R}^n$ of the equation with critical exponent

\[ (-\Delta_p)^s u =u^{\frac{n(p-1)+sp}{n-sp}}, \]

where $0< s<1$, $1< p<\frac {n}s$, see also [Reference Marano and Mosconi34]. However, both [Reference Brasco, Mosconi and Squassina3] and [Reference Marano and Mosconi34] use in a critical way the monotonicity and radial symmetry of the solutions in order to derive the asymptotic behaviour from the regularity of $u$ in the weak space $L^{r,\infty }$, where $r = \frac {n(p-1)}{n-sp}$. As it is well-known, in the Euclidean setting one can use radially decreasing rearrangement or the moving plane method to establish monotonicity and radial symmetry of solutions to variational problems and partial differential equations. These tools are not available in Carnot groups and proving the relevant symmetries of similar problems remains a very challenging task.

The result of theorem 1.2 does not rely on the symmetry of the solution, hence the method of proof is new even in the Euclidean setting. In order to obtain the optimal decay theorem 1.2 without relying on symmetry of the solution, we use a version of the local boundedness estimate given in theorem 1.1 and then obtain a new estimate of the tail term, which is particular for the fractional case.

In closing, we provide a brief description of the paper. In § 2 we introduce the geometric setting of the paper and the relevant definitions. We also prove proposition 2.1, a preparatory result which provides regularity in $L^p$ spaces for subsolutions of fractional Schrödinger equations. In § 3 we prove theorem 1.1. Finally, in § 4 we prove theorem 1.2.