2.1 Stratified nilpotent Lie groups

Let G be a (real and finite dimensional) stratified nilpotent Lie group of step s with Lie algebra $\mathfrak {g}$ . This means that we may write $\mathfrak {g}$ as a vector space direct sum $\bigoplus _{j = 1}^{s} \mathfrak {v}_{j} $ , where $[\mathfrak {v}_1, \mathfrak {v}_{j}] = \mathfrak {v}_{j + 1}$ when $1\leq j \leq s$ ; here $\mathfrak {v}_{s+1} := \{0\}$ . Let $\nu $ denote the homogeneous dimension of G; that is, $\sum _{j=1}^{s} j \dim \mathfrak {v}_{j}$ . There is a one-parameter family of automorphic dilations $\delta _t$ on $\mathfrak {g}$ , given by

$$ \begin{align*} \delta_t (\mathcal{X}_1 + \mathcal{X}_2+ \ldots + \mathcal{X}_{s}) := t\mathcal{X}_1 + t^2\mathcal{X}_2 + \ldots + t^{s} \mathcal{X}_{s}; \end{align*} $$

here each $\mathcal {X}_j \in \mathfrak {v}_{j}$ and $t \in \mathbb {R}_{+}$ . The exponential mapping $\exp \colon \mathfrak {g} \to G$ is a diffeomorphism, and we identify $\mathfrak {g}$ and G. The dilations extend to automorphic dilations of G, also denoted by $\delta _t$ , by conjugation with $\exp $ . The Haar measure on G, which is bi-invariant, is the Lebesgue measure on $\mathfrak {g}$ lifted to G using $\exp $ . Many more general facts about stratified Lie groups may be found in [Reference Folland and Stein12].

By [Reference Hebisch and Sikora15], the group G may be equipped with a smooth subadditive homogeneous norm $\rho $ , a continuous function from G to $[0,\infty )$ that is smooth on $G\setminus \{o\}$ and satisfies

1. (1) $\rho (g^{-1}) =\rho (g)$ ;

2. (2) $\rho (g^{-1}h) \leq \rho (g) + \rho (h)$ ;

3. (3) $\rho ({ \delta _t(g)}) =t\rho (g)$ for all $g\in G$ and all $t \in \mathbb {R}_{+}$ ;

4. (4) $\rho (g) =0$ if and only if $g=o$ .

Abusing notation, we set $\rho (g, g') := \rho (g^{-1} g')$ for all $g, g' \in G$ ; this defines a metric on G. We write $B(g, r)$ for the open ball with centre g and radius r with respect to $\rho $ :

$$\begin{align*}B(g, r) = g B(o,r) = g \{ h \in G \colon \rho(h) < r \}. \end{align*}$$

Then

$$\begin{align*} \delta _r(B,o,1) = B(o,r) \quad\text{and}\quad |B(o,r)| = r^\nu |B(0,1)|.\end{align*}$$

The metric space $(G,\rho )$ is geometrically doubling; that is, there exists $A \in \mathbb {N}$ such that every metric ball $B(x,2r)$ may be covered by at most A balls of radius r.

We remind the reader that a stratified Lie group is a space of homogenous type in the sense of Coifman and Weiss [Reference Coifman and Weiss3, Reference Coifman and Weiss4], and analysis on stratified Lie groups uses much from the theory of such spaces. In particular, we frequently deal with molecules, that is, functions $\zeta $ that satisfy standard decay and smoothness conditions, by which we mean that there is a parameter $\epsilon \in (0,1]$ , which we fix once and for all, such that

(2.1) $$ \begin{align} \begin{gathered} \left| \zeta(g) \right| \lesssim \frac{1}{(1+ \rho(g))^{\nu+\epsilon}}, \\ \left| \zeta(g)-\zeta(g') \right| \lesssim \frac{\rho( g' g^{-1} )^\epsilon}{(1+ \rho(g)+ \rho(g'))^{\nu+2\epsilon} } \end{gathered} \end{align} $$

for all $g, g' \in G$ . We often impose an additional cancellation condition, namely

(2.2) $$ \begin{align} \int_G \zeta(g) \,\mathrm{d} g =0. \end{align} $$

The normalized dilate $f_t$ of a function f on G by $t \in \mathbb {R}_{+}$ is given by $f_{t} := t^{-\nu }f\circ \delta _{1/t}$ , and the convolution $f\ast f'$ of suitable functions f and $f'$ on G is defined by

$$ \begin{align*} (f \ast f')(g) :=\int_Gf(h)f'(h^{-1}g)\,\mathrm{d} h =\int_Gf(gh^{-1})f'(h)\,\mathrm{d} h. \end{align*} $$

Take left-invariant vector fields $\mathcal {X}_1$ , …, $\mathcal {X}_{n}$ on G that form a basis of $\mathfrak {v}_1$ , and define the sub-Laplacian $\mathcal {L} := -\sum _{j=1 }^{n} (\mathcal {X}_{j})^2 $ . Observe that each $\mathcal {X}_{j}$ is homogeneous of degree $1$ and $\mathcal {L}$ is homogeneous of degree $2$ , in the sense that

$$ \begin{align*} &\mathcal{X}_{j} \left( f \circ \delta_{t} \right) = t \left( \mathcal{X}_{j} f \right) \circ \delta_{t}, \\ &\mathcal{L} \left( f \circ \delta_{t} \right) = t^2 \left( \mathcal{L} f \right) \circ \delta_{t} \end{align*} $$

for all $t \in \mathbb {R}_{+}$ and all $f \in \mathsf {C}^{2}(G)$ .

Associated to the sub-Laplacian, Folland, and Stein [Reference Folland and Stein12] defined the Riesz potential operators $\mathcal {L}^{-\alpha }$ , where $\alpha \in \mathbb {R}_{+}$ ; these are convolution operators with homogeneous kernels. The Riesz transformation $\mathcal {R}_{j} := \mathcal {X}_{j} \mathcal {L}^{-1/2}$ is a singular integral operator, and is bounded on $\mathsf {L}^{p}(G)$ when $1 < p < \infty $ as well as from the Folland–Stein Hardy space $\mathsf {H}^1(G)$ [Reference Folland and Stein12] to $\mathsf {L}^{1}(G)$ .

The Hardy–Littlewood maximal operator $\mathcal {M}$ on G is defined using the metric balls:

where the “average integral” is defined by

For future use, we note that the layer cake formula implies that, if $\mu $ is a radial decreasing function on G (that is, $\mu (g)$ depends only on $\rho (g)$ and decreases as $\rho (g)$ increases), then

(2.3) $$ \begin{align} \big(\left| f \right| \ast \mu_\epsilon\big) (g) \leq \left\Vert \mu \right\Vert{}_{\mathsf{L}^{1}(G)} \mathcal{M}f(g) \qquad\forall g \in G. \end{align} $$

2.2 Functional calculus for the sub-Laplacian

The sub-Laplacian $\mathcal {L}$ has a spectral resolution:

$$\begin{align*}\mathcal{L} (f)=\int_{\mathbb{R}_{+}}\lambda \,\mathrm{d} \mathcal{E}_{\mathcal{L} }(\lambda) f \qquad\forall f\in \mathsf{L}^{2}(G), \end{align*}$$

where $\mathcal {E}_{\mathcal {L}}(\lambda )$ is a projection-valued measure on $[0,\infty )$ , the spectrum of $\mathcal {L}$ . For a bounded Borel function $m\colon [0,\infty )\to \mathbb {C}$ , we define the operator $F(\mathcal {L})$ spectrally:

$$ \begin{align*} m(\mathcal{L})f := \int_{\mathbb{R}_{+}} m(\lambda)\,\mathrm{d} \mathcal{E}_{\mathcal{L}}(\lambda) f \qquad\forall f\in \mathsf{L}^{2}(G). \end{align*} $$

This operator is a convolution with a Schwartz distribution on G.

2.3 The heat and Poisson kernels

Let $h_t$ and $p_{t}$ , where $t \in \mathbb {R}_{+}$ , be the heat and Poisson kernels associated with the sub-Laplacian operator $\mathcal {L}$ , that is, the convolution kernels of the operators $e^{t\mathcal {L}}$ and $e^{t \sqrt {\mathcal {L}}}$ on G. We write ${{q}}_{t}$ for $t\partial _t p_{t}$ . We warn the reader that $p_{t}$ and ${{q}}_{t}$ are the normalized dilates of $p_1$ and ${{q}}_{1}$ by the factor t, but $h_t$ is the normalized dilate of $h_1$ by a factor of $t^{1/2}$ . Let $\nabla $ denote the horizontal subgradient on G and denote the gradient $(\nabla , \partial _{t})$ on $G\times \mathbb {R}_{+}$ .

Lemma 2.1 The kernels $h_t$ and $p_{t}$ are $\mathbb {R}_{+}$ -valued. Further, $h_t$ and $p_{t}$ have integral $1$ , while ${{q}}_{t}$ has integral $0$ for all $t \in \mathbb {R}_{+}$ . Finally, there exists a constant c such that

for all $g\in G$ and $t \in \mathbb {R}_{+}$ .

Proof For the heat kernel estimates, see [Reference Varopoulos, Saloff-Coste and Coulhon30, Theorem IV.4.2]. Note that the first estimate has a version with the opposite inequality, with a different constant c.

The estimates for $p_{t}$ and ${{q}}_{t}$ follow from the subordination formula

$$ \begin{align*} e^{-t\sqrt{\mathcal{L}}} =\frac{1}{2\sqrt{\pi}}\int_{\mathbb{R}_{+}} \frac{te^{-{t^2}/{4v}}}{\sqrt{v}}e^{-v \mathcal{L}}\frac{\,\mathrm{d} v}{v}. \end{align*} $$

We leave the details to the reader.

This lemma shows that the heat kernel $h_1$ and the Poisson kernel $p_1$ and their derivatives satisfy the standard decay and smoothness conditions (2.1); their derivatives also satisfy the cancellation condition (2.2).

2.4 Systems of pseudodyadic cubes

We use the Hytönen–Kairema [Reference Hytönen and Kairema17] families of “dyadic cubes” in geometrically doubling metric spaces. We state a version of [Reference Hytönen and Kairema17, Theorem 2.2] that is simpler, in that we work on metric spaces rather than pseudometric spaces. The Hytönen–Kairema construction builds on seminal work of Christ [Reference Christ7] and of Sawyer and Wheeden [Reference Sawyer and Wheeden29].

We write $\mathscr {Q}(G)$ for the union of all $\mathscr {Q}_k(G)$ , and call this a system of pseudodyadic cubes. Given a cube $Q \in \mathscr {Q}_k(G)$ , we denote the quantity $\kappa ^k$ by $\operatorname {\ell }(Q)$ , by analogy with the side-length of a Euclidean cube.

A finite collection $\{\mathscr {Q}^{\tau }\colon {\tau }=1,2,\dots ,\mathrm {T}\}$ of systems of pseudodyadic cubes is called a collection of adjacent systems of pseudodyadic cubes with parameters $C'$ , c, C and $\kappa $ , if it has the following properties: individually, each $\mathscr {Q}^{\tau }$ is a system of pseudodyadic cubes with parameters c, C and $\kappa $ as in Theorem 2.2; collectively, for each ball $B(x,r)\subseteq G$ such that $\kappa ^{k+3}<r\leq \kappa ^{k+2}$ , where $k\in \mathbb {Z}$ , there exist ${\tau } \in \{1, 2, \dots , \mathrm {T}\}$ and $Q\in \mathscr {Q}^{\tau }_k$ with centre $z(Q)$ such that $d(x, z(Q)) < 2\kappa ^{k}$

(2.4) $$ \begin{align} B(x,r)\subseteq Q\subseteq B(x,C'r). \end{align} $$

The following construction is due to [Reference Hytönen and Kairema17].

Theorem 2.3 Suppose that $(G,\rho )$ is a metric stratified group. Then there exists a finite collection $\{\mathscr {Q}^{\tau }\colon {\tau } = 1,2,\dots ,\mathrm {T}\}$ of adjacent systems of pseudodyadic cubes with parameters $C'$ , c, C and $\kappa $ , where $\kappa := 1/100$ , $c := 12^{-1}$ , $C := 4$ and $C' := 8 \times 10^{6}$ . For each ${\tau }\in \{1,2,\dots ,\mathrm {T}\}$ , the centres $z(Q)$ of the cubes $Q \in \mathscr {Q}^{\tau }_k$ have the properties that

$$\begin{align*}d(z(Q), z(Q')) \geq \kappa^k /4 \qquad\text{when }Q\neq Q' \end{align*}$$

and

$$\begin{align*}\min \{ d(x, z(Q)) \colon Q \in \mathscr{Q}^{\tau}_k \} < 2\kappa^k \qquad\forall x\in G. \end{align*}$$

From [Reference Kairema, Li, Pereyra and Ward21, Remark 2.8], the number $\mathrm {T}$ of adjacent systems of pseudodyadic cubes in Theorem 2.3 may be taken to be at most $A^6 \kappa ^{-\log _2(A)}$ , where A is the geometric doubling constant of G. The constants c, C, $C'$ and $\kappa $ do not depend on the choice of the metric stratified group $(G,\rho )$ .

2.5 Products of stratified groups

We equip the product of two stratified groups $G_1$ and $G_2$ with a product structure. We carry forward the notation from Section 2.1, adding a subscript i or superscript $[i]$ to clarify that we are dealing with $G_i$ ; the parameter i is always $1$ or $2$ . To shorten the formulae, we often use bold face type to indicate a product object: thus we write $\mathbf {G}$ , $\mathbf {T}$ , $\mathbf {g}$ , $\mathbf {r}$ and $\mathbf {t}$ in place of $G_1 \times G_2$ , $\mathbb {R}_{+} \times \mathbb {R}_{+}$ , $(g_1,g_2)$ , $(r_1,r_2)$ and $(t_1,t_2)$ . For example, $B_i (g_i, r_i)$ denotes the open ball in $G_i$ with centre $g_i$ and radius $r_i$ , with respect to the homogeneous norm $\rho _i $ , and we write $\mathbf {P}(\mathbf {g},\mathbf {r})$ for the product $B_1(g_1, r_1) \times B_2(g_2, r_2)$ . We write $\pi _i$ for the projection of $\mathbf {G}$ onto $G_i$ .

Products of balls are basic geometric objects and we write $\mathscr {P}(\mathbf {G})$ for the family of all such products. In addition we deal with rectangles, by which we mean products of pseudodyadic cubes. We write $\mathscr {R}(\mathbf {G})$ for the family of all rectangles, and for adjacent pseudodyadic systems as constructed in Theorem 2.3, we let $\mathscr {R}^{{\tau }_1,{\tau }_2}(\mathbf {G})$ be the family of rectangles $\{Q_1\times Q_2\colon Q_i\in \mathscr {Q}^{{\tau }_i},\ {\tau }_i=1,2,\dots ,\mathrm {T}_i\}$ . We let $\operatorname {\boldsymbol {\ell }}\colon \mathscr {R}(\mathbf {G}) \to \mathbf {T}$ be the function such that $\operatorname {\ell }_i(Q_1 \times Q_2) = \operatorname {\ell }(Q_i)$ , the “side-length” of $Q_i$ . We also let $\mathbf {z}(R)$ be the center of R, that is, $\mathbf {z}(R)=(z_1(R), z_2(R))$ , where, when $R=Q_1\times Q_2$ , we set $z_j(R)=z(Q_j)$ , with $z(Q_j)$ as defined in Theorem 2.2.

The element of Haar measure on $\mathbf {G}$ is denoted $\,\mathrm {d}\mathbf {g}$ , but is often written $\,\mathrm {d} g_1\,\mathrm {d} g_2$ for calculations. The convolution $f\ast f'$ of suitable functions f and $f'$ on $\mathbf {G}$ is defined by

$$ \begin{align*} (f\ast f')(\mathbf{g}) :=\int_{\mathbf{G}}f(\mathbf{h})f'(\mathbf{h}^{-1}\mathbf{g}) \,\mathrm{d} \mathbf{h}. \end{align*} $$

The strong maximal operator $\mathcal {M}_{s}$ is defined by

for all $f \in \mathsf {L}^{1}_{\mathrm {loc}}(\mathbf {G})$ . It is evident that $\mathcal {M}_{s}$ is dominated by the composition of the Hardy–Littlewood maximal operators in the factors:

$$\begin{align*}\mathcal{M}_{s}{f} \leq \mathcal{M}_1 \mathcal{M}_2 (f) \qquad\text{and}\qquad \mathcal{M}_{s}{f} \leq \mathcal{M}_2 \mathcal{M}_1(f). \end{align*}$$

When $1 < p \leq \infty $ , the operators $\mathcal {M}_1$ and $ \mathcal {M}_2$ in the factors are $\mathsf {L}^{p}$ -bounded, so the iterated maximal operators and hence the strong maximal operator are also $\mathsf {L}^{p}$ -bounded.

Given an open subset U of $\mathbf {G}$ with finite measure $\left | U \right | $ , we define the enlargement $U^{*}$ of U using the strong maximal operator $\mathcal {M}_{s}$ :

$$ \begin{align*} U^{*} := \Big\{ \mathbf{g} \in \mathbf{G} \colon \mathcal{M}_{s} \chi_{U}(\mathbf{g})> \alpha \Big\} , \end{align*} $$

for some $\alpha \in (0,1)$ that varies from instance to instance. It is easy to see that $U \subset U^*$ , while the $\mathsf {L}^{2}(\mathbf {G})$ boundedness of the strong maximal operator together with Chebyshev’s inequality shows that $| U^*| \lesssim |U|$ . We write $\mathscr {M}(U)$ for the family of maximal rectangles contained in U.

If $\varphi ^{[1]}$ on $G_1$ and $\varphi ^{[2]}$ on $G_2$ both satisfy the decay and smoothness conditions (2.1) and have integral $1$ , then

(2.5) $$ \begin{align} \left| ( f \ast \varphi_{\mathbf{t}})(\mathbf{g}) \right| \lesssim \mathcal{M}_{s}(f)(\mathbf{g}) \qquad\forall \mathbf{g} \in \mathbf{G}, \quad\forall f \in \mathsf{L}^{1}(\mathbf{G}), \end{align} $$

much as argued to prove (2.3), but with “biradial” in place of “radial”.

Recall from (1.6) that ${{q}}_{\mathbf {t}}$ denotes $t_1\partial _{t_1} p_{t_1}^{[1]} \otimes t_2\partial _{t_2} p^{[2]}_{t_2}$ and

The properties of the Poisson kernel are important for the next lemma.

Proof The function $(\mathbf {t}, \mathbf {h}) \mapsto (f \ast {{q}}_{\mathbf {t}} )(\mathbf {h})$ is smooth in $ \mathbf {T} \times \mathbf {G}$ , and so the hypothesis $\mathcal {S}_{q,\eta }(f)(\mathbf {g}) = 0$ implies that $(f \ast {{q}}_{\mathbf {t}} )(\mathbf {h}) = 0$ for all $(\mathbf {t}, \mathbf {h}) \in \Gamma ^\eta (\mathbf {g})$ . Hence for all $\mathbf {h} \in \mathbf {G}$ , the function $\mathbf {t} \mapsto (f \ast p_{\mathbf {t}})(\mathbf {h})$ is constant once $t_1$ and $t_2$ are both large enough. However,

$$\begin{align*}\left| (f \ast p_{\mathbf{t}})(\mathbf{h}) \right| \leq \left\Vert f \right\Vert{}_{\mathsf{L}^{2}(\mathbf{G})} \left\Vert p_{\mathbf{t}} \right\Vert{}_{\mathsf{L}^{2}(\mathbf{G})} \to 0 \end{align*}$$

as $t_1$ or $t_2$ tends to infinity, so the only possible value for $(f \ast p_{\mathbf {t}})(\mathbf {h})$ when $t_1$ and $t_2$ are both large is $0$ . Since

$$\begin{align*}(-\partial_{t_1}^2f + \mathcal{L}^{[1]})( f \ast p_{\mathbf{t}}) = (- \partial_{t_2}^2f + \mathcal{L}^{[2]}) (f \ast p_{\mathbf{t}}) = 0, \end{align*}$$

analytic hypoellipticity implies that $f \ast p_{\mathbf {t}} = 0$ in $\mathbf {T} \times \mathbf {G}$ and hence $f = 0$ .

The hypothesis of analytic hypoellipticity certainly holds when $\mathbf {G} = \mathbb {R}^{m} \times \mathbb {R}^{n}$ , but in general it is not satisfied; see [Reference Helffer16] for more information.

2.6 Journé’s covering lemma

We recall a covering lemma for product spaces. Journé [Reference Journé20] first established this result on $\mathbb {R}\times \mathbb {R}$ . The fourth author [Reference Pipher28] extended it to higher dimensional Euclidean spaces and an arbitrary number of factors $\mathbb {R}^{n_1}\times \mathbb {R}^{n_2}\times \ldots \times \mathbb {R}^{n_m}$ . This was extended to products of spaces of homogeneous type in [Reference Han, Li and Lin14, Lemma 2.1].

Take $\alpha \in (0,1/2)$ . Let U be an open subset of $\mathbf {G}$ of finite measure and let $\mathscr {M}(U)$ be the set of all maximal subrectangles of U. Given $R=Q_1\times Q_2\in \mathscr {M}(U)$ , let $\tilde {Q}_2$ be the biggest pseudodyadic cube containing $Q_2$ such that

$$\begin{align*}\big|\big( Q_1\times \tilde{Q}_2\big) \cap U\big|> \alpha |Q_1\times \tilde{Q}_2|. \end{align*}$$

Similarly, given $R=Q_1\times Q_2\in \mathscr {M}(U)$ , let $\tilde {Q}_1$ be the biggest pseudodyadic cube containing $Q_1$ such that

$$\begin{align*}\big|\big( \tilde{Q}_1\times Q_2\big) \cap U\big|>\alpha |\tilde{Q}_1\times Q_2|. \end{align*}$$

We then set

$$\begin{align*}\gamma_1(R) := \frac{\operatorname{\ell}(\tilde{Q}_1)}{\operatorname{\ell}(Q_1)} \qquad\text{and}\qquad \gamma_2(R) := \frac{\operatorname{\ell}(\tilde{Q}_2)}{\operatorname{\ell}(Q_2)}. \end{align*}$$

Here is our Journé-type covering lemma on $\mathbf {G}$ .

Lemma 2.5 Suppose that U is an open subset in $\mathbf {G}$ of finite measure and $\delta \in \mathbb {R}_{+}$ . Then

$$ \begin{align*} \sum_{R\in \mathscr{M}(U)}\left| R \right| \left( \gamma_1(R)^{-\delta} + \gamma_2(R)^{-\delta} \right) \lesssim_\delta |U|. \end{align*} $$

2.7 The Orlicz space L log L

We recall the definition of an Orlicz space on $\mathbf {G}$ . A Young function is a continuous, convex, increasing bijective function $\Phi \colon [0,\infty )\to [0,\infty )$ . To a Young function $\Phi $ , we associate the nonlinear functional $F_\Phi $ , given by

$$\begin{align*}F_{\Phi}(f) = \int_{\mathbf{G}} \Phi(|f(\mathbf{g})|) \,\mathrm{d} \mathbf{g} \qquad\forall f \in \mathsf{L}^{1}_{\mathrm{loc}}(\mathbf{G}). \end{align*}$$

The convexity of $\Phi $ implies that $\{ f \in \mathsf {L}^{1}_{\mathrm {loc}}(\mathbf {G}) \colon F_\Phi (f) \leq 1 \}$ is a closed convex symmetric set, and so the Luxemburg norm, given by

$$\begin{align*}\|f\|_{\mathsf{L}^{\Phi}(\mathbf{G})} := \inf\left\{ \lambda\in \mathbb{R}_{+} \colon F_{\Phi}(f/\lambda) \leq 1 \right\}\hspace{-1pt}, \end{align*}$$

is indeed a norm, and $\mathsf {L}^{\Phi }(\mathbf {G})$ , the set of functions f for which $\|f\|_{\mathsf {L}^{\Phi }(\mathbf {G})}$ is finite, is a Banach space. The sets $\{ f \in \mathsf {L}^{1}_{\mathrm {loc}}(\mathbf {G}) \colon F_\Phi (f) \leq 1 \}$ and $\{ f \in \mathsf {L}^{1}_{\mathrm {loc}}(\mathbf {G}) \colon \left \Vert f \right \Vert {}_{\mathsf {L}^{\Phi }(\mathbf {G})} \leq 1 \}$ coincide.

Suppose that $\Phi $ and $\Psi $ are Young functions such that

(2.6) $$ \begin{align} st \leq \Phi(s)+\Psi(t) \qquad\forall s,t \in \mathbb{R}_{+}. \end{align} $$

Then

(2.7) $$ \begin{align} \left| \int_{\mathbf{G}} f(\mathbf{g}) h(\mathbf{g}) \,\mathrm{d} \mathbf{g} \right| \leq \int_{\mathbf{G}} \Phi(f(\mathbf{g})) \,\mathrm{d} \mathbf{g} + \int_{\mathbf{G}} \Psi( h(\mathbf{g})) \,\mathrm{d} \mathbf{g} , \end{align} $$

and it follows that the corresponding Luxemburg norms are related by the generalized Hölder inequalities, namely,

(2.8) $$ \begin{align} \Big|\int_{\mathbf{G}} f(\mathbf{g}) h(\mathbf{g}) \,\mathrm{d} \mathbf{g}\Big| \leq 2\|f\|_{\mathsf{L}^{\Phi}(\mathbf{G})} \|h\|_{\mathsf{L}^{\bar\Phi}(\mathbf{G})}. \end{align} $$

We are particularly interested in a special pair of Young functions. Henceforth,

(2.9) $$ \begin{align} \Phi(s):=s[\log(\mathrm{e} + s)]\qquad\text{and}\qquad \Psi(t) := \exp(t)-1. \end{align} $$

By maximizing in s, it is straightforward to show that $1 + st - s \log (\mathrm {e}+s) \leq \mathrm {e}^t$ , so these functions satisfy (2.6), whence (2.7) and (2.8) hold. For this choice of $\Phi $ and $\Psi $ , the corresponding spaces are denoted by $\mathsf {L\,log\,L}(\mathbf {G})$ and $\mathrm {e}^{\mathsf {L}^{}}(\mathbf {G})$ . Clearly $\mathsf {L\,log\,L}(\mathbf {G}) \subseteq \mathsf {L}^{1}(\mathbf {G})$ . Further, $\Phi (\lambda t) \leq \Phi (\lambda ) \Phi (t)$ , whence

(2.10) $$ \begin{align} F_\Phi(\lambda f) \leq \Phi(\lambda) F_\Phi(f) \qquad\forall \lambda\in \mathbb{R}_{+} \quad\forall f \in \mathsf{L\,log\,L}(\mathbf{G}). \end{align} $$

This implies that $f \in \mathsf {L\,log\,L}(\mathbf {G})$ if and only if $F_\Phi (f)$ is finite, and that

(2.11) $$ \begin{align} F_\Phi(f/\lambda) \leq \Phi(1/\lambda) F_\Phi(f) , \end{align} $$

which we used in the introduction to prove (1.4) from (1.3).

Finally, since $\Psi $ is a Young function,

(2.12) $$ \begin{align} \Psi(t/\lambda) \leq \Psi(t)/\lambda \qquad\forall \lambda\in [1,\infty). \end{align} $$

For all this and much more about Orlicz spaces, see [Reference Luxemburg24].

We need a density result.

Proof For $f\in \mathsf {L\,log\,L} (\mathbf {G})$ , we define, for all $N \in \mathbb {N}$ ,

$$\begin{align*}f_N := \chi_{\{\mathbf{g}\in \mathbf{G}\colon |f(\mathbf{g})| \leq N\}} f. \end{align*}$$

Then $f_N\in \mathsf {L\,log\,L} (\mathbf {G})$ , $|f_N| \leq |f|$ , and $f_N \to f$ almost everywhere as $n \to \infty $ . Next,

$$ \begin{align*} \int_{\mathbf{G}} |f_N(\mathbf{g})|^2 \,\mathrm{d} \mathbf{g} \leq \int_{\mathbf{G}} N |f(\mathbf{g})| \log(\mathrm{e} + |f(\mathbf{g})|) \,\mathrm{d} \mathbf{g} = N F_{\Phi}(f), \end{align*} $$

which shows that $f_N \in \mathsf {L}^{2}(\mathbf {G})$ for all $N \in \mathbb {Z}^+$ . Finally, if $\lambda < 1$ , then

$$ \begin{align*} F_{\Phi}((f-f_N)/\lambda) &= \frac{1}{\lambda} \int_{\mathbf{G}} |f(\mathbf{g})-f_N(\mathbf{g})| \log{\Big(}\mathrm{e} + |(f(\mathbf{g})-f_N(\mathbf{g}))/\lambda)|\Big) \,\mathrm{d} \mathbf{g}\\ &\leq \frac{1}{\lambda} \int_{\{\mathbf{g}\in \mathbf{G}\colon |f(\mathbf{g})|>N\}} |f(\mathbf{g})| \left( \log(\mathrm{e} + |f(\mathbf{g})|) + \log(1/\lambda) \right) \,\mathrm{d} \mathbf{g} \\ &\to 0 \end{align*} $$

as $N \to \infty $ . Consequently, $\left \Vert {f-f_N}\right \Vert {}_{\mathsf {L\,log\,L}(\mathbf {G})}$ tends to zero as N tend to infinity.

We also have the following auxiliary result.

Lemma 2.7 For every $f \in \mathsf {L\,log\,L}(\mathbf {G})$ ,

$$\begin{align*}\sum_{k \leq 0} 2^{k}F_\Phi(2^{-k}f)^{1/2} \lesssim F_\Phi(f)^{1/2}. \end{align*}$$

Proof By definition, if $k\leq 0$ , then

$$ \begin{align*} 2^kF_{\Phi}(2^{-k}f)^{1/2} &= 2^{k} \bigg(\int_{\mathbf{G}} 2^{-k} |f(\mathbf{g})| \log\Big(\mathrm{e} + 2^{-k}|(f(\mathbf{g})|\Big) \,\mathrm{d} \mathbf{g}\bigg)^{1/2}\\ &\leq 2^{k / 2} \bigg(\int_{\mathbf{G}} |f(\mathbf{g})| \log\Big( 2^{-k}(\mathrm{e} + |(f(\mathbf{g})|)\Big) \,\mathrm{d} \mathbf{g}\bigg)^{1/2} \\ &= 2^{k / 2} \bigg(\int_{\mathbf{G}} |f(\mathbf{g})|\Big[ \log 2^{-k}+ \log\big( \mathrm{e} + |(f(\mathbf{g})|\big) \Big]\,\mathrm{d} \mathbf{g}\bigg)^{1/2} \\ &\leq (-k)^{1 / 2}2^{k / 2} (\log 2)^{1 / 2} \bigg(\int_{\mathbf{G}} |f(\mathbf{g})| \,\mathrm{d} \mathbf{g}\bigg)^{1/2} \\ &\qquad\qquad\qquad+ 2^{k / 2} \bigg(\int_{\mathbf{G}} |f(\mathbf{g})| \log\big( \mathrm{e} + |(f(\mathbf{g})|\big) \,\mathrm{d} \mathbf{g}\bigg)^{1/2}. \end{align*} $$

Summing over all such k yields the required estimate.

4.1 Hyperweak boundedness and the area function

The first step is to apply Corollary 3.4.

Proposition 4.1 The nontangential maximal operator $\mathcal {M}_{p,\eta }$ is hyperweakly bounded. That is,

$$\begin{align*}\big| \big\{\mathbf{g} \in \mathbf{G}\colon |\mathcal{M}_{p,\eta } f(\mathbf{g})|>\lambda \big\}\big| \lesssim F_\Phi(f/\lambda) \qquad\forall \lambda \in \mathbb{R}_{+} \end{align*}$$

for all $f \in \mathsf {L\,log\,L}(\mathbf {G})$ .

This weak type estimate for the Poisson maximal operator implies a similar estimate for the Lusin area function , which is the key to establishing the atomic decomposition for $\mathsf {L\,log\,L} (\mathbf {G})$ functions. More explicitly, we denote the tensor by and define

where

(4.1)

Theorem 4.2 The following estimate holds:

for all $f \in \mathsf {L\,log\,L}(\mathbf {G})$ .

Proof A recent result of Fan, Yan and the first and third authors [Reference Cowling, Fan, Li and Yan6] shows that for f such that $\left \Vert \mathcal {M}_{p,\eta } f \right \Vert {}_{\mathsf {L}^{1}(\mathbf {G}) }<\infty $ ,

More precisely, define the sublevel set $L_{\eta }(\lambda ) :=\left \{ \mathbf {g}\in {\mathbf {G}}\colon \mathcal {M}_{p,\eta }(f)(\mathbf {g}) \leq \lambda \right \}$ . Then it is shown that, when $\eta $ is sufficiently large,

(4.2)

for all $\lambda \in \mathbb {R}_{+}$ . Now $\big | L_{\eta }(\lambda )^c \big | \lesssim F_\Phi (f/\lambda )$ by Proposition 4.1, so the layer-cake formula and (2.10) show that

This implies that the right-hand side of (4.2) is finite. By repeating the argument of [Reference Cowling, Fan, Li and Yan6], we see that (4.2) also holds for $f\in \mathsf {L\,log\,L} (\mathbf {G})$ , and we conclude that

for all $f \in \mathsf {L\,log\,L}(\mathbf {G})$ , as required.

Recall that ${{q}}_{\mathbf {t}}$ denotes $t_1\partial _{t_1} p_{t_1}^{[1]} \otimes t_2\partial _{t_2} p^{[2]}_{t_2}$ and

Corollary 4.3 The area function $\mathcal {S}_{q,1} (f)$ satisfies the estimate:

$$ \begin{align*} \big|\big\{ \mathbf{g}\in \mathbf{G}\colon \mathcal{S}_{q,1} (f)(\mathbf{g})>\lambda \big\} \big| \lesssim F_\Phi(f/\lambda) \qquad\forall \lambda \in \mathbb{R}_{+} \end{align*} $$

for all $f \in \mathsf {L\,log\,L}(\mathbf {G})$ .

4.2 The atomic decomposition for L log L

Recall that the $q^{[i]}_1$ satisfy the standard smoothness, decay and cancellation conditions (2.1) and (2.2), and by [Reference Geller and Mayeli13] there exist compactly supported smooth functions $\varphi ^{[i]}$ on $G_i$ with integral $0$ such that

$$ \begin{align*} f &= \int_{\mathbf{T}} f \ast {{q}}_{\mathbf{t}} \ast \varphi_{\mathbf{t}} \,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}} \end{align*} $$

for all $f\in \mathsf {L}^{2}(\mathbf {G})$ , where $\varphi _{\mathbf {t}}=\varphi _{t_1}\otimes \varphi _{t_2}$ . This is the Calderón reproducing formula. We suppose that $\operatorname {supp}(\varphi ^{[i]}) \subseteq B_i(o,1)$ , by rescaling if necessary.

Theorem 4.4 (Atomic decomposition for $\mathsf {L\,log\,L} $ )

Let $f\in \mathsf {L}^{2}(\mathbf {G}) \cap \mathsf {L\,log\,L} (\mathbf {G})$ . Then we may write

$$\begin{align*}f=\sum_{k \in \mathbb{Z}} a_k, \end{align*}$$

where the sum converges unconditionally in $L^2(\mathbf {G})$ and the atoms $a_k$ satisfy:

1. (1) $a_k$ vanishes outside a subset $U^{\dagger }_k$ of $\mathbf {G}$ such that $|U^{\dagger }_k|\lesssim F_\Phi (2^{-k}{f})$ ;

2. (2) $\|a_k\|_{\mathsf {L}^{2}(\mathbf {G})}^2 \lesssim 2^{2k}F_\Phi (2^{-k}f)$ ;

3. (3) each $a_k$ can be further decomposed: $a_k=\sum _{R\in \mathscr {M}(U^{*}_k)}a_{k,R} $ , where the sum converges unconditionally in $\mathsf {L}^{2}(\mathbf {G})$ , and

   1. (a) $\operatorname {supp} a_{k,R} \subseteq \beta R$ , for a suitable $\beta \in \mathbb {R}_{+}$ ;

   2. (b) $\int _{G_1}a_{k,R}(g_1,g_2) \,\mathrm {d} g_1=0$ for all $g_2 \in G_2$ ;

   3. (c) $\int _{G_2}a_{k,R}(g_1,g_2) \,\mathrm {d} g_2=0$ for all $g_1 \in G_1$ ;

   4. (d) $\sum _{R\in \mathscr {M}(U^*_k)} \left \Vert a_{k,R} \right \Vert {}^2_{\mathsf {L}^{2}(\mathbf {G})}\lesssim 2^{2k}F_\Phi (2^{-k} f)$ .

The sets $U^{\dagger }_k$ and $U^*_k$ are defined in the proof.

Proof From Corollary 4.3, $\mathcal {S}_{q,1}$ is hyperweakly bounded. Assume that $0 \neq f\in \mathsf {L}^{2}(\mathbf {G}) \cap \mathsf {L\,log\,L} (\mathbf {G})$ . Then $\mathcal {S}_{q,1} (f)\in \mathsf {L}^{2}(\mathbf {G})$ . Now for all $k\in \mathbb {Z}$ , we set

(4.3) $$ \begin{align} U_k &:= \{ \mathbf{g}\in\mathbf{G}\colon \mathcal{S}_{q,1}(f)(\mathbf{g})>2^k \}; \end{align} $$

(4.4) $$ \begin{align} U^{*}_k &:= \{\mathbf{g}\in\mathbf{G}\colon \mathcal{M}_{s} (\chi_{U_k})(\mathbf{g})> \alpha \}; \end{align} $$

(4.5) $$ \begin{align} U^{\dagger}_k &:= \bigcup\nolimits_{S \in \mathscr{M}(U^*_k)} \beta S; \end{align} $$

(4.6) $$ \begin{align} \mathscr{B}_k &:= \{ S \in \mathscr{R}(\mathbf{G}) \colon \left| S\cap U_k\right|> \left| S \right|/2, \left| S\cap U_{k+1}\right| \leq \left| S \right|/2 \}; \end{align} $$

here $\alpha $ and $\beta $ are positive numbers that will be specified during the proof. For each rectangle $S \in \mathscr {R}(\mathbf {G})$ , we define the tent $S_+$ over S to be the set

(4.7) $$ \begin{align} \Big\{ (\mathbf{g},\mathbf{t}) \in \mathbf{G} \times \mathbf{T}\colon \mathbf{g}\in S, 4C\operatorname{\ell}_1(S) < t_1 \leq 8C\operatorname{\ell}_1(S), 4C\operatorname{\ell}_2(S) < t_2 \leq 8C\operatorname{\ell}_2(S)\Big\}. \end{align} $$

The constant C is as in Theorem 2.2. This definition implies that, if $(\mathbf {h},\mathbf {t}) \in S_+$ , then

(4.8) $$ \begin{align} S \subseteq B_1(h_1,4C\operatorname{\ell}_1(S)) \times B_2(h_2,4C\operatorname{\ell}_2(S)) \subseteq B_1(h_1,t_1) \times B_2(h_2,t_2), \end{align} $$

and so $S_+ \subseteq \Gamma (g)$ for all $g \in S$ .

Further, the “upper half space” $\mathbf {T} \times \mathbf {G}$ is the disjoint union of all tents $S_+$ as S runs over $\mathscr {R}(\mathbf {G})$ .

By definition $U_k \supseteq U_{k+1}$ , so $\left | S\cap U_k\right | / \left | S \right | \geq \left | S\cap U_{k+1}\right | / \left | S \right |$ for all $k \in \mathbb {Z}$ . This implies that the sets $\mathscr {B}_k$ are pairwise disjoint. Indeed, if $S \in \mathscr {B}_k$ , then $\left | S\cap U_k\right |/ \left | S \right |> 1/2$ , which implies that $\left | S\cap U_j\right |/ \left | S \right |> 1/2$ whenever $j \leq k$ , so $S \notin \mathscr {B}_j$ if $j < k$ . Likewise, $\left | S\cap U_{k+1} \right |/ \left | S \right | \leq 1/2$ , which implies that $\left | S\cap U_j\right |/ \left | S \right |> 1/2$ whenever $j \geq k+1$ , so $S \notin \mathscr {B}_j$ if $j> k$ .

By Chebyshev’s inequality, $\left | U_k \right | \lesssim 2^{-2k}$ , so $\lim _{k \to \infty } \left | S\cap U_k\right |/ \left | S \right | =0$ for all $S \in \mathscr {R}(\mathbf {G})$ . If $\mathbf {G} \setminus \bigcup _{k\in \mathbb {Z}} U_k$ were a null set, then it would follow that $\lim _{k \to -\infty } \left | S\cap U_k\right |/ \left | S \right | =1$ for all $S \in \mathscr {R}(\mathbf {G})$ , and $\mathscr {R}(\mathbf {G})$ would be the disjoint union of the sets $\mathscr {B}_k$ as k runs over $\mathbb {Z}$ . However, except in special cases, we do not know that $\mathbf {G} \setminus \bigcup _{k \in \mathbb {Z}} U_k$ is null, and we must take into account the possibility that some $S \in \mathscr {R}(\mathbf {G})$ do not belong to any $\mathscr {B}_k$ . When $S \notin \bigcup _{k \in \mathbb {Z}} U_k$ , it is evident that $\left | S\cap U_{k} \right |/ \left | S \right | \leq 1/2$ for all $k \in \mathbb {Z}$ , or equivalently,

$$\begin{align*}\left| S\cap U_k^c\right|> \left| S \right|/2 \qquad\mathrm{for\ all\ } k\in\mathbb Z. \end{align*}$$

We claim that, if $S \notin \bigcup _{k \in \mathbb {Z}} U_k$ , then $(f \ast {{q}}_{\mathbf {t}} )(\mathbf {h}) =0$ for almost all $(\mathbf {h},\mathbf {t})\in S_+$ . To see this, we define the set $Z =\{ \mathbf {g}\in \mathbf {G}: \mathcal {S}_{q,1}(f)(\mathbf {g})=0\}$ , and note that $\bigcap _{k\in \mathbb Z} U_k^c=Z$ . Since $\mathcal {S}_{q,1}(f)$ is lower semicontinuous, Z is closed. Then

$$\begin{align*}|S\cap Z| = \lim_{k\to-\infty }|S\cap U_k^c|\geq \frac{\left| S \right|}{2}. \end{align*}$$

By definition, for every $\mathbf {g}\in Z$ ,

which implies that $(f \ast {{q}}_{\mathbf {t}} )(\mathbf {h})=0$ for almost all $(\mathbf {h},\mathbf {t})\in \Gamma (\mathbf {g})$ . Hence

(4.9) $$ \begin{align} \begin{aligned} &\frac{\left| S \right|}{2}\iint_{S_+} \left| (f \ast {{q}}_{\mathbf{t}} )(\mathbf{h}) \right|{}^{2} \,\mathrm{d}\mathbf{h} \,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}} \\ &\qquad\leq \left| S \cap Z \right| \iint_{S_+} \left| (f \ast {{q}}_{\mathbf{t}} )(\mathbf{h}) \right|{}^{2} \,\mathrm{d}\mathbf{h} \,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}}\\ &\qquad\leq\int_{S \cap Z} \iint_{\Gamma(\mathbf{g}) } \left| (f \ast {{q}}_{\mathbf{t}} )(\mathbf{h}) \right|{}^{2} \,\mathrm{d}\mathbf{h} \,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}}\,\,\mathrm{d}\mathbf{g}\\ &\qquad\leq \int_{Z } \iint_{\Gamma(\mathbf{g})}\left| (f \ast {{q}}_{\mathbf{t}} )(\mathbf{h}) \right|{}^{2} \,\mathrm{d}\mathbf{h} \,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}}\,\,\mathrm{d}\mathbf{g} \\ &\qquad=0, \end{aligned} \end{align} $$

so that $(f \ast {{q}}_{\mathbf {t}} )(\mathbf {h}) =0$ for almost all $(\mathbf {h},\mathbf {t})\in S_+$ , as claimed.

Since $f\in \mathsf {L}^{2}(\mathbf {G})$ , the reproducing formula implies that

$$ \begin{align*} f(\mathbf{g}) &= \int_{\mathbf{T}} (f \ast {{q}}_{\mathbf{t}} \ast \varphi_{\mathbf{t}})(\mathbf{g}) \,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}} \\ &= \int_{\mathbf{T}} \int_{\mathbf{G}} (f \ast {{q}}_{\mathbf{t}})(\mathbf{h}) \varphi_{\mathbf{t}}(\mathbf{h}^{-1}\mathbf{g}) \,\mathrm{d} \mathbf{h} \,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}} \\ &= \sum_{S\in \mathscr{R}(\mathbf{G})} \iint_{S_+} (f \ast {{q}}_{\mathbf{t}})(\mathbf{h}) \varphi_{\mathbf{t}}(\mathbf{h}^{-1}\mathbf{g}) \,\mathrm{d} \mathbf{h} \,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}} \\ &= \sum_{k \in \mathbb{Z}}\sum_{S\in \mathscr{B}_k} \iint_{S_+} (f \ast {{q}}_{\mathbf{t}})(\mathbf{h}) \varphi_{\mathbf{t}}(\mathbf{h}^{-1}\mathbf{g}) \,\mathrm{d} \mathbf{h} \,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}} \\ &= \sum_{k \in \mathbb{Z}} a_k(\mathbf{g}). \end{align*} $$

The third equality holds since the upper half-space $\mathbf {T} \times \mathbf {G}$ is the disjoint union of the tents $S_+$ . The fourth equality holds because the $\mathscr {B}_k$ are disjoint, and if $S \in \mathscr {R}(\mathbf {G}) \setminus \big (\bigcup _{k\in \mathbb {Z}} \mathscr {B}_k \big )$ , then $(f \ast q_{\mathbf {t}})(\mathbf {h}) = 0$ for almost all $(\mathbf {t}, \mathbf {h}) \in S_+$ .

If $\mathbf {g} \in S \in \mathscr {B}_k$ , then $|S \cap U_k | / \left | S \right |> 1/2$ so $\mathbf {g} \in U^{*}_k$ by Lemma 3.1 provided that $\alpha < C(\mathbf {G})/2$ ; coupled with (4.6), this shows that

(4.10) $$ \begin{align} \left| S \cap (U^{*}_{k} \setminus U_{k+1}) \right| = \left| S \setminus (S \cap U_{k+1}) \right| \geq \frac{1}{2} \left| S \right|. \end{align} $$

For future use, we note that, for any measurable set V in $\mathbf {G}$ ,

(4.11) $$ \begin{align} \begin{aligned} (\chi_V * \chi_{\mathbf{P}(\mathbf{o},\mathbf{t})})(\mathbf{h}) &= \int_{\mathbf{G}} \chi_V(\mathbf{g}) \chi_{\mathbf{P}(\mathbf{o},\mathbf{t})} (\mathbf{g}^{-1}\mathbf{h}) \,\mathrm{d}\mathbf{g} = \int_{\mathbf{G}} \chi_V(\mathbf{g}) \chi_{\mathbf{P}(\mathbf{h},\mathbf{t})} (\mathbf{g}) \,\mathrm{d}\mathbf{g} \\ &= \left| \mathbf{P}(\mathbf{h}, \mathbf{t}) \cap V \right|. \end{aligned} \end{align} $$

Next, for all $S\in \mathscr {B}_k$ , choose $\tilde {S}\in \mathscr {M}(U^{*}_k)$ such that $S\subseteq \tilde {S}$ . Then for all $R\in \mathscr {M}(U^{*}_k)$ , set $a_{k,R}=\sum _{S \in \mathscr {B}_k\colon \tilde {S}=R } b_{k,S}$ , where

$$\begin{align*}b_{k,S}(\mathbf{g}) = \iint_{S_+} (f \ast {{q}}_{\mathbf{t}})(\mathbf{h}) \varphi_{\mathbf{t}}(\mathbf{h}^{-1}\mathbf{g}) \,\mathrm{d} \mathbf{h} \,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}}. \end{align*}$$

By construction,

$$\begin{align*}a_k= \sum_{R\in \mathscr{M}(U^{*}_k)} a_{k,R}. \end{align*}$$

Now

$$\begin{align*}\operatorname{supp}(\varphi_{\mathbf{t}}) \subseteq B(o,t_1) \times B(o,t_2)\subseteq B_1(o,8C\operatorname{\ell}_1(S)) \times B_2(o,8C\operatorname{\ell}_2(S)) , \end{align*}$$

and hence

$$ \begin{align*} \operatorname{supp}( b_{k,S}) &\subseteq B_1(z_1(S),10C\operatorname{\ell}_1(S)) \times B_2(z_2(S),10C\operatorname{\ell}_2(S)) \\ &= B_1(z_1(S),\beta c\operatorname{\ell}_1(S)/3) \times B_2(z_2(S),\beta c\operatorname{\ell}_2(S)/3) \\ &\subseteq \beta S, \end{align*} $$

where $\beta = 30 C/c$ . Thus $a_k$ vanishes on $(U^{\dagger }_k)^c$ (recall that $U^{\dagger }_k$ is defined in (4.5)). By Lemmas 3.5 and 3.1 and Corollary 4.3,

(4.12) $$ \begin{align} |U^{\dagger}_k|\lesssim|U^{*}_k|\lesssim|U_k|\lesssim F_\Phi(2^{-k}f). \end{align} $$

Next, we claim that for all $k\in \mathbb {Z}$ ,

(4.13) $$ \begin{align} \|a_k\|_{\mathsf{L}^{2}(\mathbf{G})}^2\lesssim 2^{2k} F_\Phi(2^{-k} f). \end{align} $$

To see this, take $h \in \mathsf {L}^{2}(\mathbf {G})$ such that $\|h\|_{\mathsf {L}^{2}(\mathbf {G})}=1$ ; then, writing $\check \varphi $ for the reflected version of $\varphi $ , that is, $\check \varphi (\mathbf {g}) = \varphi (\mathbf {g}^{-1})$ , we see that

$$ \begin{align*} \big|\langle a_k,h \rangle\big| &= \bigg|\sum_{S\in \mathscr{B}_k} \iint_{S_+} (f \ast {{q}}_{\mathbf{t}})(\mathbf{g}) (h \ast \check\varphi_{\mathbf{t}})(\mathbf{g}) \,\mathrm{d} \mathbf{g} \,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}}\bigg|\\[5pt] &\leq \bigg( \sum_{S\in \mathscr{B}_k} \iint_{S_+} |(f \ast {{q}}_{\mathbf{t}})(\mathbf{g})|^2 \,\mathrm{d} \mathbf{g}\,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}} \bigg)^{1/2} \bigg( \sum_{S\in \mathscr{B}_k} \iint_{S_+} |(h \ast \check\varphi_{\mathbf{t}})(\mathbf{g})|^2 \,\mathrm{d} \mathbf{g} \,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}} \bigg)^{1/2} \\[5pt] &\lesssim \bigg( \sum_{S\in \mathscr{B}_k} \iint_{S_+} |(f \ast {{q}}_{\mathbf{t}})(\mathbf{g})|^2 \,\mathrm{d} \mathbf{g} \,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}} \bigg)^{1/2}. \end{align*} $$

By definition, (4.10), (4.7), (4.8), a convolution identity and (4.12),

$$ \begin{align*} &\sum_{S\in \mathscr{B}_k} \iint_{S_+} |(f \ast {{q}}_{\mathbf{t}})(\mathbf{h})|^2 \,\mathrm{d} \mathbf{h}\,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}} \\ &\qquad\leq 2\sum_{S\in \mathscr{B}_k} \iint_{S_+} \frac{\big| R \cap (U^{*}_{k} \setminus U_{k+1}) \big|}{\left| S \right|} \left| (f \ast {{q}}_{\mathbf{t}})(\mathbf{h})\right|{}^2 \,\mathrm{d} \mathbf{h}\,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}} \\ &\qquad\lesssim \sum_{S\in \mathscr{B}_k} \iint_{S_+} \frac{\big| \mathbf{P}(\mathbf{h},\mathbf{t}) \cap (U^{*}_{k} \setminus U_{k+1}) \big|}{|\mathbf{P}(\mathbf{o},\mathbf{t})|} |(f \ast {{q}}_{\mathbf{t}})(\mathbf{h})|^2 \,\mathrm{d} \mathbf{h}\,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}} \\ &\qquad {\ \leq} \iint_{\mathbf{G} \times \mathbf{T}} \frac{\big| \mathbf{P}(\mathbf{h},\mathbf{t}) \cap (U^{*}_{k} \setminus U_{k+1}) \big|}{|\mathbf{P}(\mathbf{o},\mathbf{t})|} |(f \ast {{q}}_{\mathbf{t}})(\mathbf{h})|^2 \,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}} \,\mathrm{d}\mathbf{h}\\ &\qquad= \iint_{\mathbf{G} \times \mathbf{T}} \Big(\chi_{U^{*}_{k} \setminus U_{k+1}} * \frac{\chi_{\mathbf{P}(\mathbf{o},\mathbf{t})}}{|\mathbf{P}(\mathbf{o},\mathbf{t})|}\Big)(\mathbf{h}) |(f \ast {{q}}_{\mathbf{t}})(\mathbf{h})|^2 \,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}} \,\mathrm{d}\mathbf{h} \\ &\qquad= \int_{\mathbf{G}}\chi_{U^{*}_{k} \setminus U_{k+1}} (\mathbf{h}) \int_{\mathbf{T}} \bigg(\left| f \ast {{q}}_{\mathbf{t}}\right|{}^2 * \frac{\chi_{\mathbf{P}(\mathbf{o},\mathbf{t})}}{|\mathbf{P}(\mathbf{o},\mathbf{t})|}\bigg)(\mathbf{h}) \,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}} \,\mathrm{d} \mathbf{h} \\\ &\qquad=\int_{U^{*}_k\backslash U_{k+1}} |\mathcal{S}_{q,1}(f)(\mathbf{h})|^2\,\mathrm{d} \mathbf{h} \\ &\qquad\leq 2^{2k}|U^{*}_k|\\ &\qquad\lesssim 2^{2k}F_\Phi(2^{-k}f). \end{align*} $$

This shows that $|\langle a_k,h\rangle |^2 \lesssim 2^{2k}F_\Phi (2^{-k}f)$ , whence $\|a_k\|_{\mathsf {L}^{2}(\mathbf {G})}^2\lesssim 2^{2k}F_\Phi (2^{-k}f)$ , and (4.13) holds.

We now claim that

(4.14) $$ \begin{align} \sum_{R\in \mathscr{M}(U^*_k)} \|a_{k,R}\|^2_{\mathsf{L}^{2}(\mathbf{G})} &\lesssim 2^{2k}F_\Phi(2^{-k}f). \end{align} $$

Arguing as above, we see that, if $h\in \mathsf {L}^{2}(\mathbf {G})$ and $\|h\|_{\mathsf {L}^{2}(\mathbf {G})}=1$ , then

$$ \begin{align*} \big|\langle a_{k,R},h\rangle\big| &= \bigg| \sum_{S \in \mathscr{B}_k(R)} \iint_{S_+} (f \ast {{q}}_{\mathbf{t}})(\mathbf{h})\ (h \ast \check\varphi_{\mathbf{t}})(\mathbf{h}) \,\mathrm{d} \mathbf{h} \,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}}\bigg|\\[5pt] &\leq \bigg( \sum_{S \in \mathscr{B}_k(R)} \iint_{S_+} |(f \ast {{q}}_{\mathbf{t}})(\mathbf{h})|^2 \,\mathrm{d} \mathbf{h} \,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}} \bigg)^{1/2} \\&\hspace{2.5cm}\times \bigg( \sum_{S \in \mathscr{B}_k(R)}\iint_{S_+} |(h \ast \check\varphi_{\mathbf{t}})(\mathbf{h})|^2 \,\mathrm{d} \mathbf{h} \,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}} \bigg)^{1/2}\\[5pt] &\lesssim \bigg( \sum_{S \in \mathscr{B}_k(R)} \iint_{S_+} |(f \ast {{q}}_{\mathbf{t}})(\mathbf{h})|^2 \,\mathrm{d} \mathbf{h}\,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}} \bigg)^{1/2}, \end{align*} $$

where $\mathscr {B}_k(R) := \{ S \in \mathscr {B}_k\colon \tilde {S}=R \}$ . Hence

$$\begin{align*}\left\Vert a_{k,R} \right\Vert{}_{\mathsf{L}^{2}(\mathbf{G})}\leq\bigg( \sum_{S \in \mathscr{B}_k(R)}\iint_{S_+} |(f \ast {{q}}_{\mathbf{t}})(\mathbf{h})|^2 \,\mathrm{d}\mathbf{h} \,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}} \bigg)^{1/2}, \end{align*}$$

which implies that

$$\begin{align*}\sum_{R\in \mathscr{M}(U^*_k)} \left\Vert a_{k,R} \right\Vert{}^2_{\mathsf{L}^{2}(\mathbf{G})} \leq \sum_{S\in \mathscr{B}_k} \iint_{S_+} |(f \ast {{q}}_{\mathbf{t}})(\mathbf{h})|^2 \,\mathrm{d} \mathbf{h} \,\frac{\mathrm{d}\mathbf{t}}{\mathbf{t}}. \end{align*}$$

Then, much as argued to prove (4.13), we deduce (4.14). The unconditional convergence of the sum follows as in [Reference Chen, Cowling, Lee, Li and Ottazzi2].

5.1 Proof of Theorem 1.1 for area functions $\mathcal {S}_{\psi ,\eta }(f)$

Recall from Section 4.2 that $\mathsf {L}^{2}(\mathbf {G}) \cap \mathsf {L\,log\,L} (\mathbf {G})$ is dense in $ \mathsf {L\,log\,L} (\mathbf {G})$ . Now we take a general $\psi $ satisfying the decay, smoothness and cancellation conditions (2.1) and (2.2), and prove that

(5.3) $$ \begin{align} \big|\big\{ \mathbf{g}\in \mathbf{G}\colon \mathcal{S}_{\psi,\eta }(f)(\mathbf{g})>\lambda \big\}\big| \lesssim F_\Phi(f/\lambda) \qquad\forall \lambda \in \mathbb{R}_{+} \end{align} $$

for all $f\in \mathsf {L\,log\,L} (\mathbf {G})$ . By sublinearity and density, it suffices to prove that

(5.4) $$ \begin{align} \big|\big\{ \mathbf{g}\in \mathbf{G}\colon \mathcal{S}_{\psi,\eta }(f)(\mathbf{g})>1 \big\}\big| \lesssim F_\Phi(f) \end{align} $$

for all $f\in \mathsf {L}^{2}(\mathbf {G}) \cap \mathsf {L\,log\,L} (\mathbf {G})$ . We may and shall suppose that $\eta> 1$ .

From Theorem 4.4, we may write

$$\begin{align*}f=\sum_{k\in \mathbb Z}a_k , \end{align*}$$

where the $a_k$ satisfy the conditions of the lemma.

Observe that

$$ \begin{align*} \Big\Vert \sum_{k \leq 0}a_k\Big\Vert_{\mathsf{L}^{2}(\mathbf{G})} &\le \sum_{k \leq 0} \left\Vert a_k \right\Vert{}_{\mathsf{L}^{2}(\mathbf{G})} \lesssim \sum_{k \leq 0} 2^{k}F_\Phi(2^{-k}f)^{1/2} \lesssim F_\Phi(f)^{1/2}, \end{align*} $$

where the last inequality follows from Lemma 2.7.

Thus, by the $\mathsf {L}^{2}(\mathbf {G})$ boundedness of $\mathcal {S}_{\psi ,\eta }$ ,

(5.5) $$ \begin{align} \bigg|\bigg\{\mathbf{g}\in \mathbf{G}\colon \mathcal{S}_{\psi,\eta }\bigg(\sum_{k \leq 0}a_k\bigg)(\mathbf{g})>1\bigg\}\bigg| \lesssim \Big\Vert \sum_{k \leq 0}a_k\Big\Vert_{\mathsf{L}^{2}(\mathbf{G})}^2 \lesssim F_\Phi(f). \end{align} $$

To handle $\mathcal {S}_{\psi ,\eta }(\sum _{k \geq 1} a_k)$ , we use the fine structure of the atoms $a_k$ : each $a_k$ is supported in $U_k^\dagger $ , where $|U_k^\dagger |\lesssim 2^{-k}F_\Phi (f)$ , and $a_k=\sum _{R\in \mathscr {M}(U^*_k)} a_{k,R}$ .

For all $R = {Q_1} \times {Q_2} \in \mathscr {M}(U^*_k)$ , let ${\tilde {Q}_1}$ be the biggest pseudodyadic cube containing ${Q_1}$ such that ${\tilde {Q}_1}\times {Q_2}\subset U^{**}_k$ , where $U^{**}_k$ is defined in (5.1). Next, let ${\tilde {Q}_2}$ be the biggest pseudodyadic cube containing ${Q_2}$ such that ${\tilde {Q}_1}\times {\tilde {Q}_2} \subseteq U^{***}_k$ , where $U^{***}_k$ is defined in (5.2). Finally, let $R^{\dagger }$ be $100\beta _k({\tilde {Q}_1} \times {\tilde {Q}_2})$ , where $\beta _k = 2^{k/ (2\nu _1+2\nu _2)}$ . By Lemmas 3.5 and 3.1 and Theorem 4.4,

(5.6) $$ \begin{align} \begin{aligned} \bigg| \bigcup_{R \in \mathscr{M}(U^*_k)} R^{\dagger}\bigg| &\lesssim2^{k/2} |U_k^{\dagger}| \lesssim2^{k/2} |U_k^{***}| \lesssim2^{k/2} |U_k^{**}| \\ &\lesssim2^{k/2} |U_k^*| \lesssim2^{k/2} |U_k| \lesssim 2^{k/2} F_\Phi(2^{-k} f). \end{aligned} \end{align} $$

We claim that there exists $\delta \in \mathbb {R}_{+}$ such that

(5.7) $$ \begin{align} \sum_{R\in \mathscr{M}(U^*_k)} \int_{(R^{\dagger})^c} |\mathcal{S}_{\psi,\eta }(a_{k,R})(\mathbf{g})|\,\mathrm{d}\mathbf{g} \lesssim 2^{-\delta k} F_\Phi(f) \qquad\forall k \in \mathbb{N}. \end{align} $$

Assume (5.7) for the moment, and let $E^\dagger :=\bigcup _{k \geq 1}\bigcup _{R \in \mathscr {M}(U^*_k)} R^{\dagger }$ . Then on the one hand,

$$ \begin{align*} |E^\dagger| = \Big|\bigcup_{k \geq 1}\bigcup_{R \in \mathscr{M}(U^*_k)} R^{\dagger}\Big| &\lesssim\sum_{k \geq 1} 2^{k/2} F_\Phi(2^{-k} f) \lesssim F_\Phi(f) \end{align*} $$

by (5.6). On the other hand,

$$ \begin{align*} &\int_{(E^\dagger)^c}\Big|\mathcal{S}_{\psi,\eta }\Big( \sum_{k \geq 1}a_k\Big)(\mathbf{g})\Big|\,\mathrm{d} \mathbf{g} \\ &\qquad\lesssim \int_{(E^\dagger)^c}\Big|\mathcal{S}_{\psi,\eta }\Big( \sum_{k \geq 1}\sum_{R\in \mathscr{M}(U^*_k)} a_{k,R}\Big)(\mathbf{g})\Big|\,\mathrm{d} \mathbf{g} \\ &\qquad\lesssim \sum_{k \geq 1}\sum_{R\in \mathscr{M}(U^*_k)} \int_{(E^\dagger)^c}\Big|\mathcal{S}_{\psi,\eta }\Big( a_{k,R}\Big)(\mathbf{g})\Big|\,\mathrm{d} \mathbf{g} \\ &\qquad\lesssim F_\Phi(f), \end{align*} $$

where the last inequality follows by summing (5.7) over positive k.

By Chebychev’s inequality,

$$ \begin{align*} \Big|\Big\{ \mathbf{g} \in \mathbf{G} \colon\mathcal{S}_{\psi,\eta }\Big( \sum_{k \geq 1} a_k(\mathbf{g}) \Big)>1 \Big\} \Big| &\leq |E^\dagger|+ \int_{(E^\dagger)^c}\Big|\mathcal{S}_{\psi,\eta }\Big( \sum_{k \geq 1}a_k\Big) (\mathbf{g})\Big|\,\mathrm{d} \mathbf{g} \lesssim F_\Phi(f). \end{align*} $$

Together with (5.5), this implies (5.3).

It remains to prove (5.7). Now

$$ \begin{align*} &\int_{(R^{\dagger})^c} |\mathcal{S}_{\psi,\eta }(a_{k,R})(g_1, g_2)|\,\mathrm{d} g_2 \,\mathrm{d} g_1\\ &\quad\leq \int_{(100\beta_k{\tilde{Q}_1})^c}\int_{100{Q_2}}|\mathcal{S}_{\psi,\eta }(a_{k,R})(g_1, g_2)|\,\mathrm{d} g_2 \,\mathrm{d} g_1 \\ &\qquad +\int_{(100\beta_k{\tilde{Q}_1})^c}\int_{(100{Q_2})^c}|\mathcal{S}_{\psi,\eta }(a_{k,R})(g_1, g_2)|\,\mathrm{d} g_2 \,\mathrm{d} g_1 \\ &\qquad +\int_{(100\beta_k{\tilde{Q}_2})^c}\int_{100{Q_1}} |\mathcal{S}_{\psi,\eta }(a_{k,R})(g_1, g_2)| \,\mathrm{d} g_1 \,\mathrm{d} g_2 \\ &\qquad +\int_{(100\beta_k{\tilde{Q}_2})^c}\int_{(100{Q_1})^c} |\mathcal{S}_{\psi,\eta }(a_{k,R})(g_1, g_2)| \,\mathrm{d} g_1 \,\mathrm{d} g_2 \\ &\quad= \mathrm{I}_1(R) + \mathrm{I}_2(R) + \mathrm{I}_3(R) + \mathrm{I}_4(R), \end{align*} $$

say, where $\beta _k=2^{k/(2\nu _1+2\nu _2)} \eta $ . It suffices to control $\mathrm {I}_1(R)$ and $\mathrm {I}_2(R)$ , as the other two terms are similar.

By Hölder’s inequality and the $ \mathsf {L}^{2}(G_2)$ -boundedness of $\mathcal {S}_{\psi ^{[2]},\eta }$ ,

$$ \begin{align*} \mathrm{I}_1(R) &\lesssim |{Q_2}|^{1/2} \int_{(100\beta_k {\tilde{Q}_1})^c} \bigg(\int_{100{Q_2}} \big( \mathcal{S}_{\psi^{[1]},\eta }(a_{k,R})(g_1,g_2)\big) ^2 \,\mathrm{d} g_2\bigg)^{1/2} \,\mathrm{d} g_1. \end{align*} $$

We use the cancellation and support restrictions on $a_{k,R}$ in the first variable, (2.1), homogeneity, the geometry (namely, $h_1, z({Q_1}) \in {Q_1}$ and $g_1 \in (100\beta _k {\tilde {Q}_1})^c$ ) and Hölder’s inequality to deduce that

$$ \begin{align*} &|a_{k,R} \ast_1 \psi_t^{[1]}(g_1,g_2)| \\ &\qquad=\bigg|\int_{G_1} a_{k,R}(h_1,g_2)t^{-\nu_1} \big(\psi^{[1]}(\delta_{1/t}(h_1^{-1}g_1))-\psi^{[1]}(\delta_{1/t}(z({Q_1})^{-1}g_1))\big) \,\mathrm{d}{h}_1\bigg|\\ &\qquad\lesssim \int_{G_1} \left| a_{k,R}(h_1,g_2)\right| \frac{t^{-\nu_1} \big( \rho(\delta_{1/t}(z({Q_1})^{-1}g_1)\delta_{1/t}(h_1^{-1}g_1)^{-1}) \big)^{\epsilon}} {\big( 1 + \rho(\delta_{1/t}(h_1^{-1}g_1)) + \rho(\delta_{1/t}(z({Q_1})^{-1}g_1))\big)^{\nu_1+2\epsilon}} \,\mathrm{d}{h}_1 \\ &\qquad= \int_{G_1} \left| a_{k,R}(h_1,g_2)\right| \frac{t^{\epsilon} \big( \rho( z({Q_1})^{-1} h_1) \big)^{\epsilon}} {\big( t + \rho(h_1^{-1}g_1) + \rho(z({Q_1})^{-1}g_1)\big)^{\nu_1+2\epsilon}} \,\mathrm{d}{h}_1 \\ &\qquad\lesssim \int_{G_1} \left| a_{k,R}(h_1,g_2)\right| \frac{t^{\epsilon} \operatorname{\ell}({Q_1})^{\epsilon}} {\big( t + \rho(z({Q_1})^{-1}g_1)\big)^{\nu_1+2\epsilon}} \,\mathrm{d}{h}_1 \\ &\qquad\eqsim \frac{t^{\epsilon} \operatorname{\ell}({Q_1})^{\epsilon}} {\big( t + \rho(z({Q_1})^{-1}g_1) \big)^{\nu_1+2\epsilon}} \left\Vert a_{k,R}(\cdot,g_2) \right\Vert{}_{\mathsf{L}^{1}(G_1)}, \end{align*} $$

where $z({Q_1})$ is the center of ${Q_1}$ . Hence

(recall that denotes the “average integral”). This estimate implies that

$$ \begin{align*} \mathrm{I}_1(R) &= \int_{(100\beta_k {\tilde{Q}_1})^c}\int_{100{Q_2}}\left|\mathcal{S}_{\psi,\eta }(a_{k,R})(g_1,g_2)\right|\,\mathrm{d} g_2 \,\mathrm{d} g_1 \\ &\leq \int_{(100\beta_k {\tilde{Q}_1})^c} |100{Q_2}|^{1/2} \left( \int_{100{Q_2}}|\mathcal{S}_{\psi,\eta }(a_{k,R})(g_1,g_2)|^2\,\mathrm{d} g_2 \right) ^{1/2} \,\mathrm{d} g_1\\ &\leq \int_{(100\beta_k {\tilde{Q}_1})^c} |100{Q_2}|^{1/2} \left( \int_{G_2}|\mathcal{S}_{\psi,\eta }(a_{k,R})(g_1,g_2)|^2\,\mathrm{d} g_2 \right) ^{1/2} \,\mathrm{d} g_1\\ &\lesssim \int_{(100\beta_k {\tilde{Q}_1})^c} |{Q_2}|^{1/2} \frac{\operatorname{\ell}({Q_1})^{\epsilon} |{Q_1}|^{1/2} }{\rho_1(z({Q_1})^{-1}g_1)^{\nu_1+\epsilon}} \left\Vert a_{k,R} \right\Vert{}_{\mathsf{L}^{2}(\mathbf{G})} \,\mathrm{d} g_1\\ &= \operatorname{\ell}({Q_1})^{\epsilon} \int_{(100\beta_k {\tilde{Q}_1})^c} \frac1{\rho_1(z({Q_1})^{-1}g_1)^{\nu_1+\epsilon}} \,\mathrm{d} g_1 \left| R \right|{}^{1/2} \left\Vert a_{k,R} \right\Vert{}_{\mathsf{L}^{2}(\mathbf{G})} \\ &\eqsim_\epsilon \beta_k ^{-\epsilon} \gamma_1(R)^{-\epsilon} \left| R \right|{}^{1/2} \left\Vert a_{k,R} \right\Vert{}_{\mathsf{L}^{2}(\mathbf{G})}. \end{align*} $$

To estimate $\mathrm {I}_2(R)$ , we argue as follows. For $g_1\notin 100\beta _k {\tilde {Q}_1}$ and $g_2\notin 100 {Q_2}$ ,

and because of the cancellation of $a_{k,R}$ and properties of $\psi ^{[1]}$ and $\psi ^{[2]}$ ,

$$ \begin{align*} &\big| a_{k,R}\ast\psi_{\mathbf{t}}(\mathbf{g}) \big| \\ &\qquad= \bigg| \int_{\mathbf{G}} a_{k,R}(\mathbf{h}) \psi_{\mathbf{t}}(\mathbf{h}^{-1}\mathbf{g}) \,\mathrm{d} \mathbf{h} \bigg| \\ &\qquad= \bigg| \int_{\mathbf{G}} a_{k,R}(h_1,h_2) \psi^{[1]}_{t_1}(h_1^{-1}g_1) \psi^{[2]}_{t_2}(h_2^{-1}g_2) \,\mathrm{d} h_2 \,\mathrm{d} h_1 \bigg| \\ &\qquad= \bigg| \int_{\mathbf{G}} a_{k,R}(h_1,h_2) \big( \psi^{[1]}_{t_1}(h_1^{-1}g_1) - \psi^{[1]}_{t_1}(z({Q_1})^{-1}g_1) \big)\\ &\qquad\qquad\qquad\qquad \times\big( \psi^{[2]}_{t_2}(h_2^{-1}g_2) -\psi^{[2]}_{t_2}z({Q_2})^{-1}g_2)\big)\,\mathrm{d} h_2 \,\mathrm{d} h_1 \bigg| \\ &\qquad\leq \left\Vert a_{k,R} \right\Vert{}_{\mathsf{L}^{1}(\mathbf{G})} \sup\left\{ \left| \psi^{[1]}_{t_1}(h_1^{-1}g_1) - \psi^{[1]}_{t_1}(z({Q_1})^{-1}g_1) \right| \colon h_1 \in {Q_1}, g_1 \in (100b {Q_1})^c \right\}\\ &\qquad\qquad\qquad \times\sup\left\{ \left| \psi^{[2]}_{t_2}(h_2^{-1}g_2) - \psi^{[2]}_{t_2}(z({Q_2})^{-1}g_2) \right| \colon h_2 \in {Q_2}, g_2 \in (100b {Q_2})^c \right\}\\ &\qquad\lesssim \left\Vert a_{k,R} \right\Vert{}_{\mathsf{L}^{1}(\mathbf{G})} \frac{t_1^{\epsilon} \operatorname{\ell}({Q_1})^{\epsilon}}{\big( t_1 + \rho_1(g_1^{-1}z({Q_1}))\big)^{\nu_1+2\epsilon}} \frac{t_2^{\epsilon}\operatorname{\ell}({Q_2})^{\epsilon}}{\big( t_2 + \rho_2(g_2^{-1}z({Q_2}))\big)^{\nu_2+2\epsilon}} \,, \end{align*} $$

by the same geometrical arguments as used to treat $\mathrm {I}_1(R)$ , applied in both variables. Hence $\big (\mathcal {S}_{\psi ,\eta }(a_{k,R})(\mathbf {g})\big )^2$ is dominated by a multiple of $\left \Vert a_{k,R} \right \Vert {}_{\mathsf {L}^{1}(\mathbf {G})} $ by

Our estimation of $\mathrm {I}_2(R)$ concludes with the observation that

$$ \begin{align*} \mathrm{I}_2(R) &= \int_{(100\beta_k {\tilde{Q}_1})^c}\int_{(100{Q_2})^c}| \mathcal{S}_{\psi,\eta }(a_{k,R})(g_1,g_2)|\,\mathrm{d} g_2 \,\mathrm{d} g_1 \\ &\lesssim \left\Vert a_{k,R} \right\Vert{}_{\mathsf{L}^{1}(\mathbf{G})} \int_{(100\beta_k {\tilde{Q}_1})^c}\int_{(100{Q_2})^c}\\&\qquad \frac{ \operatorname{\ell}({Q_1})^{\epsilon}}{\big( \rho_1(g_1^{-1}z({Q_1}))\big)^{\nu_1+\epsilon}} \frac{ \operatorname{\ell}({Q_2})^{\epsilon}}{\big( \rho_2(g_2^{-1}z({Q_2}))\big)^{\nu_2+\epsilon}} \,\mathrm{d} g_2 \,\mathrm{d} g_1 \\ &\lesssim \frac{ \operatorname{\ell}({Q_1})^{\epsilon}}{(100 \beta_k \operatorname{\ell}({\tilde{Q}_1}))^{\epsilon}} \frac{ \operatorname{\ell}({Q_2})^{\epsilon}}{(100 \operatorname{\ell}({Q_2}))^{\epsilon}} \left| R \right|{}^{1/2} \left\Vert a_{k,R} \right\Vert{}_{\mathsf{L}^{2}(\mathbf{G})} \\ &\eqsim \beta_k ^{-\epsilon} \gamma_1(R)^{-\epsilon} \left| R \right|{}^{1/2} \left\Vert a_{k,R} \right\Vert{}_{\mathsf{L}^{2}(\mathbf{G})}. \end{align*} $$

Then, by combining the estimates of $\mathrm {I}_1(R)$ and $\mathrm {I}_2(R)$ , and then using Hölder’s inequality and Lemma 2.5, we see that for all $k \geq 1$ ,

$$ \begin{align*} &\sum_{R\in \mathscr{M}(U^*_k)} \mathrm{I}_1(R) + \mathrm{I}_2(R) \\ &\qquad\lesssim \beta_k ^{-\epsilon} \sum_{R\in \mathscr{M}(U^*_k)} \gamma_1(R)^{-\epsilon} \left| R \right|{}^{1/2} \left\Vert a_{k,R} \right\Vert{}_{\mathsf{L}^{2}(\mathbf{G})}\\ &\qquad\leq \beta_k ^{-\epsilon}\bigg(\sum_{R\in \mathscr{M}(U^*_k)} \gamma_1(R)^{-2\epsilon}\left| R \right| \bigg)^{1/2}\bigg(\sum_{R\in \mathscr{M}(U^*_k)} \left\Vert a_{k,R} \right\Vert{}^2_{\mathsf{L}^{2}(\mathbf{G})}\bigg)^{1/2}\\ &\qquad\lesssim_\epsilon \beta_k ^{-\epsilon} |U_k|^{1/2} \Big( 2^{2k} F_\Phi(2^{-k} f) \Big) ^{1/2}\\ &\qquad\lesssim \beta_k ^{-\epsilon} 2^{k} F_\Phi(2^{-k} f)\\ &\qquad\lesssim 2^{-\epsilon k/(2\nu_1+2\nu_2)} F_\Phi(f). \end{align*} $$

Similar estimates hold for $\mathrm {I}_3(R)$ and $\mathrm {I}_4(R)$ , but with $\gamma _2$ in place of $\gamma _1$ , and hence (5.7) holds, and the proof is complete.

5.2 Proof of Theorem 1.1 for square functions $\mathcal {S}_{\psi ,0}(f)$

Arguing much as in Section 5.1, we may also check that

(5.8) $$ \begin{align} \big|\big\{ \mathbf{g}\in \mathbf{G}\colon \mathcal{S}_{\psi,\eta }(f)(\mathbf{g})>\lambda \big\}\big| \lesssim F_\Phi(f/\lambda) \qquad\forall\lambda\in\mathbb{R}_{+} \end{align} $$

for all $f\in \mathsf {L\,log\,L} (\mathbf {G})$ . We leave the details to the reader.

5.3 Hyperweak boundedness of Riesz transformations

We write $\mathcal {R}$ for a double Riesz transformation $\mathcal {R}^{[1]}_{j_1} \otimes \mathcal {R}^{[2]}_{j_2}$ .

Theorem 5.1 The double Riesz transformations satisfy the endpoint estimate:

(5.9) $$ \begin{align} \big|\big\{ \mathbf{g}\in \mathbf{G}\colon \mathcal{R}(f)(\mathbf{g})>\lambda \big\} \lesssim F_\Phi(f/\lambda) \qquad\forall\lambda\in\mathbb{R}_{+} \end{align} $$

for all $f\in \mathsf {L\,log\,L} (\mathbf {G})$ .

Proof By density and sublinearity, we need only prove that, for all $f\in \mathsf {L}^{2}(\mathbf {G}) \cap \mathsf {L\,log\,L} (\mathbf {G})$ ,

(5.10) $$ \begin{align} \big|\big\{ \mathbf{g}\in \mathbf{G}\colon \mathcal{R}(f)(\mathbf{g})>1 \big\}\big| \lesssim F_\Phi(f). \end{align} $$

From Theorem 4.4, we may write

$$\begin{align*}f=\sum_{k\in \mathbb Z}a_k \end{align*}$$

where the $a_k$ satisfy the conditions specified there. By the $\mathsf {L}^{2}(\mathbf {G})$ boundedness of $\mathcal {R}$ ,

(5.11) $$ \begin{align} \Big|\Big\{\mathbf{g}\in \mathbf{G}\colon \mathcal{R}\bigg(\sum_{k \leq 0} a_k\bigg)(\mathbf{g})>1 \Big\}\Big| \lesssim \Big\|\sum_{k \leq 1}a_k\Big\|_{\mathsf{L}^{2}(\mathbf{G})}^2 \lesssim F_\Phi(f). \end{align} $$

To handle $\mathcal {R}(\sum _{k \geq 1} a_k)$ , we consider the $a_k$ in more detail. The atom $a_k$ is supported in $U^\dagger _k$ , where $|U^\dagger _k|\lesssim F_\Phi (2^{-k} f)$ , and we may write $a_k=\sum _{R\in \mathscr {M}(U^*_k)} a_{k,R}$ .

Again, for all $R = {Q_1} \times {Q_2} \in \mathscr {M}(U^*_k)$ , let ${\tilde {Q}_1}$ be the biggest pseudodyadic cube containing ${Q_1}$ such that ${\tilde {Q}_1}\times {Q_2}\subset U^{**}_k$ , where $U^{**}_k$ is defined in (5.1). Next, let ${\tilde {Q}_2}$ be the biggest pseudodyadic cube containing ${Q_2}$ such that ${\tilde {Q}_1}\times {\tilde {Q}_2} \subseteq U^{***}_k$ , where $U^{***}_k$ is defined in (5.2). Now let $R^{\dagger }$ be $100\beta _k({\tilde {Q}_1} \times {\tilde {Q}_2})$ , where $\beta _k = 2^{k/ (2\nu _1+2\nu _2)}$ . By Lemmas 3.5 and 3.1 and Theorem 4.4,

$$ \begin{align*} \bigg| \bigcup_{R \in \mathscr{M}(U^*_k)} R^{\dagger}\bigg| &\lesssim 2^{k/2} |U_k^{***}| \lesssim 2^{k/2} |U_k^{**}| \lesssim 2^{k/2} |U_k^*| \lesssim 2^{k/2} |U_k| \lesssim 2^{k/2} F_\Phi(2^{-k} f). \end{align*} $$

As in the proof of Theorem 1.1, to prove that

(5.12) $$ \begin{align} \bigg|\bigg\{\mathbf{g}\in \mathbf{G}\colon \mathcal{R}\bigg(\sum_{k \geq 1} a_k\bigg)(\mathbf{g})>1\bigg\}\bigg| \lesssim F_\Phi(f), \end{align} $$

it suffices to show that, for some $\delta \in \mathbb {R}_{+}$ ,

(5.13) $$ \begin{align} \sum_{R\in \mathscr{M}(U^*_k)} \int_{(R^{\dagger})^c} |\mathcal{R}(a_{k,R})(\mathbf{g})|\,\mathrm{d} \mathbf{g} \lesssim 2^{-\delta k} F_\Phi(f) \end{align} $$

for every $k \geq 1$ . If (5.13) holds, then (5.11) and (5.12) imply (5.10). We now prove (5.13).

$$ \begin{align*} \int_{(R^{\dagger})^c} |\mathcal{R}(a_{k,R})(\mathbf{g})|\,\mathrm{d} \mathbf{g} &\le \iint_{g_1\notin 100\beta_k{\tilde{Q}_1}} |\mathcal{R}(a_{k,R})(\mathbf{g})|\,\mathrm{d} \mathbf{g} + \iint_{g_2\notin100\beta_k{\tilde{Q}_2}} |\mathcal{R}(a_{k,R})(\mathbf{g})|\,\mathrm{d} \mathbf{g} \\ &=:\mathrm{I}_1(R) + \mathrm{I}_2(R), \end{align*} $$

say, where $\beta _k=2^{{k/(2\nu _1+2\nu _2)}}$ . Since the estimates of $\mathrm {I}_1(R)$ and $\mathrm {I}_2(R)$ are symmetric, we only estimate $\mathrm {I}_1(R)$ . Note that

$$ \begin{align*} \mathrm{I}_1(R)&=\int_{g_1\notin 100\beta_k{\tilde{Q}_1}}\int_{100{Q_2}}|\mathcal{R}(a_{k,R})(\mathbf{g})|\,\mathrm{d} \mathbf{g}+\int_{g_1\notin 100\beta_k{\tilde{Q}_1}}\int_{(100{Q_2})^c}|\mathcal{R}(a_{k,R})(\mathbf{g})|\,\mathrm{d} \mathbf{g} \\ &=:\mathrm{I}_{11}(R)+\mathrm{I}_{12}(R), \end{align*} $$

say. By Hölder’s inequality and the $\mathsf {L}^{2}(G_2)$ -boundedness of $\mathcal {R}^{[2]}_{j_2}$ ,

$$ \begin{align*} \mathrm{I}_{11}(R) &\lesssim |{\tilde{Q}_2}|^{1/2}\int_{g_1\notin 100\beta_k{\tilde{Q}_1}} \bigg(\int_{Q_2} \big( \mathcal{R}^{[1]}_{j_1}(a_{k,R})(g_1,g_2)\big) ^2 \,\mathrm{d} g_2\bigg)^{1/2} \,\mathrm{d} g_1. \end{align*} $$

By the cancellation condition on $a_{k,R}(\cdot ,g_2)$ and the smoothness of $\mathcal {R}^{[1]}_{j_1}$ ,

$$ \begin{align*} \left| \mathcal{R}^{[1]}_{j_1}(a_{k,R})(\mathbf{g}) \right| &= \left| \int_{{G_1}} \big( \mathcal{R}^{[1]}_{j_1}(g_1,h_1)-\mathcal{R}^{[1]}_{j_1}(g_1,z({Q_1}))\big) a_{k,R}(h_1,g_2) \,\mathrm{d} h_1 \right| \\ &\lesssim \frac{\operatorname{\ell}({Q_1})^{\epsilon}}{\rho_1(g_1,z({Q_1}))^{\nu_1+\epsilon}} \int_{{G_1}} a_{k,R}(h_1,g_2) \,\mathrm{d} h_1\\ &\lesssim \frac{\operatorname{\ell}({Q_1})^{\epsilon}}{\rho_1(g_1,z({Q_1}))^{\nu_1+\epsilon}} |{Q_1}|^{1/2}\bigg(\int_{{G_1}} |a_{k,R}(h_1,g_2)|^2 \,\mathrm{d} h_1\bigg)^{1/2}, \end{align*} $$

where $z({Q_1})$ is the center of ${Q_1}$ . Hence

$$ \begin{align*} \mathrm{I}_{11}(R) \lesssim \beta_k^{-\epsilon}\gamma_{11}(R)^{-\epsilon} \left| R \right|{}^{1/2} \left\Vert a_{k,R} \right\Vert{}_{\mathsf{L}^{2}(\mathbf{G})}. \end{align*} $$

To estimate $\mathrm {I}_{12}(R)$ , we use the cancellation of $a_{k,R}$ to write $\mathcal {R}(a_{k,R})(\mathbf {g})$ as

$$ \begin{align*} &\int_{\beta_k R} \big( \mathcal{R}^{[1]}_{j_1}(g_1,h_1) - \mathcal{R}^{[1]}_{j_1}(g_1, z({Q_1}))\big) \big( \mathcal{R}^{[2]}_{j_2}(g_2,h_2)\\&\quad-\mathcal{R}^{[2]}_{j_2}(g_2, z({Q_2}))\big) a_{k,R}(h_1,h_2) \,\mathrm{d} h_1 \,\mathrm{d} h_2, \end{align*} $$

where $z({Q_2})$ is the center of ${Q_2}$ . Using the smoothness of $\mathcal {R}^{[1]}_{j_1}$ and $\mathcal {R}^{[2]}_{j_2}$ , we deduce that

$$ \begin{align*} \mathrm{I}_{12}(R)&\lesssim \int_{g_1\notin 100b{\tilde{Q}_1}}\int_{(100{Q_2})^c} \frac{\operatorname{\ell}({Q_1})^{\epsilon}}{\rho_1(g_1,z({Q_1}))^{\nu_1+\epsilon}}\\&\qquad\qquad\quad \frac{\operatorname{\ell}({Q_2})^{\epsilon}}{\rho_2(g_2,z({Q_2}))^{\nu_2+\epsilon}} \,\mathrm{d} g_1 \,\mathrm{d} g_2 \left| R\right|{}^{1/2} \left\Vert a_{k,R} \right\Vert{}_{\mathsf{L}^{2}(\mathbf{G})}\\ &\lesssim b^{-\epsilon}\gamma_1(R)^{-\epsilon} \left| R \right|{}^{1/2} \left\Vert a_{k,R} \right\Vert{}_{\mathsf{L}^{2}(\mathbf{G})}. \end{align*} $$

Then, by continuing as in Section 5.1, we establish (5.13) and complete the proof.