Proof. By $(f_1)$– $(f_2)$, for any ϵ, δ > 0 and $q\in (2, 2_*)$, there exist constants $C^{\prime}_\epsilon, C_\delta \gt 0$ such that

(2.2)\begin{eqnarray} &&F_2(u)\geq -\frac{\epsilon}{2}|u|^2 -\frac{C^{\prime}_\epsilon}{q}|u|^q, \end{eqnarray}

(2.3)\begin{eqnarray} &&F(u)\leq \frac{\delta}{2}|u|^2 + \frac{C_\delta}{p}|u|^p. \end{eqnarray}

Let $u\in \mathcal{B}_\alpha$ and $v\in \partial\mathcal{B}_{2\alpha}$, where α is to be determined. By (2.2)–(2.3), along with the Gagliardo–Nirenberg inequality, we obtain

\begin{equation*} \begin{aligned} & J_1(v)-J_{\frac{1}{2}}(u)\\ &\geq \frac{1}{2}\int_\Omega |\nabla v|^2-\frac{1}{2}\int_\Omega |\nabla u|^2 -\int_\Omega F(v)+ \int_\Omega F_2(u)\\ &\geq \frac{1}{2}\int_\Omega \left( |\nabla v|^2- |\nabla u|^2 \right)dx -\frac{\delta+\epsilon}{2}c-\frac{C_\delta C_{N,p}^p}{p}c^{\frac{p(1-\gamma_p)}{2}}\|v\|^{p\gamma_p}- \frac{C^{\prime}_\epsilon C_{N,q}^q}{q}\\ &\quad \times c^{\frac{q(1-\gamma_q)}{2}}\|u\|^{q\gamma_q}\\ &\geq \frac{\alpha}{2}-\frac{\delta+\epsilon}{2}c-\frac{C_\delta C_{N,p}^p}{p}c^{\frac{p(1-\gamma_p)}{2}}(2\alpha+c)^{\frac{p\gamma_p}{2}}- \frac{C^{\prime}_\epsilon C_{N,q}^q}{q}c^{\frac{q(1-\gamma_q)}{2}}(\alpha+c)^{\frac{q\gamma_q}{2}}. \end{aligned} \end{equation*}

Due to the arbitrariness of ϵ and δ, we conclude that for some $c^* \gt 0$ and any $c\in (0, c^*)$, taking $\alpha^*= 4c$, we have

\begin{equation*} J_1(v)-J_{\frac{1}{2}}(u)\geq c-9^{\frac{p\gamma_p}{2}}\frac{C_1 C_{N,p}^p}{p}c^{\frac{p}{2}}- 5^{\frac{q\gamma_q}{2}}\frac{C_1' C_{N,q}^q}{q}c^{\frac{q}{2}} \gt 0. \end{equation*}

Hence, the conclusion follows.

Proof. Fix $c\in (0, c^*)$. By lemma 2.1, for any $\rho \in \left[\frac{1}{2},1 \right]$, we have

\begin{eqnarray*} \sup_{u\in \mathcal{B}_{\alpha^*}}J_\rho(u) \leq \sup_{u\in \mathcal{B}_{\alpha^*}}J_{\frac{1}{2}}(u) \lt \inf_{u\in \partial\mathcal{B}_{2\alpha^*}}J_1(u)\leq \inf_{u\in \partial\mathcal{B}_{2\alpha^*}}J_\rho(u). \end{eqnarray*}

Let $\{u_{\rho,n}\}\subset \mathcal{B}_{2\alpha^*}$ be a minimizing sequence for J_ρ at the level m_ρ. Clearly, $\{u_{\rho,n}\}$ is bounded in $H^1(\Omega)$. Consequently, there exist a subsequence of $\{u_{\rho,n}\}$, still denoted by $\{u_{\rho,n}\}$, and some $u^*_\rho\in H^1(\Omega)$ such that $u_{\rho,n}\rightharpoonup u^*_\rho$ in $H^1(\Omega)$ and $u_{\rho,n}\to u^*_{\rho}$ strongly in $L^r(\Omega)$ for $r\in[1,2^*)$. This implies that $u^*_{\rho}\in \mathcal{B}_{2\alpha^*}$ and thus $J_\rho(u^*_{\rho})\ge m_{\rho}.$

By $(f_1)$– $(f_2)$, together with the Hölder inequality and the Lebesgue dominated convergence theorem, we have

\begin{eqnarray*} \int_{\Omega}\left(F(u_{\rho,n})-F(u^*_{\rho})\right)dx=\int_{\Omega}F(u_{\rho,n}-u^*_{\rho})dx+o_n(1)\to 0. \end{eqnarray*}

Therefore, we obtain

\begin{equation*} J_\rho(u^*_{\rho})\le \liminf\limits_{n\to+\infty}J_\rho(u_{\rho,n})=m_{\rho}. \end{equation*}

Combining this with the previous inequality, we deduce that $J_\rho(u^*_{\rho})=m_{\rho}$ and $\|\nabla \left(u_{\rho,n}-u^*_{\rho}\right)\|_2^2\to0$. This implies that $u_{\rho,n}\to u^*_{\rho}$ in $H^1(\Omega)$ and hence $u^*_{\rho}$ is a minimizer of J_ρ. Thus, the desired conclusion follows.

Proof. Let $B_r(x)$ denote a ball in $\mathbb{R}^N$, centred at $x\in \mathbb{R}^N$ with radius r > 0. Take $\phi \in C^\infty_0(B_1(0))$ with ϕ > 0 in $B_1(0)$ such that $\int_{B_1(0)}\phi^2 =1$. For $n\in \mathbb{N}$, $x_0\in \Omega$, define

\begin{equation*} \varphi_n(x):= c^\frac{1}{2} n^\frac{N}{2} \phi (n(x-x_0)),~x\in \Omega. \end{equation*}

We can verify that $\varphi_n\in\mathcal{S}_c$ and $supp(\varphi_n)\subset B_\frac{1}{n}(x_0)\subset \Omega$ for sufficiently large n.

By $(f_1)$– $(f_2)$, taking $R_0 \gt 0$ larger if necessary, there exist constants $C_{R_0}, C^{\prime}_{R_0}, C_p \gt 0$ such that

\begin{eqnarray*} &&F_1(t)\ge \frac{C_p a_0}{2p}|t|^p,~~ \forall |t|\ge 2R_0,~~~F_1(t)\ge -C_{R_0},~~ \forall |t|\le 2R_0,\\ &&F_2(t)\ge -C^{\prime}_{R_0}, \forall t\in \mathbb{R}. \end{eqnarray*}

Set $\Omega_{n, 2R_0} :=\{x\in \Omega: |\varphi_n|\leq 2R_0 \}$. Then, for sufficiently large n such that $\max\limits_{x\in \overline\Omega}|\varphi_n| \gt 2R_0$, we have for all $\rho\in [\frac{1}{2},1]$,

\begin{eqnarray*} J_\rho(\varphi_n)&\leq& \frac{1}{2}\int_{\Omega}|\nabla \varphi_n(x)|^2 + (C_{R_0}+C^{\prime}_{R_0})|\Omega| -\rho\int_{\Omega\backslash \Omega_{n, 2R_0}} \frac{C_p a_0}{2p} |\varphi_n|^p \\ &=& \frac{cn^2}{2}\int_{B_1(0)}|\nabla \phi(x)|^2 +(C_{R_0}+C^{\prime}_{R_0})|\Omega|+ \frac{2^pC_P a_0}{4p}R_0^p|\Omega|\\ &&-\frac{C_Pa_0}{4p}c^\frac{p}{2} n^{\frac{pN-2N}{2}}\int_{B_1(0)}|\phi(x)|^p\to -\infty. \end{eqnarray*}

Hence, there exists $n_0 \gt 0 $ sufficiently large such that

\begin{equation*} J_\rho(\varphi_{n_0}) \lt J_1(u^*_{\frac{1}{2}})\le J_{\rho}(u^*_{\frac{1}{2}}), ~\forall \rho\in \left[\frac{1}{2},1 \right]. \end{equation*}

Choose $w_1=u^*_{\frac{1}{2}}$ and $w_2=\varphi_{n_0}$. Clearly, $u^*_{\frac{1}{2}}\in \mathcal{B}_{2\alpha^*}\setminus \partial\mathcal{B}_{2\alpha^*}, \varphi_{n_0}\not\in \mathcal{B}_{2\alpha^*}$. By continuity, for any $\gamma \in \Gamma$, there exists $t_\gamma \in [0,1]$ such that $ \gamma(t_\gamma)\in \partial \mathcal{B}_{2\alpha^*}$. Thus, by lemma 2.1, it follows that

\begin{align*} \max_{t\in[0,1]}J_\rho(\gamma(t))\geq J_\rho(\gamma(t_\gamma)) &\geq \inf_{u\in \partial \mathcal{B}_{2\alpha^*}}J_\rho(u) \gt \sup_{u\in \mathcal{B}_{\alpha^*}}J_{\frac{1}{2}}(u)\\ & \geq J_\rho(u^*_{\frac{1}{2}})=\max\{J_\rho(w_1), J_\rho(w_2)\}, ~\forall \rho\in \left[\frac{1}{2},1 \right]. \end{align*}

The proof is now complete.

Proof. We will apply theorem 3.4 to the family of functionals J_ρ, where $E=H^1(\Omega)$, $H=L^2(\Omega)$, $S(a)=\mathcal{S}_c$ and Γ is defined by (2.4). Specifically, we set

\begin{equation*} A(u)= \frac{1}{2}\int_{\Omega}|\nabla u|^2dx- \int_\Omega F_2(u) dx~~{\rm and }~~B(u)=\int_{\Omega} F_1(u) dx. \end{equation*}

Thus, we have $J_\rho(u)= A(u)-\rho B(u).$ Given that $u\in \mathcal{S}_c$ and considering the boundedness of $\int_\Omega F_2(u)dx$, we deduce that

\begin{equation*}A(u)\to \infty~~{\rm as }~~ \|u\|\to +\infty.\end{equation*}

Moreover, by assumptions $(f_1)$– $(f_2)$, it follows that $J^{\prime}_\rho$ and $J'^{\prime}_\rho$ are locally Hölder continuous on $\mathcal{S}_c$. By lemma 2.3, we can apply theorem 3.4 to produce a bounded Palais–Smale sequence $\{u_n\}\subset H^1(\Omega)$ for the constrained functional $J_{\rho}|_{\mathcal{S}_c}$ at level c_ρ for almost every $\rho\in [\frac{1}{2}, 1]$. Additionally, there exists a sequence $\zeta_n \to 0^+$ such that $\tilde{m}_{\zeta_n}(u_n)\leq 1$.

Since $\|J^{\prime}_\rho |_{\mathcal{S}_c}(u_n) \|\to 0$, and by the boundedness of $\{u_n\}$, there exists a sequence $\{\lambda_n\}\subset \mathbb{R}$ such that for any $\varphi\in H^1(\Omega)$, we have

(3.4)\begin{equation} \int_{\Omega} \nabla u_n \nabla \varphi dx+\lambda_n \int_{\Omega} u_n \varphi dx - \int_{\Omega} h_{\rho}(u_n) \varphi dx =o(1). \end{equation}

This implies that

\begin{equation*} \int_{\Omega}|\nabla u_n|^2 dx + \lambda_n c-\int_{\Omega} h_{\rho}(u_n)u_n dx\to 0. \end{equation*}

Using $(f_1)$– $(f_2)$ again, we deduce that $\{\lambda_n\}$ is bounded. Therefore, up to a subsequence, we may assume that $\lambda_n \to \lambda_\rho \in \mathbb{R}$ and $u_n\rightharpoonup u_{\rho}$ weakly in $H^1(\Omega)$. By (3.4), we obtain

\begin{equation*} \int_{\Omega} \nabla u_\rho \nabla \varphi dx+\lambda_\rho\int_{\Omega} u_\rho \varphi dx -\int_{\Omega}h_{\rho}(u_\rho) \varphi dx=0, \end{equation*}

which implies that u_ρ weakly solves (3.3). By the compact embedding $H^1(\Omega)\hookrightarrow L^r(\Omega)$ for $r\in[1,2^*)$ and standard arguments, we obtain that $u_n\to u_{\rho}$ strongly in $H^1(\Omega)$.

It remains to show that $m(u_\rho)\le 2$. Since $T_u\mathcal{S}_c$ has codimension 1, noting that $d^2|_{\mathcal{S}_c}J_\rho$ and $T_{u_n}\mathcal{S}_c$ vary with continuity, by $\tilde{m}_{\zeta_n}(u_n)\leq 1$ it follows that $\tilde{m}_0(u_\rho) \leq 1$. Then, we can use similar arguments as in [Reference Borthwick, Chang, Jeanjean and Soave12, proposition 3.5] to show $m(u_\rho)\leq 2$. In fact, since the tangent space $T_{u}\mathcal{S}_c$ has codimension 1, it suffices to show that $u_\rho\in \mathcal{S}_c$ has Morse index at most 1 as a constrained critical point. If this were not the case, by definition 3.3, we may assume by contradiction that there exists a subspace $W_0 \subset T_u\mathcal{S}_c$ with $dim W_0 = 2$ such that

(3.5)\begin{equation} D^2J_\rho(u_\rho)[w, w] \lt 0~\mbox{for~all}~w\in W_0\backslash \{0\}. \end{equation}

Since W ₀ is finite-dimensional, there exists a constant β > 0 such that

\begin{equation*} D^2J_\rho(u_\rho)[w, w] \lt -\beta~\mbox{for~all}~w\in W_0\backslash \{0\}~~\mbox{with}~~\|w\|=1. \end{equation*}

Using the homogeneity of $D^2J_\rho(u_\rho)$, we deduce that

\begin{equation*} D^2J_\rho(u_\rho)[w, w] \lt -\beta \|w\|^2~\mbox{for~all}~w\in W_0\backslash \{0\}. \end{equation*}

Now, since $J_\rho'$ and $J_\rho^{\prime\prime}$ are α-Hölder continuous on bounded sets for some $\alpha\in (0,1]$, it follows that there exists a sufficient small $\delta_1 \gt 0$ such that, for any $v\in \mathcal{S}_c$ satisfying $\|v-u_\rho\|\leq \delta_1$,

(3.6)\begin{equation} D^2J_\rho(v)[w, w] \lt -\frac{\beta}{2} \|w\|^2~\mbox{for~all}~w\in W_0\backslash \{0\}. \end{equation}

Hence, using the fact that $\|u_n-u_\rho\| \leq\delta_1$ for sufficiently large $n\in \mathbb{N}$, and in view of (3.5), (3.6), and $\zeta_n\to 0^+$, we obtain

\begin{equation*} D^2J_\rho(u_n)[w, w] \lt -\frac{\beta}{2} \|w\|^2 \lt \zeta_n \|w\|^2~\mbox{for~all}~w\in W_0\backslash \{0\} \end{equation*}

for any such large n. Since dim $W_0 \gt 1$, this provides a contradiction with theorem 3.4 (iv), recalling that $\zeta_n\to 0$.

Proof. We assume by contradiction that $\lambda_n\to -\infty$. Let V be a subspace of $H^1(\Omega)$ with dimension k, where k > 2. Define $\Omega_{M_0}:=\{x\in \Omega: |u_n(x)|\leq M_0\}$, where $M_0=\max\{M, R_0+1\}$ and R ₀ is given in §2. By assumptions $(f_1)$ and $(f_3)$, there exist constants $C_{M_0}, C^{\prime}_{M_0} \gt 0$ such that for any $\phi\in H^1(\Omega)$,

\begin{equation*} \begin{aligned} \int_\Omega h_{\rho_n}'(u_n)\phi^2 dx &= \int_{\Omega_{M_0}} h_{\rho_n}'(u_n)\phi^2 dx+\int_{\Omega\backslash \Omega_{M_0}}h_{\rho_n}'(u_n)\phi^2 dx\\ &\geq \int_{\Omega_{M_0}} -C_{M_0}\phi^2 dx+ \int_{\Omega\backslash \Omega_{M_0}}h_{\rho_n}'(u_n)\phi^2 dx\\ &\geq \int_{\Omega_{M_0}} -C_{M_0}\phi^2 dx+ \int_{\Omega\backslash \Omega_{M_0}}\left(\rho_n\mu |u_n|^{p-2}- C_{M_0}'\right)\phi^2 dx. \end{aligned} \end{equation*}

Taking $\varphi\in V\setminus\{0\}$, we obtain

\begin{equation*} \begin{aligned} Q_{\lambda_n,\rho_n}(\varphi;u_n;\Omega) &\leq \int_{\Omega}|\nabla \varphi |^2 dx+ \lambda_n\int_{\Omega}\phi ^2 dx +\int_{\Omega_{M_0}} C_{M_0} \varphi^2 dx\\ &-\int_{\Omega\backslash \Omega_{M_0}} \left(\mu |u_n|^{p-2}- C_{M_0}'\right)\varphi^2 dx\\ &\leq \|\varphi\|^2 + \left(\lambda_n+C_{M_0}+ C_{M_0}'-1\right)\int_{\Omega}\varphi^2 dx. \end{aligned} \end{equation*}

This implies that $Q_{\lambda_n,\rho_n}(\varphi;u_n;\Omega)$ is negative definite on V for sufficiently large n, which contradicts the fact that $m(u_n)\le 2$.

Proof. By (4.1) and assumptions $(f_1)$– $(f_2)$, there exists a constant $C_1 \gt 0$ such that

\begin{equation*} \lambda_nc\leq \int_\Omega f(u_n)u_n \leq c+\int_\Omega C_1 |u_n|^p dx\le c+C_1|\Omega|\|u_n \|_{L^{\infty }}^p. \end{equation*}

This implies that

\begin{equation*} \|u_n \|_{L^{\infty }}\geq \left(\frac{c}{C_1|\Omega |} \right)^{\frac{1}{p}} (\lambda_n-1)^{\frac{1}{p}} \to +\infty. \end{equation*}

Proof. Since u_n may change sign, P_n can be either a positive local maximum or a negative local minimum point. For simplicity, we focus on the case where P_n is a positive local maximum point; the arguments for the negative local minimum case are analogous.

By (4.1), we get

\begin{equation*} 0\leq \frac{-\Delta u_n(P_n)}{u_n(P_n)}= \frac{f_2(u_n(P_n))+\rho_n f_1(u_n(P_n))}{u_n(P_n)}- \lambda_n. \end{equation*}

Using $(f_2)$ and lemma 4.1, we deduce that

\begin{equation*}\frac{\lambda_n}{|u_n(P_n)|^{p-2}}\to \tilde{\lambda}\in [0, a_0] ~{\rm as}~n\to \infty. \end{equation*}

Next, we show that $\tilde{\lambda} \gt 0$.

Define the rescaled function

\begin{equation*} \tilde{u}_n(x):=a_0^{\frac{1}{p-2}} \tilde{\epsilon}_n^{\frac{2}{p-2}}u_n(\tilde{\epsilon}_nx+ P_n)~~{\rm for}~~ x\in \tilde{\Omega}_n:=\frac{\Omega-P_n}{\tilde{\epsilon}_n}. \end{equation*}

Clearly, $\tilde{u}_n$ satisfies

(4.8)\begin{equation} \left\{\begin{array}{ll} -\Delta \tilde{u}_n +\lambda_n \tilde{\epsilon}_n^2 \tilde{u}_n =a_0^{\frac{1}{p-2}}\tilde{\epsilon}_n^{\frac{2p-2}{p-2}}h_{\rho_n}(a_0^{-\frac{1}{p-2}}\tilde{\epsilon}_n^{-\frac{2}{p-2}}\tilde{u}_n) & {\mathrm{in}} \,~ \tilde{\Omega}_n,\\ |\tilde{u}_n(x)|\leq |\tilde{u}_n(0)|=1 & {\mathrm{in}} \,~ \tilde{\Omega}_n, \\ \displaystyle \frac{\partial \tilde{u}_n}{\partial \nu}=0 \, &{\mathrm{on}}\,~\partial\tilde{\Omega}_n. \end{array}\right. \end{equation}

Let $d_n=dist(P_n, \partial \Omega)$. Then

\begin{equation*} \frac{d_n}{\tilde{\epsilon}_n}:= L\in [0, +\infty] ~~{\rm and}~~\tilde{\Omega}_n\to \begin{cases} &\mathbb{R}^N ~~{\rm if}~~L=+\infty;\\ &\mathbb{H}~~~~{\rm if}~~L \lt +\infty, \end{cases} \end{equation*}

where $\mathbb{H}$ denotes a half-space such that $0\in \overline{\mathbb{H}}$ and $d(0, \partial \mathbb{H})=L$. By regularity arguments, up to a subsequence, $\tilde{u}_n\to \tilde{u}$ in $C^2_{loc}(\overline{D})$, where $\tilde{u}$ solves

(4.9)\begin{equation} \left\{\begin{array}{ll} -\Delta \tilde{u}+ \tilde{\lambda} \tilde{u}=|\tilde{u}|^{p-2}\tilde{u} & {\mathrm{in}} \,~ D,\\ |\tilde{u}(x)|\leq |\tilde{u}(0)|=1 & {\mathrm{in}} \,~ D, \\ \displaystyle\frac{\partial \tilde{u}}{\partial \nu}=0 \, &{\mathrm{on}}\,~\partial D, \end{array}\right. \end{equation}

where D is either $\mathbb{R}^N$ or $\mathbb{H}$.

We claim that $m(\tilde{u})\le 2$. To see this, suppose for contradiction that there exists k > 2 such that there are k positive functions $\phi_1, \ldots, \phi_k \in H^1(D)$, orthogonal in $L^2(\Omega)$, satisfying

(4.10)\begin{equation} \int_D |\nabla \phi_i|^2 dx + \tilde{\lambda}\int_D \phi^2 dx- \int_D (p-1)|\tilde{u}|^{p-2}\phi^2 dx \lt 0 \end{equation}

for every $i\in \{1,\ldots, k\}$.

Define the rescaled functions

\begin{equation*}\phi_{i,n}(x):= \tilde{\epsilon}_n^{-\frac{N-2}{2}}\phi_i \left(\frac{x-P_n}{\tilde{\epsilon}_n}\right).\end{equation*}

Additionally, let

\begin{equation*}\tilde{\Omega}_{n, M_0}:= \{x\in \tilde{\Omega}_n: |a_0^{-\frac{1}{p-2}}\tilde{\epsilon}_n^{-\frac{2}{p-2}}\tilde{u}_n|\leq M_0\}, ~~\tilde{\Omega}_{n, M_0}^c:= \tilde{\Omega}_n\backslash \tilde{\Omega}_{n, M_0},\end{equation*}

where M ₀ is as given in lemma 4.1.

By direct computations, there exist constants $\tilde{C}_{M_0}, \tilde{C}^{\prime}_{M_0} \gt 0$ such that the following estimates hold:

\begin{align*} \int_{\Omega} f^{\prime}_1(u_n) \phi_{i, n}^2 dx &= \int_{\tilde{\Omega}_n} \tilde{\epsilon}^2_n \phi_i^2 f_1'(a_0^{-\frac{1}{p-2}}\tilde{\epsilon}_n^{-\frac{2}{p-2}} \tilde{u}_n) dx\\ &=\int_{\tilde{\Omega}_{n, M_0+1}} [\tilde{\epsilon}^2_n\phi_i^2 f'(a_0^{-\frac{1}{p-2}}\tilde{\epsilon}_n^{-\frac{2}{p-2}}\tilde{u}_n)\eta(a_0^{-\frac{1}{p-2}}\tilde{\epsilon}_n^{-\frac{2}{p-2}}\tilde{u}_n)\\ &+ \tilde{\epsilon}^2_n \phi_i^2 f(a_0^{-\frac{1}{p-2}}\tilde{\epsilon}_n^{-\frac{2}{p-2}}\tilde{u}_n)\eta'(a_0^{-\frac{1}{p-2}}\tilde{\epsilon}_n^{-\frac{2}{p-2}}\tilde{u}_n) ]dx \\ &+\int_{\tilde{\Omega}_{n, M_0}^c} \tilde{\epsilon}^2_n \phi_i^2 f'(a_0^{-\frac{1}{p-2}}\tilde{\epsilon}_n^{-\frac{2}{p-2}}\tilde{u}_n)\eta(a_0^{-\frac{1}{p-2}}\tilde{\epsilon}_n^{-\frac{2}{p-2}}\tilde{u}_n) dx\\ &\geq \int_{\tilde{\Omega}_{n, M_0} \cap supp\{\phi_i\}} -\tilde{C}_{M_0} \tilde{\epsilon}^2_n \phi_i^2 dx\\ &+ \int_{\tilde{\Omega}_{n, M_0}^c} \tilde{\epsilon}^2_n \phi_i^2\mu |a_0^{-\frac{1}{p-2}}\tilde{\epsilon}_n^{-\frac{2}{p-2}}\tilde{u}_n|^{p-2} dx\\ &= \int_{\tilde{\Omega}_{n, M_0} \cap supp\{\phi_i\}} -\tilde{C}_{M_0} \tilde{\epsilon}^2_n \phi_i^2 dx\\ &\quad + \int_{\tilde{\Omega}_n} \tilde{\epsilon}^2_n \phi_i^2\mu |a_0^{-\frac{1}{p-2}}\tilde{\epsilon}_n^{-\frac{2}{p-2}}\tilde{u}_n|^{p-2} dx\\ &-\int_{\tilde{\Omega}_{n, M_0}} \tilde{\epsilon}^2_n \phi_i^2\mu |a_0^{-\frac{1}{p-2}}\tilde{\epsilon}_n^{-\frac{2}{p-2}}\tilde{u}_n|^{p-2} dx \end{align*}

and

\begin{align*} \int_{\Omega} f^{\prime}_2(u_n) \phi_{i, n}^2 dx &= \int_{\tilde{\Omega}_n} \tilde{\epsilon}^2_n\phi_i^2 f_2'(a_0^{-\frac{1}{p-2}}\tilde{\epsilon}_n^{-\frac{2}{p-2}}\tilde{u}_n) dx\\ &=\int_{\tilde{\Omega}_{n, M_0}\cap supp\{\phi_i\}} [\tilde{\epsilon}^2_n\phi_i^2 f'(a_0^{-\frac{1}{p-2}}\tilde{\epsilon}_n^{-\frac{2}{p-2}}\tilde{u}_n)\\ &\quad\times(1-\eta(a_0^{-\frac{1}{p-2}}\tilde{\epsilon}_n^{-\frac{2}{p-2}}\tilde{u}_n))\\ &- \tilde{\epsilon}^2_n\phi_i^2 f(a_0^{-\frac{1}{p-2}}\tilde{\epsilon}_n^{-\frac{2}{p-2}}\tilde{u}_n)\eta'(a_0^{-\frac{1}{p-2}} \tilde{\epsilon}_n^{-\frac{2}{p-2}}\tilde{u}_n)]dx \\ &\geq \int_{\tilde{\Omega}_{n, M_0} \cap supp\{\phi_i\}} -\tilde{C}^{\prime}_{M_0} \tilde{\epsilon}^2_n \phi_i^2 dx. \end{align*}

Note that $a_0^{-\frac{1}{p-2}}\tilde{u}_n\leq \tilde{\epsilon}_n^{\frac{2}{p-2}} M_0$ for $x\in \tilde{\Omega}_{n, M_0}$, by (4.10), it follows that

\begin{equation*} \begin{aligned} Q_{\lambda_n,\rho_n}(\phi_{i,n}; u_n; \Omega)&= \int_{\Omega}|\nabla\phi_{i,n} |^2 dx +\lambda_n \int_{\Omega} \phi_{i,n}^2 dx -\int_{\Omega}f_2'(u_n)\phi_{i,n}^2 dx - \rho_n \\ &\quad\times\int_{\Omega}f_1'(u_n)\phi_{i,n}^2 dx\\ &\leq \int_{\tilde{\Omega}_n}|\nabla \phi_{i} |^2 dx +\lambda_n \tilde{\epsilon}_n^2\int_{\tilde{\Omega}_n} \phi_{i}^2 dx\\ &\quad + \int_{\tilde{\Omega}_{n, M_0} \cap supp\{\phi_i\}} (\rho_n \tilde{C}_{M_0}+ \tilde{C}^{\prime}_{M_0}) \tilde{\epsilon}^2_n \phi_i^2 dx\\ &-\rho_n\int_{\tilde{\Omega}_n} \tilde{\epsilon}^2_n \phi_i^2\mu |a_0^{-\frac{1}{p-2}}\tilde{\epsilon}_n^{-\frac{2}{p-2}}\tilde{u}_n|^{p-2} dx+ \rho_n\\ &\quad\times\int_{\tilde{\Omega}_{n, M_0}} \phi_i^2\mu a_0^{-1}|\tilde{u}_n|^{p-2} dx \\ &\to \int_D |\nabla \phi_i|^2 dx + \tilde{\lambda}\int_D \phi^2 dx- \int_D a_0^{-1}\mu|\tilde{u}|^{p-2}\phi^2 dx\\ &\leq \int_D |\nabla \phi_i|^2 dx + \tilde{\lambda}\int_D \phi^2 dx- \int_D (p-1)|\tilde{u}|^{p-2}\phi^2 dx \lt 0. \end{aligned} \end{equation*}

This implies that $m(u_n)\geq k \gt 2$ for sufficiently large n, thereby yielding a contradiction. Thus, the claim is valid.

Having established that $\tilde u$ is a finite Morse non-trivial solution of (4.9), we can apply either [Reference Farina22, theorem 2] or [Reference Yu57, proposition 2.1]. This allows us to conclude that the occurrence of $\tilde{\lambda}=0$ is ruled out, regardless of whether D is a half-space or a whole space. Consequently, we can assert that $\tilde{\lambda}\in (0, a_0]$.

In the sequel, we consider the sequence $\{v_n\}$ defined by (4.4). Clearly, v_n satisfies

(4.11)\begin{equation} \left\{\begin{array}{ll} -\Delta v_n+ v_n=a_0^{\frac{1}{p-2}}\epsilon_n^{\frac{2p-2}{p-2}}h_{\rho_n}(a_0^{-\frac{1}{p-2}}\epsilon_n^{-\frac{2}{p-2}} v_n) & {\mathrm{in}} \,~ \Omega_n,\\ |v_n(x)|\leq |v_n(0)|=(\frac{\epsilon_n}{\tilde{\epsilon}_n})^{\frac{2}{p-2}}\to \tilde{\lambda}^{-\frac{1}{p-2}} & {\mathrm{in}} \,~ \Omega_n, \\ \displaystyle\frac{\partial v_n}{\partial \nu}=0 \, &{\mathrm{on}}\,~\partial \Omega_n. \end{array}\right. \end{equation}

Up to a subsequence, we have $v_n\to v$ in $C^2_{loc}(\overline{D})$, where D is either $\mathbb{R}^N$ or a half space $\mathbb{H}$, and v solves

(4.12)\begin{equation} \left\{\begin{array}{ll} -\Delta v+ v=|v|^{p-2}v & {\mathrm{in}} \,~ D,\\ |v(x)|\leq |v(0)|= \tilde{\lambda}^{-\frac{1}{p-2}} & {\mathrm{in}} \,~ D, \\ \displaystyle\frac{\partial v}{\partial \nu}=0 \, &{\mathrm{on}}\,~\partial D. \end{array}\right. \end{equation}

Arguing as above implies that $m(v)\leq 2$.

More precisely, we distinguish the following three cases:

1. (1) If $\limsup\limits_{n\to +\infty}\frac{dist(P_n, \partial\Omega)}{\tilde{\epsilon}_n}= +\infty$, then $D=\mathbb{R}^N$. Using $m(v)\le2$, by standard regularity arguments, $v\in C^2(\mathbb{R}^N)$, $|v(x)|\to0$ as $|x|\to+\infty$. If $v\ge0$, then using the strong maximum principle and [Reference Esposito and Petralla20, theorem 1.1] we know that $v \gt 0, m(v)=1$, and it coincides with the unique radial ground-state solution U ₀ to $-\Delta u+u=u^{p-1}$ in $\mathbb{R}^N$. If v is sign-changing, then $m(v)=2$ and v admits exactly one local maximum point and one local minimum point in Ω.

2. (2) If there exists a constant a > 0 such that

   \begin{eqnarray*} 0 \lt a\leq\limsup_{n\to +\infty}\frac{dist(P_n, \partial\Omega)}{\tilde{\epsilon}_n} \lt +\infty, \end{eqnarray*}

   we may assume that $\lim\limits_{n\to +\infty} \frac{dist(P_n, \partial\Omega)}{\tilde{\epsilon}_n}= d \gt 0$. Then, up to subsequences, $v_n\to v $ in $C^2_{loc}(\{x_N \gt -d\})$, and v weakly solves

   \begin{equation*} \left\{\begin{array}{ll} -\Delta v+ v=|v|^{p-2}v & {\mathrm{in}} \,~ \{x_N \gt -d\},\\ |v(x)|\leq |v(0)|= \tilde{\lambda}^{-\frac{1}{p-2}} & {\mathrm{in}} \,~ \{x_N \gt -d\}, \\ \displaystyle \frac{\partial v}{\partial x_N}=0 \, &{\mathrm{on}}\,~\{x_N=-d\}. \end{array}\right. \end{equation*}

   Let $\tilde{v}(x)=v(x',x_N-d)$, where $x'=(x_1,x_2,\ldots,x_{N-1})$. It is straightforward to verify that $\tilde{v}$ is a finite Morse index solution to the following system:

   \begin{equation*} \left\{\begin{array}{ll} -\Delta \tilde{v}+ \tilde{v}=|\tilde{v}|^{p-2}\tilde{v} & {\mathrm{in}} \,~ \{x_N \gt 0\},\\ |\tilde{v}(x)|\leq |\tilde{v}(0,\ldots,0,d)|= \tilde{\lambda}^{-\frac{1}{p-2}} & {\mathrm{in}} \,~ \{x_N \gt 0\}, \\ \displaystyle \frac{\partial \tilde{v}}{\partial x_N}=0 \, &{\mathrm{on}}\,~ \{x_N=0\}. \end{array}\right. \end{equation*}

   Next, we extend $\tilde{v}$ by reflection with respect to $\{x_N= 0\}$. Specifically, for $(x',x_N)\in \mathbb{R}^N$, we define

   \begin{equation*} \hat{v}(x',x_N):= \begin{cases} &\tilde{v}(x',x_N) ~~~~~~~~{\rm if}~x_N\geq 0,\\ &\tilde{v}(x',-x_N)~~~~~~{\rm if}~x_N \lt 0. \end{cases} \end{equation*}

   Consequently, $\hat{v}$ satisfies the equation

   \begin{equation*} \left\{\begin{array}{ll} -\Delta \hat{v}+ \hat{v}=|\hat{v}|^{p-2}\hat{v} & {\mathrm{in}} \,~ \mathbb{R}^N,\\ |\hat{v}(x)|\leq |\hat{v}(0,\ldots,0,-d)|= |\hat{v}(0,\ldots,0,d)|=\tilde{\lambda}^{-\frac{1}{p-2}} & {\mathrm{in}} \,~ \mathbb{R}^N, \\ \displaystyle \frac{\partial \tilde{v}}{\partial x_N}=0 \, &{\mathrm{on}}\,~ \{x_N=0\}. \end{array}\right. \end{equation*}

   Thus, $\hat{v}$ is a bounded function that solves the equation $-\Delta \hat{v}+ \hat{v}=|\hat{v}|^{p-2}\hat{v}$ in $\mathbb{R}^N$ in the weak sense. By applying Schauder interior and boundary estimates (see [Reference Gilbarg and Trudinger26]), we conclude that $\hat{v}\in C^{2,\alpha}_{loc}({\mathbb{R}^N})$ for $\alpha\in (0,1)$, and the regularity extends up to the hyperplane $\{x_N=0\}$. Since $\hat{v}$ is symmetric across $\{x_N=0\}$, it follows that the second derivatives of $\hat{v}$ are continuous everywhere, including at the boundary $\{x_N=0\}$. Consequently, by applying elliptic regularity theory and considering that $\hat{v}$ is a solution with finite Morse index, we can show that $\hat{v}\in C^2(\mathbb{R}^N)$ and $|\hat{v}(x)|\to0$ as $|x|\to+\infty$. Therefore, $\tilde{v}$ must be a sign-changing solution, a situation that can occur according to [Reference McLeod, Troy and Weissle41, theorem 1]. In fact, if $\hat{v}$ is a positive solution, then by applying [Reference Esposito and Petralla20, theorem 1.1] once again, we deduce that $\hat{v}$ coincides with the unique radial ground-state solution U ₀ to $-\Delta u+u=u^{p-1}$ in $\mathbb{R}^N$. It is well known that U ₀ has a unique global maximum and is a radially strictly decreasing function. This would contradict the fact that $\max\limits_{x\in\mathbb{R}^N} \hat{v}=|\hat{v}(0,\ldots,0,-d)|= |\hat{v}(0,\ldots,0,d)|$.

3. (3) If $\limsup\limits_{n\to +\infty}\frac{dist(P_n, \partial\Omega)}{\tilde{\epsilon}_n}= 0$, then, up to a subsequence, $\Omega_n\to \mathbb{R}^N_+(\{x_N \gt 0\})$. By similar arguments as above, $v_n\to v$ in $C^2_{loc}(\mathbb{R}^N_+)$, where v is a finite Morse index of the following equation

   \begin{equation*} \left\{\begin{array}{ll} -\Delta v+ v=|v|^{p-2}v & {\mathrm{in}} \,~ \{x_N \gt 0\},\\ \displaystyle \frac{\partial v}{\partial x_N}=0 \, &{\mathrm{on}}\,~\{x_N=0\},\\ |v(y)|\leq |v(0)|= \tilde{\lambda}^{-\frac{1}{p-2}}. \end{array}\right. \end{equation*}

   Extending v by reflection with respect to $\{x_N=0\}$ and defining

   \begin{equation*} \hat{v}(x',x_N):= \begin{cases} v(x',x_N)& ~{\rm if}~~x_N\geq 0,\\ v(x',-x_N)&~{\rm if}~~x_N \lt 0. \end{cases} \end{equation*}

   Then $\hat{v}$ satisfies

   \begin{equation*} \left\{\begin{array}{ll} -\Delta \hat{v}+ \hat{v}=|\hat{v}|^{p-2}\hat{v} & {\mathrm{in}} \,~ \mathbb{R}^N,\\ |\hat{v}(x)|\leq |\hat{v}(0)| & {\mathrm{in}} \, \mathbb{R}^N, ~\\ \displaystyle \frac{\partial v}{\partial x_N}=0 \, &{\mathrm{on}}\,~\{x_N=0\}. \end{array}\right. \end{equation*}

   Using arguments analogous to those in case (1), we deduce that if $\hat{v}$ is positive, then $m(\hat{v})=1$, and thus $\hat{v}$ coincides with U ₀. Using [Reference McLeod, Troy and Weissle41, theorem 1], it is established that $\hat{v}$ may exhibit sign-changing behaviour, and in such instance, $m(v)=2$.

All in all, we get that (ii) holds. Employing similar reasoning to [Reference Esposito and Petralla20, theorem 3.1], we conclude that (iii) and (iv) are also satisfied.

Proof. Take $P_n^1\in \overline{\Omega}$ such that $u_n(P_n^1)=\max\limits_{\overline{\Omega}}|u_n(x)|$. If (4.15) is satisfied for $P_n^1$, then we get k = 1. It is evident that $P_n^1$ satisfies (4.13).

Otherwise, if $P_n^1$ does not satisfy (4.15), we suppose that there exists δ > 0 such that

(4.16)\begin{equation} \lim_{R\to \infty}\left(\limsup_{n\to +\infty} \lambda_n^{-\frac{1}{p-2}} \max_{|x-P_n^1|\geq R\lambda_n^{-1/2}}|u_n(x)| \right)\geq 4\delta. \end{equation}

For sufficiently large R, up to a subsequence, it holds

(4.17)\begin{equation} \lambda_n^{-\frac{1}{p-2}} \max_{|x-P_n^1|\geq R\lambda_n^{-1/2}}|u_n(x)|\geq 2 \delta. \end{equation}

Let $P_n^2\in \overline{\Omega}\backslash B_{R\lambda_n^{-1/2}}(P_n^1)$ such that

\begin{equation*} |u_n(P_n^2)|=\max_{\overline{\Omega}\backslash B_{R\lambda_n^{-1/2}}(P_n^1)}|u_n|. \end{equation*}

Then, (4.17) yields that $|u_n(P_n^2)|\to +\infty$ as $n\to +\infty$.

We claim that

(4.18)\begin{equation} \lambda_n|P_n^1-P_n^2|^2\to +\infty. \end{equation}

If (4.18) is not true, then up to subsequence $\lambda_n^{\frac{1}{2}}|P_n^1-P_n^2|\to R'\geq R$. Define

\begin{equation*} v_{n,1}(x):=a_0^{\frac{1}{p-2}}\lambda_n^{-\frac{1}{p-2}}u_n(\lambda_n^{-1/2}x + P_n^1). \end{equation*}

As in lemma 4.3, we can deduce that $ v_{n,1}\to v$ in $C^2_{loc}(\overline{D})$, where $D=\mathbb{R}^N$ or D is a half-space. Then, up to subsequences,

\begin{equation*} \lambda_n^{-\frac{1}{p-2}}u_n(P_n^2)=v_{n,1}(\lambda^{1/2}(P_n^2-P_n^1)) \to v(x'),~|x'|\geq R' \gt R. \end{equation*}

Since $v(x)\to 0$ as $|x|\to \infty$, taking R larger if necessary, it follows that

\begin{equation*} |v(x)|\leq a_0^{\frac{1}{p-2}}\delta, ~\forall |x|\geq R. \end{equation*}

This contradicts to (4.17), which proves the claim (4.18).

In the following, we shall show that

(4.19)\begin{equation} |u_n(P_n^2)|=\max_{\overline{\Omega}\cap B_{R_{n,2}\lambda_n^{-1/2}}(P_n^2)}|u_n(x)|, ~{\rm for~some}~ R_{n,2}\to +\infty. \end{equation}

Let $\tilde{\epsilon}_{n,2}=a_0^{-\frac{1}{2}}|u_n(P_n^2)|^{-\frac{p-2}{2}}$. Clearly, $\tilde{\epsilon}_{n,2}\to 0$. By (4.17), we get $\tilde{\epsilon}_{n,2}\leq (2\delta)^{-\frac{p-2}{2}}\lambda_n^{-\frac{1}{2}}$. From (4.18), we can assert that

\begin{equation*} \tilde{R}_{n,2}:=\frac{|P_n^1-P_n^2|}{2\tilde{\epsilon}_{n,2}}\geq \frac{(2\delta)^{\frac{p-2}{2}}}{2}\lambda_n^{\frac{1}{2}}|P_n^1-P_n^2| \to +\infty~{\rm as }~n\to\infty. \end{equation*}

On the other hand, for any $x\in B_{\tilde{R}_{n,2}\tilde{\epsilon}_{n,2}}(P_n^2)$ and R > 0, we have

\begin{equation*} |x-P_n^1|\geq |P_n^2-P_n^1|- |x-P_n^2|\geq \frac{1}{2}|P_n^2-P_n^1|\geq R\lambda_n^{-\frac{1}{2}} \end{equation*}

for arbitrarily large n. Consequently,

\begin{equation*} \overline{\Omega}\cap B_{\tilde{R}_{n,2}\tilde{\epsilon}_{n,2}}(P_n^2)\subset \overline{\Omega}\backslash B_{R\lambda_n^{-\frac{1}{2}}}(P_n^1), \end{equation*}

which implies that

(4.20)\begin{equation} |u_n(P_n^2)|=\max_{\overline{\Omega}\cap B_{\tilde{R}_{n,2}\tilde{\epsilon}_{n,2}}(P_n^2)}|u_n|. \end{equation}

We define

\begin{equation*} \tilde{u}_{n,2}:=a_0^{\frac{1}{p-2}}\tilde{\epsilon}_{n,2}^{\frac{2}{p-2}}u_n(\tilde{\epsilon}_{n,2} x+P_n^2),~~~ x\in \tilde{\Omega}_{n,2}:=\frac{\Omega -P_n^2}{\tilde{\epsilon}_{n,2}}. \end{equation*}

Set $d_{n,2}={\rm dist} (P_n^2, \partial \Omega)$. Then, passing to subsequences if necessary,

\begin{equation*} \frac{\tilde{\epsilon}_{n,2}}{d_{n,2}}\to L_2\in [0,+\infty]~{\rm and}~\tilde{\Omega}_{n,2}\to \begin{cases} &\mathbb{R}^N ~~{\rm if}~~L_2=+\infty,\\ &\mathbb{H}~~{\rm if}~~L_2 \lt +\infty. \end{cases} \end{equation*}

Then $\tilde{u}_{n,2}$ satisfies the following equation

\begin{equation*} \left\{\begin{array}{ll} -\Delta \tilde{u}_{n,2}+ \lambda_n \tilde{\epsilon}_{n,2}^2 \tilde{u}_{n,2}=a_0^{\frac{1}{p-2}}\tilde{\epsilon}_{n,2}^{\frac{2p-2}{p-2}}h_{\rho_n}(a_0^{-\frac{1}{p-2}}\tilde{\epsilon}_{n,2}^{-\frac{2}{p-2}}\tilde{u}_{n,2}) & {\mathrm{in}} \,~ \tilde{\Omega}_{n,2},\\ |\tilde{u}_{n,2}(x)|\leq |\tilde{u}_{n,2}(0)|=1 & {\mathrm{in}} \,~ \tilde{\Omega}_{n,2}, \\ \displaystyle\frac{\partial \tilde{u}_{n,2}}{\partial \nu}=0 \, &{\mathrm{on}}\,~\partial\tilde{\Omega}_{n,2}. \end{array}\right. \end{equation*}

Since $P_n^2$ is a local maximum or a local minimum point, by lemma 4.1, we get

\begin{equation*}\frac{\lambda_n}{|u_n(P^2_n)|^{p-2}}\to \tilde{\lambda}^{(2)}\in [0, a_0],~{\rm as}~n\to \infty. \end{equation*}

Using the similar argument as in lemma 4.3, we deduce $\tilde{\lambda}^{(2)} \gt 0$, namely

\begin{equation*} \lim_{n\to +\infty}\lambda_n^{\frac{1}{2}}\tilde{\epsilon}_{n,2} \gt 0. \end{equation*}

Set

\begin{equation*} v_{n,2}:=a_0^{\frac{1}{p-2}}\epsilon_n^{\frac{2}{p-2}} u_n(\epsilon_n x+P_n^2)~{\rm for }~x\in \Omega_{n,2}:=\frac{\Omega-P_n^2}{\epsilon_n}, \end{equation*}

where $\epsilon_n=\lambda_n^{-\frac{1}{2}}$ is defined in (4.4). Up to a subsequence, there exists a function $v^{(2)}\in H^1(D)$ such that $v_{n,2}\to v^{(2)}$ in $C_{loc}^2(\overline{D})$, where D is either $\mathbb{R}^N$ or a half space. Moreover, $v^{(2)}$ solves the following problem

(4.21)\begin{equation} \left\{\begin{array}{ll} -\Delta v^{(2)}+ v^{(2)}=|v^{(2)}|^{p-2}v^{(2)} & {\mathrm{in}} \,~ D,\\ |v^{(2)}(x)|\leq |v^{(2)}(0)|= (\tilde{\lambda}^{(2)})^{-\frac{1}{p-2}} & {\mathrm{in}} \,~ D, \\ \displaystyle\frac{\partial v^{(2)}}{\partial \nu}=0 \, &{\mathrm{on}}\,~\partial D. \end{array}\right. \end{equation}

Then, by a similar discussion as above, we conclude that either $\frac{d_{n,2}}{\epsilon_n}\to +\infty$ or $\frac{d_{n,2}}{\epsilon_n}$ remains bounded. Define $R_{n,2}= \tilde{R}_{n,2}\lambda_n^{\frac{1}{2}}\tilde{\epsilon}_{n,2}$. Clearly, $ R_{n,2}\to +\infty$ as $n\to+\infty$. Hence, (4.19) holds.

If (4.15) does not hold, we can apply similar arguments as before to show that there exists $P_n^3$ such that (4.13)–(4.14) are satisfied. For $P_n^i,i=1,2,3$, applying lemma 4.3 again, we can find $\phi^i_n\in C^\infty_0(\Omega)$ with supp $\phi^i_n\subset B_{R\varepsilon_n}(P_n^i)\cap \overline{\Omega}$ for some R > 0, such that

\begin{equation*} \int_\Omega\vert \nabla\phi^i_n\vert~^2dx+\int_\Omega[(\lambda-(p-1)\rho_nu_n^{p-2})(\phi^i_n)^2]dx \lt 0. \end{equation*}

In light of (4.14), we observe that $\phi^1_n,\phi^2_n,\phi_n^{3}$ are mutually orthogonal for sufficiently large n, which implies $\lim\limits_{n\rightarrow+\infty}m(u_n)\ge3$. This leads to a contradiction with the fact that $m(u_n)\leq2$.

Proof. The proof is inspired by [Reference Li and Zhao38, lemma 2.1] and [Reference Esposito and Petralla20, theorem 3.2]. For any θ > 0, we set

\begin{equation*}\Omega_\theta:= \{x\in \overline{\Omega}: dist (x,\partial\Omega) \lt \theta\}, \end{equation*}

and define the inner normal bundle

\begin{equation*}(\partial\Omega)_\theta :=\{(x,y):x\in \partial\Omega, y\in (-\theta,0]\nu_x\}, \end{equation*}

where ν_x is the unit outer normal of $\partial\Omega$ at x.

Since $\partial\Omega$ is a smooth compact submanifold of $\mathbb{R}^N$, by the tubular neighbourhood theorem, it follows that there exists a diffeomorphism $\Phi_{Ib}$ from $\Omega_\theta$ onto $(\partial\Omega)_\theta$. More precisely, for any $x\in \Omega_\theta$, it is easily seen that there exists a unique $\bar{x}\in \partial\Omega$ such that ${\rm dist}(x, \bar{x})= {\rm dist}(x, \partial\Omega)$. Hence, we can define $\Phi_{Ib}(x):=(\bar{x}, -{\rm dist}(x, \bar{x})\nu_{\bar{x}})$ for any $x\in \Omega_\theta$. Clearly, $\Phi_{Ib}(x)=x$ for $x\in \partial\Omega$.

Similarly, let

\begin{equation*}\Omega^\theta := \{x\in \mathbb{R}^N\backslash \Omega, dist (x,\partial\Omega) \lt \theta\},\end{equation*}

and define the outer normal bundle

\begin{equation*}(\partial\Omega)^\theta:=\{(x,y):x\in \partial\Omega, y\in [0,\theta)\nu_x\}. \end{equation*}

Then there exists a diffeomorphism $\Phi_{Ob}:\Omega^\theta \to (\partial\Omega)^\theta$ defined by $\Phi_{Ob}(x):=(\hat{x}, dist(x, \hat{x})\nu_{\hat{x}}), \forall x\in \Omega^\theta $. Here $\hat{x}\in \partial\Omega$ is the unique element in $\partial \Omega$ such that ${\rm dist}(x, \hat{x})= {\rm dist}(x, \partial\Omega)$. Moreover, $\Phi_{Ob}|_{\partial\Omega}= {\rm Identity}$.

Consider the reflection $\Phi_C:(\partial\Omega)_\theta\to (\partial\Omega)^\theta$ defined by $\Phi_C((x,y)):=(x,-y)$. Then, $\Phi:=\Phi_{Ib}^{-1}\circ \Phi_C^{-1}\circ \Phi_{Ob}$ is a diffeomorphism from $\Omega^\theta$ onto $\Omega_\theta$ and $\Phi |_{\partial\Omega}={\rm Identity}$. Furthermore, we take $x=\Phi(z)=(\Phi_1(z),\ldots,\Phi_N(z))$ for $z\in\Omega^\theta$, $z=\Psi(x)=\Phi^{-1}(x)=(\Psi_1(x),\ldots,\Psi_N(x))$ for $x\in \Omega_\theta$, and

\begin{equation*} g_{ij}= \sum_{l=1}^N \frac{\partial \Phi_l}{\partial z_i}\frac{\partial \Phi_l}{\partial z_j},~~ g^{ij}= \sum_{l=1}^n \frac{\partial \Psi_i}{\partial x_l}\frac{\partial \Psi_j}{\partial x_l}(\Phi (z)). \end{equation*}

Then $g_{ij}|_{\partial \Omega}= g^{ij}|_{\partial\Omega}=\delta_{ij}$, where δ_ij is the Kronecker symbol. For simplicity, denote $G=(g^{ij})$, $g(x)=det(g_{ij})$ and $\hat{u}_n(x)=u_n(\Phi(x))$ for $x\in \Omega^\theta$. Then $\hat{u}_n$ satisfies

\begin{equation*} \left\{\begin{array}{ll} -L \hat{u}_n+ \sqrt{g}\lambda_n \hat{u}_n=\sqrt{g}h_{\rho_n}(\hat{u}_n) & {\mathrm{in}} \,~ \Omega^\theta,\\ \displaystyle\frac{\partial \hat{u}_n}{\partial \nu}=0 \, &{\mathrm{on}}\,~\partial \Omega, \end{array}\right. \end{equation*}

where

\begin{equation*}L \hat{u}_n= \sum_{i=1}^N \frac{\partial}{\partial x_i}\left(\sqrt{g}\sum_{j=1}^Ng^{ij}\frac{\partial \hat{u}_n}{\partial x_j}\right). \end{equation*}

Next we define

\begin{equation*} \bar{u}_n:= \begin{cases} u_n(x), x\in\Omega,\\ \hat{u}_n(x), x\in \Omega^\theta, \end{cases} \bar{g}_{ij}:= \begin{cases} \delta_{ij}, x\in \bar\Omega,\\ g_{ij}, x\in \Omega^\theta, \end{cases} \bar{g}^{ij}:= \begin{cases} \delta_{ij}, x\in \bar\Omega,\\ g^{ij}, x\in \Omega^\theta, \end{cases} \end{equation*}

and $\bar{g}:= det(\bar{g}_{ij})$. Let $A(x, \xi)=(A_1(x, \xi),\ldots,A_N(x, \xi))$ for $\xi=(\xi_1,\ldots,\xi_N)$ with

\begin{equation*}A_i(x,\xi)=\sqrt{\bar{g}}\sum_{j=1}^N\bar{g}^{ij}\xi_j.\end{equation*}

Then $\bar{u}_n$ weakly solves

(4.23)\begin{equation} -div\left(A(x,\nabla \bar{u}_n)\right)+ \lambda_n \sqrt{\bar{g}}\bar{u}_n= \sqrt{\bar{g}} g_{\rho_n}(\bar{u}_n)~~{\mathrm{in}}~~\overline{\Omega}\cup \Omega^\theta. \end{equation}

Given that Φ is a diffeomorphism, the functions $g_{ij}(x), \frac{\partial g_{ii}}{\partial x_i}$ and $\frac{\partial g_{ij}}{\partial x_i}$ are all smooth on the domain $\Omega^\theta$. Consequently, there exists a constant $C_0 \gt 0$ such that $|\frac{\partial g_{ii}}{\partial x_i}|, |\frac{\partial g_{ij}}{\partial x_i}|\leq C_0$ are both bounded by C ₀ for $x\in \Omega^\theta$. Moreover, according to the Taylor expansion, as θ approaches 0, the functions g_ij and g ^ij tend to 0, while g_ii and g ⁱⁱ tend to 1, and the determinant g(x) converges to 1.

For any $\tilde{x}\in \overline{\Omega}$, there exists β > 0 such that $B_\beta (\tilde{x})\subset \overline{\Omega}\cup \Omega^\theta$. For $x\in B_\beta(\tilde{x})\backslash \tilde{x}$, denote $\sigma= |x-\tilde{x}|$. Then, for any smooth increasing function $\phi(\sigma)$, we have

\begin{align*} \left|\sum_{i=1}^N \frac{\partial}{\partial x_i}(\sqrt{g}g^{ij})\frac{\partial \phi}{\partial x_j}\right|&= \left|\sum_{i=1}^N \frac{\partial}{\partial x_i}(\sqrt{g}g^{ij}) \frac{x_j-\tilde{x}_j}{\sigma} \phi'\right|\\ &\leq\sum_{i=1}^N \bigg| \max_{x\in \overline{\Omega}\cup \Omega^\theta} \frac{\partial}{\partial x_i}(\sqrt{g}g^{ij}) \bigg|\phi' \frac{diam(\overline{\Omega}\cup \Omega^\theta)}{\sigma}\\ &\le \frac{C_{\Phi ,\Omega}\phi'}{\sigma} \end{align*}

for some $C_{\Phi ,\Omega} \gt 0$, where $\phi'=\frac{d\phi(\sigma)}{d\sigma}$. Moreover, for θ > 0 small enough, there exist small constants $\delta_0, \delta_1$ such that

\begin{align*} \left|\sqrt{g} g^{ii}\frac{\partial^2 \phi}{\partial x_i^2}\right| &=\left|\sqrt{g} g^{ii} \left(\frac{\phi'}{\sigma}+\phi^{\prime\prime}\frac{(x_i-\tilde{x}_i)^2}{\sigma^2}- \phi'\frac{(x_i-\tilde{x}_i)^2}{\sigma^3}\right) \right|\\ &\leq (1+\delta_0) \left(\frac{\phi'}{\sigma}+ |\phi^{\prime\prime}|\frac{(x_i-\tilde{x}_i)^2}{\sigma^2}+\phi'\frac{(x_i-\tilde{x}_i)^2}{\sigma^3}\right), \end{align*}

and

\begin{equation*} \begin{aligned} \left|\sqrt{g} g^{ij}\frac{\partial^2 \phi}{\partial x_j \partial x_i}\right| &=\left| \sqrt{g} g^{ij} \left(\phi^{\prime\prime}\frac{(x_i-\tilde{x}_i)(x_j-\tilde{x}_j)}{\sigma^2}- \phi'\frac{(x_i-\tilde{x}_i)(x_j-\tilde{x}_j)}{\sigma^3}\right) \right|\\ &\leq |\sqrt{g} g^{ij}| \left(\frac{1}{2} |\phi^{\prime\prime}| \frac{(x_i-\tilde{x}_i)^2}{\sigma^2}+ \frac{1}{2}\phi'\frac{(x_i-\tilde{x}_i)^2}{\sigma^3}\right)\\ &+|\sqrt{g} g^{ij}|\left(\frac{1}{2} |\phi^{\prime\prime}|\frac{(x_j-\tilde{x}_j)^2}{\sigma^2}+ \frac{1}{2}\phi'\frac{(x_j-\tilde{x}_j)^2}{\sigma^3}\right)\\ &\leq \delta_1 \left(\frac{1}{2} |\phi^{\prime\prime}| \frac{(x_i-\tilde{x}_i)^2}{\sigma^2}+ \frac{1}{2}\phi'\frac{(x_i-\tilde{x}_i)^2}{\sigma^3}\right)\\ &+\delta_1 \left(\frac{1}{2} |\phi^{\prime\prime}|\frac{(x_j-\tilde{x}_j)^2}{\sigma^2}+ \frac{1}{2}\phi'\frac{(x_j-\tilde{x}_j)^2}{\sigma^3}\right), ~{\rm for}~ i\not= j. \end{aligned} \end{equation*}

Then

(4.24)\begin{equation} \left|\sum_{i=1}^N \frac{\partial}{\partial x_i}\left(\sqrt{g}\sum_{j=1}^Ng^{ij}\frac{\partial \phi}{\partial x_j}\right)\right| \leq 2 |\phi^{\prime\prime}| + \frac{2(N-1)}{\sigma}\phi' + \frac{C_{\Phi ,\Omega}\phi'}{\sigma}. \end{equation}

From now, let us fix $\theta_*\in (0, \theta)$ such that $\frac{1}{2}\leq\sqrt{g}\leq \frac{3}{2}$ and (4.24) holds. Define

\begin{equation*} A_n^*:=\{x\in \overline{\Omega}\cup \Omega^{\theta_*}: d_n(x)\geq R\lambda_n^{-1/2}\}. \end{equation*}

In view of (4.5), for every $\epsilon\in (0,1)$ small, to be chosen later, there exist $R^* \gt 0$ and $n_R\in \mathbb{N}$ large such that

\begin{equation*} \max_{\{x\in \overline{\Omega} : d_n(x)\geq R\lambda_n^{-1/2}\}}|u_n|\leq \lambda_n^{\frac{1}{p-2}}\epsilon,~\forall R \gt R^*, ~ \forall n\geq n_R. \end{equation*}

Let $A_n: =\{x\in \overline{\Omega}: d_n(x)\geq R\lambda_n^{-1/2}\}$. Clearly,

\begin{equation*} |u_n(x)|^{p-2}\leq \lambda_n\epsilon^{p-2},~x\in A_n, ~ \forall n\geq n_R. \end{equation*}

Note that there exists a constant $n_\theta \gt 0$ large enough such that for $n \gt n_\theta$, for any $i\in \{1,\ldots, k\}$, we have $B_{\lambda_n^{-1/2}R}(P_n^i)\subset \Omega^{\theta_*}\cup \Omega_{\theta_*}$. Then we can deduce that $A_n\subset A_n^*$, for $n \gt \max\{n_\theta, n_R\}:= n_{\theta R}$. From the definition of $\bar{u}_n$, we get

\begin{equation*}|\bar{u}_n(x)|^{p-2}\leq \lambda_n \epsilon^{p-2}, ~x\in A_n^*,~\forall n \gt n_{\theta R}. \end{equation*}

Using $(f_1)$– $(f_2)$, we conclude that

(4.25)\begin{equation} -div\left(A(x,\nabla \bar{u}_n)\right)+ \frac{\lambda_n}{2}\bar{u}_n\leq 0, ~\forall x\in A_n^* \end{equation}

holds for sufficiently small ϵ > 0.

For any $x_0\in A_n$ such that $B_r(x_0)\subset A_n^*$, consider the function

\begin{equation*} \phi_n(\sigma)=\phi_n(|x-x_0|)=\lambda_n^{\frac{1}{p-2}}\frac{\cosh{(\gamma \lambda_n^{\frac{1}{2}}\sigma})}{\cosh{(\gamma \lambda_n^{\frac{1}{2}}r})}. \end{equation*}

Clearly, $\phi'(\rho) \gt 0$ and $\phi^{\prime\prime}(\rho) \gt 0$. By direct computations, we obtain

(4.26)\begin{equation} \begin{aligned} & 2|\phi_n^{\prime\prime}|+\frac{2(N-1)}{\sigma}\phi_n'+ \frac{C_{\Phi,\Omega}}{\sigma}\phi_n'- \frac{\lambda_n}{2}\phi_n\\ &=\lambda_n\phi_n \left(2\gamma^2+ (2(N-1)+C_{\Phi,\Omega})\gamma^2 \frac{\tanh{(\gamma \lambda_n^{\frac{1}{2}}\sigma)}}{\gamma\lambda_n^{\frac{1}{2}}\sigma}-\frac{1}{2} \right)\\ &\leq \lambda_n\phi_n \left(2\gamma^2+ (2(N-1)+C_{\Phi,\Omega})\gamma^2 -\frac{1}{2} \right)\leq 0, \end{aligned} \end{equation}

for any $\gamma\in (0, \gamma_*]$, where $\gamma_*=(4N+2C_{\Phi,\Omega})^{-\frac{1}{2}}$. Then, fixing $\gamma\in (0, \gamma_*]$, by (4.24) and (4.26) it follows that

(4.27)\begin{equation} \begin{aligned} div(A(x, \nabla \phi_n))-\frac{\lambda_n}{2}\phi_n =\sum_{i=1}^N \frac{\partial}{\partial x_i}\left(\sqrt{\bar{g}}\sum_{j=1}^N\bar{g}^{ij}\frac{\partial \phi}{\partial x_j}\right)-\frac{\lambda_n}{2}\phi_n \leq 0, ~\forall x\in B_r(x_0). \end{aligned} \end{equation}

In addition, for $x\in \partial B_r(x_0)$, we have $\left(\phi_n -\bar{u}_n \right) \geq \lambda_n^{\frac{1}{p-2}}(1-\epsilon) \gt 0$. Then, together with (4.25) and (4.27), by the comparison principle ([Reference Pucci and Serrin52, theorem 10.1]) it follows that $\bar{u}_n\leq \phi_n~~{\mathrm{in}}~~B_r(x_0)$, which implies that $u_n(x_0)\leq \lambda_n^{\frac{1}{p-2}} e^{-\gamma\lambda_n^{\frac{1}{2}}r}$.

Take $r={\rm dist}(x_0, \partial A_n^*)$. We distinguish the following two cases:

1. (i) $dist(x_0, \partial A_n^*)= dist(x_0, B_{R\lambda_n^{-1/2}}(P_n^i))$ for some $i\in \{1,\ldots,k\}$;

2. (ii) $dist(x_0, \partial A_n^*)= dist(x_0, \partial(\overline{\Omega}\cup \Omega^{\theta_*}))$.

For case (i), we have $|x_0- P_n^i|= r+ R\lambda_n^{-\frac{1}{2}}$. Then

\begin{equation*}u_n(x_0)\leq e^{\gamma R}\lambda_n^{\frac{1}{p-2}} e^{-\gamma\lambda_n^{\frac{1}{2}}r- \gamma R}=e^{\gamma R}\lambda_n^{\frac{1}{p-2}} e^{-\gamma \lambda_n^{\frac{1}{2}}|x_0-P_n^i|}.\end{equation*}

For case (ii), we deduce that

\begin{equation*}u_n(x_0)\leq \lambda_n^{\frac{1}{p-2}} e^{-\gamma \lambda_n^{\frac{1}{2}}\theta_*}\leq \lambda_n^{\frac{1}{p-2}} e^{-\gamma \lambda_n^{\frac{1}{2}} \frac{\theta_*}{diam (\Omega)}|x_0-P_n^i|}.\end{equation*}

Hence, since x ₀ is arbitrary, we conclude that there exist $C_1, C_2 \gt 0$ such that

\begin{equation*} u_n(x)\leq C_1 e^{C_1 R}\lambda_n^{\frac{1}{p-2}}\sum_{i=1}^k e^{-C_2\lambda_n^{\frac{1}{2}}|x-P_n^i|}, ~\forall x\in A_n. \end{equation*}

This completes the proof.

Proof. First, we show that $\{\lambda_n\}$ is bounded. Suppose, by contradiction, that $\lambda_n\to +\infty$. By lemma 4.4, there exist at most k blow-up limits $\{P_n^1\},\ldots,\{P_n^k\}$ with $k\leq 2$ in $\overline{\Omega}$. In the following, we denote by $\{v_n^i \}$ the scaled sequence around $\{P_n^i\}$, and by vⁱ the limits of $\{v_n^i\}$. Note that for these blow-up points $\{P_n^1\},\ldots,\{P_n^k\}$, it may hold

\begin{equation*} \lambda_n^{\frac{1}{2}}dist(P_n^i,\partial \Omega)\to +\infty ~~{\mbox or}~~\lambda_n^{\frac{1}{2}}dist(P_n^i,\partial \Omega)\to d\geq 0. \end{equation*}

Without loss of generality, assume there exists an integer $k_1\in \{0, 1,\ldots, k\}$ such that

\begin{eqnarray*} \lambda_n^{\frac{1}{2}}dist(P_n^i,\partial \Omega)\to +\infty~~ \mbox{for every}~~ i\in \{1,\ldots, k_1\}, \end{eqnarray*}

and

\begin{eqnarray*} \lambda_n^{\frac{1}{2}}dist(P_n^j,\partial \Omega)\to d\geq 0~~ \mbox{for every}~~ j\in \{k_1+1,\ldots, k\}. \end{eqnarray*}

On the one hand, we can deduce that for any R > 0,

(5.1)\begin{equation} \bigg| \lambda_n ^{\frac{N}{2}-\frac{2}{p-2}}\int_{\Omega}u_n^2 dx-\sum_{i=1}^{k_1}\int_{B_R(0)}|v_n^i|^2 dx-\sum_{i=k_1+1}^{k}\int_{B_R(0)\cap {\Omega}_n}|v_n^i|^2 dx \bigg|\to +\infty. \end{equation}

In fact, since $p\in (2_*, 2^*)$, the first term satisfies

\begin{equation*} \bigg| \lambda_n ^{\frac{N}{2}-\frac{2}{p-2}}\int_{\Omega}u_n^2 dx \bigg|=\lambda_n ^{\frac{N}{2}-\frac{2}{p-2}}c\to +\infty. \end{equation*}

By lemma 4.4, we have

\begin{equation*} \sum_{i=1}^{k_1}\int_{B_R(0)}|v_n^i|^2 dx\to \sum_{i=1}^k \int_{B_R(0)} |v^i|^2 dx, \end{equation*}

and

\begin{equation*} \sum_{i=k_1+1}^{k}\int_{B_R(0)\cap \overline{\Omega}_n}|v_n^i|^2 dx\to \sum_{i=1}^k \int_{B_R(0)\cap \mathbb{R}_+^N} |v^i|^2 dx, \end{equation*}

which imply that (5.1) holds.

On the other hand, by lemma 4.5, there exist constants $C, C' \gt 0$ such that

\begin{eqnarray*} \begin{aligned} &\bigg| \lambda_n ^{\frac{N}{2}-\frac{2}{p-2}}\int_{\Omega}u_n^2 dx -\sum_{i=1}^{k_1}\int_{B_R(0)}|v_n^i|^2 dx-\sum_{i=k_1+1}^{k}\int_{B_R(0)\cap {\Omega}_n}|v_n^i|^2 dx \bigg|\\ &= \lambda_n^{\frac{N}{2}-\frac{2}{p-2}}\int_{\Omega\backslash \cup_{i=1}^{k} (B_{R\lambda_n^{-1/2}}(P_n^i)\cap \Omega)}u_n^2 dx\\ &\leq C_1 e^{2C_1 R}\lambda_n^{\frac{N}{2}}\sum_{i=1}^{k}\int_{\mathbb{R}^N\backslash \cup_{i=1}^{k}B_{R\lambda_n^{-1/2}}(P_n^i)}e^{-2C_2\lambda_n^{\frac{1}{2}}|x-P_n^i|} dx\\ &\leq C e^{C R}\sum_{i=1}^{k}\int_{\mathbb{R}^N\backslash B_R(0)} e^{-2C_2|y|} dy \leq C'=C'(R). \end{aligned} \end{eqnarray*}

Taking $n\to +\infty$, we obtain a contradiction to (5.1). Hence, $\{\lambda_n\}$ is bounded.

Now, we show that $\{u_n\}$ is bounded in $H^1(\Omega)$. By contradiction, we suppose that $\|u_n\|\to \infty$. Using standard arguments, we have $\|u_n\|_{L^\infty} \to \infty.$ Take $P_n\in \overline{\Omega}$ such that $|u_n(P_n)|=\|u_n\|_{L^{\infty}}$. Let $\tilde{\epsilon}_n=a_0^{-\frac{1}{2}}|u_n(P_n)|^{-\frac{p-2}{2}}$ and define

\begin{equation*} \tilde{u}_n(x):=a_0^{\frac{1}{p-2}}\tilde{\epsilon}_n^{\frac{2}{p-2}}u_n(\tilde{\epsilon}_nx+ P_n)~~{\rm for}~~ x\in \tilde{\Omega}_n:=\frac{\Omega-P_n}{\tilde{\epsilon}_n}. \end{equation*}

Clearly, $\tilde{u}_n$ satisfies (4.8). Using the boundedness of $\{\lambda_n\}$, we get $\lambda_n \tilde{\epsilon}_n^2 \to 0$ as $n\to \infty$. Then, up to a subsequence, $\tilde{u}_n\to \tilde{u}$ in $C^2_{loc}(\overline{D})$, where $\tilde{u}$ is a finite Morse index solution of

(5.3)\begin{equation} \left\{\begin{array}{ll} -\Delta \tilde{u} =|\tilde{u}|^{p-2}\tilde{u} & {\mathrm{in}} \,~ D,\\ |\tilde{u}(x)|\leq |\tilde{u}(0)|=1 & {\mathrm{in}} \,~ D, \\ \frac{\partial \tilde{u}}{\partial \nu}=0 \, &{\mathrm{on}}\,~\partial D, \end{array}\right. \end{equation}

where D is either $\mathbb{R}^N$ or a half space. By invoking [Reference Farina22, theorem 2] and [Reference Yu57, proposition 2.1], respectively, we conclude that $\tilde{u}\equiv 0$. This contradicts to the fact that $|\tilde{u}(0)|=1$. Therefore, we deduce that $\{u_n\}$ is a bounded sequence in $H^1(\Omega)$. Consequently, employing standard arguments, we establish that $\{u_n \}$ is a bounded Palais–Smale sequence for $J|_{\mathcal{S}_c}$ at level c ₁.

Completion of proof of theorem 1.1

Let $\{u_n\}$ be the sequence given in proposition 5.1 for some $c\in (0, c^*)$. By the compact embedding $H^1(\Omega)\hookrightarrow L^r(\Omega)$ for $r\in [1,2^*)$, and using similar arguments as in §3, we can deduce that $u_{n}\to u$ strongly in $H^1(\Omega)$. This, in turn, implies that u is a mountain pass type normalized solution of (1.1).