Positive ground state solutions for fractional Laplacian system with one critical exponent and one subcritical exponent

Abstract: In this paper, we consider the following fractional Laplacian system with one critical exponent and one subcritical exponent \begin{equation*} \begin{cases} (-\Delta)^{s}u+\mu u=|u|^{p-1}u+\lambda v & x\in \ \mathbb{R}^{N}, (-\Delta)^{s}v+\nu v = |v|^{2^{\ast}-2}v+\lambda u& x\in \ \mathbb{R}^{N},\ \end{cases} \end{equation*} where $(-\Delta)^{s}$ is the fractional Laplacian, $0<s\<1,\ N\>2s, \ \lambda <\sqrt{\mu\nu },\ 1<p\<2^{\ast}-1~ and~\ 2^{\ast}=\frac{2N}{N-2s}$~ is the Sobolev critical exponent. By using the Nehari\ manifold, we show that there exists a $\mu_{0}\in(0,1)$, such that when $0<\mu\leq\mu_{0}$, the system has a positive ground state solution. When $\mu>\mu_{0}$, there exists a $\lambda_{\mu,\nu}\in[\sqrt{(\mu-\mu_{0})\nu},\sqrt{\mu\nu})$ such that if $\lambda>\lambda_{\mu,\nu}$, the system has a positive ground state solution, if $\lambda<\lambda_{\mu,\nu}$, the system has no ground state solution.