Spectral Analysis of the Incompressible Viscous Rayleigh–Taylor System in $${\mathbf {R}}^3$$ R 3

The linear instability study of the viscous Rayleigh–Taylor model in the neighborhood of a laminar smooth increasing density profile $$\rho _0(x_3)$$ amounts to the study of the following ordinary differential equation of order 4 0.1 $$\begin{aligned} -\lambda ^2 [ \rho _0 k^2 \phi - (\rho _0 \phi ')'] = \lambda \mu (\phi ^{(4)} - 2k^2 \phi '' + k^4 \phi ) - gk^2 \rho _0'\phi , \end{aligned}$$ where $$\lambda $$ is the growth rate in time, and k is the wave number transverse to the density profile. In the case of $$\rho '_0\ge 0$$ compactly supported, we provide a spectral analysis showing that in accordance with the results of Helffer and Lafitte (Asymptot Anal 33:189–235, 2003), there is an infinite sequence of non-trivial solutions $$(\lambda _n, \phi _n)$$ of (0.1), with $$\lambda _n\rightarrow 0$$ when $$n\rightarrow +\infty $$ and $$\phi _n\in H^4({\mathbf {R}})$$ . In the more general case where $$\rho _0'>0$$ everywhere and $$\rho _0$$ converges at $$\pm \infty $$ to finite limits $$\rho _{\pm }>0$$ , we prove that there exist finitely non-trivial solutions $$(\lambda _n, \phi _n)$$ of (0.1). The line of investigation is to reduce both cases to the study of a self-adjoint operator on a compact set.