Cosmological instability of scalarGaussBonnet theories exhibiting scalarization
Abstract
In a subclass of scalartensor theories, it has been shown that standard general relativity solutions of neutron stars and black holes with trivial scalar field profiles are unstable. Such an instability leads to solutions which are different from those of general relativity and have nontrivial scalar field profiles, in a process called scalarization. In the present work we focus on scalarization due to a nonminimal coupling of the scalar field to the GaussBonnet curvature invariant. The coupling acts as a tachyonic mass for the scalar mode, thus leading to the instability of general relativity solutions. We point out that a similar effect may occur for the scalar modes in a cosmological background, resulting in the instability of cosmological solutions. In particular, we show that a catastrophic instability develops during inflation within a period of time much shorter than the minimum required duration of inflation. As a result, the standard cosmological dynamics is not recovered. This raises the question of the viability of scalarGaussBonnet theories exhibiting scalarization.
a]T. Anson, a,b]E. Babichev, a]C. Charmousis, c]S. Ramazanov
Prepared for submission to JCAP
Cosmological instability of scalarGaussBonnet theories exhibiting scalarization

Laboratoire de Physique Théorique, CNRS, Univ. ParisSud,
Université ParisSaclay, F91405 Orsay, France 
Sorbonne Université, CNRS, UMR7095, Institut d’Astrophysique de Paris, ,
98bis boulevard Arago, F75014 Paris, France 
CEICO, Institute of Physics of the Czech Academy of Sciences, Na Slovance 1999/2, 182 21 Prague 8, Czech Republic
Contents
1 Introduction
Scalartensor theories with higher order derivatives [1, 2, 3] have attracted a lot of attention recently, as they represent testable modifications of general relativity (GR). A particular class of scalartensor theories involve the GaussBonnet invariant, which is of second order in curvature. While per se this is a purely topological term in 4 dimensional metric theories, it becomes dynamical (and free of the Ostrogradsky ghost), when coupled to a scalar degree of freedom. The conditions imposed on the coupling function ensure that GR solutions with a trivial scalar field configuration always exist in this setup. However, this scalartensor theory may have other solutions for black holes and stars, which are different from those of GR. The reason is that the GaussBonnet term induces an effective mass for the scalar mode. When the effective mass is chosen to be tachyonic, GR solutions may become unstable in regions of strong curvature, while stable black hole or star solutions (if existent) acquire scalar hair absent in GR [4, 5, 6, 7, 8](See also [9]). This phenomenon of coexisting solutions with the same symmetry but different scalar profiles has been dubbed scalarization. In fact scalarization with the GaussBonnet term is quite similar to the wellknown scalarization in “standard” scalartensor theories [10, 11], where the role of the effective mass is played by the Ricci scalar instead of the GaussBonnet invariant. Interestingly, “standard” scalarization operates only in the presence of matter, i.e., that composing neutron stars, while the GaussBonnet term additionally allows for the scalarization of black holes. This is due to the fact that the Ricci scalar is zero in vacuum, unlike the GaussBonnet invariant.
In this paper we point out that the term leading to the scalarization of compact objects may also be dangerous for the stability of cosmological solutions due to the same effect, namely the appearance of a tachyonic mass of the scalar mode^{1}^{1}1A similar study for the case of “standard” scalarization has been done in [12].. Indeed, as we will show in Sec. 3, the mass squared of the scalar mode is positive in the case of a decelerating FriedmannLemaîtreRobertsonWalker (FLRW) Universe, while it is negative for an accelerating Universe. On the one hand, this means that the cosmological GR solution is safe at radiation and matter dominated epochs. That is, the presence of the GaussBonnet term in the action does not destabilize the GR dynamics at those times. For the presentday accelerating phase, the effective mass of the scalar becomes tachyonic. Nevertheless, given the small Hubble rate the instability is very mild: it develops at time scales largely exceeding the age of the Universe. Therefore the solution stays very close to the GR cosmological solution for a long time.
On the other hand, the instability triggered by the scalarGaussBonnet coupling is disastrous during inflation, terminating the latter well before the ‘canonical’ 60 efolds elapse. We demonstrate in Sec. 4 that the primordial quantum fluctuations of the scalar field are quickly amplified to the point where they become dominant, thus destabilizing inflation. This suggests that the theory exhibiting scalarization via the coupling to the GaussBonnet term may not be physically viable, and one needs to look for ultraviolet extensions or modifications of the theory. In Sec. 5 we conclude with discussions on possible ways to avoid this catastrophic instability of the scalar mode during inflation.
2 The model and scalarization of compact objects
We consider the following action, which includes a coupling term between a scalar field and the GaussBonnet invariant :
(2.1) 
where is the cosmological constant. The EulerLagrange equation for the scalar field reads:
(2.2) 
The variation of (2.1) with respect to the metric yields modified Einstein equations:
(2.3) 
Here is the double dual of the Riemann tensor, defined using the antisymmetric LeviCivita tensor as:
(2.4) 
Under certain conditions on the coupling function , namely
(2.5) 
for some constant , such theories were shown to exhibit spontaneous scalarization around black holes and neutron stars [4, 5, 6, 7, 8, 13]. This mechanism is similar to the one presented in [10, 11], where the scalar field is coupled to the trace of the energymomentum tensor and scalarization was shown to arise around neutron stars only. In the present case, scalarization results from the coupling of the scalar field to the GaussBonnet invariant [4, 5, 6, 7, 8, 13] and occurs for black holes as well.
The first condition on the coupling function ensures that passes through the field equations, in which case we recover the solution of general relativity (GR). The second condition is crucial, as it implies that the scalar field acquires a negative effective mass, which is responsible for the appearance of scalar hair via a tachyonic instability. Indeed, one may study a scalar perturbation on a fixed Schwarzschild geometry, as was done in [4]. Notably, equations for the scalar and metric fluctuations are decoupled to the first order for constant . The equation describing scalar perturbations is given by
(2.6) 
where the d’Alembert operator and the GaussBonnet invariant are calculated with the Schwarzschild metric. One decomposes the scalar perturbation on the static and spherically symmetric background as
(2.7) 
where are the spherical harmonics. One can rewrite Eq. (2.7) in the form of a Schrödinger equation by introducing the tortoise coordinate , where is the Schwarzschild radius of the black hole:
(2.8) 
For the effective potential reads
(2.9) 
A sufficient condition on the effective potential for the existence of an unstable mode is [14]
(2.10) 
This condition, which may be satisfied if , translates into
(2.11) 
Thus the Schwarzschild solution becomes unstable for small enough masses, and one expects scalar hair to appear in that case. Bearing in mind the possible redefinition , the function can be expanded around ,
(2.12) 
Here is some scale normally taken to be of order of the Planck mass , and the sign of the quadratic term is chosen so that the effective mass is tachyonic. The value of sets the upper bound of the mass of a black hole or a star at which scalarization may happen, For physically interesting objects, e.g., neutron stars, one easily finds
(2.13) 
Different branches of scalarized solutions were shown to exist by a numerical analysis, for specific bands of . Each branch may be labelled by an integer corresponding to the number of nodes of the radial scalar profile. It was shown in [15] that none of the branches are stable for a theory with a purely quadratic coupling function, . However, for the theory with the fundamental branch is stable for a specific range of parameters. The same can be achieved by adding a quartic term to the purely quadratic coupling function [16, 17].
In the next sections we study the cosmological stability of scalarGaussBonnet theories with values of the coupling parameter that lead to scalarized solutions around compact objects.
3 (In)stability of perturbations on FLRW background
In this section, we show that the tachyonic mass leading to the scalarization of black holes and stars is potentially dangerous on a cosmological background, and may result in a catastrophic instability of scalar modes.
Perturbing the EOM for the scalar field (2.2) one obtains
(3.1) 
where is the perturbation of the scalar field. Again, there are no terms involving in the above equation since the scalar field is constant on the background. This equation defines an effective mass for the scalar perturbation, where . Assuming the coupling (2.12), the effective mass squared can be written as
(3.2) 
As we have discussed above, the case of real corresponds to theories with scalarized stars and black holes. For a flat FLRW background with the metric
(3.3) 
where is the scale factor, the perturbation equation (3.1) takes the form
(3.4) 
Here is the Hubble parameter, and we have assumed the coupling function of the form (2.12). We now expand the scalar perturbation into Fourier modes,
to obtain the following dispersion relation:
(3.5) 
In the above equation we have neglected the slow change of on the time scales shorter than . Whether the mass of the scalar perturbations is real or tachyonic depends on the sign of for the flat FLRW metric (3.3),
(3.6) 
Thus for a decelerating Universe, , i.e., during radiation and matter dominated epochs, the mass of the scalar is real, , and no instability arises. On the other hand, for an accelerated expansion, , the mass (3.2) becomes tachyonic and the perturbations are unstable. In particular, for a de Sitter solution with constant Hubble rate , one finds
(3.7) 
The instability is extremely slow for the presentday acceleration of the Universe. Indeed, the ratio of the instability time to the the age of the Universe is estimated as
(3.8) 
so that the instability is not noticeable. However, the same estimation for inflation with the scale GeV and efoldings yields:
(3.9) 
Thus the GR solution with has a very short instability time. This suggests a catastrophic instability of the theory during the inflation era, unless the initial value of the field is tuned to be extremely small. Indeed, as the scalar field grows as starting from some initial value , one may choose arbitrarily close to 0, so that the instability does not develop during the time of inflation. In particular, if , the field will stay on the top of the potential for an infinitely long time. However, even for the solution with , quantum fluctuations of the scalar field rapidly grow and ultimately destroy the inflationary stage. This happens during a time which is much smaller than the duration of inflation, as we explicitly show in the next section.
4 Catastrophic instability of the scalar field during inflation
In this section we show that the tachyonic instability inherent to the scalar field is inconsistent with the existence of inflation in the early Universe. We approximate inflation by an exact de Sitter expansion, which is a plausible assumption away from its final stages. From (2.3) one can find that in the presence of the field coupled to the Gauss–Bonnet term, the Friedmann equation is modified by the term
(4.1) 
We require that the term (4.1) be negligible compared to the inflaton energy density dominating the evolution of the Universe, i.e.,
(4.2) 
Our goal is to show that this condition gets violated quickly, namely as soon as inflation starts. Even if the field is set at the top of the effective potential initially, , its perturbations will cause a rapid instability. Perturbations of the field obey the equation (3.4) with the effective mass given by (3.7) with being the Hubble scale during inflation. We comment on deviations of the function from a simple quadratic form at the end of this section.
In the following we will primarily use the conformal time , related to the cosmic time as
Conformal time is negative during inflation, and the scale factor verifies . The solution of the perturbation equation (3.4), assuming that the field is in the Bunch–Davies vacuum initially, is given by
(4.3) 
where we have expanded into Fourier modes. Here and are the annihilation and creation operators, respectively, and are the Hankel functions of order
Note that the evolution of inflaton perturbations is described by . In that case, perturbations get frozen as they exit the horizon. On the other hand, in the situation with —the case of our interest—perturbations grow fast beyond the horizon, as we will see shortly. This behaviour is due to the huge tachyonic mass acquired by the scalar field. From (4.3), the derivative is given by
The derivative of the Hankel function is expressed as follows:
We are working in the large regime. In this limit, the Hankel functions take the form:
(4.4) 
(minus and plus signs correspond to the Hankel functions of the first and second kind, respectively). We see that it has a very sharp dependence on the order . Hence, one can keep only Hankel functions with the largest when calculating the relevant correlation function:
(4.5) 
where we made use of the commutation relation and integrated over the directions of the momenta . As the lower limit of the above integral we choose some , which is onhorizon at some moment during inflation, i.e., , but otherwise is arbitrary. We choose , so that corresponds to the past with respect to . As for the upper limit we take . The result reads
Finally, we use that and implement condition (4.2) to obtain the ratio at which inflation gets violated:
(4.6) 
or equivalently
We see that the modes with the momenta destabilize inflation immediately after exiting the horizon. Given an essentially arbitrary choice of , we conclude that inflation cannot take place in this theory.
One remark is in order here. Recall that our calculations imply the existence of modes which start from the Bunch–Davies vacuum and exit the horizon during inflation. This seemingly modest assumption requires some justification in the situation with exponentially large . The asymptotic expansion of the Hankel functions for large arguments is given by
The Bunch–Davies vacuum is defined in the limit , when the second term in the brackets is irrelevant. In practice, however, inflation has a finite duration. Thus, the quantity is also large, but finite. Hence, the minimum value of momenta which are in the Bunch–Davies vacuum at the beginning of inflation is given by
(4.7) 
It is possible, in principle, that the modes with these large momenta never exit the horizon during inflation, formally invalidating our analysis. Let us show that this is not the case, unless the duration of inflation is tuned to its minimum value. By integrating Eq. (4.5) from to infinity (i.e. ), we get
We now apply the condition Eq. (4.2), leading to
Combining the latter with Eq. (4.7), we obtain
which should hold until the end of inflation . Unless the duration of inflation is close to efoldings, the ratio is very large, even compared to the huge value used for scalarization of the stars. Hence, the modes, which are in the Bunch–Davies initially and exit the horizon during inflation, alone destabilize inflation already efolds after it starts.
Let us also comment on a possible stabilization of the scalar field due to the presence of the quartic corrections, since one should include those anyway to make the scalarized branch of compact objects physically viable [16, 17]. It is not difficult to see, however, that the quartic terms cannot stabilize the field and prevent the catastrophic instability, because destabilization occurs at tiny values of , for which the approximation still holds. Indeed, from Eq. (4.3) we obtain
Using Eq. (4.4) and integrating over the modes in the range , we get
(4.8) 
Finally, substituting Eq. (4.6) into Eq. (4.8), we obtain the typical value of the scalar when inflation is violated:
where we used that . Given that , we conclude that the instability develops at the field values in the theory exhibiting scalarization, i.e., well before the quartic term starts to act.
5 Discussion and conclusions
In this paper we studied the stability of cosmological solutions in theories exhibiting scalarization of black holes and neutron stars due to the coupling of the scalar to the GaussBonnet invariant. The scalarization of compact objects happens because the scalar acquires a tachyonic mass and its perturbations destabilize the GR solutions. It is natural to question whether an analogous destabilization arises on FLRW backgrounds. This has been the subject of our study.
As we have shown in Sec. 3, stability of the scalar perturbations on GR cosmological solutions depends on the sign of the acceleration , see Eqs. (3.2) and (3.6). For a decelerating Universe, the mass in the equation for the scalar perturbation is real, therefore no instability arises for the GR branch. However, the mass becomes tachyonic for an accelerated expansion, and one expects the GR cosmological solution to become unstable in that case. It turns out that this instability is very slow with respect to the current acceleration, with the time of instability being much larger than the age of the Universe, see Eq. (3.8).
On the contrary, the scalarGaussBonnet coupling leads to a catastrophic instability during the inflationary epoch, due to its huge effective tachyonic mass. The characteristic instability time, at which the amplitude of a classical solution increases by the factor , is given in Eq. (3.9). This is a tiny number compared to the time of inflation. One should take into account that classically the initial conditions of the scalar field can be tuned such that the field stays on top of the potential for an infinite time, at least in principle. However, as we explicitly show in Sec. 4, quantum fluctuations of the scalar field are strong enough to trigger this instability, making any conventional inflationary scenario impossible in the theories exhibiting scalarization under study (2.1).
Throughout this study we have assumed that the scalarization field is not the inflaton field. However, the GaussBonnet scalar is an ultraviolet correction which naturally modifies gravity at early times. Indeed inflation models where the GaussBonnet scalar is coupled to the inflaton field according to our action (2.1) have been studied in the literature (see for instance [18, 19, 20]). However, in these studies, the effective sign of the coupling function in (2.1) is crucially required to be of the opposite sign to the one required for scalarization.
It would be interesting to find ways to stabilize the theories exhibiting scalarization during inflation. For instance, one may try to solve the problem by adding a coupling to the inflaton , e.g., , where is the coupling constant. In this case the inflaton expectation value serves as a stabilizing effective mass, which vanishes as inflation ends. The problem here is that the coupling constant should be huge in this approach. Indeed, to balance the large value of given by Eq. (3.7), one needs to assume that
for , , and GeV. Thus, the theory is deeply in the strong coupling regime, where no trustworthy predictions can be made.
Another idea would be to add an extra coupling of the scalar to higher powers of curvature, so that it becomes dominant during inflation. The extra term should be higher than the second order in the curvature, otherwise one risks to spoil scalarization triggered by the coupling to the Gauss–Bonnet invariant, which is of the second order in the curvature. For example, the coupling with the appropriate sign stabilizes the scalar field in the highcurvature regime. In this case, similarly to gravity, an extra scalar degree of freedom is effectively introduced^{2}^{2}2We thank Gilles EspositoFarèse for pointing this out., which is coupled to . However, the coupling between and the scalar from the gravitational sector must be huge in order to balance the term involving the Gauss–Bonnet invariant. As we have discussed above, the introduction of a strong coupling is not a viable solution.
Yet another possible way to fix the catastrophic instability during inflation is to introduce the quartic terms in the scalar, . It has been shown that such terms help to stabilize the scalarized solutions of compact objects, see, e.g., Refs. [16, 17]. However, as we have seen in Sec. 4, destabilization of inflation happens already for very small , where the quartic term does not play any role. Adding a large coupling , i.e., writing does not improve the situation for the following reason. We presume that the solutions for the scalarized compact objects will be indistinguishable from GR ones, since the scalar field will have values extremely close to zero due to the stabilizing quartic term.
Acknowledgments
We thank Gilles EspositoFarèse for interesting discussions. T.A., E.B. and C.C. acknowledge support from the research program PRC CNRS/RFBR (2018–2020) n^{o}1985 “Gravité modifiée et trous noirs: signatures expérimentales et modèles consistants”. S.R. is supported by the European Regional Development Fund and the Czech Ministry of Education, Youth and Sports through the Project CoGraDSCZ.02.1.01/0.0/0.0/15 003/0000437.
References
 [1] G. W. Horndeski, “Secondorder scalartensor field equations in a fourdimensional space,” Int. J. Theor. Phys. 10 (1974) 363–384.
 [2] C. Deffayet, G. EspositoFarese, and A. Vikman, “Covariant Galileon,” Phys. Rev. D79 (2009) 084003, arXiv:0901.1314 [hepth].
 [3] C. Deffayet, S. Deser, and G. EspositoFarese, “Generalized Galileons: All scalar models whose curved background extensions maintain secondorder field equations and stresstensors,” Phys. Rev. D80 (2009) 064015, arXiv:0906.1967 [grqc].
 [4] D. D. Doneva and S. S. Yazadjiev, “New GaussBonnet Black Holes with CurvatureInduced Scalarization in Extended ScalarTensor Theories,” Phys. Rev. Lett. 120 no. 13, (2018) 131103, arXiv:1711.01187 [grqc].
 [5] D. D. Doneva and S. S. Yazadjiev, “Neutron star solutions with curvature induced scalarization in the extended GaussBonnet scalartensor theories,” JCAP 1804 no. 04, (2018) 011, arXiv:1712.03715 [grqc].
 [6] H. O. Silva, J. Sakstein, L. Gualtieri, T. P. Sotiriou, and E. Berti, “Spontaneous scalarization of black holes and compact stars from a GaussBonnet coupling,” Phys. Rev. Lett. 120 no. 13, (2018) 131104, arXiv:1711.02080 [grqc].
 [7] G. Antoniou, A. Bakopoulos, and P. Kanti, “Evasion of NoHair Theorems and Novel BlackHole Solutions in GaussBonnet Theories,” Phys. Rev. Lett. 120 no. 13, (2018) 131102, arXiv:1711.03390 [hepth].
 [8] G. Antoniou, A. Bakopoulos, and P. Kanti, “BlackHole Solutions with Scalar Hair in EinsteinScalarGaussBonnet Theories,” Phys. Rev. D97 no. 8, (2018) 084037, arXiv:1711.07431 [hepth].
 [9] Y. S. Myung and D.C. Zou, “GregoryLaflamme instability of black hole in EinsteinscalarGaussBonnet theories,” Phys. Rev. D98 no. 2, (2018) 024030, arXiv:1805.05023 [grqc].
 [10] T. Damour and G. EspositoFarese, “Nonperturbative strong field effects in tensor  scalar theories of gravitation,” Phys. Rev. Lett. 70 (1993) 2220–2223.
 [11] T. Damour and G. EspositoFarese, “Tensor  scalar gravity and binary pulsar experiments,” Phys. Rev. D54 (1996) 1474–1491, arXiv:grqc/9602056 [grqc].
 [12] D. Anderson, N. Yunes, and E. Barausse, “Effect of cosmological evolution on Solar System constraints and on the scalarization of neutron stars in massless scalartensor theories,” Phys. Rev. D94 no. 10, (2016) 104064, arXiv:1607.08888 [grqc].
 [13] A. Bakopoulos, G. Antoniou, and P. Kanti, “Novel BlackHole Solutions in EinsteinScalarGaussBonnet Theories with a Cosmological Constant,” arXiv:1812.06941 [hepth].
 [14] W. Buell and B. A. Shadwick, “Potentials and bound states,” American Journal of Physics  AMER J PHYS 63 (03, 1995) 256–258.
 [15] J. L. BlÃ¡zquezSalcedo, D. Doneva, J. Kunz, and S. S. Yazadjiev, “Radial perturbations of the scalarized EGB black holes,” arXiv:1805.05755 [grqc].
 [16] M. Minamitsuji and T. Ikeda, “Scalarized black holes in the presence of the coupling to GaussBonnet gravity,” arXiv:1812.03551 [grqc].
 [17] H. O. Silva, C. F. B. Macedo, T. P. Sotiriou, L. Gualtieri, J. Sakstein, and E. Berti, “On the stability of scalarized black hole solutions in scalarGaussBonnet gravity,” arXiv:1812.05590 [grqc].
 [18] P. Kanti, R. Gannouji, and N. Dadhich, “GaussBonnet Inflation,” Phys. Rev. D92 no. 4, (2015) 041302, arXiv:1503.01579 [hepth].
 [19] P. Kanti, R. Gannouji, and N. Dadhich, “Earlytime cosmological solutions in EinsteinscalarGaussBonnet theory,” Phys. Rev. D92 no. 8, (2015) 083524, arXiv:1506.04667 [hepth].
 [20] G. Hikmawan, J. Soda, A. Suroso, and F. P. Zen, “Comment on âGaussBonnet inflationâ,” Phys. Rev. D93 no. 6, (2016) 068301, arXiv:1512.00222 [hepth].