Astrophysical Evidence for the Non-Hermitian but -symmetric Hamiltonian of Conformal Gravity
In this review we discuss the connection between two seemingly disparate topics, macroscopic gravity on astrophysical scales and Hamiltonians that are not Hermitian but symmetric on microscopic ones. In particular we show that the quantum-mechanical unitarity problem of the fourth-order derivative conformal gravity theory is resolved by recognizing that the scalar product appropriate to the theory is not the Dirac norm associated with a Hermitian Hamiltonian but is instead the norm associated with a non-Hermitian but -symmetric Hamiltonian. Moreover, the fourth-order theory Hamiltonian is not only not Hermitian, it is not even diagonalizable, being of Jordan-block form. With symmetry we establish that conformal gravity is consistent at the quantum-mechanical level. In consequence, we can apply the theory to data, to find that the theory is capable of naturally accounting for the systematics of the rotation curves of a large and varied sample of 138 spiral galaxies without any need for dark matter. The success of the fits provides evidence for the relevance of non-diagonalizable but -symmetric Hamiltonians to physics.
1 Introduction and Overview
On January 19, 2000 a paper by myself and Aharon Davidson entitled “Fourth order theories without ghosts” appeared on the arXiv . The abstract of the 2000 paper read:
“Using the Dirac constraint method we show that the pure fourth-order Pais-Uhlenbeck oscillator model is free of observable negative norm states. Even though such ghosts do appear when the fourth order theory is coupled to a second order one, the limit in which the second order action is switched off is found to be a highly singular one in which these states move off shell. Given this result, construction of a fully unitary, renormalizable, gravitational theory based on a purely fourth order action in 4 dimensions now appears feasible.”
Also in the 2000 paper we identified a particular Hamiltonian and noted of it: “we see that the complete spectrum of eigenstates of is the set of all states , a spectrum whose dimensionality is that of a one rather than a two-dimensional harmonic oscillator, even while the complete Fock space has the dimensionality of the two-dimensional oscillator.”
To be specific, what we had found in this paper and its follow-ups [2, 3] was that in the one-particle sector the Hamiltonian of the pure fourth-order Pais-Uhlenbeck oscillator model took the generic form
and that even though the secular equation had two solutions (both equal to ), only had one eigenvector since the eigenvalue equation
only had one eigenvector solution, viz. the one that obeyed
The one-particle sector Hamiltonian was thus of a non-diagonalizable, Jordan-block form. And with its missing an eigenvector, its eigenvector basis was incomplete (in consequence of which the two energy eigenvalues had to be equal since they had to share a common eigenvector), and could not be diagonalized by a similarity transformation. Even though the Hamiltonian was not Hermitian, nonetheless all of its eigenvalues (as well as those in the multi-particle sectors of the theory) were real. The Pais-Uhlenbeck Hamiltonian thus serves as an example of a non-Hermitian Hamiltonian that has all eigenvalues real, to thus show that while Hermiticity of an operator is sufficient to yield real eigenvalues, it is not necessary.
Aharon Davidson and I had been exploring the Pais-Uhlenbeck oscillator model because it serves as a prototype for the fourth-order conformal gravity theory that I had been advancing (see e.g. [4, 5]) as a candidate alternative to standard gravity and string theory. My interest in the conformal gravity theory was triggered by the fact that in a conformal invariant theory the cosmological constant would have to be zero, with conformal gravity thus serving as a good starting point for addressing the cosmological constant problem. Problematically though, when quantized, fourth-order theories such as conformal gravity were thought to possess negative norm states and not be unitary. However, through our work Aharon Davidson and I were able to show that the theory could solve its ghost problem by having the ghosts not appear as on-shell energy eigenstates, to thereby potentially open the road to the construction of a consistent quantum gravitational theory in four spacetime dimensions.
As such, the pure fourth-order Pais-Uhlenbeck oscillator model provides an example of a -symmetric Hamiltonian at a so-called exceptional point – a point at which the energy eigenspectrum is incomplete and the Hamiltonian cannot be brought to a diagonal form by a similarity transformation. However, back in 2000 I had no idea that the theory was a theory, let alone one at an exceptional point, and only learned of it from Carl Bender in 2007 when the two of us started to collaborate on the theory [6, 7, 8]. With Carl Bender and his collaborators only having started to develop the concept of non-Hermitian but -symmetric Hamiltonians in 1998 by showing that such Hamiltonians could have real eigenvalues (see e.g. ), my 2000 paper with Aharon Davidson thus provides one of the first examples of a theory with a non-Hermitian but -symmetric Hamiltonian at an exceptional point.
As first presented by Pais and Uhlenbeck , the Pais-Uhlenbeck model is based on an action
that describes the dynamics of a single, non-relativistic coordinate . The associated Euler-Lagrange equation of motion is of the fourth-order derivative form
and the associated propagator is of the form
viz. a propagator whose poles are entirely on the real energy axis.
Two things in the structure of the propagator immediately stand out: the presence of the relative minus sign in the unequal-frequency partial fraction decomposition and the singularity of the prefactor in this decomposition in the equal-frequency limit. We thus recognize two distinct sectors of the theory, the non-singular case of unequal frequencies () and the singular case of equal frequencies (). Prior to my work with Carl Bender the presence of the relative minus sign in the unequal-frequency propagator had always been taken to indicate that some of the states in the theory would have to have negative norm (since some of the poles would have negative residues), with the theory then not being unitary. And the result obtained by myself and Aharon Davidson was that even if there were such negative norm states in the unequal-frequency theory, because of the singular nature of the equal-frequency limit, in the limit the Hamiltonian would lose some of eigenstates, and none of the energy eigenstates that remained would have negative norm.
My collaboration with Carl Bender then went further by establishing that the unequal-frequency theory itself had a realization that was unitary, and that it too could be reinterpreted as a rather than a Hermitian theory, and that then even in the unequal-frequency case there would be no states of negative norm. Thus for both the unequal-frequency and equal-frequency theories the Hamiltonian would not be Hermitian. However, since all the poles of the propagator lie on the real axis, all energy eigenvalues would be real, with the Hamiltonian then being symmetric. At the time Carl Bender and I first studied the Pais-Uhlenbeck model we showed that the Hamiltonian was symmetric simply by examining the behavior of the fields in it under a transformation. Subsequently, we established a general theorem  that stated that if a Hamiltonian did not have a symmetry (as understood in general to mean the product of discrete linear and discrete anti-linear operators), some of its eigenvalues would have to be complex, with no non--symmetric Hamiltonian then being able to have a set of eigenvalues that was entirely real. Since the energy eigenvalues of the Pais-Uhlenbeck theory were all real and the Hamiltonian was not Hermitian, the theory thus had to be a theory.
While my own interest had primarily been in the equal-frequency theory since it is a non-relativistic prototype of pure fourth-order gravity, the unequal-frequency theory is of interest not only because it is the quantum-mechanical limit of the often studied second-order plus fourth-order gravity theory, but also because it allows one to monitor what precisely happens in the limit in which the second-order piece is switched off. If one ignores metric indices (which as we show below we can), the issues involved can be illustrated by consideration of the second-order plus fourth-order scalar field theory with action, equation of motion and propagator of the form
where . For this propagator a standard Feynman contour integration yields a positive frequency contribution of the form
where , , , . An identification of the propagator with the Green’s function would then require the closure relation for the energy eigenstates to be of the form
with the presence of negative residues in the propagator translating into the presence of negative norm states in the Hilbert space.
To avoid this outcome we must not identify the propagator with but must instead seek some other identification for it, one in which a closure relation involving negative norm states would not then appear. That we might be able to do this at all is due to the fact that the propagator is a c-number quantity that is constructed from a c-number differential equation of the form , and as such it itself makes no reference to any Hilbert space. There is thus some freedom in relating the c-number propagator to an underlying q-number field theory, with other possible quantum-field-theoretic representations of the propagator being possible. And indeed, it is only after second-quantizing the scalar field in a very specific way, viz. the canonical where the operator associated with the negative-frequency sector is expressly the Hermitian conjugate of the operator associated with the positive-frequency sector, that we then construct the propagator as a matrix element of products of field operators in a vacuum state that annihilates on the right according to and annihilates on the left according to .
While this canonical prescription is completely standard in quantum field theory, it was established in field theories where the Hamiltonian and the fields were all Hermitian. Thus in the non-Hermitian but -symmetric case we can no longer identify the propagator this way. Rather, in the case one must distinguish between left and right vacua. Specifically, if we expand the scalar field in creation and annihilation operators according to where the scalar field is not now Hermitian and is not the Hermitian conjugate of , the left and right vacua will be defined as the states that are annihilated according to , . In this case will not be the Hermitian conjugate of , and the propagator will be given not by the canonical but by instead. In the non-Hermitian but case and while distinct are related according to , where is the operator that relates a non-Hermitian Hamiltonian to its adjoint according to (see e.g.  and references therein). In this case the closure relation and normalization conditions are of the form
and the propagator is given by . With there being no states with , there are no states of negative norm, with it instead being the presence of the operator that generates the relative minus sign in the propagator. Specifically, the operator is related to the operator of theory , an operator that commutes with the Hamiltonian and obeys , with the relative minus sign in the propagator being due to the fact that the eigenvalues of the operator are and . The relative minus sign is thus associated with the operators and not with any negative norm states. Thus what had led to the belief that the propagator had to be associated with negative norm states was the assumption that one had to identify the propagator as and had to use the Dirac norm. While such a Dirac norm is appropriate in the Hermitian case, as we see, the non-Hermitian situation is much richer, with the bra state in a scalar product not in general needing to be the Hermitian conjugate of the ket.
As can be seen from the structure of the scalar field theory action and propagator , not only, in analog to the Pais-Uhlenbeck , do we encounter both a relative minus sign (when is non-zero) and a singularity (when we set ), in addition we see that in the limit the second-order term in the action drops out, with the action becoming pure fourth order. Since the identification
(and thus ) reduces the scalar field theory to the Pais-Uhlenbeck theory, we see that study of the Pais-Uhlenbeck theory enables us to study unitarity issues both in second- plus fourth-order derivative quantum field theories and in pure fourth-order ones. We thus turn now to a detailed study of the Pais-Uhlenbeck theory, and after establishing that fourth-order theories are unitary, we make an application of the pure fourth-order conformal gravity theory to galactic rotation curves.
2 The Pais-Uhlenbeck Oscillator Model
As introduced, the Pais-Uhlenbeck action contains three degrees of freedom, , , and . This is too many degrees of freedom for one oscillator but not enough for two. Consequently the theory is constrained. To handle the constraints we replace by a new variable , and using the method of Dirac constraints obtain [1, 2] a Hamiltonian of the form
with two canonical pairs that obey
If we proceed canonically we can introduce two sets of creation and annihilation operators
to find that the Hamiltonian and commutation relations take the form
While we have thus succeeded in diagonalizing the unequal-frequency Hamiltonian via second quantization, we see that the limit of equal frequencies is singular. We will thus need to treat the equal-frequency case separately, and shall thus discuss the unequal-frequency case first. For the unequal-frequency case we note the relative minus sign between the two commutators. Thus in order to avoid negative norm states we must unconventionally define the vacuum as the one that and annihilate (we take for definitiveness), with the energy and norm of the vacuum then being given by
In such a vacuum states of the form have negative energy, with the energy spectrum being unbounded from below, just as is to be anticipated given the presence of the term in .
To avoid such an outcome we could instead work in a Hilbert space in which and annihilate the vacuum. However, then we obtain
While we no longer have any states with negative energy, we instead now have states with negative norm.
As we see, both of the above realizations of the commutation algebra are undesirable and the problem even appears to be insurmountable. However, quantum mechanics is a global theory, and we must thus take global considerations into account. To this end we make a Schrödinger wave mechanics representation of the commutation algebra of the form
to obtain a stationary state with energy , viz. the ground state in the realization of the theory where energies are bounded from below, with its wave function being given by where
As we see, while this wave function has good asymptotic behavior at large , it diverges at large , and is thus not normalizable. Since we can only represent the momentum operator as when it acts on wave functions that are well-behaved at large , we see that this representation for is invalid. In an integration by parts of we would not be able to throw away surface terms, and thus have to conclude that in the bounded from below energy sector the operator cannot be Hermitian. Similarly, in this sector the vacuum normalization evaluates to , to thus be infinite. Thus the global information that is missing from (20) is that we cannot normalize to a finite value, and that accordingly no closure relation of the form given in (11) can hold.
While is not normalizable when is real, we note that it would be nicely bounded if were pure imaginary. Hence the theory will be well-behaved if take not to be Hermitian but to be anti-Hermitian instead. With the operator then being anti-Hermitian too, the troublesome term now is bounded from below. Since we can continue to maintain Hermiticity for and , we see that because of the cross term is not Hermitian. However, its eigenvalues are real, so emerges as a -symmetric Hamiltonian. Since we had made the creation and annihilation operator expansion of (16) on the assumption that and were Hermitian, we see that (16) is not a valid expansion in the realization of the theory in which the energy is bounded from below. Since use of (16) led us to states with negative norm, we must seek an alternate formulation of the theory, one that uses some entirely different norm.
To find such a norm it is convenient to work not with anti-Hermitian and but to instead introduce Hermitian operators and according to
These operators also form a canonical pair with , with the transformation in (23) being one of a broad class of transformations that leave quantum commutators (and equally classical Poisson brackets) unchanged. Specifically, a commutator such as is left unchanged if is transformed to since is then transformed to . In and of itself such a transformation would be without content unless in varying we can transit between regions in the complex plane with differing asymptotic behavior for the wave functions. The complex plane thus breaks up into regions known as Stokes wedges  with differing asymptotic behaviors, and one is able to represent the commutator in the differential form only when it acts on states in a domain in the complex plane where is asymptotically bounded.
In terms of the operators and and an and a that obey , the Hamiltonian takes the non-Hermitian form
and its symmetry is made manifest via the assignments: and are odd under , and are even, and are odd, and are even. Finally, using the techniques of theory we introduce a Hermitian operator defined as 
to find that the unequal-frequency Hamiltonian given as
is Hermitian and diagonal, with all of its energy eigenvalues being real and positive. The unequal-frequency while not itself Hermitian can be brought to a Hermitian form by a similarity transform.
Since Hermitian theories possess no states with negative norm, it follows that the unequal-frequency theory must possess none either. To see what form the theory norms do take, we note that since is Hermitian rather than anti-Hermitian, the unequal frequency has been diagonalized by a similarity transformation that is non-unitary. Since similarity transformations do not preserve orthonormality of basis vectors, the eigenstates of and are not unitarily equivalent. Rather, if we define eigenstates of and according to and , we find that these states and their conjugates are related according to
with the Hermitian conjugate of the right-eigenstate of not being a left-eigenstate of . In terms of the operator we introduced earlier we can identify , and can check directly that just as required. (In passing we note that in this case the operator of theory can be written in terms of and the parity operator as , with it obeying since obeys .) Finally, since is unitary, we obtain normalization and closure relations of the form
with there being no states of negative norm, and with the similarity transformation thus taking us between a skew basis and an orthonormal one. In the realization of the unequal-frequency theory then there are no states of negative energy and no states of negative norm, and the theory is fully physically acceptable.
3 The Singular Equal-Frequency Limit
Inspection of (25) shows that in equal-frequency limit the operator becomes singular, with the transformation to no longer being valid. To see what happens in the limit we consider the two one-particle states of the realization of the unequal frequency theory in which the energy is bounded from below, viz.
where is the ground state wave function
with energy . As we see, in the limit not only do the two one-particle energies become equal, the two wave functions become identical with the two one-particle states having collapsed into a single state. To confirm that the Hamiltonian has thus become a non-diagonalizable, Jordan-block Hamiltonian, we note that in the unequal-frequency one-particle sector the Hamiltonian takes the form 
where we have set , . For this Hamiltonian the eigenstates are given by
corresponding to eigenenergies and .
If we introduce the operator
we find that is diagonalized by this according to
Then, in the limit, we see that becomes undefined and becomes the Jordan-block Hamiltonian
with two degenerate energies but only one eigenstate. (As noted in , by setting one can introduce creation and annihilation operators at that obey , , , to then obtain , with this form for updating the form given in  since and are not the Hermitian conjugates of and . With this form for the one-particle sector then follows.)
To ascertain what happened to the eigenstate of that went missing in the limit, we take the limit of the ground state and of two particular linear combinations of the two one-particle eigenfunctions of the unequal-frequency theory, to obtain
As we see, the non-singular combination remains an eigenstate in the limit, but the singular combination does not. However, while is not an energy eigenstate, it is a non-stationary solution to the time-dependent Schrödinger equation
Despite the fact that the non-stationary states are not energy eigenstates, the set of stationary plus non-stationary states combined is complete since together they provide just the right number of independent polynomial functions of and that are needed to form an arbitrary localized wave packet in the plane . Moreover, as shown in , the norm of any such packet is preserved in time, with time evolution being unitary. As we thus see, in the event that a Hamiltonian is not diagonalizable, unitarity is still expressed as a completeness statement. However, now it is the set of solutions to the time-dependent plus time-independent Schrödinger equations combined that needs to be complete rather than solutions to the time-independent one alone.
4 Conformal Gravity
The conformal Weyl tensor defined as
is a quite remarkable geometric object. Under a local conformal transformation of the form
with arbitrary spacetime-dependent the Weyl tensor transforms as . Thus even though the Riemann tensor and its and Ricci contractions all acquire derivatives of under the conformal transformation (each being a second-derivative function of the metric), all of these derivative terms cancel identically in the Weyl tensor. As such, the Weyl tensor behaves under a conformal transformation the same way that the Maxwell tensor behaves under a gauge transformation. It is thus very suggestive to endow gravity with such a conformal structure, so that as well as being general coordinate invariant, gravity will in addition possess local conformal invariance as well. With such additional symmetry the gravitational action is uniquely prescribed to be of the form
where the gravitational coupling constant is dimensionless, with no other terms being permitted. (To obtain a dimensionless one needs four derivatives to compensate for the measure, with the term precisely providing the needed four derivatives). Associated with the action are fourth-order derivative gravitational equations of motion of the form (see e.g. )
where is covariantly conserved and traceless and can be written in the form
As constructed, conformal gravity is thus a power-counting renormalizable theory of gravity since is dimensionless. Moreover, conformal gravity is able to control the cosmological constant since conformal invariance does not permit the presence of any term in the fundamental action. In addition, we note that since the cosmologically relevant Robertson-Walker (RW) geometry is a conformally flat geometry (viz. one in which ), in conformal cosmology (42) reduces to
so unlike the (non-conformal) double-well Higgs potential, conformal gravity knows exactly where the zero of energy is, a feature of the theory that is central to solving the cosmological constant problem [13, 5].
As well as exclude any cosmological constant term from the fundamental action, the same conformal symmetry also excludes the presence of any Einstein-Hilbert term in the fundamental action. Despite this, the theory still admits of the Schwarzschild solution [14, 15] since is an exact solution to (42) when . The conformal theory thus recovers both Newton’s Law of Gravity and its familiar general-relativistic corrections (gravitational bending of light, gravitational redshift, and modifications to planetary orbits). As noted in , Einstein gravity is thus only sufficient to give the standard Schwarzschild metric phenomenology but not necessary.
Now while the vanishing of implies the vanishing of , the converse is not true since the vanishing of can be achieved without vanishing. The vacuum vanishing of thus has other solutions [14, 15], solutions that can thus enable us to distinguish between Einstein gravity and conformal gravity. As we shall see below, these other solutions are important on galactic distance scales where they enable the conformal theory to account for galactic rotation curve systematics without the need for dark matter.
To identify the quantum structure of the conformal theory, it is convenient to linearize the theory around flat spacetime according to where is the Minkowski metric and is the fluctuation. Through second order in it is found  that (41) and (42) only depend on the traceless combination . The equations of motion are found to simplify considerably in the transverse gauge where , with the first order modification to being given by
(with both sides of (45) being traceless), and with the second-order fluctuation term in the conformal action being given by
As we see, both (45) and (46) are diagonal in the indices, and so each component of can be treated independently. Comparing with (7) and (8), we see that each independent component of obeys the same dynamics as a fourth-order scalar field with . The unitarity analysis of the scalar field theory this caries over identically to the conformal theory . Thus despite the fourth-order nature of its equations of motion, the unitarity of microscopic conformal gravity is secured. Since the macroscopic limit of the conformal theory is obtained by taking matrix elements of its quantum field operators in states with an indefinite number of gravitational quanta, the consistency of conformal gravity as a macroscopic theory is thus secured as well, and so we turn now to a study of some of its macroscopic implications.
5 Fitting Galactic Rotation Curves
To fit galactic rotation curves one needs to determine the geometry for a static, spherically symmetric system in the conformal case. To this end it was noted in  that though use of the underlying conformal symmetry the most general metric in the static, spherically symmetric case can be brought to the form
to thus be described in terms of just the one function where is the radial coordinate. Moreover, in  it was shown that in metric given in (47) the gravitational equation of motion given in (42) takes the form of a fourth-order Poisson equation
As derived, (48) is exact and without approximation, showing that despite its complexity, in the presence of sufficient symmetry (42) can be quite tractable. Moreover, for a general source function (48) can be integrated exactly to yield
where satisfies .
Comparing (50) and (52), and noting the presence of an integral that continues all the way to infinity in (50), we see that the novel feature of the fourth-order theory is that the gravitational force at radius receives contributions not just from local sources in the region but from global sources in the region as well. Thus unlike Newtonian gravity where local gravity is produced solely by local sources, in the fourth-order case global sources can contribute as well. Newtonian gravity is thus intrinsically local, while conformal gravity is intrinsically global. Since in Newtonian gravity one can only explain local gravitational observations though the use of local sources, a failure to account for observations using luminous sources alone leads to the need to postulate the presence of dark matter within any given system of interest. However, in the conformal theory there are additional contributions to the gravitational field within systems due to matter that is outside of them. As shown in [17, 18, 19, 20], these global effects can remove the need for galactic dark matter, with dark matter potentially being nothing more than an attempt to describe global physics effects in purely local terms. To determine the effect of conformal gravity on galactic motions we thus need to determine the local contribution due to the material within each given galaxy and the global contribution due to the material in the rest of the Universe.
As regards first the local contribution, we note that if in (49) we consider a purely localized source of radius , then on setting , in the region