Electroweak and Flavour Structure of a
Warped Extra Dimension with Custodial Protection
Michaela E. Albrecht, Monika Blanke, Andrzej J. Buras,
Björn Duling and Katrin Gemmler
Physik Department, Technische Universität München, D-85748 Garching, Germany
Max-Planck-Institut für Physik (Werner-Heisenberg-Institut),
D-80805 München, Germany
TUM Institute for Advanced Study, Technische Universität München,
D-80333 München, Germany
We present the electroweak and flavour structure of a model with a warped extra dimension and the bulk gauge group . The presence of implies an unbroken custodial symmetry in the Higgs system allowing to eliminate large contributions to the parameter, whereas the symmetry and the enlarged fermion representations provide a custodial symmetry for flavour diagonal and flavour changing couplings of the SM boson to left-handed down-type quarks. We diagonalise analytically the mass matrices of charged and neutral gauge bosons including the first KK modes. We present the mass matrices for quarks including heavy KK modes and discuss the neutral and charged currents involving light and heavy fields. We give the corresponding complete set of Feynman rules in the unitary gauge.
Models with a warped extra dimension, also called Randall-Sundrum (RS) models [1, 2, 3, 4], in which all Standard Model (SM) fields are allowed to propagate in the bulk, offer natural solutions to many outstanding puzzles of contemporary particle physics. In addition to providing a geometrical solution to the hierarchy problem related to the vast difference between the Planck scale and the electroweak (EW) scale, they also allow to naturally generate hierarchies in fermion masses and weak mixing angles [5, 6], suppress flavour changing neutral current (FCNC) interactions [7, 8, 9], construct realistic models of EW symmetry breaking (EWSB) [10, 11, 12, 13, 14, 15] and achieve gauge coupling unification[16, 17].
The question then arises whether some imprints of this new physics scenario could be in the reach of the LHC, while satisfying all existing experimental constraints, coming in particular from EW precision tests, from the data on FCNC processes in both quark and lepton sectors and also the data on the very highly suppressed electric dipole moments (for recent reviews, see [18, 19, 20]).
A necessary, though not always sufficient, condition for direct signals of RS models at the LHC is the existence of Kaluza-Klein (KK) modes with masses. Early studies of EW precision observables (EWPO) [10, 21] have shown that with the SM gauge group in the bulk such low masses of KK particles are inconsistent in particular with the bounds on the oblique parameter and the well-measured coupling.
and enlarging the fermion representations, so that the discrete left-right symmetry , exchanging and , is preserved. The presence of the additional gauge group implies the existence of an unbroken custodial symmetry in the Higgs sector, so that tree level contributions to the parameter can be safely neglected. The symmetry and the related enlarged fermion representations eliminate the problematic contributions to the coupling.
Interestingly, the presence of new light KK modes necessary to solve the “ problem” implies significant contributions to the parameter at the one loop level . However with an appropriate choice of quark bulk mass parameters, an agreement with the EW precision data in the presence of light KK modes can be obtained [24, 25]. In fact, while the masses of the KK gauge bosons are forced to be at least to be consistent with the data on the oblique parameter , fermionic KK modes with masses even below can be made consistent with the measured EWPO.
The suppression of FCNC transitions to an acceptable level in the presence of light KK modes turns out to be much more challenging if the hierarchy of fermion masses and weak mixings is supposed to come solely from geometry so that the fundamental 5D Yukawa couplings are anarchic. In fact, recent studies demonstrate that in this case the data on the CP-violating parameter imply a lower bound on the lightest gauge KK modes in the ballpark of [26, 27], the corresponding bound from is above [28, 29]111We would like to mention that this bound can be avoided by new choices of lepton representations . and even stronger bounds come from electric dipole moments [8, 31]. Moreover it has been pointed out in  that the flavour problem in these models becomes even more serious when and decays are considered simultaneously. Note however that the bound in question can be somewhat relaxed by appropriately chosen brane kinetic terms  and/or by allowing the Higgs boson to propagate in the bulk .
In view of this situation a number of proposals has been made in order to overcome these “FCNC problems” of RS models that are directly related to the breakdown of the universality of gauge boson–fermion couplings implied by the geometric explanation of the hierarchical structure of fermion masses and mixings. This breakdown implies the violation of the GIM mechanism  and consequently tree level FCNC transitions that are inconsistent with the data for light KK scales, provided that anarchic 5D Yukawa couplings are chosen and the relevant couplings are .
In  a class of RS models has been considered that makes use of bulk and brane flavour symmetries in order to prevent the theory from large FCNCs. It has been shown that if flavour mixing is introduced via UV brane kinetic terms, the GIM mechanism is realized and a minimal flavour violating (MFV) model [35, 36, 37, 38, 39] can be obtained. However, the natural explanation of fermionic hierarchies had to be abandoned in that setup. A different strategy has been followed in , where the field theoretical concept of MFV has been promoted to the 5D theory, i. e. the bulk mass matrices are expressed in terms of the 5D Yukawa couplings. Low energy flavour violation can be further suppressed by a single parameter that dials the amount of violation in the up or down sector. If this parameter is ensured to be small, no flavour or CP problem arises even with KK masses as low as . A more thorough analysis, including the presentation of a possible dynamical origin of such a model, has been given in . Another economical model based on a bulk flavour symmetry has been proposed in . Here the right-handed down quark bulk masses are enforced to be degenerate, so that the contributions of the operator to are generated only by suppressed mass insertions on the IR brane. A recent approach  presents a simple model where the key ingredient are two horizontal symmetries. The SM fields are embedded into the 5D fields motivated by protecting . The horizontal symmetries force an alignment of bulk masses and down Yukawas which strongly suppresses FCNCs in the down sector. FCNCs in the up sector, however, can be close to the experimental limits.
In two recent papers [27, 44] we took a different strategy and investigated to which extent a hierarchy in the 5D Yukawa couplings has to be reintroduced in order to achieve consistency with the existing data on FCNC processes in the presence of KK modes in the reach of the LHC. In particular in  we have demonstrated that there exist regions in parameter space with only modest fine-tuning in the 5D Yukawa couplings involved which allow to obtain a satisfactory description of the quark masses and weak mixing angles and to satisfy all existing and EW precision constraints for scales in the reach of the LHC. As the dominant part of the observed hierarchy in masses and mixings is still explained through the AdS geometry, the resulting hierarchies are significantly milder than in the SM and other usual 4D approaches.
Subsequently, confining the numerical analysis to the regions of parameter space allowed by observables and with only modest fine-tuning, we have presented in  a complete study of rare and meson decays including , , , , , , and .
In this context it should be emphasised that the presence of FCNC transitions already at the tree level in the model in question, as opposed to the MSSM and Little Higgs models, necessarily implies other patterns in CP-violating observables and rare decay branching ratios. In particular in RS models not only non-MFV interactions are present, like for instance in the Little Higgs models with T-Parity, but also new operators become important that are strongly suppressed in the latter. As found in [27, 44] such new contributions lead to interesting deviations from the SM and in particular from models with Constrained MFV [35, 36, 45] in observables that are still poorly measured and which allow for large new physics contributions.
The main results of  can be briefly summarised as follows:
The EW tree level contributions to observables mediated by the new weak gauge boson , while subleading in the case of and , turn out to be of roughly the same size as the KK gluon contributions in the case of physics observables.
The symmetry implies automatically the protection of flavour violating couplings so that tree level contributions to all processes in which flavour changes appear in the down quark sector are dominantly represented by couplings.
However, the tree level contributions to processes are of higher order in and can be neglected.
On the other hand the main messages from  are as follows:
New physics contributions to rare and decays, as opposed to transitions, are governed by tree level contributions from boson exchanges (dominated by couplings) with the new heavy EW gauge bosons playing a subdominant role.
Imposing all existing constraints from transitions we find that a number of branching ratios for rare decays can differ significantly from the SM predictions, while the corresponding effects in rare decays are modest. In particular the branching ratios for and can be by a factor of three and two larger than the SM predictions, respectively. The latter enhancement could be welcomed one day if the central experimental value  will remain in the ballpark of and its error will decrease.
However, it is very unlikely to get simultaneously large NP effects in rare decays and , which constitutes a good test of the model.
Sizable departures from the MFV relations between and and between and the decay rates are possible.
The pattern of deviations from the SM differs from the deviations found in the LHT model .
It is interesting that in spite of many new flavour parameters present in this model a clear pattern of new flavour violating effects has been identified in [27, 44]: large effects in transitions, large effects in rare decays, small effects in rare decays and the absence of simultaneous large effects in the and system. This pattern implies that an observation of a large asymmetry would in the context of this model preclude sizable NP effects in rare decays. On the other hand, finding to be SM-like will open the road to large NP effects in rare decays, even if such large effects are only a possibility and are not guaranteed. On the other hand, an observation of large NP effects in rare decays would put this model in serious difficulties.
In [27, 44] only a brief description of the RS model in question has been presented as only gauge boson exchanges were relevant at the tree level. In particular details on the fermion sector have not been presented there. For the subsequent phenomenological studies like the and transitions it is of interest to have a more detailed presentation which is the main goal of our paper. We formulate a particular RS model based on the bulk gauge group in (1.1) and having appropriate quark representations in order to avoid tensions with EWPO. We work out the general structure of the gauge and fermion sectors, discuss the new sources of flavour violation, and we give a collection of Feynman rules222Some of these Feynman rules have already been presented in [55, 56]. that can be used to calculate all observables of interest. In fact a subset of the Feynman rules presented here has already been used in [27, 44].
Throughout our analysis we follow the perturbative approach, i. e. we first solve the 5D equations of motion and perform the KK decomposition in the absence of EWSB, as also done e. g. in [55, 57] and then treat the Higgs vacuum expectation value (VEV) as a small perturbation that induces mixing among the various modes. The complementary approach, solving the equations of motion already in the presence of EWSB, has been followed e. g. in [58, 59, 60, 61]. Recently, a very detailed theoretical discussion of the latter approach has been presented in . In Appendix C.2 we show that both approaches are indeed equivalent; for an independent discussion see also .
The present paper is organised as follows. In Section 2 we present in detail the gauge sector of the model and in particular the effects of EWSB. The final formulae for gauge boson masses and mixings in the charged and neutral sectors are collected in Appendix C. Next in Section 3 we set up the quark representations under the bulk gauge group. In Section 4, one of the main sections of our paper, we work out the flavour structure of the quark sector. After a detailed discussion of quark mass matrices and Yukawa couplings in the flavour eigenbasis we outline the diagonalisation of these matrices and study the structure of weak neutral and charged currents. Subsequently the couplings of KK gluons and photons are considered. This section forms the basis of the Feynman rules in the quark sector that are collected in Appendix D. We end this section by listing the sources of flavour violation in this model, with the pattern of flavour violation, in particular in processes, governed by the custodial protection present in the model. In Section 5 we list the parameters of the model and present a useful parameterisation for the 5D Yukawa couplings in terms of parameters accessible at low energies. In Section 6 we discuss one possible realisation of the lepton sector and present a dictionary that allows in a straightforward manner to obtain the Feynman rules for the leptons from those of quarks. We close the paper with a brief summary in Section 7.
2 Gauge Sector
We consider an gauge theory on a slice of AdS with the metric 
with the fifth coordinate being restricted to the interval , and . In order to simplify the phenomenological discussion in [27, 44] we chose to work with the sign convention for the metric, i. e. . The gauge bosons and fermions are allowed to propagate in the 5D bulk, while the Higgs field will be localised on or near the IR brane ().
where is the discrete symmetry interchanging the two groups. This means for instance that . The gauge group (2.2) is broken by boundary conditions (BCs) on the UV brane () to the Standard Model (SM) gauge group, i. e.
This breakdown is achieved by the following assignment of BCs333These BCs can be naturally achieved by adding a scalar doublet with charge on the UV brane, that develops a VEV (see [61, 3] for details).
where the first (second) sign denotes the BC on the UV (IR) brane: stands for a Neumann BC while stands for a Dirichlet BC. Furthermore and . The fields and are given in terms of the original fields and as follows:
Here, and are the 5D gauge couplings of and , respectively. Note that the BCs for a gauge field imply automatically opposite BCs for its 5th component . In what follows we choose to work in the gauge and .
The fields with BCs have, in addition to the massive KK modes, zero modes which are massless at this stage and are identified with the SM gauge bosons and of . The fields with BCs contain only massive KK modes.
for the modes with a BC on the IR brane that we are presently interested in. The accuracy of this approximate formula improves significantly with increasing , hence for the first KK modes it is safer to work with the exact KK masses. These can be found numerically to be
for gauge bosons with BCs, and
for gauge bosons with BCs. Here we have introduced the effective new physics scale and set in order to solve the hierarchy problem. The suppression in the latter case is a direct consequence of the different BC on the UV brane . Note that the KK masses for the gauge bosons depend neither on the gauge group nor on the size of the gauge coupling, but are universal for all gauge bosons with the same BCs. Only after EWSB, the weak KK gauge boson masses will receive small additional corrections. As can easily be seen from (2.9), the gauge KK modes are localised near the IR brane.
To further proceed it will be useful to follow  and define the fields
where again is given in terms of gauge couplings (see (2.8) for the definition of )
Because of the mixing between the various gauge boson zero and KK modes , but corrections appear first at order . Their impact on EW precision studies is beyond the scope of this paper and will be studied elsewhere.
We note that the above relations can be modified by the presence of additional gauge kinetic terms on the UV and IR branes, that are allowed by the symmetries of the model. In order not to complicate our analysis, we will neglect such terms and work exclusively with the action given in Appendix A. A generalisation of our results to include also the effects of possible brane terms is straightforward. In Section 4.5.7 we comment on the effects of such terms on flavour phenomenology.
2.2 Electroweak Symmetry Breaking
As discussed in the previous section, the bulk gauge symmetry in (1.1) is broken to the SM gauge group
by means of the BCs of the EW gauge bosons on the UV brane. In order to achieve the standard EWSB, , a Higgs boson is introduced that is localised either on or near the IR brane, transforming as a self-dual bidoublet of
and being a singlet under , . In the case of a 5D Higgs field living in the bulk, the whole bidoublet has to obey BCs in order to yield a light zero mode.
When its neutral component develops a 4D effective VEV, on or near the IR brane the symmetry breaking
takes place. We see explicitly that in the Higgs sector of the theory an unbroken custodial symmetry remains, being responsible for the protection of the parameter. Similarly the symmetry, protecting the coupling, remains unbroken.
Combining then the symmetry breakings by BCs on the UV brane and by the Higgs VEV in the IR, we see that the low energy effective theory is described by the spontaneous breaking
as required by phenomenology. The symmetry breaking structure of the model is displayed in Fig. 1.
Now due to the unbroken gauge invariance of QED and QCD, the gluon and photon fields including their KK modes do not couple to the Higgs boson at leading order in perturbation theory and hence do not mix with each other or with , and and the higher KK modes of and . Therefore, even after EWSB
and the corresponding states remain mass eigenstates. On the other hand the kinetic term for the Higgs field (see Appendix A)
leads to -corrections to the masses of , and as well as of , and , and mixing between states of the same electric charge is induced. Here
where is the Higgs shape function along the extra dimension. We assume to be of the form
and denotes the effective 4D VEV of the zero mode of in (2.18).
with and given explicitly in Appendix C.
In order to determine the physical mass eigenstates and the corresponding masses, and have to be diagonalised by means of orthogonal transformations:
The mass eigenstates and are then related to the gauge eigenstates of the KK modes via
The explicit form of the orthogonal matrices and can be found in Appendix C.
3 Fermion Sector – Quarks
In order to preserve the symmetry, that is necessary for the suppression of dangerous contributions to EW precision observables [10, 11, 22, 14, 15, 23], we will choose a particular simple set of representations of the group. Although in order to satisfy EW precision measurements only the third quark generation needs to preserve the symmetry, the incorporation of CKM mixing requires the same choice of representations also for the first two quark generations. This is crucial for having a custodial protection for the flavour violating couplings  as well.
In this section we restrict our attention to the quark sector of the model. The lepton sector will be discussed separately in Section 6.
The particular fermion assignment given below has been motivated by the analyses of [13, 15, 14, 23, 66]. In particular the representations given below can easily be embedded into complete multiplets used in [14, 23, 66] in the context of models with gauge-Higgs unification.
We introduce three multiplets per generation :
The corresponding states of opposite chirality are given by
The following comments are in order:
and are bidoublets of , with acting vertically and horizontally.
and are singlets of .
transform as under . The embedding of the right-handed down-type quarks into triplet representations is necessary in order to allow for a invariant Yukawa coupling.
The charges assigned to the various multiplets are charges.
The charges assigned to separate fields are electric charges, given as
where and denote the third component of the and isospins, respectively.
Only the fields obeying BCs have massless zero modes. Up to small mixing effects with other massive modes due to the transformation to mass eigenstates discussed in Section 4, these zero modes can be identified with the usual SM quarks.
The remaining fields are KK modes with approximately vectorlike couplings. We thus have in this model additional heavy fermionic states. These are
(3.8) (3.9) (3.10)
Left- and right-handed fermion fields are defined via .
3.2 KK Decomposition and Bulk Profiles
3.2.1 Zero Modes
where is the bulk mass of the 5D fermion field. In the case of right-handed zero modes, has to be replaced by in the above formula and in the discussion following below. We note that
For the normalisation factor in (3.11) is and is peaked around , i. e. fermions with bulk mass parameter are placed close to the UV brane.
For , as is the case of the top quark, the second term in the denominator of (3.11) can be neglected and we obtain
Thus the shape function is strongly peaked towards , i. e. the IR brane.
One should stress that generally the and of left- and right-handed SM fermions can differ from each other, as these fermions are zero modes of different 5D representations. As both and enter the formula for the Yukawa couplings and fermion masses, this freedom can help to satisfy certain features in EW precision studies [13, 15, 14, 23] and in flavour physics [8, 62] while keeping the fermion masses of their natural size. This is in particular relevant for the third quark generation.
3.2.2 KK Modes
where for left-(right-)handed modes, and expressions for and can be found in Appendix B. The KK masses are approximately given by
where the sign corresponds to a BC for the left-handed fermion on the IR brane. Again, as in the case of gauge KK modes, the accuracy of (3.14) improves with increasing .
The bulk mass parameter is universal for the full tower of KK modes and equal to the describing the localisation of the zero mode if such mode exists.
In spite of the same even for the form of (3.13) implies that all KK modes are localised near the IR brane. There is no freedom to delocalise the massive KK modes away from the IR brane, as was the case for the zero mode.
3.3 Yukawa Couplings and Fermion Masses
The SM fermions acquire masses via their Yukawa interactions with the Higgs in the process of EWSB. The effective 4D Yukawa matrices are then given by
where is the fundamental 5D Yukawa coupling and is the Higgs shape function along the extra dimension, as given in (2.25).
We stress that together with from (2.25) the fermionic zero mode functions with respect to the warped metric, as given in (B.13), have to be used in order to determine the effective 4D Yukawa couplings in (3.15).
In the special case of an IR brane localised Higgs we obtain
where are normalisation factors of the fermion shape functions on the IR brane. We note that in this case, up to , has a factorised form, as emphasised in . In the more general case of the Higgs field propagating in the bulk this factorisation is broken, but only weakly for a large range of .
On the other hand we stress that in the presence of non-diagonal entries of , the factorisation must always be broken as not all entries of can be equal. Indeed this special choice of 5D Yukawa couplings would lead to two zero eigenvalues of the corresponding mass matrix. In addition some non-degeneracy between the entries of is needed to cure the “ problem” identified in . Indeed this is completely analogous to the case of the Froggatt-Nielsen flavour symmetry , where also a slight structure in the Yukawa coupling matrices is needed to obtain the correct size of . A detailed discussion of the close analogy between the Froggatt-Nielsen mechanism and bulk fermions in RS models has been presented in .
4 Flavour Structure
The flavour structure of this class of models is rather complicated, although as emphasised by Agashe et al.  in certain approximations it is quite simple. For the time being we will however not make any approximations.
The procedure to find all interactions including flavour violating ones is basically the following:
We begin with the interaction terms in and in (A.1) in terms of flavour eigenstates for fermions and in the gauge eigenbasis for the gauge bosons.
To this end we start with the fundamental 5D interactions and then perform the KK decomposition as described in Appendix B. We thus obtain effective 4D couplings that are non-local quantities along the extra dimension, resulting from the overlap of the gauge boson and fermion shape functions. Schematically the interactions between different KK levels are given by
Note that only fermions within the same gauge multiplet are coupled to each other in this way, so that their bulk mass parameters are necessarily equal. In the Feynman rules collected in Appendix D these overlap integrals appear as , and .
As the gauge boson zero mode has a flat shape function, the coupling for equal fermion KK levels to the gauge boson zero mode reduces to the 4D gauge coupling , while the integrals with vanish due to the normalisation of the fermion shape function. In the same way, couplings of different fermionic KK levels to the gauge boson zero mode vanish due to the orthogonality of the fermion shape functions. However, due to the effects of EWSB, the EW gauge boson zero mode mixes with its KK modes, so that eventually flavour non-universalities in the couplings of the SM weak gauge bosons and non-zero couplings between various KK levels will arise.
A different treatment of the effects of EWSB on the gauge boson zero modes has been discussed in [58, 59, 60]. In their description the Higgs VEV is inserted already at the level of the 5D equations of motion, which leads to a distortion of the gauge boson zero mode in the vicinity of the IR brane. On the other hand no mixing between the various KK levels appears. In Appendix C we review both approaches in detail, study their advantages and shortcomings and show that both interpretations are indeed physically equivalent. A similar independent discussion has been presented in .
Next we have to consider the mass matrices of gauge bosons and fermions. Here the new aspects relative to the SM are
in the gauge sector:
The extended gauge group leads to the presence of additional (heavy) gauge bosons.
In addition, also the heavy KK modes of the SM gauge bosons, including also gluons and the photon, are present.
EWSB induces mixing of the SM zero modes with the additional heavy KK modes of the same electric charge.
in the fermion sector:
Also the heavy KK modes of the SM fermions have to be considered.
Again mixing of the SM zero modes with the heavy KK states is induced.
Finally all interactions have to be rewritten in terms of mass eigenstates for gauge bosons and fermions. Therefore the corresponding mass matrices need to be diagonalised. Note that mixing takes place not only between different flavours, but also between different KK levels444This should be contrasted with models with flat extra dimensions where the presence of KK parity has eliminated such mixing. For a recent attempt to introduce KK parity in the RS framework, see .. As the new physics scale is experimentally constrained to be , within a good approximation it is sufficient to consider only the contributions of modes and neglect all higher KK levels. Therefore in what follows we restrict our discussion to this simplified case. The generalisation of our formulae to include also higher KK modes is then straightforward.
4.2 Quark Mass Matrices
The transformation to mass eigenstates in the gauge sector is performed in Section 2 and Appendix C. The goal of the present section is to construct and diagonalise the mass matrices for the quark fields given in (3.1)–(3.6). To this end we will only consider zero modes and the lowest () KK modes. As there are only few fields among the ones in (3.1)–(3.6) that have zero modes, we will assign to them the superscript . For the excited KK modes we will just use the notation of (3.1)–(3.6), making the index implicit.
We will have to deal with three mass matrices corresponding to the electric charges , and . To this end we group the fermion modes into the following vectors:
For the charge mass matrix we have
where the flavour index runs over the three quark generations. Thus we deal with 9-dimensional vectors. Note that in this sector only massive excited KK states are present.
For the charge mass matrix the corresponding vectors read
Here the first components are zero modes, and so that we really deal with 18-dimensional vectors.
The charge vectors read
Again the first entries are zero modes, the remaining ones massive KK modes, and , so that in this case a 12-dimensional vector is obtained.
In order to construct the mass matrices let us briefly recall certain properties, known already from numerous studies in the literature:
We have three bulk mass matrices corresponding to the representations ( is the flavour index), respectively. Note that for a given multiplet with fixed flavour index all bulk mass parameters for different components of the multiplet are equal to each other.
In general, are arbitrary hermitian matrices, where corresponds to the multiplet . In the following we choose to work in the basis where they are real and diagonal, i. e. each of them is described by three real parameters , where is the flavour index. This can always be achieved by appropriate field redefinitions of the multiplets. Explicitly we then have:
and similarly for and .
The allowed Yukawa couplings, giving mass to the fermion zero modes after EWSB, have to preserve the full gauge symmetry. The possible gauge invariant terms in the full 5D theory can be found in Appendix A.
The effective 4D Yukawa matrices will involve the fermion and Higgs shape functions. We will denote the fermionic ones by and , corresponding to the -th and -th component of and in (4.2)–(4.7), respectively, and is the Higgs shape function as given in (2.25).
Having at hand this information and restricting ourselves to for simplicity, we obtain the following effective 4D Yukawa couplings
(4.9) (4.10) (4.11) (4.12)
Interestingly, the Yukawa coupling proportional to , connecting with and being thus responsible for the SM down quark Yukawa coupling, leads to mass terms not only for the charge quarks, but simultaneously also to mass terms for the and quarks. This is a direct consequence of and being placed in the adjoint representations of and , respectively, as seen in (3.3), (3.6).
On the other hand, the term proportional to , connecting with and being thus responsible for the SM up quark Yukawa coupling, contributes only to the mass matrix for the charge quarks.
Finally the fermionic KK masses, which can be obtained from solving the bulk equations of motion, have to be included in the mass matrices. Note that both the fermion shape function and the KK mass depend on the bulk mass parameter and on the BCs.
These three matrices have to be diagonalised via a bi-unitary transformation to find the quark mass eigenstates. Due to the large size of the mass matrices this diagonalisation has to be done numerically.
Let us make a few remarks:
The mass eigenstates of charge are all heavy.
In and the off-diagonal entries in the first column and row lead to mixing between light zero modes and heavy KK modes. This mixing will be suppressed by .
In the case of the Higgs field being confined exactly to the IR brane, only Yukawa couplings to those fermion modes are non-vanishing that obey a BC on the IR brane. In that case some of the entries in the above mass matrices in (4.14)–(4.16) vanish:
(4.17) (4.18) (4.19)
We can then diagonalise the , and charge matrices by