LHC Signatures of Two-Higgs-Doubletswith Fourth Family

LHC Signatures of Two-Higgs-Doublets
with Fourth Family

Ning Chen   and  Hong-Jian He 
 Institute of Modern Physics and Center for High Energy Physics,
  Tsinghua University, Beijing 100084, China.
 Center for High Energy Physics, Peking University, Beijing 100871, China.
 Kavli Institute for Theoretical Physics China, CAS, Beijing 100190, China.
hep_nchen@tsinghua.edu.cn, hjhe@tsinghua.edu.cn

On-going Higgs searches in the light mass window are of vital importance for testing the Higgs mechanism and probing new physics beyond the standard model (SM). The latest ATLAS and CMS searches for the SM Higgs boson at the LHC (7 TeV) found some intriguing excesses of events in the    channels () around the mass-range of   GeV. We explore a possible explanation of the and signals from the light CP-odd Higgs or CP-even Higgs from the general two-Higgs-doublet model with fourth-family fermions. We demonstrate that by including invisible decays of the Higgs boson or to fourth-family neutrinos, the predicted and signals can explain the observed new signatures at the LHC, and will be further probed by the forthcoming LHC runs in 2012.

Higgs Physics, Beyond Standard Model
JHEP (2012), in Press, [ arXiv:1202.3072 ]

1 Introduction

The LHC searches for Higgs boson(s) in the light mass window have vital importance for testing the Higgs mechanism [1] and probing new physics beyond the SM. The most recent results from the LHC (7 TeV) have constrained the light Higgs boson of the standard model (SM) into the mass-range  (115.5 GeV,  131 GeV) by ATLAS [2] and  (115 GeV,  128 GeV) by CMS [3], at  C.L.111From the latest updates at the Moriond conference [4],  ATLAS further confined the allowed light SM Higgs mass ranges into  (117.5 GeV,  118.5 GeV) and  (122.5 GeV,  129 GeV) at  C.L., while CMS gave the improved Higgs mass limits of  (114.4 GeV,  127.5 GeV). In particular, the ATLAS observed an intriguing excess of events for a Higgs boson with mass close to [2]. The three most sensitive channels in this mass range, ,  ,  and  ,  contribute to the excess with local significances of ,  ,  and ,  respectively. If this would be confirmed by the upcoming LHC data in 2012, a Higgs boson of mass around 126 GeV does call for new physics beyond the SM due to the vacuum instability [5]. Furthermore, the observed excess in the channel by ATLAS is also higher than the expected signals of the pure SM Higgs boson (with the same mass) by a factor-2 [2], which again points to new physics.

In this work, we investigate a simple SM-extension as the new physics — the generic two-Higgs-doublet model (2HDM) with fourth-family SM fermions (4F2HDM). It contains the minimal extension in the SM Higgs sector with one more doublet and in the SM fermion sector with one more family. With such a truly simple addition, we study distinct new signatures of the light CP-odd Higgs or CP-even Higgs at the LHC, and analyze the implications for the latest ATLAS and CMS Higgs searches [2][3]. We consider the 4F2HDM in both type-I and type-II, with CP-conserving Higgs potential. Such 2HDMs contain four physical Higgs states with masses  .  Due to the additional contributions from heavy fourth-family quarks , the gluon-fusion production cross sections of at the LHC are generally much enhanced relative to in the SM, and thus may be easily excluded by the current LHC data. In the present study, we demonstrate that the invisible Higgs decays into the light fourth-family neutrinos, ,  can become the major channel, and play a key role to properly suppress   rates for the consistency with the existing LHC data. Especially, we show that such a light Higgs boson or with mass around  GeV can nicely explain the observed event excesses by ATLAS [2] and CMS [3].

2 Signals of CP-Odd in 4F2HDM with Invisible Decays

We start with the analysis of CP-odd Higgs boson .  The general 2HDM allows to be the lightest Higgs boson for proper parameter space of the Higgs potential, unlike the minimal supersymmetric SM (MSSM) where the lightest Higgs boson is always .  An explicit realization of such 2HDMs is given by the dynamical top-seesaw model [6], where the light mass of the composite pseudo-scalar is induced by the topcolor instanton effect [7] and thus can naturally serve as the lightest state in the Higgs spectrum. Since has no cubic gauge couplings at tree-level, it mainly decays into the SM fermion pairs and the final states (via triangular fermion-loops). So the decay channel could be important for detecting such a light boson at the LHC. However, it was recently found [8, 9] that a light in the presence of fourth-family is excluded due to the enhanced cross section and unsuppressed decay branching ratio of . We note that this exclusion holds only in certain parameter region. In the following, we will include the invisible decays   for the fourth-family neutrinos being lighter than half of ,  and study the distinct new LHC signatures of the Higgs boson.

The Higgs potential of the general 2HDM contains two characteristic input parameters, the   as the ratio of two Higgs vacuum expectation values (VEVs), and the mixing angle   from diagonalizing the mass-matrix of neutral Higgs bosons   .  It was shown [10] that such 2HDM with fourth-family fermions is consistent with the electroweak precision constraints. Ref. [11] also found that within broad parameter regions, the 4F2HDM can satisfy the   and   mixing constraints. For the present study, we focus on two types of CP-conserving 2HDMs without tree-level flavor-changing neutral currents (FCNC) [12], the type-I and type-II 2HDMs including the fourth-family. By definition, the type-I 2HDM assigns the first Higgs doublet (with VEV ) to couple with all fermions via Yukawa interactions and generate their masses, but the second Higgs doublet (with VEV ) does not. The type-II 2HDM has couple to all up-type fermions and to all down-type fermions. The most general Yukawa interactions for the pseudo-scalar in the 4F2HDM can be expressed as,


where the couplings in the 4F2HDM-I and -II are summarized in Table 1.

Table 1: Yukawa couplings of the CP-odd Higgs boson in the 4F2HDM-I and 4F2HDM-II.

The major production channel of at the LHC is the gluon-fusion process, and its cross section differs from that of the SM Higgs boson (coupled to three families of SM fermions) through the ratio,


Here the form factors for the CP-even and CP-odd Higgs bosons take the forms [13],


with .  Notice that the ratio of the on-shell production cross sections (2) is clearly independent of the center-of-mass energy of the LHC. This is also true for the ratio of the corresponding signal event numbers, as the integrated luminosity is the same for both cross sections. Hence our predicted ratio of signals for either production cross sections or number of events should also apply to the forthcoming LHC runs with higher collision energies and/or higher luminosities [14]. Then, we compute the ratio (2) for the inputs and in Fig. 1. For larger vlaues, the ratio (2) for 4F2HDM-II receives an enhancement from fourth-family quark  .  Thus the production cross section   is more enhanced for large relative to that of the SM Higgs boson. On the other hand, all type-I Yukawa couplings are controlled by an overall factor as shown in Table 1. So the fourth-family quarks give contributions proportional to a uniform factor  .  Obviously, the fourth-family corrections to the gluon-fusion cross section is enhanced by in 4F2HDM-II while suppressed by in 4F2HDM-I for .  Fig. 1 shows that for ,  the production is always enhanced in 4F2HDM-II, and the enhancement factor is about for in the mass-range GeV.  In contrast, the production in 4F2HDM-I is moderately enhanced by a factor of for and GeV,  which is much lower than that of 4F2HDM-II with the same .  Due the opposite signs between the up-type and the down-type Yukawa couplings and of 4F2HDM-I (Table 1), a cancellation appears between their contributions to the ratio (2). This cancellation becomes maximal when the two heavy quarks are degenerate. So the cross sections in (2) are dominated by the third-family top-quark-loop and the inequality for a given Higgs mass determines the final enhancement of the ratio (2) for ,  as shown in Fig. 1 for 4F2HDM-I. We also see that for a larger ,  such as ,  the production in the 4F2HDM-I is suppressed by about a factor-10 relative to that of the in the SM3.

In general, the type-II Higgs sector is more nontrivial and interesting than the type-I, it is also well motivated for the fermion mass generations. In the natural parameter-space of ,  it is very challenging to make the 4F2HDM-II safe from the LHC constraints as noted before [8][9]. We have to sufficiently reduce the signals by suppressing the relevant decay branching fractions of .  For this purpose, we propose a new resolution by exploring the invisible decays of into light fourth-family neutrinos,  .

Figure 1: Ratio of for   (solid lines) and   (dashed lines).

Generally, the fourth-family neutrinos    have both Dirac and Majorana mass-terms which form the seesaw mass-matrix,


After the diagonalization into mass-eigenbasis   ,  their mass-eigenvalues are determined by the two mass-parameters and ,


with the mixing angle defined as,


where must hold due to .  The case of corresponds to , leading to two degenerate states of pure Dirac neutrinos. The limit of   is unphysical since it gives .  The LEP precision data on invisible decays constrain ,  while the naturalness requires Yukawa couplings to be of and thus the Dirac mass GeV.  Hence, our parameter space for the mixing angle   is confined into the range of  .  The fourth-family neutrino can be stable on the collider lifetime, and the current experimental lower limits on stable neutral heavy lepton mass is as low as  GeV at 95% C.L., as inferred from the invisible width [16, 17]. Taking into account of the mixing between two Majorana neutrinos and , this bound may be further reduced to  GeV [15]. Such light fourth-family neutrinos will open up new invisible decay channels for both and  .  The lower limit on the mass of fourth-family charged lepton is about  GeV, as given by the LEP-II direct searches [18]. These limits show that the fourth-family neutrinos and leptons () can be much lighter than the fourth-family quarks . For short-lived    with prompt decays of and ,  the current searches at the LHC (7 TeV) places the following lower bounds (95% C.L.),  GeV from the CMS with [19] or  GeV from the ATLAS with [20], and  GeV [21]. Meanwhile, the latest analysis from the Tevatron searches [22] using an integrated luminosity of   for both CDF and D0 places the lower mass limits,  GeV  and   GeV  at 95% C.L. For illustration in the following analysis, we will uniformly take a sample input of fourth-family fermion masses,  GeV,  unless specified otherwise.

Figure 2: Decay branching ratios of the CP-odd Higgs boson   as functions of its mass   for the 4F2HDM-II. The other input parameters are fixed as,  .

For the present analysis, we will systematically explore the new decay channels of   in the 4F2HDM, ,  as well as ,  when is heavier than twice of () or .  The invisible decay widths of are computed at the tree-level,


where the second channel (9) is open when .  The fourth-family fermions also contribute to the loop-induced decay widths for as follows,


where and denote the electric charge and color-factor for each fermion species. Besides, ,  and    with being the weak mixing angle. All decay widths in our analysis are computed by including the relevant NLO QCD corrections as in Ref. [23]. The form factor   in (12) is given by


with  .  Since the form factors   and are positive for the fermionic contributions, all three decay widths in (10)-(12) are larger than in the SM3. Including the new invisible decay channels of   with decay rates in (8)-(9) ,  it is possible to suppress all SM decay branching fractions for the low-mass range of .  In Fig. 2, we show the decay branching ratios in a wide mass-range of  GeV  for the 4F2HDM-II. We take the sample inputs of   ,  where   corresponds to the case of   being pure Dirac neutrino. Fig. 2 shows that the invisible decay   can dominate over all other channels for  , while the and   channels become dominant for  .  In particular, the diphoton channel   can be suppressed by a factor of for  ,  as compared to the diphoton branching fraction of the SM Higgs boson in the same mass range. Combining this with the enhanced cross sections in Fig. 1, we see that the new invisible decays of play a key role to bring down the signals for being consistent with the current LHC searches. Moreover, has vanishing cubic couplings with gauge bosons and thus no final states will be produced. It is clear that the current limits on the mass-range of will be much weaker than that of the conventional SM Higgs boson (mentioned at the beginning of Sec. 1).

Figure 3: Decay branching ratios of the CP-odd Higgs boson with mass  GeV  for the 4F2HDM-II. Plot-(a): Br as a function of the fourth-family neutrino mixing angle ,  with fixed  . Plot-(b): Br as a function of  ,  with fixed  . 

Motivated by the latest ATLAS data [2], we focus on the case of a light with mass  GeV. The invisible decay mode is kinematically allowed for ,  where the lower limit comes from the LEP constraints [16, 17]. For the invisible decay rates (8)-(9), we have included both Dirac and Majorana neutrino masses. In the pure Dirac-mass limit ,  the two fourth-family neutrinos become degenerate and thus their decay rates are equal,  .

Figure 4: Predicted signals from the CP-odd Higgs boson at the LHC, computed as the ratio . Plot-(a) shows this ratio versus ,  for .  Plot-(b) displays the same ratio as a function of  ,  for (red curves) and (blue curves).  For both type-I and type-II, we show the predictions for two different masses:   (solid curve) and (dashed curve). The horizontal black solid-line indicates that the prediction coincides with the SM3 (with the same Higgs mass), while the black dashed-line denotes the signal prediction being twice of the SM3.

In Fig. 3(a), we analyze the decay branching fractions of versus the mixing angle of the fourth-family neutrinos. It shows that the invisible decay channel always dominates over all other channels for the full allowed range .  The branching ratios for all other SM decay channels are maximized around ,  at which the second invisible decay channel   is kinematically forbidden. In Fig. 3(b), we further analyze various decay branching fractions versus   for the 4F2HDM-II. The invisible decay branching ratio  Br[]  is maximized for small  .  When   gets larger, it gets reduced and no longer dominates over other channels; this could potentially cause too large signals at the LHC for the 4F2HDM-II. Such a danger is absent for the 4F2HDM-I, where all the partial decay widths are suppressed by due to  .  Besides, in the 4F2HDM-I the production cross section of   gets suppressed for larger  ,  as shown in Fig. 1.

Now we are ready to evaluate the signal predictions of for in the 4F2HDM (type-I and type-II), and then derive the ratio  ,  where the denominator is the corresponding signals in the SM3 with the same input of Higgs mass as .  We present our results in Fig. 4 for the LHC (7 TeV). It shows that for the 4F2HDM-I, the predictions are always significantly below that of the SM3. For the 4F2HDM-II with ,  we find that the   signals can be moderately larger than that of the SM3 in the parameter region,   with   [Fig. 4(a)], or   with   [Fig. 4(b)]. But, larger values of   in Fig. 4(a) would cause too much excess of signals, and are excluded by the current data. With the data set, ATLAS collaboration observed   excess of   events at the invariant-mass GeV,  while the expected SM Higgs signal is   above the SM backgrounds, which is about a factor-2 smaller than what ATLAS observed [2]. The signal-reduction-rate due to various cuts and detection efficiencies should be roughly the same for both the SM Higgs boson and the CP-odd boson. When the predicted ratio   in the 4F2HDM-II, the   signals can nicely explain the   excess of ATLAS observation at GeV.  For instance, this is realized at   in Fig. 4(a).

As shown by Figs. 2-3, the invisible decays    can dominate over all other channels in the relevant parameter regions. The suppression on the fermionic decay branching ratios (such as  ) appears moderate in comparison with the SM3 case. Nevertheless, the Higgs searches in the   and   decay modes [3] are made through the vector boson associated production and the vector boson fusion processes, respectively, which receive no new enhancement from the fourth-family fermions. This is consistent with the present observations of ATLAS [2] and CMS [3], which found no excess from the   and   final states. Similar reasoning also holds for the detection of the CP-even Higgs boson   via the   and   channels (cf. Sec. 3). Furthermore, the CMS detector showed no new signals in the final states, and ATLAS analysis only indicated a smaller excess in the events. It is very likely that the channels contains only the SM backgrounds. If so, this is again consistent with our analysis of the Higgs boson, since the CP-odd has no tree-level gauge couplings with and thus the   decays are forbidden.

Note that in each plot of Fig. 4, the two solid curves correspond to  GeV,  and the two dashed curves to  GeV.  We do not show a curve for  GeV since it almost overlaps with that of  GeV.  So the parameter space between the two adjacent curves in each set (either red or blue) in Fig. 4 essentially represent that of the mass-range GeV.  From Fig. 4, we see that should the present ATLAS excess at GeV be disconfirmed by this summer with more LHC data, our 4F2HDM-II can predict new Higgs signals in other   values around  GeV, either above or below the SM3 Higgs rates. This will be further probed by the LHC Higgs searches.

3 Signals of CP-Even in 4F2HDM with Invisible Decays

In this section, we turn to the analysis of the CP-even Higgs boson   in the 4F2HDM, and study the impacts of the invisible decays on the LHC discovery. Due to the mixing between the two CP-even neutral states, we have the mixing angle as a new input parameter. Unlike ,  the CP-even Higgs boson   also has additional decay channels of at tree-level. We will present a benchmark model for the 4F2HDM, and analyze the production and decays of at the LHC. We further compare our predictions to that of the SM Higgs coupled with four families of fermions (SM4), by including the invisible decay channel of .

Table 2: Yukawa couplings of the SM fermions to the lighter CP-even Higgs boson for the 4F2HDM-I and 4F2HDM-II.

In the 4F2HDM, the analysis of production and decays of   are more complicated than ,  due to the additional decay channels in the   and   final states, as well as the mixing parameter associated with two CP-even states . The Yukawa interactions of   can be generally expressed as follows,


where the Yukawa couplings for the 4F2HDM-I and 4F2HDM-II are summarized in Table 2. The   production via the gluon-fusion process receives new contributions from the fourth-family quarks , which are enhanced by the Yukawa couplings of relative to that of the SM top quark (). We compute the ratio of production cross sections between the 4F2HDM and SM3 with the same mass of ,


which is found to be generally larger than unity. In Fig. 5, we present the enhancement factors (15) for 4F2HDM-I (blue curve) and 4F2HDM-II (red curve) with the sample input .  The enhancement (15) is moderate since   in (15) can be smaller than one, as compared to the SM4 with .  For the SM4, we see from the purple curve in Fig. 5,   holds in the limit of light Higgs mass   with the loop-contributions from the heavy quarks.

Figure 5: Predicted ratio   for the 4F2HDM-I (blue curve) and 4F2HDM-II (red curve) with the sample input .  As a comparison, the ratio   is shown for the SM4 (purple curve).
Figure 6: Decay branching ratios of CP-even Higgs boson in the 4F2HDM-II for the mass-range  GeV  and sample input   .
Figure 7: Decay braching ratios of CP-even Higgs boson with mass  GeV. Plot-(a): Br versus with .  Plot-(b): Br versus , with . In both plots   is fixed.

We analyze the impact of fourth-family fermions on the Higgs boson   decays. For relatively light fourth-family neutrinos and leptons, new decay channels of   can be open, in addition to the conventional SM decay modes whose partial widths have rescaling factors   and   for the fermionic and final states, respectively. Therefore, we shall rewrite the loop-induced decay rates in terms of the modified couplings for both bosonic and fermionic contributions,


with the form factors,


The charged Higgs loops may also contribute to the (17) and (18). For our illustration, we consider the large limit where the contributions are negligible. Such large limit is also fairly reasonable from the flavor physics constraints, including the leptonic decay of mesons , loop-induced transitions, and the mass difference as measured in the mixing [25, 26]. For lighter ,  the inclusion of charged Higgs loop will not affect our physical conclusion. In Fig. 6, we presented a sample of decay branching fractions for   as a function of its mass in the 4F2HDM-II, by including the new invisible decay modes. It clearly shows that the invisible decays   can suppress the other decay channels in the light mass region of .  For ,  they no longer dominate because of   .  From Fig. 5-6, we see that for a light   with mass ,  its production and decays in the 4F2HDM are very different from that of the SM Higgs boson due to the fourth-family quark contributions and the new channels of invisible decays.

In Fig. 7(a), we present the decay branching fractions of   as a function of   with   ,  while in Fig. 7(b) we display  Br  as a function of   with  .  We find that  Br  is sensitive to ,  and for the invisible decays dominate over all other SM channels in the full range of  .

Figure 8: Predicted signal ratios of in the processes   and   as functions of  ,  with Higgs mass for both 4F2HDM-I and 4F2HDM-II. We have take the sample inputs  . 
Figure 9: Predicted signal ratios of for   and   as functions of ,  with Higgs mass for both 4F2HDM-I and 4F2HDM-II. The neutrino mixing angle is taken to be   in plot-(a) and   in plot-(b), while the Higgs mixing angle   in both plots. The horizontal dashed-line in each plot corresponds to the ratio,  .
Figure 10: Predicted signal ratios of in the processes   and   as functions of Higgs mass   for the SM4. The fourth-family neutrino mass  GeV in plot-(a) and  GeV in plot-(b) with are taken.

In Fig. 8-9, we present the predicted for the processes   and   in both 4F2HDM-I and 4F2HDM-II.  In particular, we show the ratio  ,  for the comparison to that of the SM Higgs boson, with the same mass GeV.  In Fig. 8, we plot the ratios   for the 4F2HDM-I and 4F2HDM-II as functions of the neutrino mixing parameter ,  with a sample input of  .  The predictions of the 4F2HDM-I are generally suppressed in comparison with the SM3, thus they cannot be observed from the current LHC data. To detect in the 4F2HDM-I thus requires higher integrated luminosities at the LHC. For predictions of the 4F2HDM-II, Fig. 8 shows interesting excess of signals above that of the SM Higgs boson for both and   channels in the parameter range (with ), where the   signals are significantly higher than the signals.  We note that such 4F2HDM-II model with is quite generic for the dynamical fourth-family models [24]. Combined with the invisible decay channels, this can nicely explain why ATLAS experiment [2] has detected sizable excess of events in the   mode but not the and final states. Fig. 9(a)-(b) depict these signal ratios as functions of for the neutrino mixing parameter and , respectively. We find that the 4F2HDM-II always predicts larger signals than the SM3 in the   channel for ,  while the   signals are mainly suppressed except for (with ).

For comparison with our above 4F2HDM studies, we also analyze the signals in the   and   channels from the one-Higgs-doublet SM including fourth-family (SM4), with relatively light  .  Ref. [27] showed signals from in the SM4 without including invisible decays, while the effect of invisible decays for the SM4 Higgs was discussed in [28] for the LEP searches and in [15, 29] for the LHC searches. In Fig. 10, we present the predicted signal ratios of   in the processes   and   as functions of Higgs mass   for the SM4. For comparison, we assign the fourth-family neutrino mass,  GeV in Fig. 10(a) and  GeV in Fig. 10(b), respectively. From these plots, we see that the   signals are always much more suppressed than the   signals. For the   final states, besides the overall suppression from invisible decays ,  the fourth-family fermions further suppress the decay width of   due to the enhanced fermion-loop contributions that cancel against the -loop in (17). Hence, for the Higgs detection in the SM4, the LHC should observe significantly larger signals than the signals; this is just opposite to the most recent ATLAS and CMS observations [2][3]. Furthermore, Fig. 10(a) shows that the SM4 predictions for the light mass-range GeV are generally lower than that of the conventional SM3 through all three decay channels of ,    and   .  Hence, higher integrated luminosities at the LHC are required for its detection. For Fig. 10(b) with a larger fourth-neutrino mass  GeV, this window shifts to GeV. Fig. 10(b) shows that the SM Higgs boson with GeV or GeV is clearly excluded by the current LHC data due to excessive signals in the    final states. If the recent event excesses around mass-values of  GeV at the LHC (7 TeV) [2, 3] are actually due to statistical fluctuations or other systematical errors, then the low Higgs-mass-ranges of the SM4, namely GeV in plot-(a) and GeV in plot-(b), are still viable and will be further probed at the LHC with higher luminosities.

4 Conclusions

The on-going LHC Higgs searches for the light mass window ( GeV) are crucial for testing the Higgs mechanism and probing new physics beyond the SM. In this work, we studied the new signatures of a light CP-odd Higgs   or CP-even Higgs   in the   and   channels () at the LHC (7 TeV), as predicted by the two-Higgs-doublet-model with the fourth-family fermions (4F2HDM). By including the invisible decays of Higgs boson   or   into fourth-family neutrinos ,  we demonstrated that the predicted   and    signals can explain the recently observed excesses of events in ATLAS [2] and CMS [3] detectors. Due to the absence of cubic gauge-couplings  -- ,  the decay channel   becomes unique for discovering   in the light mass-range   GeV.  Although the fourth-family quark-loops significantly enhance the production cross section of the gluon-fusion process relative to that of in the conventional three-family SM (SM3) as in Fig. 1, the invisible decay modes can properly suppress the   branching fraction (Figs. 2-3) and make the   signals mildly exceed that of the SM3 (Fig. 4). Hence, we found that for our 4F2HDM-II (with generic type-II Higgs sector), a light with mass  GeV can nicely explain the excess of   signals at the LHC [2, 3]. At the same time, the   Higgs boson gives no signal for   channels. Note that the latest ATLAS search showed lower excesses in   modes and the CMS analysis found no excess in the same channel. If a light Higgs boson indeed exists, more LHC data in 2012 will pin down the possible signals in both   and