Multi-\tau signatures at the LHC in the two Higgs doublet model

# Multi-τ signatures at the LHC in the two Higgs doublet model

Shinya Kanemura Department of Physics, The University of Toyama, Toyama 930-8555, Japan    Koji Tsumura Department of Physics and Center for Theoretical Sciences, National Taiwan University, Taipei 10617, Taiwan    Hiroshi Yokoya Department of Physics and Center for Theoretical Sciences, National Taiwan University, Taipei 10617, Taiwan National Center for Theoretical Sciences, National Taiwan University, Taipei 10617, Taiwan
July 14, 2019
###### Abstract

A detailed simulation study is performed for multi- signatures at the Large Hadron Collider, which can be used to probe additional Higgs bosons with lepton-specific Yukawa interactions. Such an extended Higgs sector is introduced in some of new physics models at the TeV scale. We here consider the two Higgs doublet model with the Type-X Yukawa interaction, where nonstandard Higgs bosons predominantly decay into tau leptons. These extra Higgs bosons can be pair produced via -channel gauge boson mediation at hadron colliders; and (), where , and are CP-even, odd and charged Higgs bosons, respectively. Consequently, multi- originated signals appear in the final state as a promising signature of such a model. We find that the main background can be considerably reduced by requiring the high multiplicity of leptons and tau-jets with appropriate kinematical cuts in the final state. Thus, assuming the integrated luminosity of a hundred of inverse fb, the excess can be seen in various three- and four-lepton channels. With the integrated luminosity of thousands of inverse fb, the determination of the mass as well as ratios of leptonic decay branching ratios of these Higgs bosons would also be possible.

Higgs boson, tau lepton, hadron colliders
###### pacs:
12.60.Fr, 14.60.Fg, 14.80.Cp
preprint: UT-HET 061

## I Introduction

The gauge structure of the standard model (SM) for elementary particles has been tested precisely Ref:PDG (). A missing piece is the Higgs boson, which is responsible for electroweak symmetry breaking and mass generation mechanisms for all SM particles. It is expected that the Higgs boson will be discovered at the Large Hadron Collider (LHC) in near future. The LHC is also searching for the evidence of new physics beyond the SM Ref:LHC-SUSY (); Ref:LHC-4G (). The LHC has already clarified the absence of light new colored particles; e.g., squarks and gluinos in the supersymmetric theories Ref:LHC-SUSY (), or forth generation quarks Ref:LHC-4G (). However, a light particle without strong interactions has not been ruled out yet by the LHC data because of small production cross sections.

The Higgs sector is totally unknown, since no Higgs boson has been discovered yet Ref:LEP-Higgs (); Ref:LHC-Higgs (); Ref:TeV-Higgs (). In the SM, only one scalar iso-doublet field is introduced to spontaneously break the electroweak gauge symmetry. However, since there is no reason for the Higgs sector with only one doublet, there is a possibility of non-minimal Higgs sectors. There are two important experimental constraints on extended Higgs sectors; i.e., the flavor changing neutral current (FCNC) and the electroweak rho parameter. In the SM, these constraints are automatically satisfied: FCNC is suppressed by the Glashow-Illiopoulos-Maiani mechanism, and the rho parameter is predicted to be unity at the tree level due to the custodial symmetry. On the other hand, non-minimal Higgs sectors suffer from both of them in general. It is known that in the Higgs sector with only doublets, the rho parameter is predicted to be unity at the tree level, while Higgs models with higher representations such as those with triplet fields predict the rho parameter to be different values from unity. Therefore, two Higgs doublet models (THDMs) would be a simplest viable extension of the SM. However, in the THDM the most general Yukawa interaction predicts FCNC at the tree level, because both the doublet couples to a fermion so that the mass matrix and the Yukawa matrix cannot be diagonalized simultaneously. In order to avoid this, a discrete symmetry may be introduced under which different properties are assigned to each scalar doublet Ref:GW (). Under this symmetry, each fermion couples with only one scalar doublet, and hence there are no FCNC at the tree level even in the THDM.

There are four types of Yukawa interactions depending on the -charge assignments; i.e., Type-I, II, X and Y. Type-II is the most familiar type of Yukawa interactions in the THDM, which is the Higgs sector of the minimal supersymmetric standard model (MSSM). Another interesting possibility would be the Type-X THDM, where one Higgs doublet couples with quarks and the other does with leptons Ref:Barger (); Ref:AKTY (); Ref:TypeX (). The Type-X THDM can appear in the Higgs sector of a gauged extension of the Type-III seesaw model Ref:GaugedTypeIII (), the model of three-loop seesaw with electroweak baryogenesis Ref:AKS () and a model for positron cosmic ray anomaly Ref:Hall (). In the SM-like limit, where only one of the CP-even Higgs bosons couples to the gauge bosons, the Yukawa couplings of the other Higgs bosons tend to be lepton-specific. Since Yukawa coupling constants are proportional to the mass of fermions, these extra Higgs bosons predominantly decay into tau leptons for the wide range of the parameter space Ref:AKTY ().

The tau lepton has a relatively short lifetime as compared with the muon. It decays into lighter leptons and/or hadrons with neutrinos in the detector. The decay products always produce missing energies, which make event reconstructions rather complicated. However, for an energetic tau lepton, the missing momentum from its decay tends to be oriented to the same direction of the charged track Ref:HRZ (). Therefore, the tau lepton momentum can be approximately reconstructed by using the collinear approximation Ref:HRZ (). Furthermore, the decay of the tau lepton is correlated by its polarization, which can be used to separate leptonic decays from hadronic decays Ref:BHM ().

In this paper, we study multi- signatures at the LHC in the lepton-specific THDM in the SM-like limit. Masses of extra Higgs bosons can be of the order of a hundred GeV under the results. Then the gluon fusion mechanism for such extra Higgs bosons is suppressed at hadron colliders. In this case, extra Higgs bosons can be pair produced; and (), where , and are CP-even, odd and charged Higgs bosons, respectively. Produced Higgs bosons mainly decay into tau leptons because the Yukawa coupling constant is proportional the fermion mass. These characteristic decay modes can be observed in multi- signatures at the LHC. We perform detailed simulation studies for the pair production of the extra Higgs bosons where they subsequently decay into multi- states. It is found that the main background can be considerably reduced by requiring the high multiplicity of leptons and tau-jets with appropriate kinematical cuts in the final state. Assuming the integrated luminosity of a hundred of inverse fb, the excess can be seen in various three- and four-lepton channels.

This paper is organized as follows. In Sec. II, we summarize the Type-X THDM and give basic constraints on the model. The simulation studies of multi- signatures at the LHC in the lepton-specific THDM are presented in Sec. III. Summary and discussions are given in Sec. IV.

## Ii The model and constraints

The Higgs potential of the THDM is defined as Ref:HHG (); Ref:Djouadi2 ()

 VTHDM =+m21Φ†1Φ1+m22Φ†2Φ2−m23(Φ†1Φ2+Φ†2Φ1)+λ12(Φ†1Φ1)2+λ22(Φ†2Φ2)2 +λ3(Φ†1Φ1)(Φ†2Φ2)+λ4(Φ†1Φ2)(Φ†2Φ1)+λ52[(Φ†1Φ2)2+(Φ†2Φ1)2], (1)

where are the Higgs doublets with hypercharge . A softly broken symmetry is imposed in the model to forbid FCNC at the tree level, under which the Higgs doublets are transformed as and  Ref:GW (). The soft-breaking parameter and the coupling constant are complex in general. We here take them to be real assuming that CP is conserved in the Higgs sector.

The Higgs doublets can be written in terms of the component fields as

 Φi=⎛⎝iω+i1√2(vi+hi−izi)⎞⎠, (2)

where the vacuum expectation values (VEVs) and satisfy GeV and . The mass eigenstates are obtained by rotating the component fields as

 (h1h2)=R(α)(Hh),(z1z2)=R(β)(zA),(ω+1ω+2)=R(β)(ω+H+), (3)

where and are the Nambu-Goldstone bosons, , , and are respectively two CP-even, one CP-odd and charged Higgs bosons, and

 R(θ)=(cosθ−sinθsinθcosθ). (4)

The eight parameters and are replaced by the VEV , the mixing angles and , the Higgs boson masses and , and the soft breaking parameter . The coupling constants of the CP-even Higgs bosons with weak gauge bosons and are proportional to and , respectively. When , only couples to the gauge bosons while decouples. We call this limit as the SM-like limit where behaves as the SM Higgs boson Ref:GunionHaber (); Ref:KOSY ().

Assuming the discrete symmetry (see TABLE 1), there can be four types of Yukawa interactions in the THDM, i.e., Type-I, II, X and Y Ref:Barger (); Ref:AKTY ();

 LTHDMyukawa= −¯¯¯¯QLYu˜ΦuuR−¯¯¯¯QLYdΦddR−¯¯¯¯LLYℓΦℓℓR+H.c., (5)

where ( or ) is either or .

In the Type-I THDM, all fermions obtain their masses from the VEV of . In the Type-II THDM, masses of up-type quarks are generated by while those of down-type quarks and charged leptons are acquired by . The Higgs sector of the MSSM is a special THDM, whose Higgs potential is determined by gauge coupling constants and whose Yukawa interaction is of Type-II Ref:HHG (). In the Type-X Yukawa interaction, all quarks couple to while charged leptons couple to . Remaining one is referred to the Type-Y.

The Yukawa interactions are expressed in terms of mass eigenstates of the Higgs bosons as

 LTHDMyukawa= −∑f=u,d,ℓ[+mfvξfh¯¯¯ffh+mfvξfH¯¯¯ffH−imfvξfA¯¯¯fγ5fA] −{+√2Vudv¯¯¯u[+muξuAPL+mdξdAPR]dH++√2mℓξℓAv¯¯¯¯¯¯νLℓRH++H.c.}, (6)

where are projection operators for left-(right-)handed fermions, and the factors are listed in TABLE 2.

Experimental constraints on masses of Higgs bosons , , in THDMs depend on the type of the Yukawa interaction. Masses of neutral bosons have been bounded by LEP experiment to be  GeV and  GeV in the MSSM (Type-II THDM with additional relations) Ref:PDG (). At the LHC, neutral Higgs bosons can be produced via gluon fusion  gf () where we define , , associated production with heavy quarks ,  ffH () and the weak boson mediated processes  Ref:GunionHaber () and  AH+ (). For the large region, stronger mass bounds can be obtained from these production processes at the Tevatron and the LHC Ref:SUSYHiggsTeV (); SUSYHiggsLHC (). However, if the Yukawa interactions of and are quarkophobic which is realized in the wide parameter space in the Type-X THDM, these Higgs bosons are less constrained. The search for such Higgs bosons at the LEP experiments is found in Ref. Ref:LEP4tau (). On the other hand, direct search bounds on charged Higgs boson mass has also been set by LEP experiments as  GeV by assuming  Ref:LEP2tau2v (). Further stronger bound can be obtained in the Type-II(Y) THDM Ref:Barger (); bsg () from the results as  GeV bsg2 (). The observation of decay also constrains the mass of charged Higgs bosons for the large region btaunu (). The LHC can also search for charged Higgs bosons in various production processes such as  H+H- (),  gbH+ () and  WH+ (). However, these bounds depend on the types of the Yukawa interactions. Therefore, the relatively light charged Higgs boson is still allowed by experimental data in the Type-I(X) THDM. In the Type-X THDM, the charged Higgs boson mass can be constrained by the leptonic decays of tau leptons tauleptonicdecay (); tauleptonicdecay2 (). Extra Higgs boson searches in the MSSM (Type-II THDM) have been well studied so far in the literature. Fermiophobic Higgs scenario in the Type-I THDM has also been discussed Ref:Fermiphobic ().

In this paper, we focus on the Higgs boson search in the Type-X THDM, which is less constrained by decay data. In this model, more than of and decay into pairs of tau leptons for in the SM-like limit;  Ref:AKTY (). The neutral Higgs bosons would be produced in pair by process at the LHC. These Higgs bosons predominantly decay into a four- state, , which is the characteristic signal of the Type-X THDM. The tau leptons further decay into leptons or hadrons with neutrinos. Consequently, there are several four-lepton final states such as and with missing energies, where denotes an election or a muon , and is hadronic decay products of the tau lepton. Although the branching ratios of the decay are not large; namely , the decay process would also be a clear signature, since the invariant mass of the muon pair has a peak at and . This signature results in , and final states. We will give the results for simulations of these signals in the next section.

As for the charged Higgs boson associated production, the processes would be dominant. For , more than of charged Higgs bosons decay into a tau lepton with a neutrino and does into a muon with a neutrino. Therefore, the characteristic signatures for this process consist of three tau leptons , one tau lepton with two muons or two tau leptons with one muon . These signals result in the three-lepton final states; , , and with the large missing energy. In the next section, we study the collider signature of these final states in details.

## Iii Multi-τ signatures

In this section, we present the results of our simulation study for the multi- signatures at the LHC in order to probe the production of neutral and charged Higgs bosons, which predominantly decay into tau leptons and occasionally into muons.

First, we explain the framework of our simulation and event analysis. Second, we present studies for the pair production process of the neutral Higgs bosons. Finally, we present studies for the charged Higgs bosons associated production with the neutral Higgs bosons.

### iii.1 Framework for event generation and pre-selection

The signal events are generated by using MadGraph/MadEvent Alwall:2011uj (), where the decay of tau leptons is simulated by using TAUOLA Jadach:1993hs (). The partonic events are passed to PYTHIA Sjostrand:2006za () for parton showering and hadronization. Initial-state-radiation (ISR) and final-state-radiation (FSR) effects are included. We choose the collision energy to be  TeV, and use the CTEQ6L parton distribution functions Pumplin:2002vw (). Throughout this paper, we set the masses of extra Higgs bosons to GeV, GeV and GeV, and take the SM-like limit; . The total cross section for is estimated to be  fb at the tree level Ref:AKTY (). For the charged Higgs boson associated production, , the total cross sections are estimated to be  fb for production, and  fb for production. These mass splitting among the extra Higgs bosons are allowed by electroweak precision data in the SM-like limit Ref:THDMEW1 (); Ref:THDMEW2 (). Background events for (, and ), processes where the weak bosons decay leptonically and hadronically, and jets ( and ) processes followed by leptonic decays of weak bosons are generated by PYTHIA, where the decays of tau leptons are also handled by TAUOLA. The total cross sections for these processes are given as  pb,  pb and  nb, respectively for , and +jets production processes by PYTHIA. We ignore -factor corrections for all the signal and background processes, for simplicity.

First, we perform the pre-selection of the events in various four- and three-lepton channels. In order to take into account the detector availability, muons and electrons are required to be isolated111The isolation condition for muons is given by where is the sum of the magnitude of the transverse momentum of the particles inside the cone around the muon. The isolation condition for electrons is given by , where is the transverse momentum of the jet which contains the electron itself. The jet is constructed from the final state hadrons, electrons, photons and non-isolated muons. For our isolation conditions, the finding efficiency of muons is slightly better than that of electrons. and have  GeV and , where is the transverse momentum, the pseudorapidity is defined as from the scattering angle in the laboratory frame. Those muons and electrons are counted in the events. Then, we construct primal jets from the final state hadrons by anti- algorithm Cacciari:2008gp () with using the FastJet package Cacciari:2005hq (). Among the constructed primal jets, we identify the tau-jet candidates by the following criteria;

a jet with  GeV and which contains or charged hadrons in a small cone () centered at the jet momentum direction with the transverse energy deposit to this small cone more than % of the jet.

The cone of the primal jet acts as an isolation cone to reduce the mis-tagging probability for non-tau jets. We present an estimation of the tau-tagging efficiency and the mis-tagging probability in Appendix. A. The other jets with  GeV and are regarded as a hard jet, which are used to estimate the hard QCD activity of the event. Finally, the missing transverse momentum is calculated as a negative vector sum of the transverse momentum of visible particles, .

We assume that electric charges of tau-jets are measurable. Using the charges of tau-jets, we require the charge sum of the four leptons in the four-lepton channels to vanish to reduce the contributions from the background with mis-identified tau-jets. After the pre-selection of the signal and background events, we perform further event analysis for each channel.

### iii.2 Neutral Higgs boson pair production

Here, we present the results of simulation studies for the neutral Higgs boson pair production process. In the Type-X scenario, this process is characterized by the four-lepton signature, where the leptons can be charged leptons , and also the tau-jet . There are fifteen kinds of the four-lepton channels in total. For the convenience of our analysis, we divide the leptonic channels into three categories; the channels which contain two or more muons, those which contain two or more tau-jets, and those with two or more electrons. However the last channels are difficult to be utilized due to the limited statistics and the negligibly small branching ratio of the Higgs bosons into electrons.

#### iii.2.1 Four-lepton channels with two or more muons

As we have mentioned at the end of Sec. II, the dimuon from the direct decay of the Higgs bosons would be a clear signature in the Type-X THDM. Although the decay branching ratio is only , there is no way to ignore this signature. First, we explain the study for the channel in detail, and then we comment and summarize the other channels with two or more muons.

After the pre-selection, the obtained numbers of events for the channel are , , and for the , , and jets production processes, respectively, in our simulation assuming the integrated luminosity of  fb. Notice that at the pre-selection, the charges of the muons and the tau-jets are not required to be opposite in each to collect all the signals through the decay of the Higgs boson pairs. Instead, we require the charge sum of the four leptons to vanish. The number of signal events are about two orders of the magnitude smaller than background contributions.

In the left panel in FIG. 1, we show distributions of the invariant mass of a muon pair in the channel after the pre-selection. The solid black line is for the signal production, the dashed red line is for the production, the dotted green line is for the production and the dot-dashed blue line is for the jets production. Numbers of events for each process are normalized so as to correspond to the integrated luminosity of  fb. The invariant mass distribution of the dimuon for the production shows two sharp peaks at  GeV and  GeV which corresponds to a pair of primary muons from the decay of extra neutral Higgs bosons. Due to the secondary muon from the decay of tau leptons, the signal events behave a broad distribution peaked around  GeV as well. The dimuon invariant mass distributions for the and jets processes have a peak at through the decay of the boson into the dimuon. The numbers of events in the distributions for the and jets processes decrease rapidly at high mass regions, but that for the process decreases only slowly.

In FIG. 2, we show the distributions of the transverse momentum for a muon, that for a tau-jet, that for a tau-jet pair, the missing transverse energy , the hadronic and leptonic scalar sum of the transverse momentum and , respectively, in the channel after the pre-selection. In all panels in FIG. 2, numbers of the signal and background events are normalized to be unity. Line descriptions follow those in FIG. 1. Taking into account the difference of distributions between signal and background events in FIG. 2, we employ the following selection cuts;

 pτhT>40 GeV, (7a) ET>30 GeV, (7b) HjetT<50 GeV, (7c) HlepT>250 GeV, (7d) ∣∣Mμμ−mZ∣∣>10 GeV. (7e)

Here, we briefly explain the background reduction strategy with these cuts. The transverse momentum of tau-jets in signal events is relatively larger than that in the background processes due to the heavier mass of Higgs bosons than those of and bosons. Therefore a relatively high cut for tau-jets could enhance the signal-to-background ratio. We note that the high requirement of tau-jets is also suitable for the stable tau-tagging efficiency at hadron colliders Aad:2011kt (). Background events from the jets process contain two mis-identified tau-jets from the ISR jets, with a muon pair which comes from the decay. Therefore, jets background events tend to have small , and the cut on is expected to reduce the jets background significantly. The background contribution from the events can be reduced by using the cut on , because the events tend to contain many jets due to the quark fragmentation and ISR/FSR, even though two of them are mis-identified as tau-jets. The cut on can reduce the and jets backgrounds significantly. Furthermore, the events which contain can be reduced by rejecting the events with the invariant mass of the muon pair close to .

The results of the signal/background reduction are summarized at each step in TABLE 3. We show the expected numbers of events for the integrated luminosity of  fb for each process. In the table, we also include the signal events from the charged Higgs boson associated production process . The signal-to-background ratio is evaluated at each step of the cuts, where and represent the numbers of signal and background events, respectively, taking all the and production processes as the signal events. In order to evaluate the signal significance, we use the significance estimator defined as Ref:CMS-TDR ()

 S =√2[(s+b)ln(1+s/b)−s], (8)

which is also given at each step of the selection cuts. The significance is proportional to the square root of the integrated luminosity. We choose the selection cuts basically to enhance . However, since our background events are estimated based on the leading order cross sections and distributions with limited statistics, the ratio should not be small but preferably to make our results conservative. The largest significance can be obtained after the -window cut, where the number of the signal events is expected to be about while that of background events is about giving and for  fb.

In the right panel in FIG. 1, we show the dimuon invariant mass distributions after the selection cuts (7a), (7b), (7c) and (7d). After these cuts, the signal process dominates the total events in the channel. Thus the two resonant peaks are more enhanced. The mass resolution is expected to be quite well due to the fine resolution of the muon momentum measurement. On the other hand, the expected number of events for  fb is not sufficient to observe the peaks in the distribution.

Now let us consider the detail of the events for the signal process from production. As we have mentioned, the final state arises through the and decays of the Higgs bosons. Assuming the dominant branching ratio of , the probability to get to the final state from the former route is expressed as , where the binomial coefficient . On the other hand, the probability through the latter route is . Thus, the ratio of the probabilities through the two routes is , which is predicted to be about in the Type-X THDM. However, in the actual events, the ratio of the expected numbers of the events through the two routes suffers from the acceptance cuts on the muons. Especially, the cut on the muons is expected to reduce the ratio, since the muons from the decay of tau leptons have relatively small transverse momentum. From the left panel in FIG. 1, the ratio of the number of the excess events at the sharp peaks to the number of the events in the continuous distribution is found to be . After the selection cuts up to the cut on in (7d), the ratio becomes (see the right panel in FIG. 1). Thus, we find our selection cuts do not significantly modify the ratio. With this fact, the branching ratios of can be measured as

 B(ϕ0→μ+μ−)=Number of % the excess events at Mμμ peaksNumber of events in the continuous dist.×3[B(τ→μ)]2×ϵ, (9)

where is the correction factor which reflects the difference of the kinematical distributions of the muons between the and decays of Higgs bosons resulting the difference of the kinematical acceptance of the muons. It could be simulated by using the kinematical distributions of the muons calculated theoretically and the acceptance cuts for muons. In our simulation, we get , thus the ratio of the muon acceptance in the decay to that in the decay is .

Once the peaks in the distribution are observed with the sufficient number of events, the measurement of the branching ratio into the dimuon is possible. If we consider the case where the Higgs bosons decay into the other modes as well, such like weak bosons or quarks, the left-hand side of Eq. (9) is replaced by the ratio of the branching ratios or the partial decay widths for muons and tau leptons, which is exactly the ratio of the square of the Yukawa couplings to muons and tau leptons.

Next, we turn our interest to the tau-jet observables. One of the attractive features of the channel is that the four momenta of tau leptons are reconstructable, if the muon pair comes from the direct decay of one of the neutral Higgs bosons. In such a case, the missing momentum in the event is expected to come from the hadronic decay of tau leptons, and then the full kinematics of tau leptons can be reconstructed by using the collinear approximation for the relation of the momenta of the tau leptons and the tau-jets. To enhance the signal events where one of the Higgs bosons decays directly into the dimuon, we require the muon pair to have opposite-sign charges. Then, we apply the collinear approximation for the tau-jets to determine the four momenta of the tau leptons.

Here, we briefly explain the collinear approximation to calculate the four momenta of the tau leptons. If tau leptons are energetic, the missing momentum from its decay would be along the direction of the charged track (either a charged hadron (hadrons) or a charged lepton), , where , are the momenta of the neutrino and the charged track, respectively. The proportionality constant can be determined by fixing . Accordingly, the momentum of the decaying tau lepton can be approximately reconstructed as , where is the momentum fraction of the charged track from the parent tau lepton. At hadron colliders, the transverse components of the missing momentum can be measured. Assuming that the missing transverse momentum of the event is accounted solely by the missing particles in the decays of tau leptons, and applying the collinear approximation for both the tau leptons, the missing transverse momentum can be expressed by the momenta of charged tracks, as . Unknown parameters and are determined by solving simultaneous equations. Using the resulting values of and , the invariant mass of the tau leptons pair is related with that of the tau-jets pair as . The fractions and should be between 0 and 1. In our analysis, we set a 10% margin for the upper bound on the cut of the momentum fractions, i.e., . This is to take into account the resolution of the momentum measurements and also the limitation of the validity of the collinear approximation. Actually, by taking the 10% margin the number of the signal events which have the solution is enhanced by about 10%, although and are almost unchanged. The lower limit of is not significant because of the high cut on the tau-jets.

In the left panel in FIG. 3, we show the distributions, after requiring opposite-sign for the muon pair in TABLE 3. In the right panel in FIG. 3, the distributions of the pair of the reconstructed tau leptons from the two tau-jets are shown, while the events without the solution of the equations in the ranges are rejected. Similarly to the distribution, two resonant peaks can be observed with relatively wide widths at the masses of the neutral Higgs bosons. Only a few backgrounds from and production are expected. The background can be further reduced by the cut on the -window for the reconstructed .

In the upper panels of FIG. 4, the double invariant mass distribution of (left) and (right) for the signal events are shown. In the distribution in the top-left panel, we can see two bands along the -axis at  GeV and  GeV. In the distribution in the top-right panel, two peaks can be found around and [GeV]. In the lower panels we show the distributions of the sum of the background processes for the reference. The background events after solving the tau lepton momenta mostly have . Thus by the -window cut on , more rejections of the background are expected. This two dimensional invariant mass distribution gives an evidence for the pair production of neutral Higgs bosons which dominantly decay into the pair of tau leptons and occasionally into the pair of muons directly. Accurate mass measurement is also possible, when we have sufficient statistics.

Similar analysis to the channel can be performed for the other four-lepton channels with two or more muons, which are , , , and channels. In the channels where there are three or more muons, combinatorial complexity may be avoided by choosing the pair of opposite-sign muons which gives the largest transverse momentum of the muon pair. The signal-to-background ratio and the expected significance for these channels by applying the selection cuts similar to those for the channel are summarized in TABLE 4 for  fb. The cut variables are not optimized for each channel. The cut on is not imposed for the channels with one or less tau-jet, since for these channels process are negligible from the pre-selection level. The collinear approximation is applied for the two leptons other than the two opposite-sign muons. The large and can be observed for the channel as well, where the background is less significant than for the channel.

By the requirement of opposite-sign charges in the dimuon, the background processes hardly lose events, while the signal process reduces one third of events which is consistent with a naive expectation by counting combinatory. It means, in the other word, that the requirement of same-sign charges in the dimuon extracts the signal events with only small background contribution mainly from the process. This is also useful to find the evidence on top of the SM process in the various channels with two muons or also two electrons. Those channels also appear in the latter.

#### iii.2.2 Four-lepton channels with two or more tau-jets

Due to the dominant branching ratio into tau leptons, a large number of events is expected in the channels with high tau-jet multiplicity, thus an excess beyond the SM expectation may be detectable in these channels. On the other hand, four-lepton channels without a primary muon pair from the decays of neutral Higgs bosons are hard to be kinematically reconstructed.

First, we consider the channel. Since this channel contains only hadronic objects but no leptonic objects in an event, the triggering efficiency may not be as good as those in the channels with muons. However, the requirement of the high- tau-jet shall stabilize the abstraction of the events in this channel, and we expect at least relative information would be available such as kinematical distributions and the signal-to-background ratio. Then, we present results and comments for the other four-lepton channels with two or more tau-jets. In the end of this subsection, we also comment on the remaining four-lepton channels.

After the pre-selection with the requirement of the vanishing charge sum of the four hadronic tau-jets, the expected numbers of events are 324, 147, 797 and 5105 for , , and jets production processes, respectively, for  fb. The dominant background contributions come from jets production followed by the decay and the hadronic decays of the tau leptons with two more tau-jets which are mis-identified from the ISR jets.

In FIG. 5, we show distributions of the transverse momentum of the tau-jet, that for the tau-jet pair, the missing transverse energy and the scalar sum of the hadronic and leptonic transverse momentum, and , respectively. Distributions for all the signal and background processes are normalized to be unity. Line descriptions follow those in FIG. 1. In order to reject backgrounds, the following selection cuts are applied;

 pτhT>40 GeV, (10a) ET>30 GeV, (10b) HjetT<50 GeV, (10c) HlepT>350 GeV. (10d)

Reductions of the signal and background events are summarized at each cut in TABLE 5. For  fb, an expected significance is about at the pre-selection level, and reaches to around after the selection cuts. The signal-to-background ratio rises to about 8 at the end of all the selection cuts. Thus, finding the evidence in this channel seems very promising. It may be even possible at more earlier stage at the LHC. The more tight cut for the of the tau-jets can enhance the signal-to-background ratio, while slightly reduces the significance. We note that we have not estimated the background contributions from pure QCD processes as well as the jet production process followed by hadronic decays of weak gauge bosons, which could survive after the selection cuts if all the four tau-jets are mis-identified. We expect that a severe cut should sufficiently reduce the number of events for these processes.

Since in the events all the four hadronic decays of tau leptons could yield missing momenta, the collinear approximation analysis used in the channel cannot be applied, and the masses of the Higgs bosons cannot be reconstructed kinematically. However, as we will see below, distributions of the invariant mass of the tau-jet pair are useful to obtain the information of the mass of the Higgs bosons. To resolve combinatorial complexity from the four tau-jets, one invariant mass is constructed from the tau-jet pair with opposite-charges which gives the largest , and the other is constructed from the remaining tau-jet pair. Then, the invariant masses of the two tau-jet pairs, and , are defined by .

In FIG. 6, distributions of (left) and (right) are shown for the signal events from production, while the background contributions are omitted because they are negligible. In the distribution in the left panel, we can see an endpoint of the distribution at  GeV which corresponds to  GeV. Another mass information for  GeV can be seen as an endpoint in the distribution in the right panel or may be found as a bump in the distribution. Thus, if sufficient numbers of events are obtained by accumulating the integrated luminosity such like a few thousand fb at the LHC, it should be possible to extract the mass of the Higgs bosons by these distributions. In FIG. 7, we also present the two dimensional distribution of the invariant masses and for the signal events in the channel after the selection cuts.

Similar analysis to the channel can be performed for the other four-lepton channels with two or more hadronic tau-jets; such as , , and channels. Due to the expected large number of events, many of them can give large with large by the similar selection cuts for  fb. Measurements of these channels would provide a test of the signatures of tau leptons in the extensive channels, and thus they are useful to probe the production of the tau-lepton-specific Higgs bosons. The endpoint study such like in FIG. 6 may not be suitable for the invariant mass distributions including or , because the momentum fraction for the leptonic decays of tau leptons distributes mainly in a small region Ref:BHM (). The rest of the four-lepton channels are , , and . However, except the channel, the expected numbers of events are too small; a few events or less for  fb. The signal-to-background ratio and the estimated significance by applying the selection cuts for these channels are summarized in TABLE 7 and 7 assuming the integrated luminosity of 100 fb.

Before closing the subsection for the four-lepton channels, we point out an interesting comparison between the and channels. Theoretically, the difference of the two channels arises from the direct decay of Higgs bosons into dimuons through the Yukawa couplings of Higgs bosons to muons. On the other hand, those to electrons are negligible, since . The invariant mass distribution of the dielectron in the channels is nothing but that of the dimuon in the channel except the absence of the two resonant peaks seen at and . The common part of the distributions between the and channels should be originated from the decay, while the excess in the channel relative to the channel should be originated from the decay of the Higgs bosons directly into muons.

Experimentally, the efficiencies of finding electrons and muons are different due to the different isolation condition and acceptance cuts. In our simulation study, the normalization of the events could be corrected by naively multiplying for the number of events in the channel by , which is about the ratio of the finding efficiency for muons to that for electrons obtained by comparing the numbers of events in the and channels.

### iii.3 Charged Higgs boson associated production

At the hadron collider, charged Higgs bosons can be produced in association with the neutral Higgs bosons via the processes AH+ (). In the Type-X THDM, more than of charged Higgs bosons decays into a tau lepton and a neutrino, and does into a muon and a neutrino for  Ref:AKTY (). Therefore, the characteristic signatures for this process would be the three-lepton channels including many tau leptons or muons accompanied with large missing momentum.

There are ten kinds of the three-lepton channels. Following the analysis of the four-lepton channels, we first present an analysis for the three-lepton channels with two or more muons, then we present analysis for those with two or more tau-jets. Brief summary of the analysis for the rest of the three-lepton channels is presented after a while.

#### iii.3.1 Three-lepton channels with two or more muons

In the channel, the two muons can be originated from the decay of or , or one muon comes directly from and the other comes secondarily through the leptonic decay of the tau leptons. In the former case, the two muons must have opposite charges, and the distribution of their invariant mass shows resonance peaks at the mass of neutral Higgs bosons.

In the left panel in FIG. 8, we plot the invariant mass distribution of the muon pair for the channel after the pre-selection. The distributions are scaled to the expected number of the events for  fb for each signal and background process. The expected number of events from the signal (and ) process is four orders of magnitude smaller than that for the background processes, where the dominant contributions come from the jets process.

To extract the signal events, we consider the selection cuts on this channel. In FIG. 9, we plot distributions of the transverse momentum for a muon, that for a tau-jet and that for a tau-jet pair, the missing transverse energy , and the scalar sum and of hadronic and leptonic transverse momentum, respectively, in the events after the pre-selection. Looking at the distributions in FIG. 9, we perform the following selection cuts to extract signal events in the channels;

 pτhT>40 GeV, (11a) ET>80 GeV, (11b) HjetT<30 GeV, (11c) HlepT>350 GeV, (11d) |Mμμ−mZ|>10 GeV. (11e)

Reductions of the number of events are summarized in each step in TABLE 8.