# Phase separation instabilities and pairing modulations in

## Abstract

There is growing evidence that the unconventional spatial inhomogeneities in the doped high-Tc superconductors are accompanied by the pairing of electrons, subsequent quantum phase transitions (QPTs), and condensation in coherent states. We show that these superconducting states can be obtained from phase separation instabilities near the quantum critical points. We examine electron coherent and incoherent pairing instabilities using our results on exact diagonalization in pyramidal and octahedron Hubbard-like clusters under variation of chemical potential (or doping), interaction strength, temperature and magnetic field. We also evaluate the behavior of the energy gap in the vicinity of its sign change as a function of out-of-plane position of the apical oxygen atom, due to vibration of apical atom and variation of inter-site coupling. These results provide a simple microscopic explanation of (correlation induced) supermodulation of the coherent pairing gap observed recently in the scanning tunneling microscopy experiments at atomic scale in . The existence of possible modulation of local charge density distribution in these materials is also discussed.

###### pacs:

71.10.-w, 73.22.Gk, 74.20.-z, 74.72.-h## I Introduction

Since the discovery of the high-temperature (high-) superconductors in 1986, there have been many advances in the field of superconductivity due to intense worldwide research efforts. Several sets of ceramic cuprate materials have been synthesized and the transition temperature in some of these has exceeded . They all share a common quasi-two-dimensional structure and the superconductivity is believed to originate from the electron correlations in the planes. In spite of intensive study of these compounds during the past decade, there is no consensus on a possible mechanism for superconductivity. At the same time, researchers have focused on out-of-plane effects as an explanation in the cuprate superconductors, because all the cuprate families share the same plane, but the maximum varies dramatically from one cuprate family to another (1). This variation is unlikely to be caused by the electronic interactions in the plane, but it is most probably due to different atomic structures between the planes and their indirect effect on the electronic structure in the plane.

Recently, it has been argued that the non-planar apical oxygen atoms in the pyramid or octahedron play an important role in the out-of-plane atomic-scale lattice effects in high superconductors (HTSCs) (1); (2); (3); (4). More systematic experimental studies have been done on (7); (6); (5) (-). Because of the natural cleavage plane between two layers, the above system has been ideal for probing the planar electronic structure. This material also has a complicated superstructure modulation (supermodulation) which is known to have a strong effect on the position of the apical oxygens but almost no effect on the plane (5). Slezak et al. measured the local superconducting gap using STM with atomic precision and found that the energy gap also has a modulation with the same periodicity as the structural supermodulation (7).

Superconductivity in cuprate materials is often discussed in the context of the quantum critical point that controls the optimal doping and maximum in the transition temperature (8); (9). Most measurements of magnetic correlation length near putative quantum critical point (QCP) reveal that it remains finite. Scanning probes ubiquitously reveal electronic nanoscale inhomogeneity (See Ref. (7) and references therein). All these findings cause us to question the assumption of a smooth evolution to the quantum critical point. A possible alternative explanation of these findings is that the electronic matter develops a spontaneously inhomogeneous modulations (stripes, checkerboards, etc.) near QCP (10). This propensity to spontaneous charge inhomogeneity can significantly benefit from the electronic modulations induced by the apical oxygen positions. From this perspective it is natural to expect that apical oxygen position modulations provide a perturbation that leads to in-plane charge and spin modulation and therefore would lead to a spatially modulated superconducting state with finite correlation lengths both for charge and spin excitations. In addition, scanning tunneling microscopy (STM) measurements in underdoped - provide information about two energy scales or gaps, coherent vs. incoherent pairings, metal-insulator transition, etc. driven by temperature and electron concentration (11); (12); (13); (14); (15).

For most HTSCs, the changes in physical properties are obtained by doping carriers through changes of composition. Apart from the usual doping cases, some HTSCs with Bi, such as -, can experience self-doping, where the ions in non-conducting layers can act as a charge reservoir, which provides free carriers to the plane (17). Here we investigate possible supermodulation of a charge redistribution (self-doping) between the charge reservoir and the plane at fixed stoichiometry. Inspired by the observation of gap modulation, several theoretical proposals have been put forward by introducing variations of coupling constants on a phenomenological level, or applying the conventional BCS theory (18); (19); (20); (21); (22). Here we report the results of a microscopic study of pairing gap modulation for repulsive electrons within individual unit cells in real space for various spatial displacements of the apical atoms on superconducting pairing.

In this paper we discuss the physical nature of the observed local pairing gap modulation and possible charge redistribution in connection with the structural changes, the change of chemical potential and self-doping effect. We argue that the superconducting pairing correlations are strongly driven by pairing modulations while charge density and the chemical potential in the supermodulation phase can change slightly with the periodicity of structural changes. The paper is organized as follows: Following the introduction to the various aspects of pairing modulations, we present the microscopic model and methodology of exact cluster calculations of pyramidal structures in section III. In section IV, we show the results of charge gap variations and pairing modulation in . The concluding summary constitutes Section V. In Appendixes, we present the details of calculations related to various aspects of modulation properties.

## Ii Model

Slezak et al. showed a direct link between the spatial modulation of the out-of-plane atomic structure within an individual unit cell and the variation of the local gap maximum (7). The non-planar atoms, especially the apical oxygen atoms, are playing an indirect but important role on modulating the local electronic structure of the plane. Merz et al. (3) concluded from their X-ray absorption spectroscopy that increasing dopant can induce holes both on oxygens in plane and the apical oxygen atom. This result suggests that both the planar and apical oxygens can be electronically connected and might be controlling the local electronic state by exchanging holes between plane and other planes. This could be the origin of interlayer interactions and local gap variations. Therefore, in order to simulate the spatial gap modulation, we need a model which can treat the local electronic state exactly and simultaneously include an out-of-plane interaction that can exchange electrons/holes with the basal atoms. This requirement suggests the use of a quantum cluster treatment to deal with local correlations and an extra out-of-plane site to control the electron/hole concentration on the plane.

We begin with the canonical single band Hubbard model on a pyramid cluster (as shown in Fig. 1).

(1) |

Here, () is the electron destruction (creation) operator at the basal sites with spin (or magnetic sublevel), while () is the same operator at the apical site. The first term indicates the hopping between the nearest basal sites and ( denotes the nearest neighbor sites) with the same hopping parameter . The second term describes hopping between the apical site and the basal site with a hopping amplitude . In addition, is the on-site Coulomb interaction. In practice, the hopping within the plane should be much bigger than that of the out-of-plane atom, i.e. . The energies are measured with respect to , unless otherwise stated. The cluster can be solved exactly for all the eigenvalues and eigenvectors (23). The apical site can exchange holes with all the basal sites, so displacements of the top site within the cluster will affect the planar electronic structures.

In the absence of electron-hole symmetry, the sign of the hopping amplitude between the nearest neighbor sites leads to distinct changes of the electronic structure (24). In nonbipartite structures, such as the pyramid or tetrahedron, the results strongly depend on the sign of and . Here the pyramidal non-bipartite geometry is defined by choosing as the hopping connecting square (plaquettes) sites in the basal plane, and we consider various coupling parameters as hoppings between the apical site and the planar site in the square base (Fig. 1).

## Iii Methodology

An essential element for understanding our results on pairing modulation is the formation of coherent pairing state of electrons with a negative charge gap and positive spin gap obtained earlier in Refs. (23); (24); (26); (27); (28); (29); (25) and references therein. For completeness here we first briefly summarize the key results of exact diagonalization in small clusters at weak and moderate values. The square lattices, generated recently on optimized (finite) Betts unit cells, provide strong evidence for the phase separation instabilities found in generic , and other clusters (28); (29).

### iii.1 Charge and spin gaps

Numerical (exact) calculations of the energy levels for the relevant electron number and a given spin are used to define charge and spin gaps. The charge excitation gap at finite temperature can be written as an energy difference of canonical energies, , where and . The charge gap determines the stability of an electron state compared to an equal admixture of and electron states (the average number of electrons for this mixture is still ). Depending on the strength of the on-site electron-electron repulsion , this charge excitation gap can be positive or negative . The pair binding energy determines whether a total 2 electrons on pair of isolated clusters prefer to distributed uniformly () or give rise to local charge inhomogeneity (). For odd this corresponds also to an effective electron pairing, since clusters are more likely have an even number of electrons. When the charge gap is negative, the system energetically prefers the mixed state with and electron configurations.

The spin gaps , defined in the grand canonical ensemble, can provide essential insights into the response of the system to applied magnetic field . The magnetic susceptibility is defined as a function of the magnetic field near the critical chemical potential (say ) and, in the negative charge gap region (), we introduce the spin gap as the magnetic field which gives rise to the first magnetic susceptibility peak, i.e., the magnetic field required to drive the system into its lowest exited state.

### iii.2 Quantum critical points

The zeros of canonical and the grand canonical gaps as a function of at describe QCPs for continuous and abrupt quantum phase transitions. The change of sign in canonical gaps at the level crossing point away from half filling is crucial for electron (charge) phase separation and pairing instabilities. As temperature approaches zero, the possible sign change in canonical gap at signifies the existence of a QCP related to first order (dramatic) changes. This QCP describes an abrupt (discontinuous) change in the ground state of a many-body system due to strong quantum fluctuations. The negative charge gap () shows the regions of electron instability with a tendency toward phase separation. For one hole off half filling the Mott-Hubbard like insulating ground state for electron-hole pairing with is stable at all , while for , the ground state consists of paired electrons (). This pairing instability which redistributes electrons signals the formation of spatially inhomogeneous, hole-rich and hole-poor regions.

### iii.3 Phase separation instabilities

Many cooperative phenomena invoked in the approximate treatments of the “large” concentrated systems, are also seen in exact analysis of pairing instabilities in the canonical and grand canonical ensembles of the small clusters in the thermodynamic equilibrium. The negative sign of the canonical charge gap () is a signature of phase separation instabilities. However, instead of a full phase separation, the spin gap rigidity () provokes local inhomogeneities in electron distribution. The spatial inhomogeneities in the charge distribution imply different electron configurations, close in energy. The quantum mixing of these hole-rich and hole-poor clusters for one hole off the half filling case provides a stable spatial inhomogeneous media that allows the pair charge to fluctuate. Thus, the negative sign of charge gap implies phase (charge) separation of the clusters on hole-poor and hole-rich regions.

### iii.4 Coherent pairing

We find that at rather low temperatures the calculated positive gap in grand canonical method can have equal amplitude with a negative charge gap derived in the canonical method, i.e., with identical gaps, . Such behavior is similar to the existence of a single quasiparticle gap in the conventional BCS state. We identify such an opposite spin (singlet) coupling and electron charge pairing as spin coherent electron pairing. This is an important characteristic of (quantum) phase coherence in the ground state associated with simultaneous coherent pairing of independent charge and spin (Fermion) entities and the corresponding full Bose-Einstein condensation of these “preformed” bosons with a single energy gap. Such behavior is similar to coherent pairing with a unique quasiparticle gap in the conventional BCS theory.

### iii.5 Incoherent pairing

However, unlike in the BCS theory, the charge gap differs significantly from the spin gap as temperature increases above the superconducting temperature, . It is important to note that the coherent pairing formation through separate condensation of bound holes and coupled opposite spins (both boson entities) at different critical temperatures and is somewhat different from the Cooper pairs (composite bosons) condensation with a unique critical temperature. The magnitudes of the spin gap and charge gap also differ with increasing temperature here, unlike in the BCS case. This implies the emergence of two gaps and two energy scales or two characteristic critical temperatures leading correspondingly to incoherent pairing of pairs with unpaired (opposite spins) far above seen in HTSCs using STM measurements (12); (13); (14); (15). Microscopic spatial inhomogeneities and incoherent pairing detected in these experiments are correlated remarkably with predictions of charged pairs without quantum phase coherence above in nanoscale. The positive change gap describes charge redistribution and Mott-Hubbard like transition above . The positive spin gap calculated in the grand canonical approach implies coherent pairing with a homogeneous spatial distribution below . This picture is consistent with some STM measurements in (16).

## Iv Results

### iv.1 Square pyramids

High temperature superconductors have complex layered structures. For example, there are various layers, bilayers or trilayers that affect the superconductivity attributed to the Cu-O planes acting as reservoirs of electrons or holes. Here we attempt to model such complex structures with a simple pyramid having a square base and and an out-of-plane atom. As discussed in our previous publications, a negative charge gap is essential for having coherent pairing states at zero temperature . We first consider the simplest case to testify the validity of the model: a symmetric pyramidal cluster with equal hopping terms between the apex atom and the all the basal atoms. The charge gap at as a function of coupling parameter in Fig. 2 is shown at various values and low temperatures, (in units of ) close to optimal doping for one hole of half filling, (). The energy gap vanishes in a multiparameter space of and , where its sign changes define the quantum critical points for electron charge and spin pairing instabilities. In the Fig. 2 plots with provides strong evidence for the existence of a QCP of level crossing instability, associated with phase separation instability. For example, the charge gap at vanishes () at . This QCP describes an abrupt change in the ground state of a many-body system due to strong quantum fluctuations, i.e., a transition from electron pairing into Mott-Hubbard behavior. The negative charge gap () at corresponds to coherent electron pairing in a vertically deformed square pyramid (with in the basal plane), while for , the Mott-Hubbard insulating ground state for electron-hole pairing with is stable.

As shown in Fig. 2, as the hopping increases but still small (for example, for ), the value of the charge gap changes very slowly, which corresponds to the case with the top site far from the basal plane. The result indicates that the presence of the top site induces very small perturbations on the electronic state at the planar electronic state when it is far away. However, when is large enough (for example, for ), there is a dramatic change and the charge gap becomes positive. This illustrates the case with the top site close enough to the base, so the electronic state on the top site greatly affects the electronic state on the basal plane and tends to destroy the electron pairing on the basal as the apical atom gets closer to the plane. Comparing with results of the elementary square geometries (24), we can conclude that the electronic structure of the basal sites contributes significantly to the formation of electron pairs, while the apical site tries to destroy such pairing by “sharing” holes/electrons with the basal sites. This result is consistent with the conclusion of Slezak et al. (7) who stated that the “pair density” is anti-correlated with the height of apical atom. Given the consistency of this simple result, the pyramid may be a good candidate for studying local gap modulations. The pairing gap has the maximum magnitude and the widest range at , so we here consider the gap modulation for only .

Another feature of the pairing gap behavior in the pyramidal structure is that when the variations of are small, the gap varies linearly with a very small slope. It suggests that the apical effects at small values can be treated by perturbations. This feature resembles the effect of small next nearest neighbor hopping in the square lattice. Actually, the two effects are equivalent when is small (See Appendix A). This equivalence indicates that the influence of a far-away top site is like a bridge connecting the next nearest neighbor sites so that it perturbs the electronic state at the plane but does not significantly destroy the paired state. Slezak et al. observed a gap modulation of about in their experiments which is large enough to address the destruction of electron pairs (7). Such a big gap modulation is not suitable for a perturbation treatment.

### iv.2 Displacement of apical site

From the analysis of the previous section, we conclude that the best simulation conditions for the charge gap are and . The next step is to find out the position dependence of the hopping parameters. The hopping parameter between the top and the base is closely related to the distance between the apical site and the basal site, i.e. , where is the distance between the apical site and the basal site . In an ideal crystal, there is an equilibrium position with the hopping parameter that makes the system most stable. When the layers of atoms are present above the Cu-O planes, the position of the top site can be perturbed with a lattice distortion represented by a very small displacement . The new hopping can be expanded as

(2) |

where is the slope of : , so the value of is strongly dependent on the functional form of and the value of . Notice, when the variation of the position of the top site is relatively small, the change of the hopping parameter can be well approximated by a linear function. By adjusting the value of and range of , we can easily model a gap variation.

The variation of the apical site can be the displacement along vertical and/or horizontal directions. A vertical displacement is equivalent to the case discussed in the Sec. IV.1 except for converting the dependence of the hopping parameters to that of a spatial displacement. Because ,

(3) |

where . is the gradient of hopping along the vertical () direction. The pairing gap can be calculated as a function of the vertical displacement based on Eq. (3). We choose to be and confine the change of the position of apical site . Fig. 3 plots the charge of gap as a function of at different values. In order to simulate the sinusoidal gap behavior, we consider as a linear function of . The plots in Fig. 3 show two regions with different slopes, but only in the region with higher slope, the apical atom is close enough to interact with the basal electronic pairing states. Therefore the useful region for our simulation is only confined to a small region of displacement . As discussed in Appendix B, we need to have the range of linear function smaller than in the simulation. According to Fig. 3, at , the linear range with higher slope up to (approximately) achieves an amplitude about of the equilibrium gap magnitude, which is close to the result in the experiment (), so is probably the suitable region for in the simulation.

At fixed the horizontal distortion provide the relationship between the distance change of apical atom with regard to the atoms in basal plane or corresponding planar and displacements

(4) |

and the corresponding hopping parameters between the apex and the basal sites are reduced to

(5) |

where . is the gradient of hopping on the and direction of the basal plane. We confine the change of and to a small range and and to be for the same reason. We calculated the charge gap of the system at different horizontal positions with in Fig. 4. The gap behaviors with horizontal displacements are not linear and the amplitude of the gap change is also about of the gap magnitude at position. By Comparing Fig. 3 and Fig. 4, the displacements on vertical and horizontal directions have almost the same effects on the basal electronic states, so the gap change due to the planar displacements appears to be equally important to that caused by vertical distortions. The simulation become rather cumbersome by combining linear vertical and nonlinear horizontal gap behaviors. However, the measurements clearly show the sustained square atomic structure in the plane while other layers are strongly distorted by supermodulation (5). This implies that the planar structure of our model should somehow preserve a square symmetry. Therefore, the horizontal displacements of the apical site must be strongly limited to maintain such a square symmetry. Since the vertical distortions basically do not influence the planar square symmetry, then only vertical displacements of the apical site are essential in practice for simulation of pairing gap modulation.

### iv.3 Pairing modulation

Below we show the results of numerical calculations of pairing gap variation within individual unit cells for various displacements of the apical sites on superconducting pairing. The pairing (sinusoidal variation of gap) modulation has been found in using scanning tunneling microscopy. The local pyramidal structures affect the superconducting crystalline “supermodulation” of the energy-gap in bulk . Such supermodulation displacements can be seen for dopant apical oxygen displacement in both the “a” and “c” directions and tilt the pyramid. The local gap modulations here can merely be related to the dopant density modulation with the same period as the supermodulation. The pyramidal coordination of oxygen atoms surrounding each copper atom in (see Fig. 1 in Ref. (7)) is equivalent to the square pyramid geometry built on the basal squares by a rotation of 90 degrees with respect to the vertical axis). Thus a translation along “a” direction for in is equivalent to displacements along the diagonal direction in the basal Cu unit cell. The local supermodulation displacements in both the “a” and “c” directions are equivalent to vibrational distortions along the diagonal directions in the square pyramid and vertical “c” direction. Experimental results show explicitly that variations found near the dopant atoms as the primary effect of the interstitial dopant atom which is to displace the apical oxygen so as to diminish the distance “d” between atoms or distort (slightly tilt) the pyramid. We apply our exact results on the square pyramids near the optimal values to fit into the data for the experimentally observed variation of gap with the unit cell distortions.

We study the impact of the apical atom on superconductivity by monitoring the local negative gap as a function of the vertical displacement of the apical atom from its average position. Experiments have shown that the modulation of the lattice structure can be expressed as a sinusoidal wave function with respect to a phase which is defined according to the period of the supermodulation. To simulate the modulation, we introduce the modulation of top site along the vertical direction: , where is the equilibrium position of the apical atom where the hopping between top and base is set to . here is a phase according to the period of the modulation of the top site and is a constant phase shift. We use the absolute value of charge gap as the pairing gap, so we can derive the pairing gap modulation as a function of and by properly setting parameters we can fit our results into experimental data. The fitting with experimental observation of the sinusoidal gap behavior is shown in Fig. 5 for optimally doped conditions. The details of the fitting procedure are given in Appendix B. Our calculations are consistent with results in the Ref. (7) which shows a modulation of average gap value for optimally doped . The fitting parameters are and . , one of the free parameters, is within the range so it is consistent with our discussion in the Sec. IV.2. The value of , which is the other free parameter, is close to values derived from other theoretical methods (30); (31). This provides additional evidence for the validity of our model.

One interesting question has to do with the intrinsic origin of this pairing gap modulation. There are several possible answers here: the modulation of (1) electron/hole pair strength, (2) the electronic density and (3) the electron/hole pair density. In order to verify the first option, we calculate the change of the chemical potential of the system with the modulation, where is calculated as an average of and (defined in Sec. III): . The chemical potential represents the average energy needed to break an electron/hole pair, so it shows the average binding energy of the hole/electron pairs. We find that modulation amplitude of is smaller than of the average chemical potential. This result indicates that although the pairing gap experiences a strong modulation, the binding energy of the hole/electron pairs is not affected, i.e., the pairing gap modulation is a strong coherent effect not strictly relevant to the modulation of (hole/electron) pairing strength. We have also studied the change of electron density on the apical site with modulation. When the apical site is very far away from basal plane (all ), the gap is maximum and there is exactly one electron on the apical atom, i.e., decoupling of the apical atom with the plane restores the two-dimensional character of electron pairing. For our simulated case, when the apical site is spatially close to the plane, we find that the electron density of the top site increases, which supports the experimental observation that the dopant on oxygen can provoke a transfer of holes/electrons on both plane and atoms. However, our calculations show that the amplitude of modulations of the electronic density on apical site is smaller than of the average value. This result indicates that the electronic density on the pairing plane is insignificantly affected by modulation of the out-of-plane structure. Therefore, the gap modulation does not originate from the modulated electronic density. Therefore, we propose the modulation of electron/hole pair density is the source of the pairing gap modulation. This kind of modulation of pair density is somewhat similar to the hole-pair density wave proposed in Ref. (32).

## V Summary

The effects of disorder in the position of the apical atom outside the CuO plane on the superconducting gap have been investigated in (Bi2212) within the framework of small single band Hubbard clusters. We have shown that exact cluster calculations in appropriate geometries can quantitatively describe the displacements of the atoms in the bulk of material with its well-known supermodulation behavior. Our calculations at low temperatures can be used to fit and understand experimental data under variation of the position of the apex atom in the vertical “c” direction or along “a”, “b” or square diagonal directions. Independent of the character of the quasiperiodic distortion, we find modulation in the variation of the pairing gap around its central position. The theory provides strong evidence that the supermodulation is correlated with the out-of-plane displacement of the apical site and shows how variations of the inter-atomic distances affect the coupling and maximum of the superconducting energy gap. The current, exact solution-based study offers strong evidence that the inhomogeneity observed in the STM experiments is driven by distortions of the dopant oxygen atoms, located away from the CuO plane, whose primary effect is to suppress local pair formation.

The structural changes in our microscopic model in the vicinity of QCP provide the locally enhanced pairing modulation where the nanoscale variation of the pairing strength in the superconducting state have a dramatic effect on the pairing gap. type materials contain the CuO planes and a ”reservoir” of carriers in other layers. A charge redistribution may occur between plane and nonplanar atoms driven by supermodulation, i.e., out-of-plane displacements of the apical atom. Usually one discusses the charge redistribution by changing composition, such as oxygen stoichiometry, etc. Thus, one of the most direct ways to determine the structural changes in couplings can be done through a detailed analysis of the charge distribution and the valence states of the respective out-of- and in-plane ions. In particular, the position of the apex oxygen or the corresponding bond length (coupling) is a sensitive probe of these effects. To summarize, this effect may explain structural changes such as the shift of the apical oxygen atom under modulation, and therefore one would expect that there should be a charge density modulation accompanying the formation of a modulated gap in type materials. However, our results strongly suggest that there should not be significant charge transfer from the reservoir into the superconducting plane. Probably the most direct confirmation of the rigidity of charge density distribution can be obtained for the hole concentration from positron annihilation experiments or X ray absorption spectra (33); (34).

The gap modulation or corresponding quantum oscillations in - can be attained by tuning pressure or doping due to the direct access they provide to the structural changes that can be traced in the vicinity of QCP. These calculations can also be extended to finite temperatures, different doping levels or electron concentrations or can be applied to slightly different (octahedron, tetrahedron, etc.) geometries. The obvious next step is to see how the charge gap variations change with temperature and its relationship with the critical temperature T of the cuprate superconductors to explore methods of enhancing and maximizing T. In a nutshell, the spectroscopy assessed by cluster studies of repulsive electrons possesses both energy and symmetry information on the local pairing interaction and consequently contains the essential elements for understanding the variations of superconducting gap, coherent pairing and modulation in inhomogeneous, large systems. Heavily relying on gained insights from superconductivity effects driven by out-of-plane distortions of the apical atoms, one can understand also the role of local out-of-plane impurities tied to the mechanisms of superconductivity in other layered high T superconductors, impurities in graphene intercalated compounds, etc.

## Vi Acknowledgments

We are grateful to A. Bishop, J.C. Davis, K. Fujita, H. She, and J.X. Zhu for useful discussions. The authors acknowledge the computing facilities provided by the Center for Functional Nanomaterials, Brookhaven National Laboratory supported by the U.S. Department of Energy, Office of Basic Energy Sciences, under Contract No.DE-AC02-98CH10886. The work was performed also, in part, at the Center for Integrated Nanotechnologies, a U.S. Department of Energy, Office of Basic Energy Sciences user facility at Los Alamos National Laboratory (Contract DE-AC52-06NA25396) and Sandia National Laboratories (Contract DE-AC04-94AL85000).

## Appendix A The relationship between the apical and next nearest couplings

The effect on the charge gap of a small apical hopping is similar to that of a small next nearest hopping. Here are a little simple investigates of the relation between these two small extra hopping based on the first order perturbation. For the simplest case, assume all the apical hoppings are the same , and is used to denote next nearest hopping. Both and are small relative to .

Assume that the ground state of a four-site cluster without next nearest neighbor is and the ground state of an isolated apical atom is , so the first order correction due to the next nearest hopping is

(6) |

where means summation over all the next nearest neighbors. While, the first order perturbation due to the apical hopping is

(7) |

where is the state of the top atom. Comparing Eq. 7 and Eq. 6, multiplication is the only difference. If , the two equation are equivalent. Usually, can be state with no electron , one up spin , one down spin , or two spins and these four states build up a complete basis, so . However, only include three terms , , and , so

(8) |

Therefore, as long as the ground state of apical atom is not a two spin state, the effect on the ground state energy within the first order correction due to the apical atom is same as the next nearest hopping. It is hard to determine all the cases that the condition is satisfied, but some specific cases can be empirically found: () when hopping between the apex and the plane is small, the holes have very small probability to occupy the apical atom at the same time; () When the electron interaction is strong enough, the double occupancy is prohibited, but this case also causes the spin frustration because of the non-bipartite structure.

## Appendix B Modulation fit with experiment

The experiment (see (7)) shows a cosinusoidal superconducting gap behavior. The average gap value at optimal doping is about meV and the (peak-to-peak) amplitude of gap modulation is about of the average gap value. The sinusoidal behavior is caused by the super modulation in the structure of the BSCCO system. The detail of the modulation structure can be found in Ref. (5). From these papers, The layer does not change too much from the original square structure, but the and layers are strongly distorted. Therefore, the modulation is mainly coming from the changes of and layers and the modulation of the and atoms are found to be combination of sinusoidal waves.

In our model, the effects of the and layers are included in the apical site of the pyramid Hubbard model. The distortion of the two layers can be simulated by the displacement of the apical site along vertical and horizontal direction. As mentioned in Sec. IV.2, only the vertical displacement is essential in our model due to the requirement of a strong square symmetry, so the horizontal distortions can be ignored. The equilibrium hopping term between top and base is set to at which the pairing gap is about . In the experiment the average gap value is , so . Experiments (7); (5); (35) shows that the vertical amplitudes of the modulation layers are about where is the vertical lattice constant. The amplitude of vertical displacement of the apical site in our model should be much smaller than experimental results because the simplified apical site averages in matter of fact describe displacements of several (coupled) layers in real material. In the paper, we set the amplitude to . The fitting with experimental results also provides and by the amplitude and the phase shift of gap modulation.

### References

- H. Eisaki, N. Kaneko, D. L. Feng, A. Damascelli, P. K. Mang, K. M. Shen, Z.-X. Shen, and M. Greven, Phys. Rev. B 69, 064512 (2004)
- Hua Zhou, Yizhak Yacoby, Vladimir Y. Butko, Gennady Logvenov, Ivan Božović, and Ron Pindak, Proc. Natl. Acad. Sci. 107, 8103 (2010)
- M. Merz, N. Nücker, P. Schweiss, S. Schuppler, C. T. Chen, V. Chakarian, J. Freeland, Y. U. Idzerda, M. Kläser, G. Müller-Vogt, and Th. Wolf, Phys. Rev. Lett. 80, 5192 (1998)
- Y. Ohta, T. Tohyama, and S. Maekawa, Phys. Rev. B 43, 2968 (1991)
- X. B. Kan, S. C. Moss, Acta Crystallogr. B48, 122 (1992)
- J. Lee, K. Fujita, K. McElroy, J. A. Slezak, M. Wang, Y. Aiura, H. Bando, M. Ishikado, T. Masui, J.-X. Zhu, A. V. Balatsky, H. Eisaki, S. Uchida, and J. C. Davis, Nature 442, 546 (2006)
- J.A. Slezak, J. Lee, M. Wang, K. McElroy, K. Fujita, B.M. Andersen, P.J. Hirschfeld, H. Eisaki, S. Uchida, J.C. Davis, Proc. Natl. Acad. Sci. 105, 3203 (2008)
- Subir Sachdev, Science 288, 475 (2000)
- S. L. Sondhi, S. M. Girvin, J. P. Carini and D. Shahar, Rev. Mod. Phys. 69, 315 (1997)
- Jian-Huang She, Jan Zaanen, Alan R. Bishop, and A.V. Balatsky, Phys. Rev. B 82, 165128 , (2010)
- S. E. Sebastiana,, N. Harrison, M. M. Altarawneh, C. H. Mielke, R. Liang, D. A. Bonnc, W. N. Hardyc, and G. G. Lonzaricha, Proc. Natl. Acad. Sci. 107, 60175 (2010)
- K. K. Gomes, A. N. Pasupathy, A. Pushp, S. Ono, Y. Ando and A. Yazdani, Nature 447 (2007) 569
- M. C. Boyer, W. D. Wise, K. Chatterjee, Ming Yi, Takeshi Kondo, T. Takeuchi, H. Ikuta and E. W. Hudson, Nat. Phys. 3 802 (2007)
- K. Tanaka, et al., Science, 314 1910 (2006)
- W. S. Lee, et al., Nature (London) 450 81 (2007)
- M.C. Boyer, Nat. Phys. 3 802 (2007)
- D. I. Khomskii, arXiv:cond-mat/0101164v1 (2001)
- Tamara S. Nunner, Brian M. Andersen, Ashot Melikyan, and P. J. Hirschfeld, Phys. Rev. Lett. 95, 177003 (2005)
- Brian M. Andersen, P. J. Hirschfeld and James A. Slezak, Phys. Rev. B 76, 020507(R) (2007)
- K.-Y. Yang, T. M. Rice, and F.-C. Zhang, Phys. Rev. B 76, 100501(R) (2007)
- Y. He, S. Graser, P. J. Hirschfeld, and H.-P. Cheng, Phys. Rev. B 77, 220507(R) (2008)
- T. Machida, T. Kato, H. Nakamura, M. Fujimoto, T. Mochiku, S. Ooi, A. D. Thakur, H. Sakata, and K. Hirata, Phys. Rev. B 84, 064501 (2011)
- A. N. Kocharian, G. W. Fernando, K. Palandage, and J. W. Davenport, Phys. Rev. B 74 024511 (2006)
- A. N. Kocharian, G. W. Fernando, K. Palandage, and J. W. Davenport, Phys. Rev. B 78, 075431 (2008)
- A. N. Kocharian, G. W. Fernando, C. Yang 2010 Spin and charge pairing instabilities in nanoclusters and nanomaterials, In Scanning Probe Microscopy in Nanoscience and Nanotechnology (Springer) p 507-570.
- A. N. Kocharian, G. W. Fernando, K. Palandage and J. W. Davenport, Phys. Lett. A 373, 1074 (2009)
- G. W. Fernando and K. Palandage, A. N. Kocharian, J. W. Davenport, Phys. Rev. B 80, 014525 (2009)
- Kun Fang, G. W. Fernando, A. N. Kocharian, Phys. Lett. A 376, 538 (2012).
- A. N. Kocharian, Kun Fang, G. W. Fernando, J. Magn. Magn. Mater. 324, 3427 (2012)
- A. K. McMahan, Richard M. Martin and S. Satpathy, Phys. Rev. B 38, 6650 (1988)
- D. M. Newns, P. C. Pattnaik, and C. C. Tsuei, Phys. Rev. B 43, 3075 (1991)
- Han-Dong Chen, Jiang-Ping Hu, Sylvain Capponi, Enrico Arrigoni, and Shou-Cheng Zhang, Phys. Rev. Lett. 89, 137004 (2002)
- Y. C. Jean, C. S. Sundar, A. Bharathi, J. Kyle, H. Nakanishi, P. K. Tseng, P. H. Hor, R. L. Meng, Z. J. Huang, C. W. Chu, Z. Z. Wang, P. E. A. Turchi, R. H. Howell, A. L. Wachs, and M. J. Fluss , Phys. Rev. Lett. 64, 1593 (1990).
- Yasuharu Hirai, Izumi Waki, Atsushi Momose, Tokuumi Fukazawa, Toshiyuki Aida, Kazumasa Takagi and Tatsumi Hirano, Phys. Rev. B,45, 2573 (1992)
- A. Bianconi, M. Lusignoli, N. L. Saini, P. Bordet, A. Kvick and P. G. Radaelli, Phys. Rev. B 54, 4310 (1996)