Ground state of the S=1/2 triangular lattice Heisenberg-like antiferromagnet Ba3CoSb2O9 in an out-of-plane magnetic field

Spin-1/2 triangular lattice Heisenberg antiferromagnet has been accepted as an ideal system for quantum magnetism studies and quantum simulations. This system, for which the classical ground state degeneracy is lifted by quantum fluctuations, exhibits a series of novel spin structures for a field applied in-plane and out-of-plane. It has been found that both anisotropy and interlayer interaction play an important role in the stabilization of the spin configurations in a magnetic field. Conversely, the phase transitions and spin-state evolution in a field along various orientations can provide a deep insight into physics of the triangular lattice Heisenberg antiferromagnet system. While the quantum magnetization process in an in-plane field has been studied extensively, the ground state evolution in the field along the c axis requires further investigation. Here we performed high field magnetization and neutron scattering investigations on a model system of spin-1/2 triangular lattice Heisenberg antiferromagnet Ba3CoSb2O9 with field along c axis and with a small offset angle. For H parallel to c, the magnetization reveals a narrow plateau prompting a UUD-like phase, which could be suppressed by tilting the field away from the c axis. From the neutron data, a phase transition mu H-0(c1) similar to 12 T is detected and interpreted as a transition from an umbrella to a coplanar phase. Around about 22.5 T (mu H-0(c2)) for H parallel to c, another transition is observed which might be attributed to a transition between the coplanar V and V' phases based on a comparison with the calculations and previous results. Theoretical calculations using the large-size cluster mean-field plus scaling method predicts a similar phase evolution as the previous semiclassical analysis, and agree with experiment well. The discrepancies between theory and experiment are also discussed, suggesting the physics of a triangular lattice Heisenberg antiferromagnet in a field along c axis has not been fully unraveled.