Molecular Gas Excitation of the Massive Dusty Starburst CRLE and the Main-sequence Galaxy HZ10 at z=5.7 in the COSMOS Field

We report CO(5 -> 4) and CO(6 -> 5) line observations in the dusty starbursting galaxy CRLE (z = 5.667) and the main-sequence (MS) galaxy HZ10 (z = 5.654) with the Northern Extended Millimeter Array. CRLE is the most luminous z > 5 starburst in the COSMOS field and HZ10 is the most gas-rich "normal" galaxy currently known at z > 5. We find line luminosities for CO(5 -> 4) and CO(6 -> 5) of (4.9 +/- 0.5) and (3.8 +/- 0.4) x 10(10) K km s(-1) pc(2) for CRLE and upper limits of < 0.76 and < 0.60 x 10(10) K km s(-1) pc(2) for HZ10, respectively. The CO excitation of CRLE appears comparable to other z > 5 dusty star-forming galaxies. For HZ10, these line luminosity limits provide the first significant constraints of this kind for an MS galaxy at z > 5. We find the upper limit of L'(5 -> 4)/L'(2 -> 1) in HZ10 could be similar to the average value for MS galaxies around z approximate to 1.5, suggesting that MS galaxies with comparable gas excitation may already have existed one billion years after the Big Bang. For CRLE we determine the most likely values for the H-2 density, kinetic temperature, and dust temperature based on excitation modeling of the CO line ladder. We also derive a total gas mass of (7.1 +/- 1.3) x 10(10) M-circle dot. Our findings provide some of the currently most detailed constraints on the gas excitation that sets the conditions for star formation in a galaxy protocluster environment at z > 5.