Comparison of similar Mueller and Jones matrix method in catheter based polarization sensitive optical coherence tomography

•We theoretically illustrate the discrepancy between the eigenvalue decomposition in Similar Jones matrix (SJM) and the polar decomposition in similar Mueller matrix (SMM).•We find that the polar decomposition in SMM can separate diattenuation carrying different optical axis with phase retardation on sample, namely asymmetric non-coaxial diattenuation effect.•We present a real incoherent averaging method in SMM and ind that incoherent averaging in SMM avoids a time-consuming global phase collection and more effectively reduce noise compared with complex coherent averaging in SJM.