Vibration mechanism and improved phenomenological model of planetary gearbox with broken sun gear fault

•A method for estimating the fault size of gear tooth is proposed.•The law between the exit time of the faulty gear tooth and the fault size is deduced.•An improved phenomenological model of planetary gearbox vibration signal is established.•The assisted phase for verifying the correctness of the signal model is proposed.