Multi-scale aggregation network for temporal action proposals

•We propose a network with temporal multi-layer dilated convolution for TAP.•We propose the unit level and proposal level multi-scale aggregation strategies.•We propose to take the soft labelling to facilitate action boundary unit prediction.