Deep compact polyhedral conic classifier for open and closed set recognition

•We introduce a new deep neural network classier that simultaneously maximizes the inter-class separation and minimizes the intra-class variation.•The proposed method uses the polyhedral conic classification function.•The proposed method has one loss term that allows the margin maximization to maximize the inter-class separation and another loss term that controls the compactness of the class acceptance regions.•The experimental results show that the proposed method typically outperforms other state-of-the-art methods, and becomes a better choice com- pared to other tested methods especially for open set recognition type problems.