Abstract: Linear discriminant analysis (LDA) is a very efficient image feature extraction technique in the supervised scenario. However, LDA often leads to over-fitting when using small scale training samples, and simultaneously might not show an intuitive explanation for the learnt projections from the view of human cognition. To handle these problems, especially for the discovery of those interpretability structures, a called structured sparse LDA (SSLDA) method is proposed by employing the linear regression model of LDA and the structured sparse L\-2,1 mixed norm. Furthermore, to remove the correlations of the learnt linear transforms, the orthogonalized SSLDA (OSSLDA) is also proposed to learn more subtle textural structure information from face images. To solve both two proposed models: SSLDA and OSSLDA, we further introduce a simply and efficient half-quadratic optimization algorithm, which incorporates an auxiliary variable into the objective function and then alternately optimizes between the projecting variable and the auxiliary variable. To evaluate our proposed method, SSLDA and OSSLDA, we conduct extensive experiments on three public face datasets, AR, Extended Yale B and MultiPIE, for the face recognition task by comparing LDA and its several classical variants. The experimental results show the benefits of the proposed methods on both classification accuracy and interpretability.