The main issue of hyperspectral image data (HSI) is its high dimensionality which conducts challenge in high dimensional data analysis community. Popular linear approaches can work effectively when the data is unimodal Gaussian class conditional independently distributions. Yet, they usually fail when applied to HSI data since the distribution of HSI data is usually unknown in reality. Locality preserving projection (LPP) addresses this problem approvingly, where the neighborhood information can be preserved in the reduced space. Based on typical behaviors of Fisher's linear discriminant analysis (LDA), a novel discriminant analysis framework under penalty constraint(PFDA), which extends the ideas of LDA and LPP, is developed in this paper. Benefiting from different construction of affinity matrix, our method can also preserve the locality embedding information effectively in the reduced space. Four types of PFDA are analyzed in this paper and the efficiency and effectiveness of proposed methods under penalty framework are demonstrated by both synthesis data and real hyperspectral remote sensing image data set.