Intelligent fault diagnosis of complex machinery is crucial for industries to reduce the maintenance cost and to improve fault prediction performance. Acoustic signal is an ideal source for diagnosis because of its inherent characteristics in terms of being non-directional and insensitive to structural resonances. However, there are also two main drawbacks of acoustic signal, one of which is the low signal to noise ratio (SNR) caused by its high sensitivity and the other one is the low computational efficiency caused by the huge data size. These would decrease the performance of the fault diagnosis system. Therefore, it is significant to develop a proper feature extraction method to improve computational efficiency and performance in both periodic and irregular fault diagnosis. To enhance SNR of the acquired acoustic signal, the correlation coefficient (CC) method is employed to eliminate the redundant intrinsic mode functions (IMF), which comes from the decomposition procedure of pre-processing known as ensemble empirical mode decomposition (EEMD), because the higher the correlated coefficient of an IMF is, the more significant fault signatures it would contain, and the redundant IMF would compromise both the SNR and the computational cost performance. Singular value decomposition (SVD) and sample Entropy (SampEn) are subsequently used to extract the fault feature, by exploiting their sensitivities to irregular and periodic fault signals, respectively. In addition, the proposed feature extraction method using sparse Bayesian based pairwise coupled extreme learning machine (PC-SBELM) outperforms the existing pairwise-coupling probabilistic neural network (PC-PNN) and pairwise-coupling relevance vector machine (PC-RVM) by 1.8% and 2%, respectively, to achieve an accuracy of 93.9%. The experiments conducted on the periodic and irregular faults in the gears and bearings have demonstrated that the proposed hybrid fault diagnosis system is effective.