According to the limited molecular information reflected by single spectroscopy, and the complementarity of FTIR spectroscopy and Raman spectroscopy, we propose a novel diagnostic technology combining multispectral fusion and deep learning. We used serum samples from 45 healthy controls, 44 non-small cell lung cancer (NSCLC), 38 glioma and 37 esophageal cancer patients, and the Raman spectra and FTIR spectra were collected respectively. Then we performed low-level fusion and feature fusion on the spectral, and used SVM, Convolutional Neural Network-Long-Short Term Memory (CNN-LSTM) and the multi-scale convolutional fusion neural network (MFCNN). The accuracy of low-level fusion and feature fusion models are improved by about 10% compared with single spectral models.