Over the past decades, conventional methods and molecular assays have been developed for the detection of tuberculosis (TB). However, these techniques suffer limitations in the identification of Mycobacterium tuberculosis (Mtb), such as long turnaround time and low detection sensitivity, etc., not even mentioning the difficulty in discriminating antibiotics-resistant Mtb strains that cause great challenges in TB treatment and prevention. Thus, techniques with easy implementation for rapid diagnosis of Mtb infection are in high demand for routine TB diagnosis. Due to the label-free, low-cost and non-invasive features, surface enhanced Raman spectroscopy (SERS) has been extensively investigated for its potential in bacterial pathogen identification. However, at current stage, few studies have recruited handheld Raman spectrometer to discriminate sputum samples with or without Mtb, separate pulmonary Mtb strains from extra-pulmonary Mtb strains, or profile Mtb strains with different antibiotic resistance characteristics. In this study, we recruited a set of supervised machine learning algorithms to dissect different SERS spectra generated via a handheld Raman spectrometer with a focus on deep learning algorithms, through which sputum samples with or without Mtb strains were successfully differentiated (5-fold cross-validation accuracy = 94.32%). Meanwhile, Mtb strains isolated from pulmonary and extra-pulmonary samples were effectively separated (5-fold cross-validation accuracy = 99.86%). Moreover, Mtb strains with different drug-resistant profiles were also competently distinguished (5-fold cross-validation accuracy = 99.59%). Taken together, we concluded that, with the assistance of deep learning algorithms, handheld Raman spectrometer has a high application potential for rapid point-of-care diagnosis of Mtb infections in future.