The application of matrix-assisted laser desorption/ionization time-of-flight mass spectrometry (MALDI TOF MS) in glycan was limited due to their poor ionization efficiency, compared with biomolecules such as proteins and peptides. Aiming to improve the ionization efficiency and simplify preparation procedure simultaneously during MALDI MS analysis, an on-target derivatization method using 3-aminoquinoline (3-AQ)/α-cyano-4-hydroxycinnamic acid (CHCA) as matrix was employed and it was conducted both in the positive and negative ion MALDI TOF MS. Results indicated that after on-target derivatization, the ions generated had substantially improved S/N ratios and sensitivity in the tandem mass spectra. The B/Y− type ions of 3-AQ-labeled glycans could be easily recognized, and cross-ring A− type ions provided additional information to reveal the linkage patterns. Specifically, positive ion mass spectra with protonated adduct as precursor ion produced a simple fragmentation pattern benefited for sequencing and observation of branches. Furthermore, this method was successfully applied in polysaccharides analysis, including arabinoxylan, xylan, arabinogalactan and dextran after enzymatic or acid degradation. This study demonstrated that it was feasible to analyze higher molecular weight polysaccharides by MALDI TOF MS using 3-AQ/CHCA matrix through appropriate hydrolysis, and it allowed much efficient structural interpretation with increased sensitivity and characteristic fragment ions.