Peptide-based therapeutics hold immense promise for the treatment of various diseases. However, their effectiveness is often hampered by poor cell membrane permeability, hindering targeted intracellular delivery and oral drug development. This study addressed this challenge by introducing a novel graph neural network (GNN) framework and advanced machine learning algorithms to build predictive models for peptide permeability. Our models offer systematic evaluation across diverse peptides (natural, modified, linear and cyclic) and cell lines [Caco-2, Ralph Russ canine kidney (RRCK) and parallel artificial membrane permeability assay (PAMPA)]. The predictive models for linear and cyclic peptides in Caco-2 and RRCK cell lines were constructed for the first time, with an impressive coefficient of determination (R²) of 0.708, 0.484, 0.553, and 0.528 in the test set, respectively. Notably, the GNN framework behaved better in permeability prediction with larger data sets and improved the accuracy of cyclic peptide prediction in the PAMPA cell line. The R increased by about 0.32 compared with the reported models. Furthermore, the important molecular structural features that contribute to good permeability were interpreted; the influence of cell lines, peptide modification, and cyclization on permeability were successfully revealed. To facilitate broader use, we deployed these models on the user-friendly KNIME platform (https://github.com/ifyoungnet/PharmPapp). This work provides a rapid and reliable strategy for systematically assessing peptide permeability, aiding researchers in drug delivery optimization, peptide preselection during drug discovery, and potentially the design of targeted peptide-based materials.