Clustering divides objects into groups based on similarity. However, traditional clustering approaches are plagued by their difficulty in dealing with data with complex structure and high dimensionality, as well as their inability in solving multi-objective data clustering problems. To address these issues, an evolutionary state-based novel multi-objective periodic bacterial foraging optimization algorithm (ES-NMPBFO) is proposed in this article. The algorithm is designed to alleviate the high-computing complexity of the standard bacterial foraging optimization (BFO) algorithm by introducing periodic BFO. Moreover, two learning strategies, global best individual (gbest) and personal historical best individual (pbest), are used in the chemotaxis operation to enhance the convergence speed and guide the bacteria to the optimum position. Two elimination-dispersal operations are also proposed to prevent falling into local optima and improve the diversity of solutions. The proposed algorithm is compared with five other algorithms on six validity indexes in two data clustering cases comprising nine general benchmark datasets and four credit risk assessment datasets. The experimental results suggest that the proposed algorithm significantly outperforms the competing approaches. To further examine the effectiveness of the proposed strategies, two variants of ES-NMPBFO were designed, and all three forms of ES-NMPBFO were tested. The experimental results show that all of the proposed strategies are conducive to the improvement of solution quality, diversity and convergence.