It is significant to evaluate the air quality scientifically for the management of air pollution. As an air quality comprehensive evaluation problem, its uncertainty can be effectively addressed by the Dempster–Shafer (D–S) evidence theory. However, there is not enough research on air quality comprehensive assessment using D–S theory. Aiming at the counterintuitive fusion results of the D–S combination rule in the field of comprehensive decision, an improved evidence theory with evidence weight and evidence decision credibility (here namely DCre-Weight method) is proposed, and it is used to comprehensively evaluate air quality. First, this method determines the weights of evidence by the entropy weight method and introduces the decision credibility by calculating the dispersion of different evidence decisions. An algorithm case shows that the credibility of fusion results is improved and the uncertainty is well expressed. It can make reasonable fusion results and solve the problems of D–S. Then, the air quality evaluation model based on improved evidence theory (here namely the DCreWeight model) is proposed. Finally, according to the hourly air pollution data in Xi’an from June 1, 2014, to May 1, 2016, comparisons are made with the D–S, other improved methods of evidence theory, and a recent fuzzy synthetic evaluation method to validate the effectiveness of the model. Under the national AQCI standard, the MAE and RMSE of the DCreWeight model are 1.02 and 1.17. Under the national AQI standard, the DCreWeight model has the minimal MAE, RMSE, and maximal index of agreement, which validated the superiority of the DCreWeight model. Therefore, the DCreWeight model can comprehensively evaluate air quality. It can provide a scientific basis for relevant departments to prevent and control air pollution.