Heating, ventilation, and air conditioning (HVAC) systems in buildings have great potential to provide regulation capacity that is leveraged to maintain the balance of supply and demand in the power system. In order to make full use of HVAC’s regulation capacity, it is important to accurately evaluate it ahead of time. Because physical model-based approaches are hard to implement and highly personalized for each building, data-driven approaches are preferable for this capacity evaluation. However, given the insufficient data for individual buildings and buildings’ potential unwillingness to share their data because of privacy concerns, it is extremely challenging to build a highperformance data-driven regulation capacity evaluation model. In this paper, we propose a privacy-preserving framework that combines federated learning and transfer learning to evaluate the regulation capacity of HVAC systems in heterogeneous buildings. Specifically, a classified federated learning algorithm is proposed to build capacity evaluation models of HVAC systems for different building types. Each building trains its model locally without sharing data with other buildings to preserve privacy. The algorithm also tackles data insufficiency and achieves high evaluation accuracy. In addition, we design a cross-type transfer learning algorithm to enhance model generalization and further address data deficiency. A protocol is created for the above two algorithms to protect privacy and security. Finally, numerical case studies are conducted to validate the proposed framework.