Smart meter data holds tremendous application potential in improving the efficiency and stability of power systems. However, existing studies usually propose a specific data-driven method, which is only suitable for a single application and not others. This will aggravate the cost and difficulty of smart meter data applications, which limits the full data exploitation. In this paper, we propose a general method based on the pre-training model for multiple smart meter data applications. Specifically, we develop a pre-training task for the pre-training model to learn the generic knowledge of smart meter data in an unsupervised learning way. Furthermore, we design a novel pre-training model based on bidirectional Transformer, which can efficiently and effectively extract the temporal dependencies of load data. In this way, after pre-training via the developed task, the designed model can be used for different smart meter data applications by fine-tuning. Case studies based on public datasets validate the effectiveness of the proposed method.