Encrypted traffic identification is important to maintain the cybersecurity. Recently, as the SSL/TLS encryption protocols are widely used in modern Internet environment, how to identify the encrypted traffic become a big challenge. The traditional payload-based methods are usually used to identify the unencrypted traffic, but is no longer effective for the encrypted traffic. To solve the enrypted traffic identification problem, researchers tried to use machine learning methods to model the flow features of encrypted traffics and have made some progress. However the identification accuracy is still not high as these methods usually use the high-level hand-designed features which may loss a lot of important information. To overcome this limitation, in this paper, we design PEAN - a Packet-level End-to-end Attentive Network for encrypted traffic identification. PEAN uses the information such as raw bytes and length sequence as the model input rather than using the traditional hand-designed features. Then, we use an unsupervised network traffic pre-training model to better model the traffic bytes. A self-attention mechanism is also designed to better learn the deep relationship among traffic packets. Experiments on a real trace set demonstrate the effectiveness of PEAN.