Aspect-based sentiment analysis aims to predict sentiment polarities for given aspect terms in a sentence. Previous work typically encodes the aspect and the sentence separately, with either RNNs or CNNs along with sophisticated attention mechanisms. However, CNNs and RNNs suffer from problems such as restricted local receptive field and long-term dependency, respectively. Besides, separately encoding aspects and sentences also results in problems such as the aspect has no context information and neighboring aspects are not considered. To address these problems, we propose a novel approach that conducts an extract-attend-predict process with deep self-attention for aspect-based sentiment analysis. Unlike previous methods that use either RNNs or CNNs as the basic encoder, we utilizes a pre-trained deep self-attention encoder to avoid the difficulty in capturing long-distance words. Moreover, instead of performing separately encoding, our model directly extracts the aspect representation from contextualized sentence representations based on the span boundary of target aspect. A multi-granularity attending mechanism is further applied to capture the interaction between aspects and sentences, which is later used to predict the sentiment polarity. We conduct experiments on two benchmark datasets and the results show that our approach outperforms previous state-of-the-art models.