EEG-based Emotion recognition is a crucial link in Human-Computer Interaction (HCI) application. Nowadays, Convolutional Neural Network (CNN) and its related CNN-hybrid approaches have achieved the state-of-art accuracy in this field. However, most of these existing techniques employ large-scale neural networks which cause performance bottleneck in portable systems. Moreover, traditional convolution kernel confuses EEG multiple frequency bands information, which is critical for investigating emotion status. To improve these issues, firstly, we extract power spectral features from four frequency bands (θ,α,β,γ) and transform obtained features into cortex-like frames while preserving spatial information of electrodes position, so that the multi-channel, multi-frequency bands and time series EEG signals can be efficiently represented. Then, we design a shallow depthwise parallel CNN inspired by Mobilenet technique to learn spatial representation from labeled frames. Segment-level emotion recognition experiments are implemented to verify the proposed architecture with DEAP database. Our approach achieves the competitive accuracy of 84.07% and 82.95% on arousal and valence respectively. Besides, the experimental results prove the computation-effectiveness of the proposed method. Compared with the state-of-art approach, our approach saves 69.23% GPU memory and reduces 30% GPU peak utilization with only 6.5% accuracy drop. Therefore, our method shows extensive application prospects for EEG-based emotion recognition on resource-limited devices.