Music genres are useful for indexing, organizing, searching, and recommending songs and albums. Therefore, the automatic classification of music genres is an essential part of almost all kinds of music applications. Recent works focus on exploiting text, audio, or multi-modal information for genre classification, without considering the influence of the artists' and listeners' preference. However, intuitively, artists have their composing preferences, and listeners also have their music tastes. Both of them provide helpful hints to the music genre from different views, which are crucial to improve classification performance. In this paper, we make use of both artist-music and listener-music preference relations to construct a heterogeneous preference graph. Then, we propose a novel graph-based neural network to automatically encode the global preference relations of the heterogeneous graph into artist and listener representations. We construct a graph to capture the correlations among genres and apply a graph convolutional network to learn genre representation from the correlation graph. Finally, we combine artist, listener, and genre representations for multi-label genre classification. Experimental results show that our model significantly outperforms the state-of-the-art methods on two public music genre classification datasets.