This paper proposes a framework combing a fully convolutional network with broad learning system for license plate recognition. The fully convolutional network, which is designed as a pixel-level two-class classification method, is proposed for random-positioned object detection by the fusion of multi-scale and hierarchical features. For character segmentation, a trained AdaBoost cascade classifier is employed to locate a key character representing for an administrative area. We design a symmetric region horizontal projection method to estimate the license plate slant angles, and an approach based on vertical projection without hyphens to solve the problem of touching characters. For character recognition, the broad learning system with stacked auto-encoder of mapped feature nodes is proposed, and two structures are explored to recognize letters and digits, respectively. Experiments conducted on Macau license plates show that the proposed method outperforms some state-of-the-art approaches. The compatibility and generality can be expected by applying the proposed method to other regions or countries.