In this paper, a novel telescopic broad Bayesian learning (TBBL) is proposed for sequential learning. Conventional broad learning suffers from the singularity problem induced by the complexity explosion as data are accumulated. The proposed TBBL successfully overcomes the challenging issue and is feasible for sequential learning with big data streams. The learning network of TBBL is reconfigurable to adopt network augmentation and condensation. As time evolves, the learning network is augmented to incorporate the newly available data and additional network components. Meanwhile, the learning network is condensed to eliminate the network connections and components with insignificant contributions. Moreover, as a benefit of Bayesian inference, the uncertainty of the estimates can be quantified. To demonstrate the efficacy of the proposed TBBL, the performance on highly nonstationary piecewise time series and complex multivariate time series with 100 million data points are presented. Furthermore, an application for long-term structural health monitoring is presented.