We propose, in this paper, a framework for time series and nonlinear system modeling, called the basis function matrix-based flexible coefficient autoregressive (BFM-FCAR) model. It has very flexible nonlinear structure. We show that many famous nonlinear time series models can be derived under this framework by choosing the proper basis function matrices. Some probabilistic properties (the conditions of geometrical ergodicity) of the BFM-FCAR model are investigated. Taking advantage of the model structure, we present an efficient parameter estimation algorithm for the proposed framework by using the variable projection method. Finally, we show how new models are generated from the proposed framework.