Data-driven classification approaches have substantially boosted the classification performance in steady-state visual evoked potentials (SSVEP)-based brain-computer interface (BCI). However, as a tradeoff to classification accuracy, a long calibration session is required to collect training data, which greatly reduces the applicability of BCI. In order to minimize the calibration effort while retaining good performance, this paper considers the problem of transferring knowledge from historical subjects to new subject, i.e., subject-independent SSVEP-based BCI. To tackle the problem, we propose a novel way to learn the transferable spatial filters by estimating the invariant task-related spatial filter subspace. The bases of the invariant subspace, which we call prototype spatial filters, are robust estimation of the task-related spatial filters. They can be generalized to the unseen subject for better recovering the latent signals. A new classification approach based on the prototype filters, namely transfer template and filter canonical correlation analysis (ttf-CCA), is then proposed and compared with the state-of-art approaches on the SSVEP benchmark data set. The feasibility of the proposed method is validated by the significant improvement on the classification accuracy and information transfer rate (ITR).