While our recent deep fuzzy classifier DSA-FC, which stacks adversarial interpretable Takagi-Sugeno-Kang (TSK) fuzzy sub-classifiers, shares its promising classification, its training speed will become very slow and even intolerable for large-scale datasets, due to successive training on all training samples with their random gradient based updates along each layer of its stacked structure. In order to circumvent this bottleneck issue, a fast training algorithm FTA is developed in this study by downsizing fuzzy rules with the proposed gradient guided learning for each sub-classifier at each layer of DSA-FC on large-scale datasets. The core of FTA is to assure fast training of each sub-classifier at each layer of DSA-FC, which first generates first-order smooth gradient guided information by means of the proposed top-k fuzzy rules selected from all fuzzy rules in each sub-classifier, and then quickly updates the current inputs in terms of such information, which will be taken as the inputs of the sub-classifier at the next layer. Our theoretical analysis reveals that the proposed gradient guided learning indeed enhances the generalization capability of a deep fuzzy classifier with or without adversarial attacks on outputs. Experimental results on large datasets demonstrate that FTA indeed trains the deep fuzzy classifier DSA-FC quickly with enhanced generalization capability.