Wind power interval prediction is an effective technique for quantifying forecasting uncertainty caused by the intermittent and fluctuant characteristics of wind energy. Valid coverage and short interval length are the two most critical targets in interval prediction to attain reliable and accurate information, providing effective support for decision-makers to better control the risks in the power planning. This paper proposes a novel interval prediction approach named conformalized temporal convolutional quantile regression networks (CTCQRN) which combines the conformalized quantile regression (CQR) algorithm with a temporal convolutional network (TCN), without making any distributional assumptions. The proposed model inherits the advantages of quantile regression and conformal prediction that is fully adaptive to heteroscedasticity implicated in data, and meets the theoretical guarantee of valid coverage. As opposed to conventional RNN-based approaches, the adopted TCN architecture frees from suffering iterative propagation and gradient vanishing/explosion, and can handle very long sequences in a parallel manner. Case studies on two different geographical wind power datasets show that the proposed model has a distinct edge over benchmark models in goals of valid coverage and narrow interval bandwidth, which can help to ensure the economic and secure operation of the electric power system.