This paper considers the leader-following consensus problem of multi-agent systems, where agents are described by high-order dynamics under a directed graph. An algorithm depending on a distributed control protocol with time delay and intermittent sampled position information only is proposed. The protocol makes use of past position data. We show that such protocol cannot work if not enough past position information is taken into account. For nth-order system, we take n past position data. To achieve consensus, a necessary and sufficient condition depending on communication width, time delay, and sampling period is derived. Furthermore, the control protocol can be extended to leaderless consensus problem. Numerical simulations illustrate that appropriately chosen control gains can verify the theoretical analysis.