The electric vehicle (EV) market has witnessed a continuous and steady increase in the past few years. The benefits of lower energy costs and less greenhouse gas (GHG) emission have been widely recognized by customers. In addition to these benefits for the transportation sector, EVs are also considered a critical supplementary resource for building a sustainable energy system in a smart grid environment. The applications of EVs in a smart grid have attracted wide attention in recent years. One appealing application is to use aggregated EVs as either energy sources or sinks to provide a service of frequency regulation for the request signals from the grid. A real-time decision-making model is proposed in this paper for the EV aggregator to dynamically control the energy flow between the grid and each individual EV in the aggregated group as an effective response to the signals of frequency regulation issued by the grid using Markov Decision Process. The aggregator's benefit is maximized through the identification of a set of optimal charging/discharging decisions for the aggregated EVs. A semi-online solution strategy is also proposed to find the near-optimal decisions on a real-time basis. A numerical case study is used to illustrate the effectiveness of the proposed model.