This paper addresses the obstacle avoidance problem of formation control for the multi-agent systems modeled by double integrator dynamics under a directed interconnection topology. The control task is finished by a leader-follower formation scheme combined with an artificial potential field (APF) method. The leader-follower scheme is carried out by taking the desired trajectory with the desired velocity as virtual leader, while the APF method is carried out by dealing with the obstacles as the high potential points. When the obstacle avoidance tasks are finished, the artificial potential forces degrade the formation performance, so their undesired effects are treated as disturbances, which is analyzed by the robust H performance. Based on Lyapunov stability theory, it is proved that the proposed formation approach can realize the control objective. The result is also extended to the switching multi-agent formation. The effectiveness of the proposed formation scheme is further confirmed by simulation studies.