A novel nonlinear sliding mode control approach dealing with the formation control of under-actuated ships is presented in this paper. To avoid the singularity problem, state space of the system is partitioned into two regions, with one region bounded for terminal sliding mode control and its complement singular for that. And a linear auxiliary sliding mode controller is designed for system trajectories starting from the complement region. With the application of nonlinear sliding mode control approach and finite-time stability theory, a distributed controller is designed for individual under-actuated ship to achieve the given formation pattern within a finite time. Finally, two simulation examples are provided to verify the effectiveness and performance of the proposed approach.