Recently many researchers invented a wide variety of meta-heuristic optimization algorithms. Most of them achieved remarkable performance results by infusing the natural phenomena or biological behaviors into the search logics of the optimization algorithms, such as PSO, Cuckoo Search and so on. Although these algorithms have promising performance, there still exist a drawbackit is hard to find a perfect balance between the global exploration and local exploitation from the traditional swarm optimization algorithms. Like an either-or problem, algorithms that have better global exploration capability come with worse local exploitation capability, and vice versa. In order to address this problem, in this paper, we propose a novel Dynamic Group Search Algorithm (DGSA) with enhanced intra-group and inter-group communication mechanisms. In particular, we devise a formless group concept, where the vectors of solutions can move to different groups dynamically based on the group best solution fitness, the better group has the more vectors. Vectors inside a group mainly focus on the local exploitation for enhancing its local search. In contrast, inter-group communication assures strong capability of global exploration. In order to avoid being stuck at local optima, we introduce two types of crossover operators and an inter-group mutation. Experiments using benchmarking test functions for comparing with other well-known optimization algorithms are reported. DGSA outperforms other algorithms in most cases. The DGSA is also applied to solve welded beam design problem. The promising results on this real world problem show the applicability of DGSA for solving an engineering design problem.