Mobile edge computing emerges to serve mobile users with low-latency computation offloading in edge networks, which are resource-constrained with massive users and workloads. However, existing communication and computing resource allocation schemes for offloaded tasks aren't efficient enough, where finished tasks still occupy resources, wasting constrained resources. Besides, the multi-user offloading is usually for scenarios of one task per user, ignoring real-world multi-task offloading scenarios where each user has multiple tasks, lack generality and flexibility. Meanwhile, local computing resource allocation schemes in multi-task scenarios ignore resource readjustment, causing low resource utilization. To solve these problems, we propose ECO-GAME, an efficient multi-task offloading scheme, which dynamically allocates bandwidth and computing resources to unfinished tasks, resulting in high resource utilization. We initially formulate the multi-task offloading problem as the game minimizing each user's cost, which is NP-hard. Thus we re-formulate the game utilizing potential games to optimize user's objective either locally or globally, and prove the existence of its Nash equilibrium. We then design an efficient multi-task offloading algorithm to obtain an approximate solution in polynomial time, together with computational complexity analysis. We further conduct performance evaluation on ECO-GAME utilizing price of anarchy. Numerical results demonstrate the efficiency of ECO-GAME, and show ECO-GAME reduces 49.2% cost over the state-of-the-art work, and scales well with the increasing number of tasks and users.