Non-orthogonal multiple access (NOMA)-aided mobile edge computing (MEC) system can enhance the spectral-efficiency with massive tasks offloading. However, with more dynamic devices and the uncontrollable stochastic channel environment, it is even desirable to deploy appealing technique, i.e., intelligent reflecting surfaces (IRS), in the MEC system to flexibly adjust the communication environment and improve the system energy-efficiency. In this paper, we investigate the joint offloading, communication and computation resource allocation for IRS-assisted NOMA-aided MEC system. We firstly formulate a mixed integer energy-efficiency maximization problem with the system queue stability constraint. We then propose a Het-erogeneous Multi-agent Lyapunov-function-based Mixed Integer Deep Deterministic Policy Gradient (HMA-LMIDDPG) algorithm which is based on the multi-agent reinforcement learning (MARL) framework with homogeneous edge devices (EDs) and heterogeneous base station (BS) as heterogeneous multi-agent. Numerical results show that our proposed algorithms can achieve superior energy-efficiency performance to the benchmark algorithms while maintaining the queue stability.