To address the scarcity of spectrum resource incurred by explosively increasing traffic load in existing fifth generation (5G) networks, we consider exploiting Reconfigurable intelligent surface (RIS) to harmonize the coexistence of cellular networks and WiFi system on the unlicensed bands. In particular, RIS is capable of not only improving the performance of new radio on unlicensed bands (NR-U) by increasing the signal to interference and noise ratio (SINR) of cellular users, but also decreasing the interference to the existing WiFi system caused by NR-U. Thus, we investigate the performance of RIS aided coexisting networks both in single cell and multiple cell scenarios. For the single cell scenario, we aim to improve the worst transmission rate of UEs in NR-U network. The problem is solved by a proposed alternating optimization-based solution. For the multiple cell scenario, cell cooperation is exploited jointly with multiple RISs to maximize the sum-rate of all UEs. Although the problem is more challenging than that of single cell case, we propose a multi-agent reinforcement learning (MARL) algorithm to solve it. Numeral results verify the effectiveness of our proposed schemes and show that deploying RISs is a promising solution for the enhancement of NR-U networks.