In this study, a waste heat recovery method based on the thermoelectric effect was proposed to improve the thermal performance of electric water heaters. The method collected the surplus thermal energy in the high-temperature zone and released heat for reuse in the low-temperature zone through the thermoelectric element. A heat-recovery device using a thermoelectric element was designed and conducted experimentally to verify the method's feasibility and energy conservation effect. The device performance of heat absorption and heat release were investigated under various working thermal states. The experimental results showed that the heating COP of the system could be improved to 1.2–2.3 under different temperature conditions, which means the device can save energy than using electric power directly. The heat flux through the device can be actively controlled for an optimal effect of heat recovery. The source-sink matching effect of the device was investigated, and the results indicated that the device's energy efficiency could be higher than 75%. The thermal performance of an improved electric water heater using the heat recovery device was investigated numerically under a specific working condition. The results verified that the heat recovery device reduced heat dissipation and simultaneously increased the temperature of the water outlet.