Thermocells, capable of converting temperature-dependent electrochemical redox potentials into electrical power, can harvest waste or low-grade heat in an economical and continuous approach with zero carbon emission. However, the power density and conversion efficiency of thermocells are hindered by a narrow operation window and low ion conductivity of the electrodes, especially in freezing weather conditions. Herein, highly efficient cylindrical thermocells, working in a wide operation window of cold temperatures, are developed. A eutectic electrolyte consisting of formamide and water is formulated with a high ion conductivity, which is retained at a significantly extended lower limit of the operation window from conventional 0 to −40 °C. In parallel, an electrode material based on anisotropic holey graphene aerogel is synthesized with improved ion conductivity, especially at temperatures below 0 °C, due to its aligned graphene sheets and pores. By taking the advantages of both components, the power density and the Seebeck coefficient of a single-cylinder thermocell reaches an exceptionally high value, i.e., 3.6 W m and 1.3 mV K, respectively. Moreover, assembled thermocells in series packaging substantially enhance the voltage of the open-circuit, i.e., from 140 mV (1-cylinder thermocell) to 2.1 V (15-cylinder thermocells).