In this work, we report a comprehensive study on a magnesium phosphate cement (MPC) based composite as the construction material for high performance bipolar plates of fuel cells. MPC with partial replacement of fly ash was employed as the binding matrix. Some carbon-based materials, such as graphite, carbon black, carbon fiber, and multi-walled carbon nanotubes were used to construct the conductive phase. A simple hot-press process was applied to produce the composite. The formula and the structure of the composite was modified and adjusted to optimize the properties of the composite to meet the US DOE 2015 technical targets, including the introducing of a reinforcement support. Finally, all the technical targets such as electrical conductivity (>100 S cm), the flexural strength (>25 MPa), the corrosion resistance (<1 μA cm), and gas permeability (<10 cm (s cm)) were achieved as well as low cost (<5 $ per kW). The optimized formula and the detailed procedures to fabricate the MPC based composite were concluded.