Developing an easily-prepared, sensitive, and accurate point-of-need immunochromatographic assay (ICA) is significant in food safety screening, clinical diagnosis, and environmental monitoring. However, the current single-modal ICAs are limited in certain instinct drawbacks that restrict analytical performances. Herein, we introduce an ultrasensitive dual-modal colorimetric/reversed ratiometric fluorescence ICA based on facilely prepared immunoprobes with a high loading capacity of red quantum dots and AuNPs. By smartly integrating these red-colored/fluorescent signal probes with an immobilized green quantum dot antigen on the test lines, discrete "turn-on" visual inspection and reversed ratiometric quantification via a portable smartphone-based analyzer were accomplished. As an application, this method was employed to detect 11 phosphodiesterase-5 inhibitors in health foods with ultralow detection limits (0.0028-0.045 ng/mL), high repeatability (coefficient of variations of 0.3-1.91%), and reasonable accuracy (recoveries of 86.6-107%). The proposed method was further validated by the authorized liquid chromatography with tandem mass spectrometry method in actual sample detection. This new assay format can be extended to ultrasensitive flexible detection of other food contaminants, environmental pollutants, or tumor biomarkers within minutes, and it just requires simply prepared signal reporters, easy-to-operate procedures, and a low-cost miniaturized analyzer.