How can a target’s differential image of multiple components at a certain degree of freedom (DOF) of light (e.g., wavelength, polarization, and position) be obtained? Previous schemes often need two steps, i.e., first collecting each component of the target simultaneously with multiple detectors or sequentially with a single detector, then performing the difference operation in the digital postprocessing. Based on the principle of computational ghost imaging (GI), we here take advantage of the natural complementarity in space to design an illumination mode, which is no longer the spatiotemporal fluctuation of photon number (light intensity) but the photons with different components under the same DOF. By applying this engineered illumination to computational GI, termed ghost difference imaging (GDI), we can achieve differential imaging of multiple components of a certain DOF in a single-round acquisition using only one single-pixel detector with no extra digital-subtraction operations but higher SNR. Both multiwavelength-difference GDI and position-difference GDI are well demonstrated in simulations and experiments. Further, the GDI scheme reduces the number of sampling times for differential imaging while also increasing photons’ utilization. Our work, therefore, provides an insight for GI modality, along with a straightforward and low-cost update of the light path, which is suitable for all computational GI systems using the digital micromirror device.