This paper proposes a simple but efficient selective filtering algorithm (SFA) for removing the impulse noise in images. Integrating the noise detector with the relationship between a pixel and its neighbors, the SFA is able to efficiently detect and remove noise pixels while well preserving information pixels. Experimental results and comparisons demonstrate that the proposed SFA outperforms several existing denoising methods with respect to the visual effects and quantitative measure results. © 2013 IEEE.