In this study, we propose a novel structural condition assessment method based on adaptive vision feature extractions and reinforced learning-assisted evolution. First, the ‘features from accelerated segment test’ (FAST) algorithm cooperating with the Kanade-Lucas-Tomasi algorithm synergistically captures the displacements from the video clips. However, the fixed threshold values in the FAST algorithm may not satisfy the pixel requirements for different images. Second, for any evolutionary algorithms (EAs), their search modes significantly affect the optimization performance but are relatively single and monotonous. Therefore, they may perform poorly for some high-dimensional and complicated multi-objective functions. To resolve these two critical problems, firstly, we propose an adaptive feature points extraction strategy during the displacements acquisition stage. Secondly, a novel local search framework subjected to the reinforced learning framework is designed for EAs as an improvement. The proposed structural condition assessment method is used to evaluate a space frame structure by optimizing the vibration data-based multi-sample objective function. The damaged locations and severities of the frame can be well identified.