Smart factory is a new digital and intelligent platform requiring high throughput and ultra-reliable low-latency communications (URLLC). Industrial communications at sub-6 GHz faces spectrum congestion and bandwidth limitations, which seriously jeopardize the high data rate requirement of smart factory. Recently, millimeter wave (mmWave) and Terahertz technologies have become enablers for high speed communications and intelligent manufacturing in Industry 4.0 and beyond. However, the sensitivity of mmWave signals to blockage and the overhead of large-scale antenna beam sweeping pose serious challenges to the reliability and the latency of wireless networks in these frequency ranges. In this paper, we propose a vision-aided URLLC framework for smart factory that does not incur any overhead from channel training and beam sweeping. In particular, we design a feature extraction method to obtain communications-related features in environmental images for blockage prediction, reference signal receiving power (RSRP) prediction, and beam selection. Then, we construct a joint image-channel dataset covering images, annotations, blockage, and wireless channels based on Blender and Wireless Insite software. Simulations show that the accuracy of blockage prediction 400 ms ahead reaches 99.9%, the root mean square error (RMSE) of RSRP prediction 400 ms ahead reaches 2.78 dB, and the Top-5 accuracy of beam selection reaches 91.8%. Blockage and RSRP prediction can assist base station (BS) handover to avoid communications interruption, while beam selection can eliminate the overhead of channel training and beam sweeping. Hence, the proposed study provides a promising direction for enabling URLLC under mmWave and even Terahertz bands in smart factory of Industry 4.0.