Reconfigurable intelligent surface (RIS) has been anticipated to be a novel cost-effective technology to improve the performance of future wireless systems. In this paper, we investigate a practical RIS-aided multiple-input-multiple-output (MIMO) system in the presence of transceiver hardware impairments, RIS phase noise and imperfect channel state information (CSI). Joint design of the MIMO transceiver and RIS reflection matrix to minimize the total average mean-square-error (MSE) of all data streams is particularly considered. This joint design problem is non-convex and challenging to solve due to the newly considered practical imperfections. To tackle the issue, we first analyze the total average MSE by incorporating the impacts of the above system imperfections. Then, in order to handle the tightly coupled optimization variables and non-convex NP-hard constraints, an efficient iterative algorithm based on alternating optimization (AO) framework is proposed with guaranteed convergence, where each subproblem admits a closed-form optimal solution by leveraging the majorization-minimization (MM) technique. Moreover, via exploiting the special structure of the unit-modulus constraints, we propose a modified Riemannian gradient ascent (RGA) algorithm for the discrete RIS phase shift optimization. Furthermore, the optimality of the proposed algorithm is validated under line-of-sight (LoS) channel conditions, and the irreducible MSE floor effect induced by imperfections of both hardware and CSI is also revealed in the high signal-to-noise ratio (SNR) regime. Numerical results show the superior MSE performance of our proposed algorithm over the adopted benchmark schemes, and demonstrate that increasing the number of RIS elements is not always beneficial under the above system imperfections.