We investigate the multiple-input multiple-output (MIMO) transceiver design using a weighted mean-square-error (MSE) minimization approach. A novel weighted MSE model is proposed, and it is defined as a linear matrix function with respect to the traditional data detection MSE matrix. The new model can be interpreted an extension of the weighting operation from vector field to matrix field. Based on the proposed weighting operation, a general transceiver design is proposed, which aims at minimizing an increasing matrix-monotone function of the output of the previous linear matrix function. The structure of the optimal solutions is also derived. Furthermore, two important special cases of the matrix-monotone functions are discussed in detail. It is revealed that these two problems are equivalent to the transceiver design of sum MSE minimization and capacity maximization for dual-hop amplify-and-forward (AF) MIMO relaying systems, respectively. Finally, it is concluded that the AF relaying can be interpreted as a specific application of the proposed weighting operation. © 2013 IEEE.