We consider difference schemes for nonlinear time fractional Klein-Gordon type equations in this paper. A linearized scheme is proposed to solve the problem. As a result, iterative method need not be employed. One of the main difficulties for the analysis is that certain weight averages of the approximated solutions are considered in the discretization and standard energy estimates cannot be applied directly. By introducing a new grid function, which further approximates the solution, and using ideas in some recent studies, we show that the method converges with second-order accuracy in time.