An advanced procedure is introduced in this study for determining the transient probability density function (PDF) of the stochastic oscillator with various nonlinearity under combined harmonic and modulated random stimulations, which is an enhancement of the exponential-polynomial-closure (EPC) methodology. An evolutionary exponential polynomial function with time-dependent undefined parameters is selected to represent the transient probabilistic solution. A bunch of ordinary differential equations can be formulated by integrating the weighted residual error where the weight functions are specially selected as a number of independent evolutionary base functions that span the Rn space. The undefined parameters can be fixed by utilizing numerical approaches to solve those ordinary differential equations. By comparing the results with those acquired by Monte Carlo simulation (MCS), four numerical cases demonstrate that the transient PDF solutions of the nonlinear oscillators can be acquired effectively and efficiently, especially in the probabilistic tails which play a key role in reliability analysis. In addition, the results indicate that the transient responses of the stochastic oscillators are of nonzero means due to the influence of harmonic stimulation. Meanwhile, the transient PDFs are also asymmetrical at the nonzero means resulted by the coupled impacts of the combined harmonic and modulated random stimulations. Moreover, utilizing the advanced procedure can lead to a significant decrease in computational effort without sacrificing the solution accuracy in contrast to MCS.