This paper proposes a new damage detection method by integrating the advantage of transmissibility function (TF) as a health index sensitive to damage but robust to excitation and agglomerative hierarchical clustering (AHC) with intuitive explanation and visualization but avoiding specifying the number of clusters. Different from conventional AHC-based damage detection methods utilizing deterministic distance as a similarity metric and ignoring the distribution of structural features, a multivariate probabilistic distance-based similarity metric is proposed in this study to account for the uncertainty and correlation of multiple TFs following multivariate complex-valued Gaussian ratio distribution. To realize this, an analytically tractable approximation of the multivariate probabilistic distance is derived by Laplace’s asymptotic expansion to avoid high-dimensional numerical integration. To accelerate the computation of probabilistic distances over a wide frequency band that are fused to formulate the similarity metric in AHC, a function vectorization scheme is proposed to avoid the time-consuming loop operation among different frequency points. A threshold is established via bootstrapped Monte Carlo simulation to cut the dendrogram produced by AHC. Two case studies are used to validate the performance of the proposed method, indicating that, compared to the damage detection methods based on the deterministic distance of the TF, the proposed method exhibits better performance due to improving the similarity metric based on multivariate probabilistic distance properly accommodating the correlation of different TFs.