State-of-the-art unsupervised object re-identification (Re-ID) methods conduct model training with pseudo labels generated by clustering techniques. Unfortunately, due to the existence of inter-ID similarity and intra-ID variance problems in vehicle Re-ID, clustering sometimes mixes different similar vehicles together or splits images of the same vehicle in different views into different clusters. To enhance the model's ID discrimination capability in the presence of such kinds of label noise, we propose an inter-intra cluster reorganization approach (ICR) to reorganize the relationship between instances within and between clusters, which can provide higher-quality contrastive learning guidance based on existing clustering results. In the intra-cluster reorganization, we design a camera-aware maximum reliability sub-cluster organization approach, which reorganizes each cluster into several intersecting sub-clusters of higher quality based on the finer intra-camera clustering results. We further design a novel metric called centroid reliability to measure the reliability of intra-cluster contrastive learning. In the inter-cluster reorganization, we propose an ambiguous cluster discrimination criterion to measure the probability that two clusters belong to the same vehicle. Based on this criterion, we design a focal contrastive loss to adaptively re-organize the contribution of ambiguous clusters in model training to perform better contrastive learning. Extensive experiments on VeRi-776 and VERI-Wild demonstrate that ICR is effective and can achieve state-of-the-art performance.