Enrichment of indigenous microbial consortia from contaminated environments is very crucial for developing bioremediation strategies. In this study, xenobiotic degrading consortia from seven diverse contaminated soils or sediments were enriched under aerobic conditions for over a year using different xenobiotics, to develop robust consortia for xenobiotic bioremediation. The 16S rRNA gene amplicon sequencing revealed that enriched prokaryotic consortia were predominated by Proteobacteria (84.7%), whereas the xenobiotic degraders varied primarily dependent on contaminants, accounting for 91.1% and 79.4% of variance at the phylum and genus level, respectively, with Pseudomonas spp. (Gammaproteobacteria) and Paracoccus spp. (Alphaproteobacteria) dominating treatments in the presence and absence of BTEX, respectively. The origin of the microbial source also significantly affects degraders community composition but only explained a small proportion of the variance (4.7% phylum and 7.0% genus). Significant BTEX biodegradation occurred in 7 days of incubation with the enriched indigenous consortia, and the results suggest that Pseudomonas spp. are the major aerobic BTEX degraders. This conclusion is supported by the metabolic potential of BTEX for those isolated Pseudomonas strains in the literature. Taken together, these results provide evidence about the primary influence of contaminants on the selection of xenobiotic degrading prokaryotic communities and highlight a promising strategy through bioaugmentation for the cleanup of xenobiotic contaminated sites.