We present FedSM, a Federated Spectrum Management architecture to increase channel utilization (CU) and reduce latency, while protecting users' data privacy. We employ hedonic coalition formation game for spectrum allocation. Within each coalition, we design a bandit learning algorithm to share spectra and adjust resource usage. Preliminary simulation results show FedSM increases CU to 93.51% and reduces latency to 248.68 ms compared to three privacy-preserving dynamic spectrum management architectures.