Bayesian deep learning (BDL) provides a principled probabilistic foundation for uncertainty quantification and robust decision-making in neural networks. However, applying BDL in real-world distributed and dynamic environments remains challenging due to client heterogeneity, computational scalability, data sparsity, and temporal dependencies. These challenges call for unified frameworks that integrate Bayesian inference with scalable and flexible modeling across both federated and temporal settings.

This thesis advances Bayesian deep learning through four complementary frameworks that address these challenges. For federated modeling, Bayesian Personalized Federated Learning (BPFed) introduces a hierarchical Bayesian framework to jointly capture shared and personalized uncertainties, while Personalized Federated Learning with Subnetwork Inference (FedSI) proposes a subnetwork-based Bayesian inference strategy that achieves scalable and efficient uncertainty quantification across heterogeneous clients. For temporal modeling, Federated Neural Nonparametric Point Process (FedPP) formulates a privacy-preserving federated learning framework for sparse event data through divergence-based aggregation, and Marked Temporal Bayesian Flow Point Process (BMTPP) establishes a probabilistic flow-based model that captures dynamic dependencies between timestamps and marks.

Comprehensive theoretical analysis and extensive experiments on benchmark datasets demonstrate that the proposed frameworks consistently improve uncertainty quantification, robustness, and scalability over existing approaches. Collectively, these contributions form an integrated Bayesian deep learning paradigm that unifies federated and temporal modeling under uncertainty, providing both theoretical insights and practical guidance for scalable probabilistic intelligence.