Deep learning has transformed medical image analysis. However, building effective models for volumetric data, such as CT, MRI, and PET scans, presents a new set of challenges. These include high computational costs, limited annotated datasets, and the need for architectures that can process volumetric data directly. This survey provides a comprehensive overview of recent advances in deep neural network architectures specifically designed for 3D medical imaging. We analyze the progression from early 3D CNNs to hybrid and Transformer-based models, highlighting their structural innovations and applicability. In addition, we review training strategies such as supervised pretraining, self-supervised learning, and the development of general-purpose 3D foundational models. Also, a comprehensive overview of 3D medical imaging datasets and their associated tasks is presented in the supplementary materials, highlighting the diverse clinical objectives that deep learning models can support across healthcare and diagnostic applications. A dedicated section addresses the emerging field of explainability in volumetric contexts, emphasizing the limitations of adapting 2D explainability-focused tools and the importance of 3D-native explanation frameworks. We conclude by outlining the key trends in the field, including the growing availability of multiple and functionally diverse datasets; novel architectural designs tailored for volumetric learning; the transition from convolutional to attention-based models; increased reliance on transfer learning from both labeled and unlabeled data; and the emergence of 3D-native XAI methods that provide clinically meaningful and trustworthy explanations.