Despite significant progress has been made to facilitate learning in CNNs, such as via pre-training, the AFs used in CNNs and related architectures have remained almost unaltered. The continued reliance on non-optimal AFs has impacted the performance of CNNs for medical diagnostic applications. This work is focused on DL-based image classification to aid medical diagnosis of COVID-19 and Parkinson’s Disease (PD).