Reproducibility is fundamental to scientific research, as it underpins trust, progress, and credibility. In machine learning (ML), achieving reproducibility is difficult due to variability in algorithms, implementations, and observational factors. This presentation explores key contributors to irreproducibility in ML, including algorithmic factors like hyperparameter tuning and random weight initialization, implementation differences in software and hardware, and observational factors such as dataset bias and data preprocessing. It emphasizes the need to view ML model performance as a distribution, not a single metric or average of results, and clarifies the difference between reproducibility and portability. The goal is to guide researchers on improving ML reproducibility and identifying the critical information necessary for replicating experimental outcomes.

Kevin Coakley is a Computational and Data Science Research Specialist at the San Diego Supercomputer Center and UC San Diego focusing on AI reproducibility. Kevin holds a MAS in Architecture-based Enterprise Systems Engineering and Leadership from UC San Diego and is pursuing a PhD in Computer Science at the Norwegian University of Science and Technology. Kevin specializes in training and evaluating machine learning models for accuracy and reproducibility in applications like image recognition, time series prediction, and natural language processing.

Sponsored by iHARP, the NSF HDR Institute for Harnessing Data & Model Revolution in the Polar Regions

Several decades of research have led to current advances in artificial intelligence (AI), specifically machine learning (ML) and deep learning (DL). These advancements hold promise for solving major challenges facing human society – from mitigating climate change to increasing food production, designing smart cities, and optimizing scarce resources. All these problems share a common thread: they are inherently rooted in space and time. Remote sensing data serves as a prime example of spatial big data. NASA recently collected its 10 millionth Landsat image. The coarse-resolution (30 m) Landsat collection itself surpasses a petabyte in size, while private satellite data producer MAXAR holds more than 125 petabytes of high-resolution data. Applications such as disease mapping, crop monitoring, and urban studies all rely on this data. We present recent advances in GeoAI that analyze these multimodal datasets and show their applications in various fields, including climate-smart agriculture, slum mapping, and critical infrastructure monitoring.

Dr. Raju Vatsavai is a Chancellor's Faculty Excellence Program Cluster Professor of Geospatial Analytics in the Department of Computer Science at North Carolina State University (NCSU). Prior to joining NCSU, Raju served as the Lead Data Scientist for the Computational Sciences and Engineering Division (CSED) at the Oak Ridge National Laboratory (ORNL). His research focuses on the intersection of spatial and temporal big data management, machine learning, and high-performance computing.  He has authored or co-authored over 100 peer-reviewed articles in conferences and journals. He has also edited two books on "Knowledge Discovery from Sensor Data." He actively participates in the academic community, serving on program committees for leading international conferences such as ACM KDD, ACM SIGSPATIAL GIS, ECML/PKDD, SDM, CIKM, and IEEE BigData. He has further co-chaired several workshops, including ICDM/SSTDM, ICDM/KDCloud, ACM SIGSPATIAL BigSpatial, ACM/IEEE Supercomputing/BDAC, ACM KDD/LDMTA, ACM KDD/Sensor-KDD, and SIAM DM/ACS. Dr. Raju holds a M.S. and Ph.D. degrees in computer science from the University of Minnesota.