Focus 2: Use-Inspired Research in Environmental Science

Progress:

Year 4

• Real-time U-net tornado prediction model tested in real-time using the WoFS ensemble alongside the 2023 HWT spring forecasting experiment; evaluation of results is
  in progress.
• Two papers in preparation, submission of both papers (Shotande et al., 2024 and Gagne et al., 2024) expected during summer of 2024.
• PhD Student Amanda Murphy published two manuscripts, one summarizing radar dataset
  creation for AI/ML efforts and the other summarizing diagnosed discriminatory
  differences between tornadic and nontornadic storm cells within mesoscale convective
  systems (MCSs). 
• Finalized a manuscript that was submitted, revised, and is now in press that details
  attributes that forecasters deem as essential and useful for familiarizing themself with
  new guidance and using it operationally for forecasting. 

Year 3

• Analysis of AI system provides insights into potential future improvements to WoFS. Paper in preparation (to be submitted June 2023).
• Partners at OU, CMU, NSSL, and Vaisala are collaborating to develop a AI/ML nowcasting hail product. 
• Developed cloud-based real-time tornado U-Net, tested in real-time using the WoFS
  ensemble alongside the 2023 HWT spring forecasting experiment. 
• Presented 3 presentations at AMS, published 1 review paper, 2 more papers in
  preparation.
• Completed analysis of forecaster interviews to evaluate perceptions of and attitudes towards new-to-them AI/ML-derived guidance and its trustworthiness. Draft of the full manuscript has been reviewed by the authorship team and will be submitted to Weather and Forecasting this summer.

Year 2

• Convective weather is prototype use-case for the synergistic cycle with AI & RC
• Researchers at OU, CMU, and NCAR are collaborating with IBM and Vaisala to develop a prototype short-term hail prediction system which will serve as a prototype for R2O transition for AI2ES
• NCAR AI and RC researchers collaborated to solicit forecaster feedback and significantly improve performance and visualizations of ML storm mode product for real-time testing
• Robustness testing for tornado project provided insights into model choices
• Presented 6 presentations at AMS and NOAA AI conferences and 1 review paper in preparation

Year 1

• Convective weather is our prototype use-case for the synergistic cycle with AI & risk communication
• Submitted a paper summarizing our work to data on developing and using AI for convective weather.
• Hosted two summer REU students (Lydia Spychalla and Jordan Robinson) who began the work of processing >100 TB of data to make the machine learning possible.
• Presented work on hail and tornado forecasting using U-nets at AMS 2022.Working with one undergraduate student and one graduate student to continue the work on global hail nowcasting and tornado nowcasting
• Storm centered image database extended to include ~200 severe weather days (including full 3d dual-pol measurements). This will be used for improving our deep learning tornado prediction models and for the development of XAI.

Leaders: Snook (OU), McGovern (OU)

Members: Allen (CMU), Becker (NCAR), Bostrom (UW), Demuth (NCAR), Gagne (NCAR), Grimit (Vaisala), Homeyer (OU), Kumler (NOAA), Lagerquist (CSU/NOAA), Molina (NCAR), Nozka (OU), Potvin (NOAA), Stewart (NOAA)

Progress:

Year 4

• Winter Road Weather/Decision Making:
  • Made improvements to NYSDOT camera image CNN to predict road surface condition and created a beta-visualization tool for highlighting evolving road weather conditions across the State.
  • Published our hand-labeled NYSDOT camera data set, which is now
    publicly available: Quantitative Content Analysis Data for Hand Labeling Road Surface Conditions in New York State Department of Transportation Camera Images (1.0.0). https://doi.org/10.5281/zenodo.8370665, Data set.
  • Presented work and findings at the AMS 2024 Annual Meeting. Follow on paper in preparation.
• Winter Weather: Precipitation Detection from Cameras:
  • Started work on precipitation detection using daytime NYS Mesonet camera images
  • Met with Albany WFO to assess their needs with regards to this application.
  • Established methodology for labeling the camera images for machine learning use.
  • Working with the Risk Communication team and social scientists to begin labeling and intercoder reliability trials.
• Visibility Classification in Mesonet Images:
  • Currently identifying a set of reference images for all cameras in the NY Mesonet.
    Developing a visual interface that displays estimated visibility distances to end users.
  • Two presentations at the Third Annual New York State Mesonet Symposium.
  • Master’s student graduated and thesis completed, publication in preparation.
• Winter Precipitation-Type Project:
  • Evaluated and converged on best evidential neural network for winter p-type
  • Conducted evaluations of physically-informed XAI on maps and sorting data by uncertainty.
  • Created set of hindcasts on HRRR for winter 2023-24
  • Working with Risk Comm team to develop interview protocol and visualizations for
    interviews with NWS forecasters.
  • Paper under review on evidential neural network with P-Type as example application. In-depth P-type paper in prep.
• NWP Uncertainty Quantification:
  • Paper published (Gaudet et al., 2024) on the Verification of the Global Forecast System, North American Mesoscale Forecast System, and High-Resolution Rapid Refresh Model Near-Surface Forecasts by Use of the New York State Mesonet.
  • Developed LSTM model to predict forecast error using NWP and NYSM data as inputs. Presented work at AMS 2024 Annual Meeting.

Year 3

• Leveraging social science for reproducible supervised ML hand labeling:
  • Iteratively developed, tested, and refined the formal coding scheme and achieved good intercoder reliability with the coding scheme (Krippendorff’s alpha > 0.8) for both the New York Mesonet and New York State Department of Transportation images projects
  • Designed the approach, goal, and scope of a paper on the methodology
  • Drafted a complete manuscript that details our methodology
  • Began co-author reviewing and revising
• Winter Road Weather/Decision Making:
  • Numerous collaborative meetings with DOT to gear project focus (e.g. road categories of interest, focus areas across NYS)
  • Approximately 30,000 hand-labeled DOT images from ~30 cameras across NYS
  • Completed codebook detailing exactly how to classify road conditions from DOT images
• Winter Weather: Precipitation Detection from Cameras:
  • Worked with RC team to reliably hand-label over 44,000 NYSM camera images into four classes by four individuals; achieved intercoder reliability
  • Model development on 7 different deep learning architectures with class-averaged accuracies ~95%
  • Model interpretability methods applied to predictions to understand important features
  • Apply Evidential Deep Learning to quantify uncertainty estimates
  • Comparisons among human- and rain gauge-labeled datasets: best performance onhuman-labeled
  • Operationalized model predictions as streams of images are generated for each station to be integrated into the NYSM website
• Visibility Classification in Mesonet Images:
  • Comparing the relative visibility for a query/reference pair using a CNN. Learned
    models can be used to estimate relative visibility in image pairs from previously unseen cameras
  • Inferring atmospheric visibility distance for a query image by comparing to a set of
    reference images with known visibility distances
  • Evaluation of the approach across a diverse set of cameras, including those in urban and rural environments, and a range of fields of view 31
  • Beginning to incorporate human visibility estimates in the model construction and
    evaluation process
• Winter Precipitation-Type Project:
  • Trained multiple neural networks to predict p-type probability and uncertainty
  • Identified and fixed multiple quality control issues with mPING data
  • Developed inference pipeline to generate full-grid p-type predictions for HRRR, RAP, and GFS NWP models
  • Developed interactive visualization where sounding can be altered and re-run through ML model
  • Have collaborated throughout with RC, to codevelop user-oriented research with p-type
  • Supporting multiple interns working on project
    • 4 SIParCS interns (2 2022, 2 in 2023)
• NWP Uncertainty Quantification:
  • Completed thorough analysis of errors and biases in GFS, NAM, HRRR from
    2018-2021 for NYS; paper was submitted on this work.
  • Thorough analysis of NYSM and OK Mesonet climate regions/land
    types/topographies, to present at NYSM Symposium in September.

Year 2

• Winter Road Weather/Decision Making:
  • Building archive of approximately 2400 DOT camera images starting January 2022, data stored on xCITE servers
  • Preliminary CNN model built using 3 camera sites, achieved accuracy above 90%
  • Initial model results presented to DOT and were well-received; DOT shared feedback and discussed an end-product goal that would improve their current processes. This will facilitate inclusion of Risk Communication efforts on this project.
• Winter Weather: Precipitation Detection from Cameras:
  • Successful hiring of a new Postdoc with appropriate background, and provision of training opportunities for 2 REU students
  • Worked with Risk Comm team to reliably hand-label over 44,000 NYSM camera images into four classes by four individuals
  • Achieved intercoder reliability on a subset of images, to evaluate for trustworthiness
  • Spatiotemporal matching of in situ measurements from NYSM instrumentation with camera imagery for verification
  • Model development on 7 different deep learning architectures with class-averaged accuracies above 90%
  • Model interpretability methods applied to predictions to understand important features
  • Operationalized model predictions as streams of images are generated for each station to be integrated into the NYSM website
• Visibility Classification in Mesonet Images:
  • Constructing CNN models for two individual sites (rural and urban)
  • Developed data loading/transformation pipeline to address the large size of the Mesonet data set
  • Preliminary CNN models: class-weighted validation accuracy of 72.3%
• Winter Precipitation-Type Project:
  • Established bi-weekly research group meeting on P-type which includes researchers from XAI, ES and RC
  • A research plan for the coming year has been created.
  • Trained baseline neural network models on ASOS and mPING data with pressure and height coordinate data to understand sensitivity to vertical coordinate system
  • Created project code repository: https://github.com/ai2es/ptype-physical

Year 1

Leaders: Thorncroft (Albany), Sulia (Albany), Fagg (OU), Gagne (NCAR)

Members: Bassill (Albany), Becker (NCAR), Bostrom (UW), Cabrera (OU), Demuth (NCAR), Diochnos (OU), Evans (Albany), Gantos (NCAR), Gaudet (WC), Harrison (NOAA), Horan (Albany), Kurbanovas (Albany), McGovern (OU), Rothenberger (OU), Sasser (OU), Schreck (NCAR), Shresttha (Albany), Sutter (Albany), Torn (Albany), Tyle (Albany), Wirz (NCAR)

Progress:

Year 4

• Tropical Cyclones – generating synthetic microwave imagery:
  • The algorithm is now available in real-time – new imagery every 10 minutes.
• Science discovery for Tropical Cyclones:
  • Applied Principal Component Analysis (PCA) to the synthetic microwave and are studying both the principal components and the time series of the PCA coefficient during the evolution of TCs.
  • Work was in two presentations for the AMS 2024 Annual Meeting.
• Developed an algorithm using random forest techniques to improve the prediction of rapid intensification for tropical cyclones. The results are promising and seem to improve the operational consensus model. Paper in internal review (McGraw et al.).

Year 3

• Tropical Cyclones – Improve Emulated Microwave Algorithm
  • Improved NN approaches to simulate microwave imagery, performed feature optimization to improve night predictions, and tested and evaluated approaches for estimating uncertainty.
  • Published Haynes et al. (2023) on uncertainty quantification with NNs
• Added capability to predict over the Western Pacific (WP) and Southern Hemisphere (SH) basins using Himawari
• Science discovery for Tropical Cyclones:
  • Developed an AI algorithm that emulates microwave imagery from geostationary
    satellite imagery.

Year 2

• Developed CNN approach to simulate microwave imagery and characterized CNN model behavior:
  
  • Quantified errors by brightness temperature, storm type, and distance from storm center
    
• Added capability to estimate uncertainty along with central prediction
  
  • Evaluated uncertainty estimates using spread-skill and PIT diagrams
    
  • Created method of viewing uncertainty using overlaid X’s
    
• Extending method to night-time predictions:
  
  • Started performing feature and model search to improve night-time predictions

Year 1

• Hired a research scientist (Jan 2021) and a postdoc (July 2021).
• Collected and pre-processed data. 
• Developed simple, pixel-based, fully connected neural network algorithm.  Performed hyper-parameter optimization.
• Developed CNN for image-to-image translation. Optimization in progress.
• Currently evaluating algorithms, including comparison to existing random forest algorithm. The CSU team improved their AI algorithm to generate simulated Microwave imagery from geostationary satellite imagery and presented their progress at the AMS annual meeting in Jan 2022 (Haynes et al., 2022).

Leaders: Ebert-Uphoff, Musgrave (CSU)

Members: Bostrom (UW), Demuth (NCAR), Gagne (NCAR), Griffin (Disaster Tech), Hall (NVIDIA), Haynes (CSU), Kumler (NOAA), McGraw (CSU), Stewart (NOAA), Thorncroft (Albany)

Progress:

Year 4

• Developed an interpretable AI approach for transfer learning to improve S2S prediction of
  temperature along the U.S. West Coast. The approach shows skill at S2S timescales where extensive observations are lacking for training on observations alone.
• Submitted paper to AIES on the interpretable architecture and its use for transfer learning from climate models to reanalysis (Gordillo et al., 2024).
• Used a second (new) interpretable AI approach to make skillful climate forecasts on
  subseasonal-to-decadal timescales via analog forecasting. The paper is published (Rader et al., 2023). 
• Developed a new method to predict average regional water levels along the Northwest Gulf of Mexico for S2S to multi-year predictions. Results were presented at the 2023 AGU conference and a related journal publication is provisionally accepted (Vicens-Miquel et al.).

Year 3

• Published paper on Interpretable AI using a “this looks like that” prototype network with
  applications to climate prediction (Barnes et al., 2022) – now being applied to S2S
• Successfully trained the interpretable network on CESM2 data, as well as on ERA5 data for S2S prediction of North American surface temperature providing an interpretable S2S prediction model.
• AMS 2023 Annual Meeting presentation on using an interpretable network to predict temperature anomalies weeks in advance.

Year 2

• Published two papers to JAMES on “abstention networks” for skillful forecasts of opportunity
• Paper under review on Interpretable AI using a “this looks like that” prototype network with applications to climate prediction (Barnes et al., under review)
• GRA has successfully coded an analog S2S model to act as our baseline for the ProtoLNet (interpretable prototype network)
• GRA has successfully gotten the ProtoLNet working with S2S data from the CESM2 pre-industrial control simulation to predict temperatures on S2S timescales.

Year 1

• Started obtaining hindcast and climate model data for developing, testing and implementing transfer-learning framework.
• Started research on applying transfer learning between climate models and ERA5 observations to improve S2S predictions of precipitation and temperature
• Submitted two papers to JAMES (on arxiv as well) on “abstention networks” for skillful forecasts of opportunity
• Wrote blog post sharing new abstention network research
• “This Looks Like That There” paper submitted for review in AIES. This paper lays the groundwork for the machine-learning-enabled analogue S2S forecasting approach that we are developing.
• Multiple invited talks on applying abstention networks and neural network uncertainty quantification to climate/S2S applications.

Leaders: Barnes (CSU), Tissot (TAMUCC)

Members: Bostrom (UW), Demuth (NCAR), He (NCSU), Molina (NCAR), Stewart (NOAA), Vicens-Miquel (TAMUCC), Williams (TWC)

Progress:

Year 4

• PhD Student Miranda White et al. won first place (AMS AI 2024 poster competition) for
  “AI Ensemble Predictions for Cold Stunning Events in the Shallow Laguna Madre”.

Year 3

• FogNet – Coastal Fog Predictions:
  • Pushed further application and development of XAI methods for 3D CNNs: Paper in submission.
  • Continuing development of VAE version of FogNet with IBM and NWS with R2O
    context guiding decisions.
  • Finalized and pretested all the (X)AI FogNet user interview materials (questions and AI/ML think-aloud interview guide). Submitted two conference abstracts.
• CNAPS
  • Created and shared multivariate high-resolution ocean data set for the Gulf of Mexico and U.S. east coast shelf seas that is supporting the specific use-cases and AI/ML model development.
  • Collaborative research with the Risk Communication team, to evaluate the
    trustworthiness of ocean eddy predictions with oil/gas industry users, fisheries, and
    emergency response.
  • Designed a common deep learning platform OceanNet, which allows for the
    spatio-temporal characterization of ocean circulation study and prediction while
    accommodating different spatial resolutions.
  • Presented work at 5 international and national conferences.
• Coastal Inundation Predictions:
  • System of cameras is operational, beach surveys at 2-week interval continuing, added a water level and proxy wave sensor and storing other data to calibrate coastal inundation model
  • Additional water level sensor installed in Calhoun county, a location highly impacted by coastal inundation. Includes a collaboration with NOAA Sea Grant and the local County Judge
  • Published Vicens-Miquel, M. et al. (2022). A Deep Learning Based Method to Delineate the Wet/Dry Shoreline and Compute its Elevation Using High-Resolution UAS Imagery.
  • PhD student Marina Vicens-Miquel et al. won outstanding Student Presentation award at AGU 2022 for Deep Learning Generalized Model for Wet/Dry Shoreline Detection
• Sea Turtle Conservation Models
  • Established relationships with key members of the Texas Marine Coldwater Response Collaborative leadership.
  • AI operational prediction model (MLP) was used in Dec 2022 as part of a refined
    process including more direct interaction with stakeholders, use of air temperature
    predictions from NWS and ensemble predictions from IBM, supporting an interruption of coastal activities and navigation.

Year 2

• FogNet – Coastal Fog Predictions:
  • Paper “Importance of 3D convolution and physics on a deep learning coastal fog model” applying XAI to FogNet (Kamangir et al.) accepted in  Environmental Modeling and Software (https://doi.org/10.1016/j.envsoft.2022.105424). 
  • FogNet results including XAI are used as part of a collaboration with risk communication. 
  • The collaboration with risk communication has led to shifting FogNet to probabilistic outputs.
  • As part of a collaboration with IBM a VAE alternative to FogNet is being explored (would be easier to implement operationally).
• CNAPS:
  • Have formed an AI team at NCSU that consists of two postdocs (Anna Lowe, Naz Chaichitehrani) and two graduate students (Laura McGee and Michael Gray) to work on AI2ES ocean research problems.
  • A 28-years (1993-2020) data assimilative ocean circulation reanalysis has been produced. This dataset is being used for AI model training and testing.
  • Have generated AI/ML models and initial results & analyses of 3.1 (Loop Current eddy) and 3.2 (HAB)
  • 4 meeting presentations at international and national conferences
• Coastal Inundation Predictions:
  • Created a dataset of over 3,000 drone based coastal images from 12 locations in Florida and Texas with labeled shoreline wet/dry line including elevations and Deep Learning predictions.
  • Undergraduate students developed water level predictions visualizations for stakeholders and scientists.
  • A set of cameras including a stereo camera are being installed on a local pier to provide point cloud time series
  • A peer reviewed conference paper (Vicens-Miquel et al.) “’Deep Learning Automatic Detection of the Wet/Dry Shoreline at Fish Pass, Texas” was accepted for IGARSS 2022 proceedings about to be followed by journal paper submission.
• Sea Turtle Conservation Models:
  • The AI operational prediction model (shallow neural net) was extended to 120 hours lead time facilitating the decision process ahead of cold stunnings.
  • The team provided predictions and guidance for the determination and adjustment of the start and stop of navigation interruptions during a February 2022 cold stunning event.
  • Continued preparation of the stakeholder engagement including tentative agreement from stakeholders such as the Texas Marine Cold-water Response Collaborative and Texas Parks and Wildlife to participate.

Year 1

• The paper describing FogNet (Kamangir et al.) was accepted “FogNet: A Multiscale 3D CNN with Double-Branch Dense Block and Attention Mechanism for Fog Prediction” in  Machine Learning with Applications https://doi.org/10.1016/j.mlwa.2021.100038. 
• The performance of FogNet was compared to operational models, HREF, SREF showing substantial performance improvement.
• Two other journal submissions in review
• Working with IBM on potential R2O VAE implementation of coastal fog predictions
• Tested Channel-Wise PartitionSHAP on 13-channel EuroSAT dataset (FogNet)
• Created experiment where adding an additional channel (from RGB to RGB & NIR) expected to allow CNN to learn strategy. Channel-Wise PartitionSHAP results strongly suggest that model exploited the additional channel (FogNet)
• Development on visualization tool progressing (FogNet)
• Used on analysis of FogNet paper currently under submission (FogNet)
• Labeled over 3,000 coastal images (Coastal Inundation Predictions)
• Processing of images from Florida and Texas (Coastal Inundation Predictions)
• Developed methods to label orthomosaics with ESRI software (Coastal Inundation Predictions)
• Undergraduate students developing water level predictions visualizations for stakeholders and scientists (Coastal Inundation Predictions)
• Completed first drone flight to acquire beach imagery including wet/dry line. Ready to acquire more imagery in different conditions (Coastal Inundation Predictions)
• Provided guidance including recommended start and stop of navigation interruptions in the Laguna Madre during the largest sea turtle cold stunning event in recorded US history (February 2021).
• Developed visualisations of IBM ensemble predictions and NWS predictions to help with cold stunning prediction guidance (Sea Turtle Conservation Models)
• Held meeting with state and federal agency stakeholders to review predictions. Invited presentation at industry navigation association annual meeting (Sea Turtle Conservation Models)
• Long-term ocean observations including both remote sensing and in situ ocean data have been collected. A 26-years (1993-2018) data assimilative ocean circulation reanalysis is being produced. These observations and numerical model output will be used for AI model training and testing starting in fall 2021
• Self-organizing map (SOM) method was applied, which can effectively cluster LC patterns (Meso-scale Ocean Eddy Prediction)
• A preliminary  ML (ANN) model for the LC variations and eddy shedding process has been developed (Meso-scale Ocean Eddy Prediction)
• Further validation and refinements of this LC ML model is ongoing (Meso-scale Ocean Eddy Prediction)
• One NCSU PhD student (Laura McGee) has been working on this ML technique and remote sensing data analysis as a part of her Ph.D. dissertation (Cloud-free satellite marine data reconstructions)
• Based on MODIS observations, we have successfully reconstructed a 18-year (2003-2020) cloud-free, daily time series for SST, CHL, POC, and PIC for the US Atlantic coast and the GoMEX waters (Cloud-free satellite marine data reconstructions)
• The product is being used for feature detection and extreme event prediction (Cloud-free satellite marine data reconstructions)

Leaders: He (NCSU), Tissot (TAMUCC), Williams (TWC), McGovern (OU)

Members: Bostrom (UW), Collins (NOAA), Demuth (NCAR), Gagne (NCAR), Gray (NCSU), Griffin (Disaster Tech), Hajiesmaeeli (TAMUCC),  Kamangir (TAMUCC), Kastl (TAMUCC), King (TAMUCC), Krell (TAMUCC), Lowe (NCSU), McGee (NCSU), Medrano (TAMUCC), Nguyen (TAMUCC), Starek (TAMUCC), Vicens Miquel (TAMUCC), Warrillow (NCSU), White (TAMUCC), Wu (NCSU), Zambon (NCSU), Chattopadhyay (PARC)