HortPlus has tapped into research expertise from the University of Auckland to improve the quality of weather station data by harnessing data science techniques to quickly identify station sensors in need of attention or repair.

The project has been undertaken by University of Auckland Masters student Aman Tyagi as part of a data science industry project guided by mentors from New Zealand agri-tech company HortPlus and academic supervisors.

By analysing and working with data from clusters of weather stations in HortPlus’ network, Tyagi has been able to develop a machine learning model that identifies any stations that may need urgent maintenance.

“There are always differences in weather throughout a region but by using data science, Aman has been able to develop a model that quickly and precisely identifies stations sending back leaf wetness or rainfall data that significantly differs from a normal range or the data being sent in by stations nearby,” says HortPlus director Mike Barley.

“That doesn’t automatically mean a weather station is malfunctioning but it provides an alert for someone to review it and, if needed, go and check the station to make sure its rain gauges, anemometers and other sensors are operating properly.”

Barley said the data model developed by Aman could complement data validation techniques already employed by other organisations involved in weather data collection, climate research, and weather station management, such as MetService and NIWA.

“Sensor issues are a reality of having equipment operating in the great outdoors. The sooner any of us know a sensor is playing up, the sooner we can get the issue resolved. Over time faster detection of issues can lead to improved forecasting and more robust science because less data has to be omitted.”

Tyagi said it was “like a dream come true” to be able to bring his data science expertise to bear on such a wide breadth of weather data.

HortPlus has access to a network of more than 1,000 Australasian weather stations, including more than 300 across New Zealand. This includes many MetService weather stations and nearly 100 privately owned stations on orchards and farms in both the North and South Island.

“Working with the amount of weather data HortPlus has is exciting. There are decades of weather data there so it’s an ideal testing ground for this kind of anomaly detection approach.”

Barley said the results of Tyagi’s work would have lasting effects on HortPlus’ operations.

“We are on a mission to make all stations good stations; we want as many as possible running optimally, and if repairs are required, we want them carried out quickly.

“Aman’s work is a big contribution to these efforts and has potential to be applied by other organisations across New Zealand that manage weather stations for all sorts of purposes.”