„…and then one of the musicians suddenly stopped playing.“ Can AI generate missing data from NASA’s Solar Dynamics Observatory?

„…and then one of the musicians suddenly stopped playing.“

Can AI generate missing data from NASA’s Solar Dynamics Observatory?

New York, August 19, 2020

In 2014, NASA lost an important instrument that was housed on the Solar Dynamics Observatory (SDO) satellite that measured extreme UV radiation from the sun. Faced with repair costs in the millions to billions, a team from the NASA Frontier Development Lab and IBM turned to artificial intelligence and historical data to see if a well-trained model could fill the data gap.

What if artificial intelligence could decipher more than pictures of dogs, cats and stop signs? What could we learn from looking at images of the sun?

Sunshine is good for the soul

The sun is the creator of life. There may be other similar solar systems where a star is as valuable as the Sun is to us, but the Sun’s influence in our solar system is immeasurable. Any changes in the Sun’s power, however small, be they sunspots, solar flares or coronal mass ejections, have a direct impact on the Earth. So they also affect global navigation systems (GPS), satellites, radio systems, computers, mobile phones, electrical systems, air traffic control and power supply.

For example, the last major solar storm on 23 July 2012, the strongest ever recorded, missed the Earth by about a week. If it had hit the earth directly, it could have had „catastrophic effects“ as it would have destroyed most of the world’s electrical, electronic and communication systems. A study by the National Academy of Sciences estimates that a direct impact of such a storm could cause damage of up to $2 trillion. A much weaker event in March 1989 led to a power outage of several weeks in the entire province of Quebec.

Solar Dynamics Observatory is a cool satellite

To keep an eye on such solar dangers, NASA launched the Solar Dynamics Observatory (SDO) satellite in 2010. The cost was around 850 million dollars. The SDO satellite collects various measurements from the sun in the hope of predicting solar storms and mitigating their effects in and around space.

The SDO satellite has three major instrument components:

– Atmospheric Imaging Assembly (AIA) – takes images of the Sun’s atmosphere in multiple wavelengths (up to 10) for 10 seconds at IMAX resolution (x10 times more accurate than HD images). In other words, it measures what happens in the solar atmosphere.

– EUV Variability Experiment (EVE) – measures the Sun’s extreme ultraviolet radiation (EUV) to understand its influence on the climate changes of the Earth (and near-Earth space)

– Helioseismic and Magnetic Imager (HMI) – studies the vibrations and magnetic field at the surface of the Sun, the photosphere.

Together, these three instruments have continuously monitored the Sun and produced about 1 TB of data per day.

But the cool satellite broke down!

In 2014, a critical component of the EVE instrument was destroyed and true EUV measurements were no longer available to satellite operators. That was bad news. First, the EUV fluctuates most in the solar spectrum, so a constant measurement above the Earth’s atmosphere can give us a good insight. Secondly, the EUV photons emitted by the Sun are absorbed in the upper atmosphere and in the ionosphere, so a measurement above these layers is very critical. Thirdly, these extreme fluctuations of the EUV can have dramatic effects on the Earth’s outer atmosphere. They could lead to the outer atmosphere being inflated in a much larger balloon than it normally is, which could have costly effects on all other satellites.

The solution to the problem was prohibitively expensive and ranged from sending a manned mission to the satellite (costing over $500 million) to launching a new satellite that could cost about $1 billion.

Since these options were not feasible, scientists and engineers from NASA FDL, IBM and Nimbix came up with the idea: Could AI provide a solution?

AI to the rescue

The three instruments of the SDO worked well from 2010 to mid-2014. Could deep learning neural networks predict the missing EVE data based on the analysis of terabytes of data from the last four years with hundreds of possible models and variations?

A Nasa expert described the situation with the following question: „Imagine you have listened to a symphony for four years playing music and then one of the musicians suddenly stops playing. Would you be able to mentally fill in the missing music of the performer who had fallen silent?“ This is what the NASA FDL team wanted to do with the symphony of data from NASA’s Solar Dynamics Observatory.

Fortunately for the engineers, the AIA and EVE had spent four years producing harmonic data – high-resolution images of the sun and corresponding EUV measurements – that could be used to create and test models. In the meantime, there is no doubt among AI experts that the quality of the data used to create the model/algorithm plays an important role. The research team created a „machine learning bake off“ by creating 1,000 models to validate the hypothesis. After trying out several architectures, such as Linear, Multi-Layer Perceptron (MLP), Convolutional Neural Networks (CNN) and Augmented CNN, they found that Augmented CNN met their needs exactly.

CNN is the subdivision of AI, which specializes in the analysis of visual images and „deep learning“ from images. For example, when you analyze an image, it is not enough to recognize a particular gesture, you also need to understand what that gesture means in a particular culture. Similarly, the scientists wanted to analyse the outstanding images of the Sun produced by AIA and predict the measurements of EUV radiation.

Using MacGyver-like AI tools

It was impressive that the engineers tackled the task using only common software and hardware tools: Jupyter notebook, PyTorch, NVIDIA GPUs, IBM Watson AI and the cloud provider Nimbix as host for all this. Each of these tools was chosen for a specific reason. The Jupyter notebook is the easiest way for engineers to collaborate. NVIDIA is the best graphics processor available today. IBM AI tools are designed to solve AI problems in the enterprise. And the Nimbix cloud is the best AI cloud available.

The team has broken the data set into four parts. The model was able to process the daily TB data of the first year and create a solid AI model. After several iterations and training using the data of a whole year, the engineers were able to process the data of the second year. After learning from this, they used a third year of data to retrain the model, and finally tested the final model using the fourth year’s data.

The results were 97.5% accurate. Now the AI can process the images generated by the AIA in high quality and provide EUV data for the years.

If the AI can find out the missing data from the Sun based on the current inputs, could we also predict the EUV spectra into the future with precision in advance? Predicting solar changes would have dramatic effects on the Earth.  In addition, this technique for using AI to fill „data gaps“ based on information from the environment could also be used in other applications, for example in an IoT installation when a sensor is defective or when there is a part of a customer satisfaction survey that is often, but not always, skipped by the customers of a financial services company. As is often the case with AI, the sky is the limit!