We often think of astronomy as a visual science with beautiful images of the universe. However, astronomers use a range of analytical tools beyond imaging to understand the nature on a deeper level.
Data Sonification is the process of converting data into audio. It has powerful applications in research, education, and outreach, and enables the blind and visually impaired community to make sense of plots, images, and other data.
Its use as a tool in science is still in its early stages – but astronomical groups are leading the way.
In one paper published in Nature Astronomy, my colleagues and I discuss the current state of data gain in astronomy and other fields, providing an overview of 100 sounds based on project and explore its future directions.
Cocktail party effect
Imagine this scene: you are at a rather noisy crowded party. You don’t know anyone and they’re all speaking a language you can’t understand – no good. Then you hear some dialogue in a far corner in your language. You focus on it and go over to introduce yourself.
While you may have never experienced such a party, the thought of hearing a recognizable voice or language in a noisy room is familiar. The ability of the human ear and brain to filter out unwanted sounds and capture the desired sounds is called “cocktail party effect“.
Likewise, science has always pushed the boundaries of what can be detected, which often requires extracting very faint signals from noisy data. In astronomy, we often try to find the faintest, most distant, or most fleeting signals. Data augmentation helps us push these boundaries further.
The video below provides examples of how consolidation can help researchers distinguish faint signals in data. It features a spike of nine bursts from a repeating fast radio burst known as FRB121102.
Fast radio bursts are milliseconds of radio emission that can be detected in half the universe. We still don’t know what causes them. Detecting them at other wavelengths is key to understanding their nature.
Too much of a good thing
As we explore the universe with our telescopes, we find it rife with cataclysmic explosions including the death of supernovas of stars, the merging of black holes and neutron stars creating gravitational waves, and fast radio bursts.
Here you can listen to the merger of two black holes.
And the merger of two neutron stars
These events allow us to understand extreme physics at the highest known energies and densities. They help us measure the expansion rate of the universe and the amount of matter it contains, and determine where and how elements are made, among other things.
Upcoming facilities like the Rubin Observatory and the Square Kilometer will detect tens of millions of these events every night. We use computers and artificial intelligence to deal with this large number of findings.
However, most of these events are faint explosions, and computers are only very good at finding them. A computer can pick out a fuzzy burst if it is given a sample of the “wanted” signal. But if the signals come from this expected behavior, they are lost.
And often these events are the most interesting and give the most insight into the nature of the universe. Using sonification data to verify these signals and identify differences can be very effective.
More than meeting eyes
Data sonification is useful for scientific interpretation because humans interpret audio information faster than visual information. In addition, the ear can distinguish more levels of sound than the eye can distinguish color levels (and to a wider range).
Another direction we’re exploring for data augmentation is multidimensional data analysis – which involves understanding the relationships between many different characteristics or attributes in audio.
Plotting data in ten or more dimensions simultaneously is too complicated and its interpretation too confusing. However, similar data can be understood much more easily through the comparison process.
As it turns out, the human ear can instantly tell the difference between the sound of a trumpet and a flute, even if they play the same note (frequency) at the same loudness and duration.
Why? Because each sound consists of higher order harmonics that help determine the quality of the sound, or timbre. The varying intensities of the higher harmonics allow listeners to identify instruments quickly.
Now imagine putting information – different data properties – like different magnitudes of the higher harmonics. Each subject studied will have a unique tone, or belong to a class of tones, depending on its overall properties.
With a little training, one can almost instantly hear and recognize all the properties of the object, or its classification, from a single sound.
Beyond research
Sonification also has great uses in education (Sonokids) and outreach (e.g.: Sound system and STRAUSS), and has wide applications in fields including medicine, finance, and more.
But perhaps its greatest strength is enabling the blind and visually impaired community to make sense of images and plots that help with everyday life.
It can also enable meaningful scientific research and do so quantitatively, like sonification research tool provide numeric values on the command.
This ability could help advance STEM careers among visually impaired and visually impaired people. And in doing so, we can tap into a huge pool of brilliant scientists and critical thinkers who may not yet have envisioned a path towards science.
We need government and industry support to further develop disinfection tools, improve accessibility and usability, and help set sterilization standards.
With the growing number of tools available and the growing demand in research and the community, the future of data augmentation looks bright!
Source: Swinburne University of Technology
