Entry Name:
"SMU-Tan-MC1"
VAST
Challenge 2018
Mini-Challenge 1
Team Members:
1.
Tan Yong Ying, Singapore Management University, yy.tan.2017@mitb.smu.edu.sg (Primary Contact)
2.
Kam Tin Seong, Singapore Management University, tskam@smu.edu.sg
(Professor)
Student Team:
Yes
Tools Used:
1. RStudio
2. Bird Sounds Explorer is an
R Shiny web application developed by Tan Yong Ying specifically for this
challenge. It can be accessed at https://joannetyy.shinyapps.io/VAST2018MC1/.
a. Performance note: For the
last tab on spectrogram explorer, for the steps on choosing time intervals to
zoom into, please choose a time interval of maximum 6 seconds to prevent the
application from disconnecting due to limited instance memory.
Approximately how many
hours were spent working on this submission in total?
120 hours
May we post your submission
in the Visual Analytics Benchmark Repository after VAST Challenge 2018 is
complete?
Yes
Video
Questions
1 – Using
the bird call collection and the included map of the Wildlife Preserve,
characterize the patterns of all of the bird species in the Preserve over the
time of the collection. Please assume we have a reasonable distribution of
sensors and human collectors providing the recordings, so that the patterns are
reasonably representative of the bird locations across the area. Do you detect
any trends or anomalies in the patterns? Please limit your answer to 10 images
and 1000 words.
Note: Due to insufficient data points before 2010 and
in 2018, only bird calls recorded between 2010 and 2017 were included in the
analysis.
·
Bent-beak
Riffraff: They are predominantly found in the western region, but their hangout
spots are not consistent through the years.
·
Blue-collared
Zipper: In 2014, they were only making calls. In 2015, they were not spotted in
the Preserve at all. In 2016, they returned to the Preserve and went back to
their favorite spot pre-2015.
·
Bombadil:
They stayed in the same location in the northeastern part between 2010 and
2017, and no major changes in their spatial distribution were observed.
·
Broad-winged
Jojo: They have a favorite spot at the southwestern region.
There was an anomaly in 2015, as high concentrations of them were found away
from their favorite spot and they were only making calls. They returned to
their favorite spot in 2016 and 2017, but overall their concentrations have
dropped. Something may have happened in 2015 which caused them to stop singing
and subsequently experience a drop in their numbers overall.
·
Canadian
Cootamum: They have a favorite spot in the northwestern part.
No major changes in their spatial distribution were observed.
·
Carries
Champagne Pipit: They have a favorite spot at the
southeastern part and had not moved away from it throughout the years. While
they consistently make calls in that location each year, their songs are rare
in general and they are found singing in different parts in different years.
·
Darkwing
Sparrow: They were only recorded from 2014 onwards. They had
stayed in the northwestern part in 2014 and 2015, but their concentration there
decreased in 2016 and they had mostly shifted away to the southwestern part in
2017. However, this observation may not be representative of the species’
activity since it only has four years of data as compared to other species with
eight years of data.
·
Eastern
Corn Skeet: They have a favorite spot near the center and have
not moved away from it. Sometimes they are found singing in other parts.
·
Green-tipped
Scarlet Pipit: They have a favorite spot at the
southwestern part, but are also regularly found in other regions in higher
concentrations as compared to other species. However, there is a strange
observation in 2015: while none of them were spotted at their favorite spot,
there was a high number of them at the alleged dumping site. They returned to
their spot in 2016 and 2017, but their concentrations had started to drop in
2017. Something might have happened at the alleged site in 2015 which attracted
their attention, but may have also caused the subsequent drop in their numbers.
·
Lesser
Birchbeere: They have a main favorite spot in the
southwestern region and a secondary favorite spot in the central eastern
region. They are rarely found to be singing. No major changes in their spatial
distribution or numbers were observed.
·
Orange
Pine Plover: Just like the Lesser Birchbeere, it has
two favorite hangout spots, both located in the southern regions. They also do
not see big changes in their spatial distribution or numbers. They seem to be
close neighbors with the Lesser Birchbeere, as they are found in similar
regions.
·
Ordinary
Snape: They stayed in the central-eastern part without
moving and their numbers remained consistent throughout the years.
·
Pinkfinch:
This species stands out because it does not stay in any spot regularly.
Instead, they have been found throughout the Preserve at different locations
each year. It is noted their numbers seem lower in years 2014, and decrease in
their numbers are much more obvious in years 2016 and 2017.
·
Purple
Tooting Tout: The recording distribution for this
species has more anomalies than other species. Between 2010 and 2013, they were
observed in one spot in the eastern region (orange rectangles), and observed in
a bigger area in the western region (purple rectangles).
o
However, in 2014, none were spotted in
the eastern spot. In 2015, none were spotted in the western region while they
had returned to their western hangout region. After 2015, their overall numbers
have dropped significantly throughout the Preserve. We also note that in 2014
and 2015, their calls were concentrated on two different point locations in the
Preserve, which is an anomaly because their calls distributions were more
widespread in other years.
o
Overall, these observations imply
something might have happened in years 2014 and 2015 which resulted in the
drastic change in their spatial distributions.
·
Qax:
They have one or two favorite spots near the central-western region. It seems
that no major changes in their overall spatial distribution were observed.
However, looking at the distribution of songs over time, we notice they were
not singing in 2014 as compared to other years. Once again, 2014 has turned out
to be a full of "surprises".
·
Queenscoat:
They are consistently found in the western regions without major changes in
their spatial distribution. A small note is that their number of songs had
obviously dropped in 2015.
·
Rose-crested
Blue Pipit: They have two favorite spots in the
north-eastern region; one spot is at the alleged dumping site and the other
spot is below the site. Since 2015, they were no longer found at the alleged
dumping site, and they were only spotted in their second favorite spot. We also
see that in 2014, the last they were spotted at the alleged dumping site, they
were only making calls and not singing.
·
Scrawny
Jay: They have been consistently found in the western
part, and it seems that overall, no major changes in their spatial distribution
were observed. But once we break down the recordings into calls vs songs, the
startling truth is that they have stopped singing entirely since 2015. This
also highlights the importance of a good visualization that allows the audience
to see multiple angles of the situation in one unified view; the overall
distribution alone may not tell the full story.
·
Vermillion
Trillian: They have been found in many locations around the
Preserve across the years, with a favorite hangout spot in the northern region.
No major changes in their spatial distribution or numbers were observed
overall. (1000 words)
2 – Turn
your attention to the set of bird calls supplied by Kasios. Does this set
support the claim of Pipits being found across the Preserve? A machine learning approach using the bird
call library may help your investigation. What is the role of visualization in
your analysis of the Kasios bird calls?
Please limit your answer to 10 images and 1000 words.
·
Step 1: Go to “Mean Frequency Spectra
Correlations” tab, look at the heatmap summary of mean frequency spectra
cross-correlations and shortlist a few test files with low frequency offset
against the Rose-crested Blue Pipit song files. The left two heatmaps show the
maximum correlation value between the test files and the known files from
Mistford College. The right two heatmaps show the frequency offset (shifting)
required to achieve the maximum correlation value for each pairwise correlation.
The offset is expressed in kHz.
o If the absolute value of the frequency offset is high (> 2 kHz), it
means the two files have very different frequency ranges, therefore requiring a
high frequency offset to achieve high correlation. The different significance
of high correlation values versus a low or high frequency offset value can be
illustrated below.
·
Based on the heatmap, we see test
files 2, 11 and 14 have low frequency offsets against the Rose Pipit files, so
we will compare their spectrograms with the Rose-Pipit files first.
·
Step 2: Compare the spectrograms of the
shortlisted test files against the spectrograms of the known species. After
some comparison, we see that test file 2 is visually close to the Rose-crested
Blue Pipit song.
o However, both test files 11 and 14 are visually not similar to the
Rose-crested Blue Pipit. In fact, both files have different sonic textures,
sound sequences and harmonics. This highlights the shortfall of analyzing sound
files using purely statistical methods, which are usually unable to factor in
the different shapes of bird sounds.
o Also, upon further exploration, I noticed that Test File 9 is very
similar to the Rose Pipit song file too, but the similarity is not reflected in
the heatmap. The characteristics shared by the test file and the Rose-crested
Blue Pipit recordings are:
i.
The durations between notes in the
sequence are similar.
ii.
The oscillograms show similar
patterns of relative amplitude.
iii.
The sound sequences are highly
modulated, meaning frequencies vary a lot.
iv.
They share the same frequency range
of 4kHz to 7kHz.
o I also investigated Test File 13, the only file that required a positive
frequency offset to achieve maximum correlation with the Rose-crested Blue
Pipit files. After zooming in and comparing the test file versus the Rose Pipit
files at 1-second or 2-second intervals, I found that Test File 13 shows
certain characteristics that are similar to those of a Rose-crested Blue Pipit
recording:
i.
The sounds are highly modulated,
meaning the frequencies vary a lot in a phrase and in the recording.
ii.
The sound sequence can be described
to consist of different individual notes that are inseparable, meaning it is
difficult to count the number of notes.
iii.
The notes have the same general
shape, and the sound sequences have similar patterns of going up and down.
iv.
Their frequency ranges are also
similar between 4kHz to 7kHz.
o I conclude that Test File 13 could be either
i.
A Rose-crested Blue Pipit that sang
differently compared to typical recordings, a phenomenon typically attributed
to birds living in noisy urban areas, or
ii.
A bird imitating the song of a
Rose-crested Blue Pipit
The above discussion highlights the importance of using multiple
methods, both visual and statistical, and ideally lots of experience and
knowledge on bird-sound identification to correctly identify species based on
bird sounds.
·
Step 3: Refer to Where are the test birds? tab to compare where the suspected test
files are recorded against the historical intensity of Rose-crested Blue Pipit.
In Step 2, we had shortlisted test files 2, 9 and 13 to be the only files that
are similar to the Rose-crested Blue Pipit. Looking at the map of the test
points against the Rose-crested Blue Pipit concentrations, we see that these
files are not close to where the Pipits typically live at.
To summarize, in step 1 we identified that files 2, 11 and 14 may be Rose-crested
Blue Pipits. In Step 2, we compared these test files to the Rose-crested Blue
Pipit song spectrogram, but only file 2 is similarly close. In addition, we
identified files 9 and 13 to be similar to Rose-crested Blue Pipit, although
that was not observed in Step 1. In step 3, we looked at the locations of the
three test files and compared them against historical intensities of the Pipit
over the years. Kasios claimed that the 15 files are evidence that the
Rose-crested Blue Pipits are plentiful around the Preserve, and that they were
recorded in the last few months. Looking at the map, we see that they are not
close to where the Pipits typically hang out. Overall, this set does not
support the claim that Pipits are being found across the Preserve. (790 words)
3 – Formulate
a hypotheses concerning the state of the Rose Crested Blue Pipit. What are your primary pieces of evidence to
support your assertion? What next steps
should be taken in the investigation to either support or refute the Kasios claim
that the Pipits are actually thriving across the Boonsong Lekagul Wildlife
Preserve? Please limit your answer to
500 words.
We hypothesize that the lives of Rose-crested Blue
Pipits had been affected by negative activities at their old hangout location
at the alleged dumping site. From
Task 1 we see they had been found there in large concentrations pre-2014, had
stopped singing in 2014 which shows they may have been under distress, and had
moved away from there since 2015. These observations point to the high
possibility that something bad had happened at that location in 2014 which
caused their habitat to become unlivable, thereby forcing them to move away in
2015 and never return.
On top of these observations centered on
the Pipit, it was also found that another 7 species of the 19 species had
obvious changes in their numbers and/or their spatial distribution in years
2014 and 2015, and these 7 species do not live near the dumping site.
|
|
Without Change |
With Change |
|
Consistent Spatial Distribution |
Bombadil Canadian Cootamum Carries Champagne Pipit Eastern Corn Skeet Lesser Birchbeere Orange Pine Plover Ordinary Snape Queenscoat Vermillion Trillian |
Blue-collared zipper Broad-winged Jojo Green-tipped Scarlet Pipit Pinkfinch Purple Tooting Tout Qax Rose-crested Blue Pipit Scrawny Jay |
|
Inconsistent Spatial Distribution |
Bent-beak Riffraff |
Darkwing Sparrow (4 years of data only) |
It seems the problem is more than just the
alleged dumping site and the Rose-crested Blue Pipits, as nearly half of the species
had shown signs of moving away or reductions in numbers and spatially these
effects are observed throughout the Preserve, not just at the alleged dumping
site.
From Task 2 we found that only three out
of fifteen test files (Test Files 2, 9 and 13) were recordings that resembled
those of the Rose-crested Blue Pipit, and all these files were recorded at
locations far away from the usual hangout locations of the species. Overall,
the set provided by Kasios does not support their claim that the Pipits are
being found across the Preserve.
To test the hypothesis, different methods
and packages can be considered. For Task 1, we can use the Lcross function, a
statistical test of complete spatial randomness (CSR) between two point pattern
distributions, together with a confidence envelope, to test if one point
pattern is clustered towards, dispersed away from, or randomly distributed
relative to, the other point pattern and get the statistical significance of
the result. Specifically, we can do the test for the spatial distribution of
the Pipit before 2014 versus after 2014, and see if these two distributions at
two time points are clustered towards (similar), dispersed away (dissimilar) or
randomly distributed (purely random to) each other.
For Task 2, we can use the R package
warbleR which provides many methods for calculating the similarity between two
audio files or two signals. It provides a detailed workflow of how to extract
signals automatically (syllables of bird sounds in our context) from each recording,
store the results and use them to calculate pairwise cross-correlations. These
correlation values (range of -1 to 1) can tell us how similar each test file is
compared to the identified Pipit files from Mistford College to help prove or
disprove Kasios’s claim that their set of files are recordings of the Pipit; a
higher correlation value means the test file signal and identified file signal
are more similar to each other. (486 words)
References:
Araya-Salas, M. and Smith-Vidaurre, G.
(2017), warbleR: an R package to streamline analysis of animal acoustic
signals. Methods Ecol Evol. 8, 184-191.
Brumm, H. (2004), The impact of
environmental noise on song amplitude in a territorial bird. Journal of Animal
Ecology, 73: 434-440. doi:10.1111/j.0021-8790.2004.00814.x
Dragonetti, M. (n.d.). INTRODUCING
SPECTROGRAMS. Retrieved from http://www.birdsongs.it/bioac/bioacSonograms_1.html
Hans Slabbekoorn, & Margriet Peet.
(2003). Ecology: Birds sing at a higher pitch in urban noise. Nature,
424(6946), 267-267.
Nemeth, E., Pieretti, N., Zollinger, S.
A., Geberzahn, N., Partecke, J., Miranda, A. C., & Brumm, H. (2013). Bird
song and anthropogenic noise: vocal constraints may explain why birds sing
higher-frequency songs in cities. Proceedings of the Royal Society B:
Biological Sciences, 280(1754), 20122798. http://doi.org/10.1098/rspb.2012.2798
Wroza, S. (2018). SOME FOOD FOR THOUGHTS ON
BIRD IMITATIONS. Retrieved from http://soundbirding.org/index.php/2018/04/18/some-food-for-thoughts-on-bird-imitations/
Wroza, S. (n.d.). SOUND AND SPECTROGRAMS.
Retrieved from http://soundbirding.org/index.php/sound-and-spectrograms/
Sueur, J. (2018). A very short
introduction to sound analysis for those who like elephant trumpet calls or
other wildlife sound [PDF file]. Retrieved from https://cran.r-project.org/web/packages/seewave/vignettes/seewave_analysis.pdf