Entry Name: "UKN-Metz-MC1"

VAST Challenge 2018
Mini-Challenge 1

Team Members:

Benedikt Bäumle, University of Konstanz, benedikt.baeumle@uni-konstanz.de

Ina Boesecke, University of Konstanz, ina.boesecke@uni-konstanz.de

Raphael Buchmüller, University of Konstanz, raphael.buchmueller@uni-konstanz.de

Yannick Metz University of Konstanz, yannick.metz@uni-konstanz.de PRIMARY

Juri Buchmüller, University of Konstanz, juri.buchmueller@uni-konstanz.de

Eren Cakmak, University of Konstanz, eren.cakmak@uni-konstanz.de

Wolfgang Jentner, University of Konstanz, wolfgang.jentner@uni-konstanz.de

Daniel Keim, University of Konstanz, daniel.keim@uni-konstanz.de

Student Team: YES

Tools Used:

KNIME, Audicity, Librosa, D3, Matplotlib, Excel

Approximately how many hours were spent working on this submission in total?

520

May we post your submission in the Visual Analytics Benchmark Repository after VAST Challenge 2018 is complete? YES

Video

https://vimeo.com/279896378 

Questions

1 Using the bird call collection and the included map of the Wildlife Preserve, characterize the patterns of all of the bird species in the Preserve over the time of the collection. Please assume we have a reasonable distribution of sensors and human collectors providing the recordings, so that the patterns are reasonably representative of the bird locations across the area. Do you detect any trends or anomalies in the patterns? Please limit your answer to 10 images and 1000 words.

GENERAL OVERVIEW

In this overview map of the park, we can see a density map of the whole bird population over the whole timespan, ranging from 1983 to 2018. In the background, the road network of the park is shown. As we can see, there are bird population centers throughout the park, but also large areas that show little to none activity. In the lower bar chart, we can also see the amount of recordings taken per year. Obviously, there was a long phase with a fairly small number of recordings, until around the year 2005.

Thus, the validity of this earlier data has to be assessed critically.

Figure 1.1

DISTRIBUTION PATTERN & DENSITY

In figure 2, the map of the territory is shown. Each recording within the over-all time span is visualized as a point on its prospective recording location. The color is mapped to each bird species while the size of one point reflects the actuality, more recent recordings are larger in size. Taking a look at the distribution of the bird species, three distribution patterns can be identified. In picture 1, bird species are shown that are arbitrarily distributed over the territory. In contrast, picture 2 exhibits centers at which most of each bird species are located. Picture 3 shows that some species also appear around two centers.

Figure 1.2

CO-EXISTENCES

Some bird species live among each other as some populations are recorded within the same area. In the left picture the Ordinary Snape and the Rose-crested Blue pipit share parts of their habitat as well as the populations of Bombadil, Vermillion Trillian, Scrawny Jay shown in the right picture. Some birds seem to rivalry with each other which we can see in contrary increasing and decreasing size of their population. That can be for example seen in figure 1.7.

Figure 1.3

SEASONAL

There are seasonal trends of a few species to identify. The season is visualized by the color of the border of the data points. The Ordinary Snape, the Bombadil and the Broad Winged Jojo are migratory birds. They seem to use the territory during the summer months (which are filtered in the right picture from March to August) and leave the area during the winter (appearances from September to February at the left). The Broad Winged Jojo especially returns late in the year starting from May. One exception is the year 2017, in which some birds appear in February already. A theory is, that the spring of 2017 started earlier, and warm temperatures attracted the birds sooner.

Figure 1.2

Suspicious Movements

We found some suspicions patterns concerning the bird species appearing in the region of the dumping site.

During the whole timespan we see that the species Green-tipped Scarlet Pipit, Blue-collared Zipper, Eastern Corn Skeet, Broad winged Jojo and Rose-crested Blue Pipit had part of their habitat in the region of the dumping site (picture 1). In picture 2 it can be seen, that just two of the species are still living at this location. It is also striking that the diversity of species in the south from the dumping site has decreased together with the other species the Lesser Birchbeere has vanished from this area. Picture 3 shows that Eastern Corn Skeet and the Broad-winged Jojo seem not be affected by the dumping as those species still can be found at the same regions. The Rose-Crested Blue Pipit, Blue-collared Zipper and the Green-tipped Scarlet Pipit migrated away from the dumping site. In the timespan from 2017 they were not recorded in the region of the dumping (picture 3). We assume that the region in the south was affected as well because the toxic substances of the dumping have spread.

Figure 1.5

Figure 1.6 shows the migration of the Rose-crested Blue Pipit and the Green-tippet Scarlet Pipit and the Lesser Birchbeere. The time is mapped on the opacity of the path. It is obvious that the birds have changed their habitat.

In the figure on the right we observe that the number of Rose-crested Blue pipits increases in 2015. This is the first year were the bird stayed in the south from the dumping site. At first the bird population seems to recover well. But in the years 2016 – 2018 the size of the population decreases. The population of the two, other species show a similar proceeding in their population size, except of the growing population in 2015 the species have a decreasing number in 2015 were the dumping began, a small recovering after that and then they seem to decrease again.

Those are facts support the hypotheses that the breading behavior of the species was affected.

Figure 1.6

In Figure 1.7 we can see that the populations of the Ordinary Snape and the Rose-crested Blue Pipit are diverging within the year 2015, in which the Blue Pipit stayed at one single region in the habituated area of the Ordinary Snape. As the population of the Pipit declined in 2016, the number of Ordinary Snape rose again. Our theory is that both species share the same eating habits and the Pipit seem to collect its food more efficiently or that the Ordinary Snape needs to adapt to the new competitor. Both populations decreased starting after the year of 2016. We assume that the dumping further north caused the birds as it affected the breading-behavior and probably their habitat.

Figure 1.7

BREEDING OF DARK-WINGED SPARROW

We found another suspicious coincidence. At first the Dark-winged Sparrow has vanished between the years 2008 – 2013 (picture 4). In 2013, a couple of years before the dumping, the popluation started to appear again.

The population of Dark-winged sparrows grew rapidly within the marked sector after the dumping. By comparing the population of Dark-winged sparrows and the Rose-crested blue pipits we found that both, sounds and the location of their habitat, are very similar. Our theory is that Kasio intentionally bred more individuals of Dark-winged sparrows to better fake a high quantity of the Rose-crested Blue pipits.

Figure 1.8

DAILY CALL OF BIRDS

Figure 1.9

The timeseries visualize the bird songs and calls over the time of day (y-Axis) and the date (x-Axis). In picture 1 we see the Bent-beak Riffraff. This species is mostly singing between 6:00 am and 12 am. It is remarkable that after 2011 the species is predominantly singing not calling. If we assume that the singing is a sign for pairing of the male birds. It can be explained why the number of Bent-break Riffraff is decreasing.

Picture 2 shows the Canadian Cootamum which sings and calls in the morning and in the afternoon.

In picture 3 we can see the call/ song pattern of the Pinkfinch. The Pinkfinch mostly calls between 6 am and 6 pm with some exceptions where the bird calls in the morning and in the evening as well. It is also remarkable that the number of songs is really small.

In picture 4 the songs and calls of the Green-tipped Scarlet Pipit are visualized. The species is mostly active in the late morning between 6 am and 12 am.

The last picture shows that the Queenscout has recordings from 2005 and is active in the time from 6 am to 6 pm.

2 Turn your attention to the set of bird calls supplied by Kasios. Does this set support the claim of Pipits being found across the Preserve? A machine learning approach using the bird call library may help your investigation. What is the role of visualization in your analysis of the Kasios bird calls? Please limit your answer to 10 images and 1000 words.

As beginning of the analysis of the Kasios files we had a look at the metadata. We recognized that most of the Kasios recordings are not located at the main habitat of the Rose-Crested Blue pipit.

Figure 2.1

Before using more sophisticated approaches to analyze the Kasios recordings, we first listened to the recordings and had a look on the spectrograms. By that, we already observed that recording 7 is only a concatenation of the same call:

Figure 2. 2

Recording 6 contains unnatural abrupt silences:

Figure 2.3

By listening and simply viewing the spectrogram, we were not able to get more findings from the recordings.

To further tackle the audio analysis part we built a classifier based on the 2000 audio files provided.

Before being able to build a classifier, due to the heavily varying qualities and lengths of the audio files we applied several preprocessing steps by the aid of the command line tool SOX and the python library auditok.

The following preprocessing steps were applied:

1. Noise Reduction (SOX)
2. High-Pass Filter (SOX): Sound below a frequency of 1000 Hz was completely removed by applying a High Pass-Filter
3. Segmentation: On the noise reduced and filtered audio files, we segmented the audio files into segments between 0.5s and 8s by silent moments

Example for Kasios 13:

Figure 2.4

Based on the preprocessed segments we calculated various features with the aid of the python library librosa. We extracted well-known spectral features and statistics based on fourier_coefficents on windows of lengths 1 second.

We then fed a lightgbm classifier with our calculated features, which performed with over 90% accuracy on the test data.

For our classification result, we built a small visualization tool showing spectrograms of the Kasios recording to be analyzed and overlaid the classified segments. Additionally, we added an audio player to be able to directly listen to the recordings.

Figure 2.5

Unfortunately, we were not able to classify all Kasios recordings surely by the classification visualization view. For that reason, we created several projections the data into 2-dimensional space using t-SNE based on spectrograms. By just projecting the Kasios recordings, we were able to already distinguish several clusters.

Unfortunately, we were not able to classify all Kasios recordings surely by the classification visualization view. For that reason, we created several projections the data into 2-dimensional space using t-SNE based on spectrograms. By just projecting the Kasios recordings, we were able to already distinguish several clusters:

Figure 2.6 In recording 1, 6 and 15) we suspected multiple birds are present and splitted the audio files accordingly

From figure 2.6 we conclude that not all the recordings belong to the same bird species.

Further we added an additional view to the projections to interactively brush a cluster to get a view of all spectrograms from the brushed clusters.

Figure 2.7 Here recording 8 is clustered to Lesser Birchbeere calls

With the just described approaches, we tried to classify each Kasios recording. Our classifications of the recordings are listed below:

Kasios 1: Orange Pine Plover

Kasios 2: Rose-crested Blue Pipit

Kasios 3: Darkwing Sparrow

Kasios 4: Darkwing Sparrrow

Kasios 5: Orange Pine Plover

Kasios 6: Green-tipped Scarlet Pipit

Kasios 7: We were not able to surely classify the recording and suspect it to be manipulated

Kasios 8: Lesser Birchbeere

Kasios 9: Rose-crested Blue Pipit

Kasios 10: Orange Pine Plover

Kasios 11: Orange Pine Plover, Rose-crested Blue Pipit

Kasios 12: Orange Pine Plover

Kasios 13: Rose-crested Blue Pipit

Kasios 14: We were not able to find very similar sounds

Kasios 15: Orange Pine Plover

After finishing the classification, we drew additional pie glyphs around the kasios recordings inside the map to find possible connections between the classification results and the occurences of the birds in the wildlife preserve. The pie glyphs help to clarify the classification result by providing the possibility to compare the location of the recording to the habitats of bird species spread over the park.

For example, the pie glyphs support the result of the classifier that Kasios 1, 11 and 15 are Orange Pine Plovers.

3 Formulate a hypotheses concerning the state of the Rose Crested Blue Pipit. What are your primary pieces of evidence to support your assertion? What next steps should be taken in the investigation to either support or refute the Kasios claim that the Pipits are actually thriving across the Boonsong Lekagul Wildlife Preserve? Please limit your answer to 500 words.

The Rose-Crested Blue Pipits state is alarming as the dumping has effected its breeding behavior and its habitat which leads to a decreasing number of birds in this species.

As we can see on figure 3.1 the song/ call pattern changed over time. Before 2010 the birds were mainly singing. After that the number of calls in relation to the number of songs were increasing rapidly. In the years 2015 / 2016 the growth of the population can be recognized as well. At the end of the timespan the bird is mainly calling which could be an evidence for their self-maintenance as a result of the dumping and their reaction to the toxic waste. Their struggle also leads to less pairing behavior, which can be observed in the decline of the songs after 2016.

Figure 3.1 The timeseries shows songs and calls over daytime(x-Axis) and the timespan over the years (y-Axis). Songs are represented with a circle, calls are the rectangles.

In figure 1.7 we can see that the number of Rose-Crested Blue Pipits has drastically increased in 2015. Looking at our bird map and the small multiples (figure 3.3) it can be concluded, that this was the first year the birds had one single habitat southside from the dumping side. We assume that there were better living conditions than at the old habitat and the population could recover. Figure 1.7 also shows that after 2015 the number of Rose-Crested Blue Pipits decreases. One reason can be the changed breeding behavior which could have been caused by the dumping of Kasios. Furthermore, the toxic waste of the dumping site could have spread out through the river or other environmental influences.

Fig. 3.2 The visualization maps the habitats of the species to a grid, that divides the map in 16 rectangles. The opacity of the color is mapped to the amount of appearances in the regions compared to the maximum count of the birds.

In Question 2 it was concluded that the audio data provided by Kasios was manipulated and did not match with the calls of the Rose-crested Blue Pipit. If the bird was really thriving across the Preserve they would have had no reason for faking the data.

Next steps could be that neutral ornithologists deduct the area as the dumping side as well as the rest of the preserve concerning the population of the Rose-Crested Blue Pipit. By the gathering of new data more precise assumptions about the current situation can be drawn. We suggest to track the population over the next several years to analyze the species long-term development. Biologists should focus in their investigation on the breeding-behavior.