Entry Name: "SMU-Xie Weiyi-MC2"

VAST Challenge 2019
Mini-Challenge 2

Team Members:

Xie Weiyi, Singapore Management University, wyxie.2018@mitb.smu.edu.sg PRIMARY

Student Team: YES

Tools Used:

Tableau Desktop

Tableau Prep Builder

SAS JMP Pro

Excel

Approximately how many hours were spent working on this submission in total?

80 hours

May we post your submission in the Visual Analytics Benchmark Repository after VAST Challenge 2019 is complete? YES

Video

In the submission folder – “Submission Video – SMU-Xie Weiyi-MC2”

Questions

Your task, as supported by visual analytics that you apply, is to help St. Himark's emergency management team combine data from the government-operated stationary monitors with data from citizen-operated mobile sensors to help them better understand conditions in the city and identify likely locations that will require further monitoring, cleanup, or even evacuation. Will data from citizen scientists clarify the situation or make it more uncertain? Use visual analytics to develop responses to the questions below. Novel visualizations of uncertainty are especially interesting for this mini-challenge.

1 – Visualize radiation measurements over time from both static and mobile sensors to identify areas where radiation over background is detected. Characterize changes over time. Limit your response to 6 images and 500 words.

To determine the background level of radiation in St. Himark, average of radiation measurements by both static and mobile sensors across the entire simulation period is plotted. As seen from the graph, radiation measurements are relatively stable on 6 & 7 April and moderate fluctuations spanning a few hours are observed since 8 April morning when the earthquake strikes the city. On 9 & 10 April, the radiation measurements reach the peak, indicating a major leakage from the nuclear plant. Data from the stable period (6 to 7 April) are extracted and plotted to calculate the background radiation level in the city.

Data during stable period are plotted by sensor type with upper and lower control limits. When 1 standard deviation is chosen (upper and lower limits are 1 standard deviation away from the average), a few outliers are observed; however, when it is set to 2, no outlier is observed. As most of the radiation measurements lie within 1 standard deviation from the mean, we conclude that there is no special event happen in this period and the radiation level is stable. Hence, the average values 25.9801 cpm and 14.6398 cpm are good representatives of the background level of radiation by mobile and static sensors respectively. We also notice that the background radiation level measured by mobile sensors is higher than that of static sensors. It could be due to sensor calibration. Hence, radiation measurements by mobile sensors and static sensors are separated for subsequent analysis to account for the discrepancy.

By comparing static sensor measurements over time, 2 periods where radiations are above background are identified – 1) 8 April 4pm to 9 April 5am and 2) 9 April 3pm till end of 10 April. The first leakage, starts soon after the earthquake, is small in scale and not all neighborhoods are affected. As shown by the shake intensity map provided in Mini-challenge 1, the epicentral point of the earthquake is near the North-Eastern part of the city where the nuclear plant locates. At 8 April 4pm, Safe Town shows higher-than-background radiation readings (Static Sensors 13 & 15). As time passes by, static sensor readings in adjacent neighborhoods, i.e. Sensor-id 12 & 14 start to pick up. The radiation spreads radially from the nuclear plant with time. The radiation readings from mobile sensors support the above observation by showing significant number of radiation measurements above 2 standard deviations from the background level. Though far from the nuclear plant, Terrapin Springs and Scenic Vista have high radiation measurements during that period, as circled in green.

The major leakage occurs from 9 April 3pm onwards. It is larger in scale as almost all static sensors show radiation measurements over background. According to static sensors, the badly contaminated neighborhoods are Safe Town where the nuclear plant locates and Old Town which is adjacent to Safe Town. Less movement of mobile sensors is observed after the major leakage and most of the areas in the Southern part of the city are not covered by the mobile sensor network. Besides Safe Town and Old Town, all neighborhoods covered by mobile sensors show radiation measurements over background during some periods, though not for all periods. Due to the moving nature of mobile sensors, it is hard to have a holistic understanding of radiation level across all neighborhoods.

2 – Use visual analytics to represent and analyze uncertainty in the measurement of radiation across the city.

a. Compare uncertainty of the static sensors to the mobile sensors. What anomalies can you see? Are there sensors that are too uncertain to trust?

b. Which regions of the city have greater uncertainty of radiation measurement? Use visual analytics to explain your rationale.

c. What effects do you see in the sensor readings after the earthquake and other major events? What effect do these events have on uncertainty?

Limit your responses to 12 images and 1000 words.

In general, mobile sensors have higher uncertainty than static sensors across all simulation days by showing wider whiskers. More extremes values are also observed for mobile sensors.

The uncertainty of sensors is studied by splitting stable period and the rest of the periods.

In stable period, boxplots of all static sensors are plotted. A few outliers are observed for all sensors except Static Sensor 12. Static Sensor 12 is more reliable as compared to the rest of the static sensors. In view of the large data size we have (readings are taken at 5 second intervals across 5 days), the number of outliers present in the dataset is deemed acceptable. To compare the readings by temporarily excluding the outliers, minimal variations are seen for all static sensors. Hence, we can conclude that the uncertainty of static sensors during stable period is minimal and they are all reliable to some extent.

On the contrary, mobile sensor readings exhibit more variation as compared to static sensors. Similarly, number of outliers is within acceptable range and no mobile sensor shows outstanding uncertainty during the stable period.

By excluding extreme readings and setting the upper limit as 2 standard deviations above the background, 2 mobile sensors have quite a few readings falling outside the upper limit – Mobile Sensor 18 & 20. Though readings of Mobile Sensor 37 are mostly within the limits, it is highlighted for further investigation due to its wide range.

Mobile Sensors 18 & 20

Mobile Sensor 18 mainly passes through Downtown and Southwest neighborhoods, as shown in the figure below. However, the 2 Static Sensors 4 & 6 located in the 2 neighborhoods show radiation measurements within +/- 5% of the background radiation level. As seen from the readings from other mobile sensors travelling in the 2 neighborhoods, majority of them are blue in color, indicating that the radiation readings are within the upper limit. Same observation is made for Mobile Sensor 20. Hence, Mobile Sensors 18 & 20 are unreliable as their readings are consistently higher than the rest of the mobile sensors during stable period.

Mobile Sensor 37

Though Mobile Sensor 37 travels on a similar route on both 6 & 7 April, the radiation readings vary a lot. Hence, performance of Mobile Sensor 37 is questionable in terms of time-to-time uncertainty.

After earthquake and nuclear plant leakage, higher uncertainty is observed across all sensors, both mobile and static. For instance, readings of Mobile Sensor 9 increase sharply from the background level to a few hundred cpm within a short period and fall back to normal after a while on 8 April. During this period, an earthquake hits the city and there is a small-scale radiation leakage in Old Town and Safe Town as discussed previously. By plotting the path of Mobile Sensor 9, radiation readings of the sensor increase as the sensor moves towards the nuclear plant and decrease as it drives away from the leakage point.

For static sensors, uncertainty of readings increases after the earthquake on 8 April and major leakage on 9 & 10 April, especially for Static Sensors 9, 12 & 13, which locate in neighborhoods suffering from severe radiation contaminations, as seen from the boxplot below.

With regards to uncertainty in neighborhoods, besides Old Town and Safe Town which have Static Sensors 9, 12 & 13, another neighborhood shows high uncertainty is Broadview.

Though Broadview neighborhood is far from the nuclear plant, it shows relatively high radiation readings after the first leakage. This could be due to uncertainty of the sensor itself or the uncertainty of the neighborhood environment. With the given information, it is hard to differentiate which is the true cause as there are a mixture of high and low readings from mobile sensors during the period.

By looking at changes in radiation measurements over time, the middle part of the city where there is a dense mobile sensor network is also found to exhibit great uncertainty. As discussed earlier, mobile sensors in general have higher uncertainty than static sensors. Also, their moving-around nature could also contribute to the uncertainty of the neighborhood. Another neighborhood is the bottom right-hand corner of the city. A few mobile sensors show high radiation readings when travel to this area while other sensors show normal readings.

In summary, earthquake and other major events lead to increase in sensor uncertainty. However, high uncertainty of readings does not always imply unreliability of sensors. Instead, it is proven that Mobile Sensor 9 is reliable in detecting changes in radiation level in the city. The variation in readings depend on the areas passed by the sensors and hence, increase in uncertainty for all sensors varies. Uncertainty in neighborhood depends on the proximity of the neighborhood to the leakage point as well as mixture of sensors present in the neighborhood may also affect the uncertainty level. Other factors may also lead to neighborhood uncertainty which shall not be discussed here in details.

3 – Given the uncertainty you observed in question 2, are the radiation measurements reliable enough to locate areas of concern?

a. Highlight potential locations of contamination, including the locations of contaminated cars. Should St. Himark officials be worried about contaminated cars moving around the city?

b. Estimate how many cars may have been contaminated when coolant leaked from the Always Safe plant. Use visual analysis of radiation measurements to determine if any have left the area.

c. Indicated where you would deploy more sensors to improve radiation monitoring in the city. Would you recommend more static sensors or more mobile sensors or both? Use your visualization of radiation measurement uncertainty to justify your recommendation.

Limit your responses to 10 images and 1000 words

A mobile sensor is classified as being contaminated if the sensor readings show prolonged high values. By setting the contamination threshold as 300% of the background level, a few sensors are identified contaminated.

It is discovered that these contaminated sensors mainly pass through 2 areas - 1) Safe Town & Old Town where the nuclear plant locates and 2) Wilson Forest Hwy.

Based on previous discussion, radiation level in Safe Town and Old Town is high after the leakages. Since mobile sensors passing through these 2 areas or parking in these 2 areas also show prolonged high radiation measurements, this area is deemed as contaminated. Another area of potential contamination is Wilson Forest Hwy. Readings of some sensors are much more higher than other areas and as the sensors move out of the area, radiation readings return to the background level. The graph below shows the animation history of Mobile Sensor 28. The color of the dots is represented by percentage of the radiation measurements over the background. Red means that the radiation measurements are more than 100% of the background. As seen, the sensor shows higher-than-background readings only in this area. Hence, Wilson Forest Hwy is an area potentially having some contamination.

There is a concern if the contaminated sensors may spread the contamination over to other parts of the city. As we see from the table of contaminated sensors passing through Old Town and Safe Town, these contaminated sensors eventually leave the area and their readings manage to drop back to normal level by end of the simulation. However, on the other hand, Mobile Sensors 25, 28, 29 & 45 do not leave the contaminated areas based on the last radiation measurements taken. By examining their radiation measurements on the last few readings, they are no longer contaminated. Below graph shows an example of the radiation status of Mobile Sensor 45. Hence, no special attention is required from the city government.

Mobile Sensor 20 remains contaminated although it has already left the contaminated area. Hence, St. Himark city government should take actions to quarantine the contaminated car before it spreads radiation to other part of the city. High radiation may also impose health hazard to the driver.

4 –– Summarize the state of radiation measurements at the end of the available period. Use your novel visualizations and analysis approaches to suggest a course of action for the city. Use visual analytics to compare the static sensor network to the mobile sensor network. What are the strengths and weaknesses of each approach? How do they support each other? Limit your response to 6 images and 800 words.

Readings of mobile sensors and static sensors in the last 2 hours and last 10 minutes of the simulation period are plotted. There is no significant change in radiation levels in the last 2 hours. As shown in the graph, there is still moderate level of radiation contamination across neighborhoods in St. Himark, especially neighborhoods like Old Town, Broadview and the middle part of St. Himark as both mobile and static sensors show high radiation values. Hence, the city should prioritize and take actions in these neighborhoods first. As indicated by mobile sensors, a few neighborhoods such as Northwest and Safe Town are having healthy radiation measurements. However, the two static sensors in Safe Town seem to show contradicting result. It links back to our previous discussion on the uncertainty of sensors. The city could probably consider sending experts to re-measure the radiation level in Safe Town as the nuclear plant is in the neighborhood. Close and accurate monitoring of radiation level in Safe Town is required due to its proximity to the nuclear plant.

Static and mobile sensor readings in the last 2 hours of the simulation period:

Static and mobile sensor readings in the last 10 minutes of the simulation period:

The advantage of having static sensors over mobile sensors is that they are always readily available unless there was any physical damage to the infrastructure. As mobile sensors are attached to cars, their coverage is limited to road network and cars’ travel activities. If no car travels past an area, no radiation reading will be available in that area. However, the locations of static sensors need to be pre-determined and hence it loses the flexibility that mobile sensors bring to the city. Though more static sensors can be added, in view that there are only 9 static sensors installed so far, it is reasonable to assume that static sensors are more costly to set up and maintain.

On the other hand, instead of giving us readings at a particular point at all times, mobile sensors allow us to visualize radiation readings with an additional dimension – across different points in an area in a period of time. For example, in the last 2 hours, Mobile Sensor 11 travels from Downtown to Northwest, the blue path indicates that the North part of Downtown neighborhood and majority of Northwest neighborhood are free from radiation contamination. However, since mobile sensors are moving around, once they are contaminated, they will give false indication of radiation level in the subsequent neighborhoods they travel to. If the city fails to differentiate contaminated and non-contaminated sensors, it could lead to waste of resources in rescuing people in neighborhoods that do not suffer from radiation contamination.

Moreover, movement of mobile sensors at night are minimal and hence they behave more like static sensors. St. Himark should keep this in mind and recognize the fact that the mobile sensor network does not always give us a wide coverage.

Another interesting insight provided by mobile sensors is potential damages of other infrastructures, such as roads and bridges. The figure below shows the mobile sensor coverage on 10 April. There are still active car movements in neighborhoods on top; however, on the other hand, there is almost no car movement in neighborhoods below. It is likely that the road network is disrupted during the disaster. As seen from the map, there are 2 hospitals located in Broadview and Terrapin Springs. In view of the crucial roles that hospitals play post-disasters, St. Himark probably need to take some actions to ensure these 2 hospitals are accessible as soon as possible.

In summary, both mobile sensor network and static sensor network have their own strengths and limitations. During most hours of the day, a combination of the 2 networks give us good understanding of the radiation status in the city. However, due to sensor uncertainty, contradicting readings may be shown in some neighborhoods. In such cases, the government probably needs to allocate resources to investigate the real situation on the ground.

5 –– The data for this challenge can be analyzed either as a static collection or as a dynamic stream of data, as it would occur in a real emergency. Describe how you analyzed the data - as a static collection or a stream. How do you think this choice affected your analysis? Limit your response to 200 words and 3 images.

In this study, data are analyzed as both a static collection and a dynamic stream as they both have some strengths and limitations.

As majority of radiation readings are by mobile sensors which are fast-moving, data over a short period of time may not be representative of the situation. As discussed earlier, there are contaminated cars, which would also lead to biasness in determining the state of radiation measurements in a neighborhood. Hence, we need to look at data over a longer period to discover any potential contamination of the cars and the neighborhoods. On the other hand, treating data as dynamic stream allows us to follow certain mobile sensors and study the changes in radiation measurements over time and across neighborhoods.

Moreover, there are sudden spikes in radiation measures which are found to be outliers. If the data are analyzed as a static collection, the effect of outliers can be minimized by aggregating data within a time interval, i.e. hourly average, to avoid unnecessary trigger of rescue actions by 1 or 2 outliers. For example, if radiation measurements of Mobile Sensor 8 are plotted using hourly average, an alert will signal a radiation level above background from 9 April 7am onwards. However, if the radiation readings are plotted using unit of minute, there will be a lot of false alarms before 9 April 7am when the radiation is actually at background level. The absolute readings of these false alarms are also much higher than those of the hourly averages.