WEBVTT

1
00:00:11.144 --> 00:00:14.059
<v ->I'm Steve Marcus, Associate Provost for Faculty Affairs,</v>

2
00:00:14.059 --> 00:00:18.174
and it's my distinct pleasure to welcome you,

3
00:00:18.174 --> 00:00:22.376
the first Distinguished Scholar, Teacher Lecture at the U.

4
00:00:22.376 --> 00:00:24.997
As you may or may not know, the Distinguished Scholar

5
00:00:24.997 --> 00:00:28.330
Teacher program was established in 1978.

6
00:00:31.521 --> 00:00:34.048
So it's been going on for 37 years.

7
00:00:34.048 --> 00:00:37.648
Each year, we honor a small number of faculty members

8
00:00:37.648 --> 00:00:40.382
who have demonstrated notable success

9
00:00:40.382 --> 00:00:43.132
in both scholarship and teaching.

10
00:00:46.561 --> 00:00:50.770
This year, the first lecture is being given by

11
00:00:50.770 --> 00:00:52.340
Dr. Michael Hicks.

12
00:00:52.340 --> 00:00:54.292
Sameer Khuller will tell you more about him,

13
00:00:54.292 --> 00:00:58.829
but I just wanted to say that the Distinguished Scholar

14
00:00:58.829 --> 00:01:02.526
Teacher selection committee was really impressed

15
00:01:02.526 --> 00:01:05.701
with how creative and dedicated Mike is

16
00:01:05.701 --> 00:01:07.904
in both scholarship and in teaching.

17
00:01:07.904 --> 00:01:11.987
He brings the same kind of energy and creativity

18
00:01:12.859 --> 00:01:15.269
to his teaching that he does to his scholarship,

19
00:01:15.269 --> 00:01:18.864
and he is really an exemplar kind of faculty member

20
00:01:18.864 --> 00:01:23.140
who seamlessly integrates teaching and research.

21
00:01:23.140 --> 00:01:26.638
So it's my pleasure to introduce Dr. Sameer Khuller,

22
00:01:26.638 --> 00:01:29.809
the chair of the Computer Science department and also

23
00:01:29.809 --> 00:01:33.976
a Distinguished Scholar Teacher to introduce Prof Hicks.

24
00:01:34.862 --> 00:01:37.529
(loud applause)

25
00:01:41.559 --> 00:01:44.333
<v ->This is really a real honor for me to introduce</v>

26
00:01:44.333 --> 00:01:45.743
Michael Hicks.

27
00:01:45.743 --> 00:01:48.437
Mike got his PhD at University of Pennsylvania

28
00:01:48.437 --> 00:01:52.604
in 2001 where he won the SIGPLAN Dissertation Prize

29
00:01:53.477 --> 00:01:55.058
and he is now a

30
00:01:55.058 --> 00:01:57.996
Full Professor in Computer Science in UMIACS.

31
00:01:57.996 --> 00:02:00.103
He was also the inaugural Director of the

32
00:02:00.103 --> 00:02:02.195
Maryland Cybersecurity Center.

33
00:02:02.195 --> 00:02:04.471
He's done groundbreaking research in programming language

34
00:02:04.471 --> 00:02:07.582
security, which he's gonna tell us about today.

35
00:02:07.582 --> 00:02:10.036
One of the main things that struck me about Michael

36
00:02:10.036 --> 00:02:12.431
is when I first met him when he interviewed here,

37
00:02:12.431 --> 00:02:15.765
was just the passion by which he talked about his work,

38
00:02:15.765 --> 00:02:19.285
and he explained to me how they could upgrade software

39
00:02:19.285 --> 00:02:21.302
without ever shutting the machine down,

40
00:02:21.302 --> 00:02:23.710
which at that time we were (mumbles)

41
00:02:23.710 --> 00:02:27.877
Along with advising PhD students, mentoring undergraduates,

42
00:02:30.140 --> 00:02:31.900
doing all of the regular teaching where he routinely

43
00:02:31.900 --> 00:02:35.100
(mumbles) the highest teaching evaluations

44
00:02:35.100 --> 00:02:36.370
from our students.

45
00:02:36.370 --> 00:02:40.405
He also launched a group in Software Security last year

46
00:02:40.405 --> 00:02:43.330
and recently ran the Build and Break-in Contest,

47
00:02:43.330 --> 00:02:44.762
and if we all behave ourselves,

48
00:02:44.762 --> 00:02:46.480
he might even put on a drum performance

49
00:02:46.480 --> 00:02:47.569
at the end of the talk.

50
00:02:47.569 --> 00:02:48.402
(loud laughter)

51
00:02:48.402 --> 00:02:49.444
Michael.

52
00:02:49.444 --> 00:02:52.111
(loud applause)

53
00:03:01.167 --> 00:03:05.397
<v ->Thank you very much, I'm really glad to be here.</v>

54
00:03:05.397 --> 00:03:10.054
So today, I'm gonna tell you about some of my research

55
00:03:10.054 --> 00:03:12.679
and my teaching in a nutshell

56
00:03:12.679 --> 00:03:15.673
from Penetrate and Patch to Building Security In

57
00:03:15.673 --> 00:03:18.207
and if that title doesn't make sense to you yet,

58
00:03:18.207 --> 00:03:21.624
hopefully it will by the end of the talk.

59
00:03:23.898 --> 00:03:25.385
We all know about security breaches,

60
00:03:25.385 --> 00:03:29.146
we've probably personally experienced the outcome of

61
00:03:29.146 --> 00:03:31.441
the University of Maryland breach from a year ago,

62
00:03:31.441 --> 00:03:34.639
where our personal information was stolen by hackers.

63
00:03:34.639 --> 00:03:36.293
We're not alone.

64
00:03:36.293 --> 00:03:40.460
Many, many other companies have unfortunately had attacks,

65
00:03:41.678 --> 00:03:45.964
and peoples' credit card numbers, social security numbers,

66
00:03:45.964 --> 00:03:48.696
other personal information, even all of the details of their

67
00:03:48.696 --> 00:03:52.917
security clearances in case of OPM have been stolen away.

68
00:03:52.917 --> 00:03:56.718
So this is a frightening state of affairs.

69
00:03:56.718 --> 00:04:00.330
Now many if not all of these breaches started with what's

70
00:04:00.330 --> 00:04:03.729
called the Vulnerability and Defects in a software system

71
00:04:03.729 --> 00:04:06.024
that these organizations were running

72
00:04:06.024 --> 00:04:09.024
that attackers were able to exploit.

73
00:04:11.824 --> 00:04:16.345
These design flaws, it's not surprising that a complicated

74
00:04:16.345 --> 00:04:19.395
system would have mistakes in them,

75
00:04:19.395 --> 00:04:21.075
but the interesting thing is that those mistakes

76
00:04:21.075 --> 00:04:24.358
can be twisted by a clever attacker

77
00:04:24.358 --> 00:04:27.570
to make the system do something it shouldn't,

78
00:04:27.570 --> 00:04:28.802
so that they can gain access

79
00:04:28.802 --> 00:04:31.150
and maybe steal all of those records.

80
00:04:31.150 --> 00:04:34.894
Now the amount of code in software in the world is growing

81
00:04:34.894 --> 00:04:36.538
quite a bit, so there's an article recently

82
00:04:36.538 --> 00:04:40.959
that Google has about two billion lines of programming code

83
00:04:40.959 --> 00:04:42.901
in their core services.

84
00:04:42.901 --> 00:04:45.376
Microsoft Windows has 50 million lines of code,

85
00:04:45.376 --> 00:04:48.232
and if we think that computer systems that software

86
00:04:48.232 --> 00:04:52.987
is making its way into things like cars and smart meters

87
00:04:52.987 --> 00:04:56.937
and even your coffee machine, we know that the proliferation

88
00:04:56.937 --> 00:05:00.229
of code and proliferation of places that it appears,

89
00:05:00.229 --> 00:05:02.533
that maybe we're becoming more and more vulnerable

90
00:05:02.533 --> 00:05:04.700
to these sorts of attacks.

91
00:05:06.620 --> 00:05:09.603
Just as a couple of examples, you probably are all familiar

92
00:05:09.603 --> 00:05:13.931
with Stuxnet, that was something back in I think 2011,

93
00:05:13.931 --> 00:05:17.270
where surprise-surprise, nuclear centrifuges run software

94
00:05:17.270 --> 00:05:20.257
and that software is vulnerable to attack and in this case,

95
00:05:20.257 --> 00:05:23.611
the attack exploited a bug in some of that software,

96
00:05:23.611 --> 00:05:26.711
that caused the centrifuge to blow up on itself.

97
00:05:26.711 --> 00:05:30.249
Another example, more recently a hacker showed that

98
00:05:30.249 --> 00:05:34.582
they could hack into a car while it was moving using the

99
00:05:34.582 --> 00:05:39.127
on-start or remote access capabilities of the car.

100
00:05:39.127 --> 00:05:41.572
And sadly, these are not the only examples,

101
00:05:41.572 --> 00:05:43.468
there are more.

102
00:05:43.468 --> 00:05:46.228
So as I was starting to say before, if you just think,

103
00:05:46.228 --> 00:05:47.921
hey software systems are complicated,

104
00:05:47.921 --> 00:05:49.813
50 million lines of code, of course there's gonna be

105
00:05:49.813 --> 00:05:51.313
mistakes in those.

106
00:05:53.260 --> 00:05:54.864
Why weren't we in trouble before?

107
00:05:54.864 --> 00:05:58.958
Why is it that there's this in 2014, 2013, 2015

108
00:05:58.958 --> 00:06:02.875
we seem to have so many really amazing attacks?

109
00:06:03.893 --> 00:06:07.254
I look at it this way, a normal user, if you are working

110
00:06:07.254 --> 00:06:09.954
with a software system, with your word-processor,

111
00:06:09.954 --> 00:06:12.144
and you realize that you have two windows open.

112
00:06:12.144 --> 00:06:15.014
It crashes, what do you do?

113
00:06:15.014 --> 00:06:18.319
You don't have two windows open anymore, you work around it.

114
00:06:18.319 --> 00:06:21.250
If you're a normal user, you probably don't use 70% of the

115
00:06:21.250 --> 00:06:24.679
features of your word processor, and so weird bugs that

116
00:06:24.679 --> 00:06:26.376
combine from this feature and this feature,

117
00:06:26.376 --> 00:06:27.942
you never even see.

118
00:06:27.942 --> 00:06:30.515
But that's very different than what an attacker is gonna

119
00:06:30.515 --> 00:06:33.687
try to do, instead of trying to work around it,

120
00:06:33.687 --> 00:06:35.821
they are going to try to exploit it.

121
00:06:35.821 --> 00:06:38.879
They are going to look for all the weird ways that that

122
00:06:38.879 --> 00:06:42.903
software was not tested in order to find a bug that was not

123
00:06:42.903 --> 00:06:45.344
considered by the developers and they are gonna try to

124
00:06:45.344 --> 00:06:48.697
figure out a way to not make it crash the way you might

125
00:06:48.697 --> 00:06:51.455
experience it, but instead to bend that software to their

126
00:06:51.455 --> 00:06:53.839
will as it were to get it to do something

127
00:06:53.839 --> 00:06:56.052
that it shouldn't do.

128
00:06:56.052 --> 00:06:59.602
So we all know about computer security companies

129
00:06:59.602 --> 00:07:01.983
that provide products that will hopefully make us more

130
00:07:01.983 --> 00:07:06.167
secure and they surely do, but they are not the solution

131
00:07:06.167 --> 00:07:07.726
to the problem.

132
00:07:07.726 --> 00:07:09.609
In my opinion, at least the way that they are built now,

133
00:07:09.609 --> 00:07:11.743
they will never be the solution to the problem.

134
00:07:11.743 --> 00:07:15.266
And that's because what these products do is they go after

135
00:07:15.266 --> 00:07:17.895
the attacks themselves, they do not go after

136
00:07:17.895 --> 00:07:19.910
the vulnerabilities, the defects in the systems

137
00:07:19.910 --> 00:07:21.315
in the first place.

138
00:07:21.315 --> 00:07:25.482
So the way that an attacker exploits a bug is to generate

139
00:07:26.578 --> 00:07:29.854
some input, you will see this later in my talk that causes

140
00:07:29.854 --> 00:07:32.113
that bug to do the wrong thing, and what these products do

141
00:07:32.113 --> 00:07:34.593
is they try to stop the attacks but they leave the

142
00:07:34.593 --> 00:07:37.439
vulnerabilities along, and what that means is that

143
00:07:37.439 --> 00:07:39.977
a clever attacker, in fact you don't even have to be that

144
00:07:39.977 --> 00:07:43.722
clever anymore, it can be automated, can work around

145
00:07:43.722 --> 00:07:47.295
the defense that these companies provide with their systems

146
00:07:47.295 --> 00:07:50.633
to create new variants of the attack.

147
00:07:50.633 --> 00:07:55.468
That makes it wackable, you knock down one attack,

148
00:07:55.468 --> 00:07:57.406
but then another one pops up because the underlying

149
00:07:57.406 --> 00:07:59.073
flaw is still there.

150
00:08:01.080 --> 00:08:03.817
This leads to a regime of what I call penetrate and patch.

151
00:08:03.817 --> 00:08:04.889
So what does that mean?

152
00:08:04.889 --> 00:08:08.560
It means that some way or another we find a vulnerability,

153
00:08:08.560 --> 00:08:11.484
maybe the developers find it, maybe somebody else finds it,

154
00:08:11.484 --> 00:08:15.155
and then the developers create a patch, so they realize,

155
00:08:15.155 --> 00:08:16.517
"oh yes, that's a mistake in the code,

156
00:08:16.517 --> 00:08:18.985
I'm gonna fix the code," and then they disseminate that

157
00:08:18.985 --> 00:08:21.913
patch out to the software that we run, okay great,

158
00:08:21.913 --> 00:08:25.084
so now those vulnerabilities can't be hacked any more,

159
00:08:25.084 --> 00:08:26.988
but of course, the problem is they're still vulnerabilities

160
00:08:26.988 --> 00:08:29.670
remaining in the software that have yet to be found

161
00:08:29.670 --> 00:08:31.897
by the developers and for that matter,

162
00:08:31.897 --> 00:08:33.513
software never stays still.

163
00:08:33.513 --> 00:08:37.959
When's the last time you used Microsoft Word 2008,

164
00:08:37.959 --> 00:08:39.890
you probably don't anymore, there's new versions that have

165
00:08:39.890 --> 00:08:40.810
come out since then.

166
00:08:40.810 --> 00:08:43.762
Because new versions come, new code and new mistakes

167
00:08:43.762 --> 00:08:46.944
that are there to be exploited.

168
00:08:46.944 --> 00:08:49.170
And of course, there's also the problem that the security

169
00:08:49.170 --> 00:08:52.198
products that you hope will keep you secure are themselves

170
00:08:52.198 --> 00:08:56.054
the source of attack, so even worse is you're running

171
00:08:56.054 --> 00:08:59.204
an anti-virus program that is so big and complicated now

172
00:08:59.204 --> 00:09:02.786
that it has bugs that can be exploited and now the attackers

173
00:09:02.786 --> 00:09:05.377
make it seem like you're safe because you're blocking

174
00:09:05.377 --> 00:09:08.150
all these signatures, but in fact, they're making that

175
00:09:08.150 --> 00:09:10.733
software do quite the opposite.

176
00:09:11.904 --> 00:09:15.097
What do we do to solve this problem?

177
00:09:15.097 --> 00:09:17.710
We need to make it so that these sorts of bugs that are

178
00:09:17.710 --> 00:09:20.493
exploitable, not bugs that are just inconvenient

179
00:09:20.493 --> 00:09:23.015
and irritating to users, but ones that are really security

180
00:09:23.015 --> 00:09:27.028
relevant just aren't there right from the beginning.

181
00:09:27.028 --> 00:09:30.708
We need to get away from penetrate and patch and we need

182
00:09:30.708 --> 00:09:33.558
a way to build software that doesn't have those holes in it

183
00:09:33.558 --> 00:09:34.391
in the first place.

184
00:09:34.391 --> 00:09:38.086
No holes for those holes to be perfect.

185
00:09:38.086 --> 00:09:41.628
So to make an analogy, let's think about how

186
00:09:41.628 --> 00:09:43.369
we learned how to build bridges.

187
00:09:43.369 --> 00:09:45.337
If you're going to build a bridge, you're going to

188
00:09:45.337 --> 00:09:48.234
anticipate all of the ways that that bridge is gonna be

189
00:09:48.234 --> 00:09:52.305
put to the test, all of the foot traffic, the car traffic,

190
00:09:52.305 --> 00:09:54.655
maybe you're in an earthquake zone, maybe you have extreme

191
00:09:54.655 --> 00:09:57.736
weather like lots of rain or earthquakes or things like that

192
00:09:57.736 --> 00:10:00.168
and you're gonna develop methods that allow you to build

193
00:10:00.168 --> 00:10:03.003
bridges that are gonna stand up to all of that.

194
00:10:03.003 --> 00:10:04.586
So how do you that?

195
00:10:05.902 --> 00:10:08.156
Well you're gonna study the problem and you're gonna develop

196
00:10:08.156 --> 00:10:10.019
the best methods for building bridges.

197
00:10:10.019 --> 00:10:13.646
You're gonna study mechanics and develop insights

198
00:10:13.646 --> 00:10:14.736
from physics.

199
00:10:14.736 --> 00:10:17.436
You're gonna develop the best materials that are likely to

200
00:10:17.436 --> 00:10:20.443
stand up to those earthquakes or to that weather.

201
00:10:20.443 --> 00:10:22.452
You're gonna build better tools that make it so that

202
00:10:22.452 --> 00:10:25.177
when you construct those bridges, you do it reliably,

203
00:10:25.177 --> 00:10:27.633
and keep an error where other sorts of errors won't creep

204
00:10:27.633 --> 00:10:30.110
in and create flaws in those bridges.

205
00:10:30.110 --> 00:10:32.077
And you're gonna do that right from the very beginning,

206
00:10:32.077 --> 00:10:34.460
so that when you've had your outcome, it's gonna last for

207
00:10:34.460 --> 00:10:39.442
a while, it's not gonna fall over 30 days after it deploys.

208
00:10:39.442 --> 00:10:43.609
You're not going to fail to incorporate larger lessons

209
00:10:44.805 --> 00:10:47.342
from bridges that fell down before.

210
00:10:47.342 --> 00:10:51.457
You're not gonna use, I hope not, cheap materials that

211
00:10:51.457 --> 00:10:54.119
aren't gonna stand up to the conditions that you expect.

212
00:10:54.119 --> 00:10:56.856
You're not gonna use unreliable tools.

213
00:10:56.856 --> 00:10:58.414
And you're not gonna say "we'll just put it up there and

214
00:10:58.414 --> 00:11:00.547
we'll hope to patch it later."

215
00:11:00.547 --> 00:11:02.248
Unfortunately, this is a lot of the way

216
00:11:02.248 --> 00:11:05.371
software development is today.

217
00:11:05.371 --> 00:11:06.703
We don't want our bridges to fall down,

218
00:11:06.703 --> 00:11:08.522
we don't want our software to be attacked,

219
00:11:08.522 --> 00:11:11.687
and so we need to apply this methodology to software.

220
00:11:11.687 --> 00:11:13.584
So how do we do it?

221
00:11:13.584 --> 00:11:15.400
Well we need to take the same mentality.

222
00:11:15.400 --> 00:11:18.489
We need to think from day one Securities and Problem.

223
00:11:18.489 --> 00:11:20.567
We don't think from day one I need to build something

224
00:11:20.567 --> 00:11:22.403
that my customers will buy and eventually I'll think

225
00:11:22.403 --> 00:11:24.842
about security, no I need to think about it right from

226
00:11:24.842 --> 00:11:28.509
the start, I need to put it into my design, I need to think

227
00:11:28.509 --> 00:11:31.569
about the threats my software is gonna face,

228
00:11:31.569 --> 00:11:33.874
likely and unlikely and I need to develop

229
00:11:33.874 --> 00:11:35.603
the best tools and methods

230
00:11:35.603 --> 00:11:37.797
so for software those tools are things like programming

231
00:11:37.797 --> 00:11:41.033
languages, they are development processes, they are test

232
00:11:41.033 --> 00:11:44.311
and verification technologies, so again if these buzzwords

233
00:11:44.311 --> 00:11:45.175
don't make sense to you,

234
00:11:45.175 --> 00:11:48.758
hopefully they will by the end of the talk.

235
00:11:49.955 --> 00:11:52.871
Now, all of this hopefully sounds obvious to you.

236
00:11:52.871 --> 00:11:54.261
Well of course, if you're gonna build a bridge,

237
00:11:54.261 --> 00:11:55.409
you're gonna do all those things.

238
00:11:55.409 --> 00:11:57.077
Of course, if you're gonna build good software,

239
00:11:57.077 --> 00:11:58.314
you're gonna do those things.

240
00:11:58.314 --> 00:12:00.231
Why don't people do it?

241
00:12:01.618 --> 00:12:03.884
People have been building bridges for many, many more

242
00:12:03.884 --> 00:12:06.567
millennia than building software, so there is

243
00:12:06.567 --> 00:12:10.234
some amount of ignorance about what needs to be done.

244
00:12:10.234 --> 00:12:12.936
It's also the case that we haven't developed the best tools

245
00:12:12.936 --> 00:12:16.287
yet, some of the tools we have don't actually work that well

246
00:12:16.287 --> 00:12:18.478
or they could at least work better, or maybe people

247
00:12:18.478 --> 00:12:21.513
just aren't convinced that they work that well.

248
00:12:21.513 --> 00:12:23.338
There's also this big issue of cost.

249
00:12:23.338 --> 00:12:24.550
We've written a lot of software,

250
00:12:24.550 --> 00:12:26.943
Google has 2 billion lines of code already.

251
00:12:26.943 --> 00:12:28.930
It would be unfeasible to take all the code,

252
00:12:28.930 --> 00:12:32.061
all the programs you already have and say "oh we need to

253
00:12:32.061 --> 00:12:34.494
completely throw them away and start over again,"

254
00:12:34.494 --> 00:12:35.757
right that doesn't make any sense.

255
00:12:35.757 --> 00:12:37.863
And there's also the cost of retraining our people

256
00:12:37.863 --> 00:12:39.135
to do that kind of thing.

257
00:12:39.135 --> 00:12:44.030
So there are legitimate barriers to breaching this mentality

258
00:12:44.030 --> 00:12:47.559
but what we as academics can do, what I've been trying to do

259
00:12:47.559 --> 00:12:50.439
as a researcher as an educator is to address some of these

260
00:12:50.439 --> 00:12:53.972
problems and hopefully get us to a place where we are

261
00:12:53.972 --> 00:12:57.100
as good at building software as we are at building bridges.

262
00:12:57.100 --> 00:12:59.061
One thing I'll tell you a little bit about is a new

263
00:12:59.061 --> 00:13:02.130
programming language that I co-developed called Cyclone,

264
00:13:02.130 --> 00:13:05.430
who's goal is to make many of those vulnerabilities

265
00:13:05.430 --> 00:13:09.087
that are being exploited today impossible by construction.

266
00:13:09.087 --> 00:13:10.408
If you write your program in Cyclone,

267
00:13:10.408 --> 00:13:13.414
it will be invulnerable to a large majority of attacks

268
00:13:13.414 --> 00:13:17.275
that are unfortunately successful today.

269
00:13:17.275 --> 00:13:18.893
Another thing I've been trying to do is address

270
00:13:18.893 --> 00:13:21.855
that ignorance side of things and to get the word out

271
00:13:21.855 --> 00:13:24.152
about building software the right way,

272
00:13:24.152 --> 00:13:27.117
so as we were mentioning I am teaching a Coursera

273
00:13:27.117 --> 00:13:29.916
a massively open online course on Software Security

274
00:13:29.916 --> 00:13:32.448
where I try to get across some of these points

275
00:13:32.448 --> 00:13:37.098
and I've also co-developed a contest that tries to emphasize

276
00:13:37.098 --> 00:13:40.245
the building software securely part and not just

277
00:13:40.245 --> 00:13:44.162
where are the holes, let's penetrate and patch.

278
00:13:45.774 --> 00:13:47.917
So that's the overview of my talk.

279
00:13:47.917 --> 00:13:50.096
What I'm gonna go into now is I'm gonna try to dig in a

280
00:13:50.096 --> 00:13:52.718
little bit for those of you who are not so technically savvy

281
00:13:52.718 --> 00:13:54.587
or don't know so much about software.

282
00:13:54.587 --> 00:13:57.475
How do bugs get exploited?

283
00:13:57.475 --> 00:13:59.558
How does this come to be?

284
00:14:01.351 --> 00:14:03.635
So if we step back, what is a computer?

285
00:14:03.635 --> 00:14:07.802
A computer has a processor in it, your x86, your 686,

286
00:14:08.820 --> 00:14:10.168
I don't even know what the most recent version

287
00:14:10.168 --> 00:14:12.057
is of an Intel processor

288
00:14:12.057 --> 00:14:13.591
that's running inside of your machine

289
00:14:13.591 --> 00:14:16.918
and it runs machine instructions

290
00:14:16.918 --> 00:14:20.069
and those instructions are called program.

291
00:14:20.069 --> 00:14:22.450
For some purpose, they're a word processing program

292
00:14:22.450 --> 00:14:24.855
and operating systems.

293
00:14:24.855 --> 00:14:27.245
These instructions are kept in memory, in RAM

294
00:14:27.245 --> 00:14:30.029
and the processor is going to fetch those instructions

295
00:14:30.029 --> 00:14:33.399
and execute them and it's gonna also read data from memory

296
00:14:33.399 --> 00:14:35.482
and write that data back.

297
00:14:36.775 --> 00:14:39.768
So just to give you an example of what software looks like,

298
00:14:39.768 --> 00:14:41.921
what programs look like, let's look at a very simple

299
00:14:41.921 --> 00:14:44.872
function that you can compute, which is the exponentiation.

300
00:14:44.872 --> 00:14:48.031
So I wanna go and compute X to the Y power

301
00:14:48.031 --> 00:14:52.159
and I'm gonna store the results in this variable R.

302
00:14:52.159 --> 00:14:55.303
So let's take it at a distance, if someone came up to you

303
00:14:55.303 --> 00:14:58.719
and said "how would you compute, if I said what's seven to

304
00:14:58.719 --> 00:15:00.642
the third power, what would you do?"

305
00:15:00.642 --> 00:15:02.960
Well the first thing you would do is you'd think

306
00:15:02.960 --> 00:15:04.002
well what does it mean?

307
00:15:04.002 --> 00:15:06.853
Well it means I'm gonna multiply X by itself Y times.

308
00:15:06.853 --> 00:15:08.973
So seven to the third is the same as seven times seven

309
00:15:08.973 --> 00:15:11.890
times seven even so in our program,

310
00:15:13.128 --> 00:15:15.577
we're gonna store that final result seven times seven times

311
00:15:15.577 --> 00:15:20.219
seven in this variable R which will be stored in memory.

312
00:15:20.219 --> 00:15:22.837
R is just the name for that location remember

313
00:15:22.837 --> 00:15:24.462
where we store the result.

314
00:15:24.462 --> 00:15:26.658
And we're gonna use a counter to keep track of

315
00:15:26.658 --> 00:15:28.236
how many multiplications we've done.

316
00:15:28.236 --> 00:15:30.770
So we'll start off with one and then we'll multiply it by

317
00:15:30.770 --> 00:15:33.879
seven, now it's seven, we multiply it by seven again,

318
00:15:33.879 --> 00:15:36.128
and now it's 49 and we multiply it by seven again,

319
00:15:36.128 --> 00:15:38.316
and it's whatever 49 times seven is,

320
00:15:38.316 --> 00:15:39.654
we should've picked a different example.

321
00:15:39.654 --> 00:15:41.353
(soft laughter)

322
00:15:41.353 --> 00:15:42.756
So it's just like counting on your finger,

323
00:15:42.756 --> 00:15:44.223
so the computer is just gonna keep track,

324
00:15:44.223 --> 00:15:45.967
how many times have I done this multiplication?

325
00:15:45.967 --> 00:15:47.743
My program is gonna have to do that.

326
00:15:47.743 --> 00:15:51.154
But in the end, R is gonna have the final result.

327
00:15:51.154 --> 00:15:54.530
Okay, here's what some instructions, computer-like

328
00:15:54.530 --> 00:15:58.153
instructions that have been made a little bit simpler,

329
00:15:58.153 --> 00:16:00.488
look like to compute that result.

330
00:16:00.488 --> 00:16:02.644
So the first thing that they do, you can see here that

331
00:16:02.644 --> 00:16:05.280
there's R which is gonna store the result

332
00:16:05.280 --> 00:16:09.503
and C which is the counter variable we were talking about,

333
00:16:09.503 --> 00:16:12.392
and let's just walk through these instructions to see

334
00:16:12.392 --> 00:16:15.602
how it's gonna give us the answer that we want.

335
00:16:15.602 --> 00:16:17.527
So on the left here is the data,

336
00:16:17.527 --> 00:16:20.774
this is the memory locations that are gonna store

337
00:16:20.774 --> 00:16:24.513
both the inputs X and Y, computing X to the Y power,

338
00:16:24.513 --> 00:16:27.684
but then also our result R and this counter variable C

339
00:16:27.684 --> 00:16:31.101
that we need to store it in the meantime.

340
00:16:31.980 --> 00:16:35.094
So let's compute three squared, that one's easier.

341
00:16:35.094 --> 00:16:38.173
So we start by setting R to one, that's the first step,

342
00:16:38.173 --> 00:16:42.639
so you notice now in the lower left, R is set to one.

343
00:16:42.639 --> 00:16:44.930
Next, we're going to set C to Y,

344
00:16:44.930 --> 00:16:48.155
so our counter variable starts off as Y, because we wanna do

345
00:16:48.155 --> 00:16:51.044
Y multiplications, so the counter will keep track how many

346
00:16:51.044 --> 00:16:55.014
multiplications do we have left still to do.

347
00:16:55.014 --> 00:16:56.693
Okay, is C less than or equal to zero?

348
00:16:56.693 --> 00:17:00.030
No you still have multiplications to do so we'll proceed

349
00:17:00.030 --> 00:17:04.512
on by multiplying X times R and storing the results in R,

350
00:17:04.512 --> 00:17:06.667
so R used to be one, we multiplied it by X,

351
00:17:06.667 --> 00:17:10.649
that's three times one is three, so now R has three in it.

352
00:17:10.649 --> 00:17:12.465
Okay, we've just done one multiplication, so they can

353
00:17:12.465 --> 00:17:14.240
subtract one from our counter variable,

354
00:17:14.240 --> 00:17:15.990
it's now down to one.

355
00:17:18.983 --> 00:17:22.804
C is not zero yet, so we need to continue,

356
00:17:22.804 --> 00:17:26.384
we'll do another multiplication, so now R is nine,

357
00:17:26.384 --> 00:17:29.496
we'll subtract one from C, C is now zero

358
00:17:29.496 --> 00:17:31.998
so when we get to this final spot and finish,

359
00:17:31.998 --> 00:17:34.439
and now we can see we have X at the top is three,

360
00:17:34.439 --> 00:17:38.077
Y is two, the result R is nine, three squared is nine.

361
00:17:38.077 --> 00:17:39.937
Okay, so one step at a time, it's like a recipe,

362
00:17:39.937 --> 00:17:42.764
when you follow a recipe when you're making a cake.

363
00:17:42.764 --> 00:17:45.742
Put the eggs in, put the flour in, put the batter in,

364
00:17:45.742 --> 00:17:47.949
stir it up, at the end you get a cake.

365
00:17:47.949 --> 00:17:50.017
Here when we followed these instructions, at the end,

366
00:17:50.017 --> 00:17:52.934
we got three squared which is nine.

367
00:17:55.574 --> 00:17:59.018
Okay so what I should do on the right is a little bit more

368
00:17:59.018 --> 00:18:03.040
human readable, on the left are actual machine instructions

369
00:18:03.040 --> 00:18:05.367
that are run by the same processor that's running

370
00:18:05.367 --> 00:18:08.349
that laptop that will compute exactly the same thing.

371
00:18:08.349 --> 00:18:09.996
And they look a little more complicated,

372
00:18:09.996 --> 00:18:14.578
but they're doing exactly what I just showed you.

373
00:18:14.578 --> 00:18:17.827
Nevertheless, we could understand what that six line program

374
00:18:17.827 --> 00:18:20.721
was doing but exponentiation is not a very interesting

375
00:18:20.721 --> 00:18:23.316
program compared to say the software that's running on

376
00:18:23.316 --> 00:18:24.749
my computer now that takes up

377
00:18:24.749 --> 00:18:27.964
millions and millions of lines of code.

378
00:18:27.964 --> 00:18:30.873
It would be really really hard for people to understand

379
00:18:30.873 --> 00:18:34.900
programs that were written with no sorts of instructions

380
00:18:34.900 --> 00:18:36.775
so what do they do about that?

381
00:18:36.775 --> 00:18:39.313
All the way back in the 1950s, people realized that this

382
00:18:39.313 --> 00:18:41.576
was a problem and they invented what are called

383
00:18:41.576 --> 00:18:43.431
high-level programming languages.

384
00:18:43.431 --> 00:18:46.183
So these are ways of expressing programs in language

385
00:18:46.183 --> 00:18:48.161
that's closer to human language.

386
00:18:48.161 --> 00:18:52.020
So it's still arcane, for those not in the know,

387
00:18:52.020 --> 00:18:53.882
those who got computer science degrees maybe

388
00:18:53.882 --> 00:18:57.504
or with some training, but it's much easier to understand

389
00:18:57.504 --> 00:18:59.403
than huge numbers of the instructions

390
00:18:59.403 --> 00:19:00.923
that I just showed you.

391
00:19:00.923 --> 00:19:03.905
So FORTRAN was the first high level computer language

392
00:19:03.905 --> 00:19:08.128
that was invented in 1954 and of course, since then,

393
00:19:08.128 --> 00:19:10.435
there have been many, many more languages.

394
00:19:10.435 --> 00:19:12.357
The way that these high-level languages work is that

395
00:19:12.357 --> 00:19:15.898
when I write my program in FORTRAN, say, another program

396
00:19:15.898 --> 00:19:19.114
called a compiler will translate the FORTRAN program

397
00:19:19.114 --> 00:19:21.210
into the actual machine instructions.

398
00:19:21.210 --> 00:19:22.573
So what I could do is I could write

399
00:19:22.573 --> 00:19:26.046
that exponentiation function in FORTRAN

400
00:19:26.046 --> 00:19:28.014
and then this compiler would convert it into

401
00:19:28.014 --> 00:19:29.769
the machine instructions that we looked at.

402
00:19:29.769 --> 00:19:32.406
So then I as the programmer never have to look at those

403
00:19:32.406 --> 00:19:33.791
machine instructions never again,

404
00:19:33.791 --> 00:19:37.739
I just looked at this FORTRAN program.

405
00:19:37.739 --> 00:19:40.099
Now many different languages, why not just one language?

406
00:19:40.099 --> 00:19:41.752
Well they all have different strengths

407
00:19:41.752 --> 00:19:45.492
and there's quite a lot of them in (mumbles)

408
00:19:45.492 --> 00:19:49.922
So this is a nice chart that O'Reilly put out that shows the

409
00:19:49.922 --> 00:19:53.246
generation of languages, just like Latin is the root of

410
00:19:53.246 --> 00:19:57.289
French and Spanish and Italian, there are old languages

411
00:19:57.289 --> 00:19:59.397
there that are the root of languages that we use today,

412
00:19:59.397 --> 00:20:02.079
there's a long history for these things.

413
00:20:02.079 --> 00:20:04.713
So today some of the more popular languages are listed on

414
00:20:04.713 --> 00:20:08.713
this chart, Java, C, C++, Python, Ruby

415
00:20:11.805 --> 00:20:12.638
and so on.

416
00:20:14.340 --> 00:20:16.995
So here is what our exponentiation program

417
00:20:16.995 --> 00:20:19.017
looks like written in C.

418
00:20:19.017 --> 00:20:21.370
So if you stare at it for a minute, you can see it's not

419
00:20:21.370 --> 00:20:23.205
so different than that set of instructions

420
00:20:23.205 --> 00:20:24.513
that I showed you before.

421
00:20:24.513 --> 00:20:25.847
What happens on the first line?

422
00:20:25.847 --> 00:20:28.261
Well I have this variable R that's gonna keep my result

423
00:20:28.261 --> 00:20:30.347
then I set it to one at the very beginning.

424
00:20:30.347 --> 00:20:32.179
And then I continue to iterate.

425
00:20:32.179 --> 00:20:33.733
I do that pair of instructions

426
00:20:33.733 --> 00:20:37.594
over and over again until eventually the counter which

427
00:20:37.594 --> 00:20:39.953
in this case we've said that we're using as Y gets

428
00:20:39.953 --> 00:20:41.988
down to zero, so it's doing the same sort of thing

429
00:20:41.988 --> 00:20:44.453
that we saw before, but compared to those machine

430
00:20:44.453 --> 00:20:45.774
instructions that I showed you,

431
00:20:45.774 --> 00:20:48.394
it's maybe a little more understandable.

432
00:20:48.394 --> 00:20:52.547
There's another version of the program written in Python.

433
00:20:52.547 --> 00:20:54.964
There's one written in OCaml.

434
00:20:55.895 --> 00:20:58.359
That one looks a little bit different because it's expressed

435
00:20:58.359 --> 00:21:01.631
recursively like you would in a mathematical function.

436
00:21:01.631 --> 00:21:03.826
There it's in Prolog.

437
00:21:03.826 --> 00:21:05.870
So we can see that the way of expressing the same

438
00:21:05.870 --> 00:21:10.223
computation can be very different in different languages.

439
00:21:10.223 --> 00:21:12.637
So let's go back now to this problem of bugs

440
00:21:12.637 --> 00:21:16.221
which eventually could be bugs that are exploitable.

441
00:21:16.221 --> 00:21:19.516
Programmers make mistakes, these defects, these mistakes

442
00:21:19.516 --> 00:21:21.690
that they make are called bugs.

443
00:21:21.690 --> 00:21:24.071
So here's that C program again.

444
00:21:24.071 --> 00:21:27.488
Can anybody spot the bug in this program?

445
00:21:32.326 --> 00:21:34.933
<v Attendee>It should say greater than zero.</v>

446
00:21:34.933 --> 00:21:37.050
<v ->Right, so that should be greater than zero</v>

447
00:21:37.050 --> 00:21:39.510
not greater than or equal to zero.

448
00:21:39.510 --> 00:21:41.602
Now of course we would discover this bug right away

449
00:21:41.602 --> 00:21:42.809
while we were testing it

450
00:21:42.809 --> 00:21:45.509
because we'd get the wrong answer on the first test.

451
00:21:45.509 --> 00:21:47.499
But unfortunately, not all bugs are that way

452
00:21:47.499 --> 00:21:49.726
and many, many bugs escape testing,

453
00:21:49.726 --> 00:21:52.214
and it's very difficult to find the root cause of this bug

454
00:21:52.214 --> 00:21:54.201
sometimes and so they escape and make their way into

455
00:21:54.201 --> 00:21:56.323
software that we use all the time.

456
00:21:56.323 --> 00:21:59.693
Some of those bugs are exploitable so these are bugs

457
00:21:59.693 --> 00:22:03.241
in which rather than when you run across that bug,

458
00:22:03.241 --> 00:22:05.262
you just get the wrong answer or maybe your program

459
00:22:05.262 --> 00:22:09.331
crashes, the way that you make that bug happen,

460
00:22:09.331 --> 00:22:11.556
if you're an attacker can cause the program to do

461
00:22:11.556 --> 00:22:13.891
something that you want that was not at all intended

462
00:22:13.891 --> 00:22:15.442
by the original designer.

463
00:22:15.442 --> 00:22:17.544
And I'm gonna give you a flavor of that by showing you

464
00:22:17.544 --> 00:22:21.958
a very old and unfortunately still relevant bug called

465
00:22:21.958 --> 00:22:23.625
the buffer overflow.

466
00:22:25.083 --> 00:22:28.341
So a buffer overflow is a problem that plagues programs

467
00:22:28.341 --> 00:22:31.869
written in C and C++, so you saw that those were two

468
00:22:31.869 --> 00:22:35.022
of those in the top five languages that are used today.

469
00:22:35.022 --> 00:22:39.189
And normally a program with this kind of bug would crash

470
00:22:40.036 --> 00:22:43.158
but unfortunately, adversaries were able to exploit this

471
00:22:43.158 --> 00:22:46.034
bug to steal private information, we talked about that

472
00:22:46.034 --> 00:22:48.717
at the beginning, to making corrupt information,

473
00:22:48.717 --> 00:22:51.607
to damage it to make it look like something it's not,

474
00:22:51.607 --> 00:22:55.318
or even to install code remotely that runs on your machine

475
00:22:55.318 --> 00:22:57.749
for example, without your knowledge, your own computer

476
00:22:57.749 --> 00:23:00.146
is sending spam to people on the internet

477
00:23:00.146 --> 00:23:02.098
and you didn't even know that was happening.

478
00:23:02.098 --> 00:23:05.212
Some adversary exploited a bug and say your web browser

479
00:23:05.212 --> 00:23:08.104
got that software installed, and now unknowingly

480
00:23:08.104 --> 00:23:12.094
you're participating in nefarious activities.

481
00:23:12.094 --> 00:23:14.909
So from 10,000 feet, a buffer overflow, what's a buffer?

482
00:23:14.909 --> 00:23:17.552
It's just a section of memory in your computer,

483
00:23:17.552 --> 00:23:21.719
and overflow means the program tries to put more information

484
00:23:22.837 --> 00:23:25.317
in that memory than it can actually hold.

485
00:23:25.317 --> 00:23:27.101
And so the question is what happens when you overflow

486
00:23:27.101 --> 00:23:31.597
that buffer, where does that extra data actually go?

487
00:23:31.597 --> 00:23:34.466
So let's look at an example, and I'll try to illustrate

488
00:23:34.466 --> 00:23:38.338
for you how a buffer overflow can make bad things happen.

489
00:23:38.338 --> 00:23:40.830
So here on grey are the instructions.

490
00:23:40.830 --> 00:23:43.165
The instructions are much higher level now but it's the

491
00:23:43.165 --> 00:23:45.915
same idea as the instructions we saw before,

492
00:23:45.915 --> 00:23:48.384
do this, do this, do this, do this.

493
00:23:48.384 --> 00:23:50.880
On the left, we have our data and then I have a screen

494
00:23:50.880 --> 00:23:52.796
there to show you what the input and output

495
00:23:52.796 --> 00:23:54.791
of this program might be.

496
00:23:54.791 --> 00:23:57.284
So we go to our first instruction and it prints on the

497
00:23:57.284 --> 00:24:00.810
screen password so it's prompting you for your password.

498
00:24:00.810 --> 00:24:04.434
The next instruction then asks is waiting for new user

499
00:24:04.434 --> 00:24:06.145
to type in your password.

500
00:24:06.145 --> 00:24:08.140
So let's say that I type it in,

501
00:24:08.140 --> 00:24:11.188
and then it's then read into that memory location.

502
00:24:11.188 --> 00:24:13.392
How many people think that's a good password?

503
00:24:13.392 --> 00:24:14.349
(soft laughter)

504
00:24:14.349 --> 00:24:16.002
Not a good password.

505
00:24:16.002 --> 00:24:18.288
Okay, so I typed in the password.

506
00:24:18.288 --> 00:24:20.815
Let's say for example, that that password doesn't match.

507
00:24:20.815 --> 00:24:23.922
It's not the correct password and it's not my password

508
00:24:23.922 --> 00:24:26.494
and so that takes me to the next instruction which says

509
00:24:26.494 --> 00:24:28.345
"okay, I'm gonna print fail on the screen,

510
00:24:28.345 --> 00:24:31.580
you're not allowed to log in."

511
00:24:31.580 --> 00:24:34.426
Okay, now let's suppose that this program has a buffer

512
00:24:34.426 --> 00:24:38.997
overflow where it fails, we can see, it's failing to check

513
00:24:38.997 --> 00:24:42.310
that the input that's typed in is not too big to be

514
00:24:42.310 --> 00:24:46.056
contained in the space allocated to the variable X, right?

515
00:24:46.056 --> 00:24:47.698
Maybe the variable X only can

516
00:24:47.698 --> 00:24:49.754
contain 12 character passwords.

517
00:24:49.754 --> 00:24:53.148
But we type in a password that's bigger than 12 characters.

518
00:24:53.148 --> 00:24:54.414
Okay, so let's see what happens.

519
00:24:54.414 --> 00:24:58.378
So I type in, the password prompt comes up again,

520
00:24:58.378 --> 00:25:00.837
and now I'm gonna type in my password.

521
00:25:00.837 --> 00:25:02.492
And I'm gonna type in a long password,

522
00:25:02.492 --> 00:25:04.859
and I'm gonna be very clever cause I'm an attacker,

523
00:25:04.859 --> 00:25:06.609
about what I type in.

524
00:25:10.437 --> 00:25:12.894
Okay, so now I've typed in the password in the program,

525
00:25:12.894 --> 00:25:17.562
the computer says okay, now I go to my next step.

526
00:25:17.562 --> 00:25:18.395
Logged in.

527
00:25:19.539 --> 00:25:20.908
Right, what happened there?

528
00:25:20.908 --> 00:25:24.407
What happened is the attacker exploited the idea

529
00:25:24.407 --> 00:25:27.903
that both data and programs are in memory.

530
00:25:27.903 --> 00:25:29.312
Remember what I said at the very beginning

531
00:25:29.312 --> 00:25:31.595
when I showed you the computer system.

532
00:25:31.595 --> 00:25:33.857
The processor is executing instructions.

533
00:25:33.857 --> 00:25:35.481
Those instructions live in memory.

534
00:25:35.481 --> 00:25:37.218
The data also lives in memory.

535
00:25:37.218 --> 00:25:39.596
So what the attacker is able to do in some cases is

536
00:25:39.596 --> 00:25:43.166
when they overflow a buffer in memory for data,

537
00:25:43.166 --> 00:25:46.596
they can actually corrupt code and they can therefore

538
00:25:46.596 --> 00:25:49.393
cause a program to execute code that they the attacker

539
00:25:49.393 --> 00:25:51.975
provided rather than the code that was intended

540
00:25:51.975 --> 00:25:54.003
by the software developer.

541
00:25:54.003 --> 00:25:57.492
So the first instance of this attack that gave it a name

542
00:25:57.492 --> 00:26:01.482
was perpetrated in 1988, it was called Porous Worm,

543
00:26:01.482 --> 00:26:05.595
and it was the first big breach on the internet

544
00:26:05.595 --> 00:26:10.577
and it brought to light this main attack which was

545
00:26:10.577 --> 00:26:14.744
and has been a mainstay of computer hackers ever since.

546
00:26:15.888 --> 00:26:18.195
So the key idea here what happened?

547
00:26:18.195 --> 00:26:21.308
Something that we intended to be data, me the programmer,

548
00:26:21.308 --> 00:26:23.825
I was just waiting for you to type in your password,

549
00:26:23.825 --> 00:26:25.698
the attacker was able to exploit the bug

550
00:26:25.698 --> 00:26:29.523
that was in the program to confuse that data as code,

551
00:26:29.523 --> 00:26:31.717
and then therefore, get the program to run code

552
00:26:31.717 --> 00:26:34.355
of the attacker's choice and therefore get the program

553
00:26:34.355 --> 00:26:36.329
to do what the attacker wanted it to do,

554
00:26:36.329 --> 00:26:39.088
and not what it was supposed to do.

555
00:26:39.088 --> 00:26:41.408
And it turns out that other exploits that I haven't talked

556
00:26:41.408 --> 00:26:44.953
about, one's called SQL injection or cross-site scripting

557
00:26:44.953 --> 00:26:49.270
or command injection, many attacks that are in fact

558
00:26:49.270 --> 00:26:51.725
maybe even more prominent now than buffer overflows

559
00:26:51.725 --> 00:26:53.680
basically exploit the same problem

560
00:26:53.680 --> 00:26:57.226
that you type in data, you type in information

561
00:26:57.226 --> 00:26:59.877
as an attacker that the program intends to be data,

562
00:26:59.877 --> 00:27:02.805
but it fails to really check with that data,

563
00:27:02.805 --> 00:27:05.609
doesn't know is it too big, or does it have the wrong format

564
00:27:05.609 --> 00:27:09.450
so that it can't be confused as code.

565
00:27:09.450 --> 00:27:12.000
If we were to build security in, our programs wouldn't

566
00:27:12.000 --> 00:27:15.703
have these problems, how can we do that?

567
00:27:15.703 --> 00:27:17.684
Well the first thing is to stop the attacks,

568
00:27:17.684 --> 00:27:19.071
stop the buffer overflow attack,

569
00:27:19.071 --> 00:27:21.998
we need to actually check that we don't overflow

570
00:27:21.998 --> 00:27:25.076
the buffer size, and it turns out that the C and C++

571
00:27:25.076 --> 00:27:28.557
programming languages rely on the programmer to do that.

572
00:27:28.557 --> 00:27:30.714
So every time I read some information into a buffer,

573
00:27:30.714 --> 00:27:33.526
I always have to remember to check am I not reading too

574
00:27:33.526 --> 00:27:35.384
much information, does it should always fit into the

575
00:27:35.384 --> 00:27:38.173
buffer, how big is the buffer, how much did I read in,

576
00:27:38.173 --> 00:27:39.622
is the right size?

577
00:27:39.622 --> 00:27:41.087
There's lots of other finicky details

578
00:27:41.087 --> 00:27:43.688
that you have to figure out too.

579
00:27:43.688 --> 00:27:46.605
Now other languages it turns out, once we saw

580
00:27:46.605 --> 00:27:50.438
that top five list, Python, Java, Ruby, OCaml,

581
00:27:51.296 --> 00:27:53.072
all of them actually do this for you,

582
00:27:53.072 --> 00:27:55.183
the programmer doesn't need to think of it anymore.

583
00:27:55.183 --> 00:27:56.243
When you allocate your buffer,

584
00:27:56.243 --> 00:27:59.429
when you say please make X a buffer of size 10,

585
00:27:59.429 --> 00:28:01.949
anytime from then on, you read it right from your buffer,

586
00:28:01.949 --> 00:28:04.222
if you try to overflow it,

587
00:28:04.222 --> 00:28:06.719
the language basically stops you from doing that.

588
00:28:06.719 --> 00:28:08.668
So you as the programmer is just gone,

589
00:28:08.668 --> 00:28:10.384
you can't make that mistake anymore.

590
00:28:10.384 --> 00:28:12.985
And so that means your program is no longer susceptible

591
00:28:12.985 --> 00:28:17.152
to the buffer overflow attack, that's the end of the story.

592
00:28:19.034 --> 00:28:21.405
If we were doing that in this case, the program would

593
00:28:21.405 --> 00:28:25.521
essentially look the same except now when I go into

594
00:28:25.521 --> 00:28:28.856
type in that information and store it in the variable X,

595
00:28:28.856 --> 00:28:33.798
the language has taken care for me to block any attempt

596
00:28:33.798 --> 00:28:35.506
to go beyond the amount of the buffer.

597
00:28:35.506 --> 00:28:39.150
I as the programmer don't need to remember that anymore.

598
00:28:39.150 --> 00:28:41.462
Okay, so if it's just that simple,

599
00:28:41.462 --> 00:28:43.204
why are we still using these languages

600
00:28:43.204 --> 00:28:45.600
that make this pernicious bug

601
00:28:45.600 --> 00:28:48.431
that can lead to millions of records being stolen?

602
00:28:48.431 --> 00:28:49.931
Why are we still using these languages?

603
00:28:49.931 --> 00:28:52.264
Well there are good reasons.

604
00:28:53.144 --> 00:28:58.010
One is C and C++ have been around for 40 years,

605
00:28:58.010 --> 00:29:00.272
and there are billions of lines of code written in

606
00:29:00.272 --> 00:29:02.896
C and C++ and we're not gonna throw it all away.

607
00:29:02.896 --> 00:29:05.608
A lot of what's running on that computer,

608
00:29:05.608 --> 00:29:07.750
a lot of what's running on computers all over the place

609
00:29:07.750 --> 00:29:09.354
are running C and C++

610
00:29:09.354 --> 00:29:13.194
and we can't just get rid of it and replace it.

611
00:29:13.194 --> 00:29:15.602
As a result of there being so much C and C++,

612
00:29:15.602 --> 00:29:17.451
programmers are very familiar with C,

613
00:29:17.451 --> 00:29:19.830
so like me you got your degree 20 years ago,

614
00:29:19.830 --> 00:29:21.563
you learned a couple of languages.

615
00:29:21.563 --> 00:29:24.118
Maybe you've learned some newer languages, but for sure,

616
00:29:24.118 --> 00:29:25.827
you know C and C++.

617
00:29:25.827 --> 00:29:28.840
So that makes it still a popular language for new projects

618
00:29:28.840 --> 00:29:30.529
because you know that your computing staff

619
00:29:30.529 --> 00:29:32.514
knows this language.

620
00:29:32.514 --> 00:29:35.723
Probably the best practical reason is that C and C++

621
00:29:35.723 --> 00:29:39.620
are very efficient, so they give you very fine control

622
00:29:39.620 --> 00:29:42.406
over the platform resources, and that makes them

623
00:29:42.406 --> 00:29:45.683
the go-to language for these new virgin domains

624
00:29:45.683 --> 00:29:47.657
that don't have a lot of memory, don't have a lot of

625
00:29:47.657 --> 00:29:51.449
processing power like your car, like your smart meter,

626
00:29:51.449 --> 00:29:54.037
like your coffee maker, like your refrigerator.

627
00:29:54.037 --> 00:29:56.559
A lot of that stuff, we don't have a gigabyte of RAM

628
00:29:56.559 --> 00:29:59.436
sitting in your refrigerator and so we have to be

629
00:29:59.436 --> 00:30:02.782
very careful with how you spend your resources and C and C++

630
00:30:02.782 --> 00:30:06.163
are the most careful essentially because they foist

631
00:30:06.163 --> 00:30:08.565
all of the work of allocating those resources onto

632
00:30:08.565 --> 00:30:10.513
the programmer rather than having the language

633
00:30:10.513 --> 00:30:11.846
take care of it.

634
00:30:12.942 --> 00:30:16.692
So the best current advice, I say this a lot,

635
00:30:17.578 --> 00:30:19.732
more people are saying this.

636
00:30:19.732 --> 00:30:22.199
Don't use C and C++ if you can get away with it,

637
00:30:22.199 --> 00:30:24.317
only use it when you have to.

638
00:30:24.317 --> 00:30:26.693
Unfortunately, sometimes you have to.

639
00:30:26.693 --> 00:30:28.987
So this leads to a research question.

640
00:30:28.987 --> 00:30:32.645
How do we make it so that programs that need that fine

641
00:30:32.645 --> 00:30:36.701
control that C and C++ provide are nevertheless as safe

642
00:30:36.701 --> 00:30:38.597
as programs written in those other languages

643
00:30:38.597 --> 00:30:40.514
like Python, like Ruby?

644
00:30:44.067 --> 00:30:46.929
Some collaborators and I set out to answer that question

645
00:30:46.929 --> 00:30:48.594
by doing some research.

646
00:30:48.594 --> 00:30:51.266
We developed this language called Cyclone which is intended

647
00:30:51.266 --> 00:30:54.588
to be a safe programming language that's used in the

648
00:30:54.588 --> 00:30:58.526
same context that you would normally use C and C++.

649
00:30:58.526 --> 00:31:00.576
So this was a big multi-institution project.

650
00:31:00.576 --> 00:31:04.145
It started right when I began as a professor here

651
00:31:04.145 --> 00:31:06.698
and there were several contemporary efforts that also

652
00:31:06.698 --> 00:31:08.484
looked at solving this problem.

653
00:31:08.484 --> 00:31:10.803
You can still actually download Cyclone today

654
00:31:10.803 --> 00:31:14.102
if you are curious to play with it, although it's harder

655
00:31:14.102 --> 00:31:18.967
to use because it doesn't work as well on modern platforms.

656
00:31:18.967 --> 00:31:21.565
So in building Cyclone as a replacement language,

657
00:31:21.565 --> 00:31:23.302
I'm solving a technical problem here.

658
00:31:23.302 --> 00:31:27.074
I wanna build a language that I claim is as safe, as secure

659
00:31:27.074 --> 00:31:29.668
as one of these other languages like Ocaml and Python

660
00:31:29.668 --> 00:31:34.331
and Java, and yet I also wanna claim that it has the same

661
00:31:34.331 --> 00:31:38.184
usability and efficiency as C and C++.

662
00:31:38.184 --> 00:31:40.594
So as the scientist, the way that I approached this problem,

663
00:31:40.594 --> 00:31:42.960
as a professor, is I need to provide evidence

664
00:31:42.960 --> 00:31:45.411
that I have actually succeeded in my endeavor.

665
00:31:45.411 --> 00:31:48.507
So what we did in the case of Cyclone is we first of all,

666
00:31:48.507 --> 00:31:52.098
mathematically formalized what Cyclone is.

667
00:31:52.098 --> 00:31:56.289
We wrote down in mathematical notation the semantics,

668
00:31:56.289 --> 00:31:58.961
the meaning of Cyclone programs, and we were able to

669
00:31:58.961 --> 00:32:01.698
mathematically prove that all Cyclone programs

670
00:32:01.698 --> 00:32:04.691
satisfied some basic security properties including the

671
00:32:04.691 --> 00:32:08.676
property that by indication eliminates buffer overflows

672
00:32:08.676 --> 00:32:11.863
and many other attacks as a consequence.

673
00:32:11.863 --> 00:32:15.326
Secondly, while making it so that the language is secure

674
00:32:15.326 --> 00:32:16.366
is maybe not so hard.

675
00:32:16.366 --> 00:32:18.549
As I said, other languages already do that.

676
00:32:18.549 --> 00:32:21.722
We also needed to show that Cyclone is not so hard to use.

677
00:32:21.722 --> 00:32:24.239
If it was so hard to use that only two people in the world

678
00:32:24.239 --> 00:32:26.935
could use it, that's not gonna solve our billions of lines

679
00:32:26.935 --> 00:32:30.642
of C programs' problem and it's gonna really put a dent

680
00:32:30.642 --> 00:32:33.322
in the overall problem.

681
00:32:33.322 --> 00:32:35.489
We also wanna see that it actually performs as well

682
00:32:35.489 --> 00:32:37.770
as we would hope, that it can match the same domains

683
00:32:37.770 --> 00:32:40.025
that C and C++ programs do.

684
00:32:40.025 --> 00:32:42.261
So for that, what we did is we wrote a bunch of programs

685
00:32:42.261 --> 00:32:45.629
in Cyclone, we built a Cyclone infrastructure (mumbles)

686
00:32:45.629 --> 00:32:48.336
and a development environment and we wrote a bunch of

687
00:32:48.336 --> 00:32:51.087
programs what we'd normally write in C and then we

688
00:32:51.087 --> 00:32:54.007
compared Cyclone performance to the C performance

689
00:32:54.007 --> 00:32:56.997
and we also measured the effort it took to turn a C program

690
00:32:56.997 --> 00:32:58.534
into a Cyclone program.

691
00:32:58.534 --> 00:33:00.791
So I'm gonna show you a couple of charts that were in

692
00:33:00.791 --> 00:33:04.103
academic papers that I published to show you

693
00:33:04.103 --> 00:33:06.964
how we did this evaluation.

694
00:33:06.964 --> 00:33:09.043
So this is a performance comparison.

695
00:33:09.043 --> 00:33:11.018
If you look along the x-axis there,

696
00:33:11.018 --> 00:33:12.387
the names aren't important.

697
00:33:12.387 --> 00:33:15.841
That's just the list of programs that we wrote in C,

698
00:33:15.841 --> 00:33:19.270
in Cyclone and the programming language Java.

699
00:33:19.270 --> 00:33:22.589
On the y-axis is the performance of those programs,

700
00:33:22.589 --> 00:33:25.204
and it's normalized to the performance of C.

701
00:33:25.204 --> 00:33:29.219
So the black bars there, that's the smallest part set at one

702
00:33:29.219 --> 00:33:31.836
that's C, so if the C program took four seconds,

703
00:33:31.836 --> 00:33:34.787
we just divided all of the results for C, Cyclone and Java

704
00:33:34.787 --> 00:33:36.525
by four to make it one.

705
00:33:36.525 --> 00:33:37.906
For the next part, we did the same thing

706
00:33:37.906 --> 00:33:39.653
so that C was always one.

707
00:33:39.653 --> 00:33:42.341
So each of those bars then is the relative overhead

708
00:33:42.341 --> 00:33:46.001
of Cyclone or Java compared to the C original.

709
00:33:46.001 --> 00:33:48.153
So what you can see is that the Cyclone programs

710
00:33:48.153 --> 00:33:51.757
are about 60% slower than the C original ones,

711
00:33:51.757 --> 00:33:55.111
but only require changing five to 15% of the lines

712
00:33:55.111 --> 00:33:56.524
of the original C program.

713
00:33:56.524 --> 00:34:00.608
So Cyclone looks a lot like C, a few things we had to change

714
00:34:00.608 --> 00:34:02.068
but not very many.

715
00:34:02.068 --> 00:34:05.584
Whereas, the equivalent Java program was much slower,

716
00:34:05.584 --> 00:34:07.417
eight and a half times slower on average,

717
00:34:07.417 --> 00:34:11.060
and actually these really tall bars here, there's two

718
00:34:11.060 --> 00:34:14.071
that go way off the chart, they are actually 22 times slower

719
00:34:14.071 --> 00:34:15.988
or something like that.

720
00:34:18.559 --> 00:34:21.509
As another example of an experiment that we did,

721
00:34:21.509 --> 00:34:24.866
in the chart I just showed you, our aim was to do the least

722
00:34:24.866 --> 00:34:27.363
work possible to get a C program to work

723
00:34:27.363 --> 00:34:28.855
as a Cyclone program.

724
00:34:28.855 --> 00:34:31.039
But we also designed Cyclone to have some more

725
00:34:31.039 --> 00:34:34.585
sophisticated features for power users so that they could

726
00:34:34.585 --> 00:34:37.004
make their Cyclone programs perform really well,

727
00:34:37.004 --> 00:34:39.597
maybe better than that 60% overhead.

728
00:34:39.597 --> 00:34:43.108
So that's shown in these two charts

729
00:34:43.108 --> 00:34:46.207
and the two second groupings of columns here.

730
00:34:46.207 --> 00:34:48.444
So the first grouping of columns, the C program

731
00:34:48.444 --> 00:34:50.861
shows the running time and the memory consumption

732
00:34:50.861 --> 00:34:53.478
of these five programs that are listed along the rows

733
00:34:53.478 --> 00:34:57.728
of the table, so 0.7 seconds, 12.5 megabytes of memory,

734
00:34:57.728 --> 00:35:01.895
1.23 seconds, 394 K of memory under the left-most column.

735
00:35:03.568 --> 00:35:06.388
In the middle chart, beside where it says Cyclone GC,

736
00:35:06.388 --> 00:35:08.446
that was the one with the least work possible

737
00:35:08.446 --> 00:35:12.071
to get it to work in Cyclone, and we can see that still

738
00:35:12.071 --> 00:35:13.523
we're doing pretty well compared to C.

739
00:35:13.523 --> 00:35:16.017
So the numbers in parentheses are the slow down,

740
00:35:16.017 --> 00:35:20.281
so it's 1.61 the C program execution time,

741
00:35:20.281 --> 00:35:23.602
1.05, 11.0 that's pretty good.

742
00:35:23.602 --> 00:35:25.581
On the other hand, if you look at beta ftpd,

743
00:35:25.581 --> 00:35:28.563
we can see that the memory use is 30 times more memory

744
00:35:28.563 --> 00:35:31.956
than it is for the C program and maybe that's bad,

745
00:35:31.956 --> 00:35:34.155
because I don't wanna have 30 times as much memory

746
00:35:34.155 --> 00:35:36.084
in my refrigerator, I wanna have as little memory

747
00:35:36.084 --> 00:35:37.616
as I can get away with.

748
00:35:37.616 --> 00:35:40.792
So in the final column here, we can see that using these

749
00:35:40.792 --> 00:35:43.562
extra power user features of Cyclone,

750
00:35:43.562 --> 00:35:44.942
yes it will take more effort

751
00:35:44.942 --> 00:35:47.402
but you can get that memory use down while retaining

752
00:35:47.402 --> 00:35:48.634
your safety guarantee.

753
00:35:48.634 --> 00:35:51.590
So it's a trade-off, that's how you the programmer,

754
00:35:51.590 --> 00:35:54.264
if you wanna in this case, the buzzword is we get rid

755
00:35:54.264 --> 00:35:56.609
of the process called garbage collection

756
00:35:56.609 --> 00:35:59.072
and use what's called manual memory management instead

757
00:35:59.072 --> 00:36:01.399
in Cyclone case and that's gonna allow us to save a lot

758
00:36:01.399 --> 00:36:03.859
of memory but it's gonna be more work to do that,

759
00:36:03.859 --> 00:36:04.833
but you can still convince Cyclone

760
00:36:04.833 --> 00:36:07.000
that your program is safe.

761
00:36:08.854 --> 00:36:13.021
So the takeaway of this research is to hit some of these

762
00:36:13.893 --> 00:36:17.469
points as to why people are still using C and C++

763
00:36:17.469 --> 00:36:21.992
today, why they're not building security in in general.

764
00:36:21.992 --> 00:36:24.315
One is, maybe there's not good technology that they could

765
00:36:24.315 --> 00:36:27.078
use instead of the technology they're already using.

766
00:36:27.078 --> 00:36:30.463
So Cyclone was an attempt to solve that problem by providing

767
00:36:30.463 --> 00:36:33.783
technology that could work at the same places that C and C++

768
00:36:33.783 --> 00:36:38.155
work but maybe be more secure, and that might,

769
00:36:38.155 --> 00:36:40.889
because it's close to C, we have some hope of solving

770
00:36:40.889 --> 00:36:43.277
that legacy code problem, it's less work to port that

771
00:36:43.277 --> 00:36:45.812
whole C code to Cyclone code and maybe it's not so hard

772
00:36:45.812 --> 00:36:48.144
between our staff.

773
00:36:48.144 --> 00:36:52.642
Now, Cyclone is a research project, it ended in 2006,

774
00:36:52.642 --> 00:36:56.276
although it's still technically downloadable.

775
00:36:56.276 --> 00:36:59.224
But it's impact and I would say in general the impact

776
00:36:59.224 --> 00:37:02.823
of professors, while they can go out in external companies

777
00:37:02.823 --> 00:37:05.183
and make lots of money, most of us what we do,

778
00:37:05.183 --> 00:37:07.076
is we try to influence ideas.

779
00:37:07.076 --> 00:37:09.909
What we do is we come up with an idea and we evaluate it,

780
00:37:09.909 --> 00:37:13.133
we publish a paper about it that other people can then read

781
00:37:13.133 --> 00:37:16.527
and say "yeah, that looks like a good idea," her evidence

782
00:37:16.527 --> 00:37:18.586
convinces me that maybe for my project,

783
00:37:18.586 --> 00:37:19.741
I should do that.

784
00:37:19.741 --> 00:37:22.120
And in fact, that's what happened at Mozilla Corporation,

785
00:37:22.120 --> 00:37:25.443
there's a new language that they are building for the next

786
00:37:25.443 --> 00:37:27.774
generation web browser, cause a web browser is at the

787
00:37:27.774 --> 00:37:30.845
front line of attack and this language is called Rust

788
00:37:30.845 --> 00:37:34.129
and Rust basically pull cloth takes the memory management

789
00:37:34.129 --> 00:37:37.581
ideas from Cyclone and incorporates them into its own.

790
00:37:37.581 --> 00:37:40.686
So this is really exciting for us as academics to see that

791
00:37:40.686 --> 00:37:42.393
the process of research is working.

792
00:37:42.393 --> 00:37:45.377
We publish these ideas and then practitioners come around

793
00:37:45.377 --> 00:37:48.312
to incorporate them into stuff that eventually changes

794
00:37:48.312 --> 00:37:50.028
the state of the practice.

795
00:37:50.028 --> 00:37:53.287
Intel is now adding extensions to its processors to make

796
00:37:53.287 --> 00:37:55.335
some of these checks, like not running off the end of your

797
00:37:55.335 --> 00:37:59.418
buffer keeper so that it can be done in hardware.

798
00:38:00.423 --> 00:38:01.869
When I was at Microsoft over the summer,

799
00:38:01.869 --> 00:38:04.382
I learned that Microsoft is now spinning up an effort

800
00:38:04.382 --> 00:38:07.757
to basically build Cyclone as an extension to C (mumbles)

801
00:38:07.757 --> 00:38:11.091
so that people can start programming more in Windows-like C

802
00:38:11.091 --> 00:38:13.793
language like Cyclone was than a new language

803
00:38:13.793 --> 00:38:16.293
which is the way that Rust is.

804
00:38:18.314 --> 00:38:20.812
Okay, so now I'm gonna switch gears and I'm gonna tell you a

805
00:38:20.812 --> 00:38:25.083
little bit about the educational efforts that cross-cut

806
00:38:25.083 --> 00:38:27.477
with the research efforts that I'm involved in

807
00:38:27.477 --> 00:38:29.697
that also we're trying to get across this idea of

808
00:38:29.697 --> 00:38:33.864
building security in rather than penetrate and patch.

809
00:38:35.680 --> 00:38:40.245
So a lot of students are interested in cybersecurity,

810
00:38:40.245 --> 00:38:41.853
and when they're interested in it,

811
00:38:41.853 --> 00:38:45.348
they come up to me and I say "what do you like about it?"

812
00:38:45.348 --> 00:38:47.662
"Do you like that you get to put a white hat on

813
00:38:47.662 --> 00:38:50.182
and you get to build software that's really secure?"

814
00:38:50.182 --> 00:38:53.163
"Or do you like to put a black hat on and find defects

815
00:38:53.163 --> 00:38:55.764
and vulnerabilities and hack into software and show

816
00:38:55.764 --> 00:38:59.431
why it's insecure, which makes you excited?"

817
00:39:00.576 --> 00:39:02.427
(loud laughter)

818
00:39:02.427 --> 00:39:05.167
That's right, it's much more fun to blow up bridges

819
00:39:05.167 --> 00:39:09.034
than to build them, not that I said that.

820
00:39:09.034 --> 00:39:11.442
And you can see this because there were lots of

821
00:39:11.442 --> 00:39:14.949
cybersecurity events, in particular contests like

822
00:39:14.949 --> 00:39:17.932
so-called Capture the Flag or CTF Contest which were

823
00:39:17.932 --> 00:39:20.877
entirely about here's some software, it has some bugs in it

824
00:39:20.877 --> 00:39:23.537
go find the bugs, exploit the bugs, do it faster

825
00:39:23.537 --> 00:39:26.259
than everybody else, you win the contest.

826
00:39:26.259 --> 00:39:30.682
So Defcon CTF, Cyber Defense Challenge, etcetera

827
00:39:30.682 --> 00:39:33.013
really the big part of it is hacking.

828
00:39:33.013 --> 00:39:35.734
Some of it is configuring existing systems,

829
00:39:35.734 --> 00:39:38.530
but almost none of it, in fact, maybe even zero of it,

830
00:39:38.530 --> 00:39:40.045
as far as I'm aware is about

831
00:39:40.045 --> 00:39:41.014
"well how do you actually build that

832
00:39:41.014 --> 00:39:44.123
software right in the first place?"

833
00:39:44.123 --> 00:39:48.681
So one of my students Andrew Roof and I got irritated

834
00:39:48.681 --> 00:39:50.683
enough at this state of affairs that we decided to do

835
00:39:50.683 --> 00:39:53.303
something about it which is to make our own contest

836
00:39:53.303 --> 00:39:56.093
that emphasizes the building part of security

837
00:39:56.093 --> 00:39:57.333
and not just the breaking part

838
00:39:57.333 --> 00:39:59.487
because what are all of us as taxpayers doing

839
00:39:59.487 --> 00:40:01.357
when we're paying professors

840
00:40:01.357 --> 00:40:04.208
and companies to make things more secure?

841
00:40:04.208 --> 00:40:05.966
We're not paying them to find more and more bugs

842
00:40:05.966 --> 00:40:07.827
in the crappy software we already have.

843
00:40:07.827 --> 00:40:09.487
We're paying them to figure out how to build software

844
00:40:09.487 --> 00:40:10.975
that's right in the first place.

845
00:40:10.975 --> 00:40:12.427
So we gotta close that loop.

846
00:40:12.427 --> 00:40:14.357
Yeah it's great to find vulnerabilities,

847
00:40:14.357 --> 00:40:16.131
but we need to figure out a way to build systems

848
00:40:16.131 --> 00:40:18.414
that don't have vulnerabilities in the first place.

849
00:40:18.414 --> 00:40:22.581
And so that's what this contest is attempting to promote.

850
00:40:25.837 --> 00:40:28.381
So the basic idea of the contest is that there are two

851
00:40:28.381 --> 00:40:30.920
parts to it, one is a building phase and another is a

852
00:40:30.920 --> 00:40:34.837
breaking phase and these are scored separately.

853
00:40:35.734 --> 00:40:38.614
In the building phase, you are asked to build some software.

854
00:40:38.614 --> 00:40:43.085
Your team, you provide a specification, build this,

855
00:40:43.085 --> 00:40:44.816
and we'll tell you exactly what we want you to build

856
00:40:44.816 --> 00:40:46.321
and we'll tell you what its security properties

857
00:40:46.321 --> 00:40:48.133
are supposed to be.

858
00:40:48.133 --> 00:40:51.736
Your software will be scored on does it perform well,

859
00:40:51.736 --> 00:40:54.843
does it have extra features that you didn't have to add.

860
00:40:54.843 --> 00:40:56.387
Now why am I scoring it this way

861
00:40:56.387 --> 00:40:57.994
if I'm interested in security?

862
00:40:57.994 --> 00:41:01.149
Well because this is what the marketplace rewards.

863
00:41:01.149 --> 00:41:04.044
It's not that people don't know necessarily about security

864
00:41:04.044 --> 00:41:07.739
as being important, you just can't sell stuff that secure.

865
00:41:07.739 --> 00:41:09.357
We have to sell stuff that's cool,

866
00:41:09.357 --> 00:41:12.108
and then hopefully it's secure at the same time.

867
00:41:12.108 --> 00:41:15.372
So to try to show that there is that cost if your software

868
00:41:15.372 --> 00:41:17.172
is breached then you lose millions of records,

869
00:41:17.172 --> 00:41:19.967
that's not good either, you're gonna lose points

870
00:41:19.967 --> 00:41:23.124
if vulnerabilities are found in your software.

871
00:41:23.124 --> 00:41:26.801
Now the break it score gives the same teams the ability

872
00:41:26.801 --> 00:41:30.386
to find vulnerabilities in the other teams' submissions.

873
00:41:30.386 --> 00:41:32.630
So I have 10 teams, they all submit their versions

874
00:41:32.630 --> 00:41:35.227
of the software and now in Round Two, the break-it teams

875
00:41:35.227 --> 00:41:37.831
are gonna look at the other teams' software and try to find

876
00:41:37.831 --> 00:41:40.602
those vulnerabilities and prove that they exist.

877
00:41:40.602 --> 00:41:43.496
If they do, they will gain points whereas the teams that had

878
00:41:43.496 --> 00:41:46.962
those vulnerabilities will lose points.

879
00:41:46.962 --> 00:41:50.951
So the thing that I think is cool about this is first of all

880
00:41:50.951 --> 00:41:53.708
it's aiming to educate its participants.

881
00:41:53.708 --> 00:41:57.586
Just by building software that you know next week is gonna

882
00:41:57.586 --> 00:41:59.126
be hacked by somebody else,

883
00:41:59.126 --> 00:42:00.877
immediately changes your mentality.

884
00:42:00.877 --> 00:42:02.858
If you're sitting in your cube at work and you're writing

885
00:42:02.858 --> 00:42:05.175
a bunch of code, and your manager they wanna care

886
00:42:05.175 --> 00:42:07.946
whether the cool animation in the cool display

887
00:42:07.946 --> 00:42:08.994
is actually working,

888
00:42:08.994 --> 00:42:11.212
they don't care anything about your security,

889
00:42:11.212 --> 00:42:13.348
whereas here we wanna emphasize no, no there is someone

890
00:42:13.348 --> 00:42:17.202
that's gonna care, you will lose points in two weeks' time

891
00:42:17.202 --> 00:42:18.472
if you don't get this right.

892
00:42:18.472 --> 00:42:19.887
So I think that's really important for people

893
00:42:19.887 --> 00:42:21.554
to have that hat on.

894
00:42:22.736 --> 00:42:24.718
The other thing that's cool is that we can analyze

895
00:42:24.718 --> 00:42:27.130
the outcomes of the contest and we can figure out

896
00:42:27.130 --> 00:42:28.368
well what actually worked.

897
00:42:28.368 --> 00:42:30.675
Students are now afforded the opportunity

898
00:42:30.675 --> 00:42:33.687
to make the choices they think will most likely

899
00:42:33.687 --> 00:42:34.526
lead to security.

900
00:42:34.526 --> 00:42:37.643
For example, they could choose not to use C and C++,

901
00:42:37.643 --> 00:42:41.422
they could also choose to use many other techniques,

902
00:42:41.422 --> 00:42:44.357
methods and so on that they hope will make

903
00:42:44.357 --> 00:42:46.524
their systems more secure.

904
00:42:48.161 --> 00:42:51.236
Therefore we can perform science while performing education

905
00:42:51.236 --> 00:42:53.900
at the same time by looking at the exam problems,

906
00:42:53.900 --> 00:42:56.074
analyzing the data and then turning around

907
00:42:56.074 --> 00:42:57.922
what we find into what we teach

908
00:42:57.922 --> 00:43:00.627
and then into what you learn.

909
00:43:00.627 --> 00:43:03.287
So we run this contest two times.

910
00:43:03.287 --> 00:43:05.299
I'm just gonna tell you a little bit about the one they ran

911
00:43:05.299 --> 00:43:08.793
in Spring, this was just part of my Coursera class

912
00:43:08.793 --> 00:43:10.351
that I told you about before,

913
00:43:10.351 --> 00:43:11.844
I'll give you one slide about it later.

914
00:43:11.844 --> 00:43:14.966
There were 98 registered teams.

915
00:43:14.966 --> 00:43:18.802
79 of them attempted to make some kind of submission,

916
00:43:18.802 --> 00:43:22.452
and 62 succeeded, so there was a set of baseline tests

917
00:43:22.452 --> 00:43:25.326
that you had to pass before you were considered qualified,

918
00:43:25.326 --> 00:43:27.370
and 62 teams managed to do that.

919
00:43:27.370 --> 00:43:31.975
And then 66 teams found at least one bug in the submissions

920
00:43:31.975 --> 00:43:36.142
of the other teams, in fact, they in total found 9,128 bugs.

921
00:43:37.183 --> 00:43:38.137
So that's alarming.

922
00:43:38.137 --> 00:43:40.318
Now it turns out that most of those were not unique,

923
00:43:40.318 --> 00:43:42.685
so many of those bugs were duplicates from bugs found

924
00:43:42.685 --> 00:43:45.538
by other teams or in fact bugs found by the same time.

925
00:43:45.538 --> 00:43:49.652
They just didn't realize it was the same root cause.

926
00:43:49.652 --> 00:43:53.037
On the bottom there, 36 bugs were actually privacy-relevant,

927
00:43:53.037 --> 00:43:56.124
these could be used to exploit and steal information

928
00:43:56.124 --> 00:43:57.359
and the other ones, there were some that were

929
00:43:57.359 --> 00:43:59.417
integrity-relevant where information

930
00:43:59.417 --> 00:44:02.648
that was stored by the system could be corrupted.

931
00:44:02.648 --> 00:44:04.573
So these are the winners.

932
00:44:04.573 --> 00:44:06.723
The thing that's cool about the Coursera class is it's

933
00:44:06.723 --> 00:44:09.654
a worldwide contest and so I don't believe anybody

934
00:44:09.654 --> 00:44:12.565
on this picture lived in the same country.

935
00:44:12.565 --> 00:44:15.029
They found each other through surveys that we provided

936
00:44:15.029 --> 00:44:17.740
for team formation and nevertheless found a way to work

937
00:44:17.740 --> 00:44:21.214
together the build-it side of the contest.

938
00:44:21.214 --> 00:44:23.721
The Break-it Winners they also didn't know each other

939
00:44:23.721 --> 00:44:25.088
before, they were all from London

940
00:44:25.088 --> 00:44:27.921
and the second guy was from Italy.

941
00:44:30.477 --> 00:44:33.030
Just to focus on, there were many reasons why people did

942
00:44:33.030 --> 00:44:34.990
or didn't succeed, I don't have time to go into them

943
00:44:34.990 --> 00:44:36.731
but just for fun, to look at language again

944
00:44:36.731 --> 00:44:38.872
since I've been talking about that.

945
00:44:38.872 --> 00:44:41.493
Many languages were chosen as part of the contest.

946
00:44:41.493 --> 00:44:46.012
C and C++, Java, ones we've seen before, also many others.

947
00:44:46.012 --> 00:44:48.927
The language Python turned out to be the most popular.

948
00:44:48.927 --> 00:44:51.761
Python allows you to write code very quickly,

949
00:44:51.761 --> 00:44:53.255
it's also very slow.

950
00:44:53.255 --> 00:44:55.664
And so what these people, basically their value proposition

951
00:44:55.664 --> 00:44:58.447
was it's more important to me to get code that works

952
00:44:58.447 --> 00:45:01.734
and then I can confidently secure that code that works best.

953
00:45:01.734 --> 00:45:04.042
They were willing to sacrifice those performance points

954
00:45:04.042 --> 00:45:06.184
in order to gain security points.

955
00:45:06.184 --> 00:45:09.133
And interestingly, they lost a value proposition,

956
00:45:09.133 --> 00:45:11.801
because the top four scorers were not Python programmers,

957
00:45:11.801 --> 00:45:15.968
they were Java and other faster programming languages.

958
00:45:17.047 --> 00:45:19.604
So this chart which unfortunately we can't see very well

959
00:45:19.604 --> 00:45:22.018
shows the scores over time.

960
00:45:22.018 --> 00:45:24.399
So as we move from left to right,

961
00:45:24.399 --> 00:45:26.237
we see the time for the contest.

962
00:45:26.237 --> 00:45:28.021
The very beginning of the contest is the left,

963
00:45:28.021 --> 00:45:30.105
in the very end is the right.

964
00:45:30.105 --> 00:45:32.883
So at the beginning, that line before everyone's score

965
00:45:32.883 --> 00:45:35.113
shoots up, that's the score at zero

966
00:45:35.113 --> 00:45:36.549
As people are implementing their projects,

967
00:45:36.549 --> 00:45:37.791
they're gaining more and more points

968
00:45:37.791 --> 00:45:39.836
for performance and for optional features,

969
00:45:39.836 --> 00:45:41.719
until they kind of plateau there.

970
00:45:41.719 --> 00:45:43.962
And then during the break-it phase, you can see everybody's

971
00:45:43.962 --> 00:45:46.390
score plummets, in fact I don't think anybody had a

972
00:45:46.390 --> 00:45:50.322
non-negative score while there was 9,123 bugs

973
00:45:50.322 --> 00:45:51.331
that were being submitted.

974
00:45:51.331 --> 00:45:54.144
So peoples' scores went way down.

975
00:45:54.144 --> 00:45:56.548
Then we had the final phase of the contest, the fix-it stage

976
00:45:56.548 --> 00:45:58.969
where people were able to fix the bugs that we had their

977
00:45:58.969 --> 00:46:00.345
opponents found.

978
00:46:00.345 --> 00:46:02.411
And it was when they did that that we discovered that many

979
00:46:02.411 --> 00:46:04.396
of those bugs that were discovered were actually the

980
00:46:04.396 --> 00:46:06.612
same bug cause when we fixed the flaw,

981
00:46:06.612 --> 00:46:09.642
100 test cases started to pass, which gave us evidence

982
00:46:09.642 --> 00:46:11.121
that those test cases were actually

983
00:46:11.121 --> 00:46:13.028
just evidence of the same bug.

984
00:46:13.028 --> 00:46:13.861
So if you had a bug,

985
00:46:13.861 --> 00:46:16.270
if I had a bug in my exponentiation routine,

986
00:46:16.270 --> 00:46:18.120
like maybe that bug that I showed you before,

987
00:46:18.120 --> 00:46:19.465
where it was greater than or equal to zero

988
00:46:19.465 --> 00:46:20.958
instead of greater than zero,

989
00:46:20.958 --> 00:46:23.419
I can make a million tests that would cause that same bug

990
00:46:23.419 --> 00:46:26.130
to fail, three squared would fail, 2/3 would fail,

991
00:46:26.130 --> 00:46:27.381
4/3 would fail.

992
00:46:27.381 --> 00:46:29.880
All different tests but all the same bug.

993
00:46:29.880 --> 00:46:33.251
So the fix-it phase is a way to normalize those results

994
00:46:33.251 --> 00:46:36.029
so we get better scores.

995
00:46:36.029 --> 00:46:38.456
So one interesting thing is after analyzing the data,

996
00:46:38.456 --> 00:46:41.116
what we found surprise, surprise is that if you program

997
00:46:41.116 --> 00:46:42.976
in C or C++,

998
00:46:42.976 --> 00:46:45.918
that was correlated with you having a lower level of score.

999
00:46:45.918 --> 00:46:47.687
And unfortunately, you can't see that on the slide,

1000
00:46:47.687 --> 00:46:51.072
but the lines there in red are the C and C++ lines,

1001
00:46:51.072 --> 00:46:54.239
and every other line is blue, those are the non-C and C++

1002
00:46:54.239 --> 00:46:57.347
lines and using what's called Spearman coefficient

1003
00:46:57.347 --> 00:46:59.852
we're able to show that in a statistically significant

1004
00:46:59.852 --> 00:47:03.916
way using C and C++ for your presentation was bad.

1005
00:47:03.916 --> 00:47:07.598
Now interestingly knowing C or C++ or on your survey,

1006
00:47:07.598 --> 00:47:09.849
if you said "oh yes, I'm very familiar with C or C++,

1007
00:47:09.849 --> 00:47:13.111
I'm a good C or C++ programmer," if you said that but then

1008
00:47:13.111 --> 00:47:15.103
didn't use C or C++,

1009
00:47:15.103 --> 00:47:17.955
that was highly correlated with success.

1010
00:47:17.955 --> 00:47:19.635
This makes sense when you think about it, right?

1011
00:47:19.635 --> 00:47:21.820
Because if you know all of the pitfalls of

1012
00:47:21.820 --> 00:47:23.986
using this language, because you've programmed in it

1013
00:47:23.986 --> 00:47:26.689
quite a bit, then you think "boy, I don't wanna get trapped

1014
00:47:26.689 --> 00:47:30.052
by those pitfalls and so I won't use the language."

1015
00:47:30.052 --> 00:47:32.383
And there's a bunch of other interesting things you see that

1016
00:47:32.383 --> 00:47:36.054
if there's time afterwards I'll get to them.

1017
00:47:36.054 --> 00:47:37.530
Okay, so what was our goal here

1018
00:47:37.530 --> 00:47:40.160
with engendering the build-it-in mentality?

1019
00:47:40.160 --> 00:47:42.667
Well getting rid of the little bit of ignorance hopefully

1020
00:47:42.667 --> 00:47:45.556
that building is just as important as breaking,

1021
00:47:45.556 --> 00:47:47.692
trying to create that incentive that people wanted the

1022
00:47:47.692 --> 00:47:50.227
good builders and not just good breakers.

1023
00:47:50.227 --> 00:47:52.982
Also it creates a great experiment for proving which

1024
00:47:52.982 --> 00:47:56.655
technologies work, this is providing some direct evidence

1025
00:47:56.655 --> 00:47:59.530
that maybe C and C++ are not your best choices.

1026
00:47:59.530 --> 00:48:01.808
I could've said that to you before, I did,

1027
00:48:01.808 --> 00:48:04.414
but now I actually have some data that shows with 60 teams

1028
00:48:04.414 --> 00:48:08.258
competing, yeah actually 15 teams they used C and C++,

1029
00:48:08.258 --> 00:48:11.696
they did statistically worse than the rest.

1030
00:48:11.696 --> 00:48:15.267
Finally the concerns about cost, we're creating the

1031
00:48:15.267 --> 00:48:18.502
market in a microcosm and what we're doing here,

1032
00:48:18.502 --> 00:48:21.129
we're showing people making that value judgment.

1033
00:48:21.129 --> 00:48:23.836
Here we saw that for example Python could do so well,

1034
00:48:23.836 --> 00:48:26.905
so maybe it's the savings for using a very high-level

1035
00:48:26.905 --> 00:48:30.051
language or really slow language is not worth it,

1036
00:48:30.051 --> 00:48:32.965
but on the other hand, maybe you need somewhere in between

1037
00:48:32.965 --> 00:48:36.382
like using Java, maybe that's good to do.

1038
00:48:39.869 --> 00:48:42.799
In that last five minutes, I'll tell you a little bit

1039
00:48:42.799 --> 00:48:46.306
finally about the outreach that I've been doing.

1040
00:48:46.306 --> 00:48:48.716
So I'm a programming languages researcher,

1041
00:48:48.716 --> 00:48:50.717
people think of me as a cybersecurity person

1042
00:48:50.717 --> 00:48:53.188
like Sameer said I was the director of our cybersecurity

1043
00:48:53.188 --> 00:48:56.500
person but really my research area is what we call

1044
00:48:56.500 --> 00:48:59.812
programming languages and programming languages researchers

1045
00:48:59.812 --> 00:49:03.496
look at how programming languages can help solve

1046
00:49:03.496 --> 00:49:05.281
software quality problems.

1047
00:49:05.281 --> 00:49:08.031
So we just saw an example of how the language really

1048
00:49:08.031 --> 00:49:10.843
influenced certain feature of the programs that are written

1049
00:49:10.843 --> 00:49:13.634
in it which is to say their security and languages can have

1050
00:49:13.634 --> 00:49:16.907
lots of benefits beyond, you just buffer overflows,

1051
00:49:16.907 --> 00:49:19.768
but other security problems as well.

1052
00:49:19.768 --> 00:49:22.637
So there's a great community of programming languages

1053
00:49:22.637 --> 00:49:25.670
researchers and they do fantastic work,

1054
00:49:25.670 --> 00:49:29.503
it's a really tight community to be a part of.

1055
00:49:32.520 --> 00:49:35.172
One way that I'm trying to get the word out about

1056
00:49:35.172 --> 00:49:38.882
the research that I do and the research actually not so much

1057
00:49:38.882 --> 00:49:40.963
that I do but the research that the community does

1058
00:49:40.963 --> 00:49:43.079
in this space is to write a blog.

1059
00:49:43.079 --> 00:49:46.765
So I have a blog called the PL Enthusiast,

1060
00:49:46.765 --> 00:49:50.619
if you (mumbles) to google for PL Enthusiast you'll find it.

1061
00:49:50.619 --> 00:49:52.956
And I started this a little over a year ago,

1062
00:49:52.956 --> 00:49:57.147
Since then, my co-blogger and I have written

1063
00:49:57.147 --> 00:49:58.931
about 45 blog posts.

1064
00:49:58.931 --> 00:50:02.548
And we do interviews, we do discussions of research,

1065
00:50:02.548 --> 00:50:07.256
we do tutorials, it's aimed at a general, technical,

1066
00:50:07.256 --> 00:50:10.035
regular, non-researchers, not PL (mumbles),

1067
00:50:10.035 --> 00:50:14.602
but normal people that know something about programming,

1068
00:50:14.602 --> 00:50:18.023
and it's been pretty successful.

1069
00:50:18.023 --> 00:50:19.604
The thing I'm most pleased about is that

1070
00:50:19.604 --> 00:50:22.461
if you type in phrases like probabilistic programming

1071
00:50:22.461 --> 00:50:25.651
into Google, one of my blog posts would be the second hit.

1072
00:50:25.651 --> 00:50:28.319
If you type in type safety or memory safety,

1073
00:50:28.319 --> 00:50:30.382
once again my blog post would be the second hit.

1074
00:50:30.382 --> 00:50:31.972
So Google thinks that I am authoritative,

1075
00:50:31.972 --> 00:50:34.139
so Google must be correct.

1076
00:50:36.582 --> 00:50:38.593
Also, I've been working on this MOOC, that's a massively

1077
00:50:38.593 --> 00:50:41.727
open online course, so that's been a very eye-opening

1078
00:50:41.727 --> 00:50:45.314
experience so this course is on software security,

1079
00:50:45.314 --> 00:50:47.317
and of course, I'm trying to get across many of the same

1080
00:50:47.317 --> 00:50:49.630
ideas I'm trying to get across to you today

1081
00:50:49.630 --> 00:50:51.037
but in a more technical sense.

1082
00:50:51.037 --> 00:50:54.482
So it's targeted at computer science majors who wanna

1083
00:50:54.482 --> 00:50:57.198
do better at building secure software.

1084
00:50:57.198 --> 00:51:00.451
So far about 3000 people have taken and passed the class.

1085
00:51:00.451 --> 00:51:02.112
That's more people than I have ever taught

1086
00:51:02.112 --> 00:51:06.287
at my 15 years in Maryland and that was just in one year.

1087
00:51:06.287 --> 00:51:09.096
90000 people have looked at lectures, have dabbled

1088
00:51:09.096 --> 00:51:11.929
with the course about 3000 people.

1089
00:51:12.810 --> 00:51:14.525
In the capstone project for this class

1090
00:51:14.525 --> 00:51:17.442
it's the build-it break-it contest.

1091
00:51:18.557 --> 00:51:22.832
If you look ahead, things are getting better.

1092
00:51:22.832 --> 00:51:26.386
There are more ways to build, it's not like we're totally

1093
00:51:26.386 --> 00:51:28.560
unaware that there is problem, there are people like me

1094
00:51:28.560 --> 00:51:31.501
stand up, they're saying we need to build security in,

1095
00:51:31.501 --> 00:51:33.066
there's more people than just me.

1096
00:51:33.066 --> 00:51:35.138
I threw a couple of things up here,

1097
00:51:35.138 --> 00:51:37.740
Professor Bill Pugh, who's an emeritus professor here

1098
00:51:37.740 --> 00:51:41.148
has a tool that analyzes source code called Find Bugs

1099
00:51:41.148 --> 00:51:44.423
that finds security vulnerabilities in your programs.

1100
00:51:44.423 --> 00:51:46.503
Coverity is a company that provides a (mumbles) tool

1101
00:51:46.503 --> 00:51:48.647
for C and C++ programmers.

1102
00:51:48.647 --> 00:51:50.600
This thing on the right here is called PSim,

1103
00:51:50.600 --> 00:51:53.731
it's an effort by a company by Cigital for documenting

1104
00:51:53.731 --> 00:51:55.522
best practices in security.

1105
00:51:55.522 --> 00:51:57.842
And these people are beating the drum too along with me,

1106
00:51:57.842 --> 00:51:58.921
so that's good.

1107
00:51:58.921 --> 00:52:00.986
And some traditional domains like web browsers

1108
00:52:00.986 --> 00:52:03.714
and desktop computers are getting somewhat more secure

1109
00:52:03.714 --> 00:52:05.953
at least in some ways.

1110
00:52:05.953 --> 00:52:07.854
Now on the other hand, things are getting worse too,

1111
00:52:07.854 --> 00:52:09.987
and that's because of this proliferation of code

1112
00:52:09.987 --> 00:52:11.488
and the amount of code.

1113
00:52:11.488 --> 00:52:13.658
Code is growing and the programs are ending up in places

1114
00:52:13.658 --> 00:52:18.396
where they didn't use to end up before like automobiles

1115
00:52:18.396 --> 00:52:20.541
(mumbles) actually, this picture you can see right here

1116
00:52:20.541 --> 00:52:22.373
is baby monitors.

1117
00:52:22.373 --> 00:52:24.615
There were several instances where people were able to

1118
00:52:24.615 --> 00:52:28.760
hack into a baby monitor and then listen in on the baby

1119
00:52:28.760 --> 00:52:31.940
or even speak through the speaker of the baby monitor,

1120
00:52:31.940 --> 00:52:34.204
so some parents were really freaked out when some random

1121
00:52:34.204 --> 00:52:36.454
person said "wake up baby,"

1122
00:52:38.433 --> 00:52:40.350
and pretty scary stuff.

1123
00:52:41.480 --> 00:52:44.553
But that's because these vulnerabilities that we know about

1124
00:52:44.553 --> 00:52:47.514
and maybe are more aware of in desktop software,

1125
00:52:47.514 --> 00:52:49.228
the people writing the baby monitor software

1126
00:52:49.228 --> 00:52:52.044
are just not thinking about that

1127
00:52:52.044 --> 00:52:54.245
so we're gonna continue to have these problems until

1128
00:52:54.245 --> 00:52:57.460
it's more in our habit from the experienced folks

1129
00:52:57.460 --> 00:52:59.637
all the way down to the less-experienced ones,

1130
00:52:59.637 --> 00:53:02.214
so there's more work to do.

1131
00:53:02.214 --> 00:53:04.405
I wanna say thanks to so many people

1132
00:53:04.405 --> 00:53:07.418
for making this moment possible.

1133
00:53:07.418 --> 00:53:11.221
I've worked with so many great students.

1134
00:53:11.221 --> 00:53:13.482
I have a lot of great graduate students who have done

1135
00:53:13.482 --> 00:53:14.920
this research with me.

1136
00:53:14.920 --> 00:53:17.944
My undergraduate students who keep me honest,

1137
00:53:17.944 --> 00:53:19.738
because they come into my office and they say,

1138
00:53:19.738 --> 00:53:21.883
"I don't understand your crappy project write-up,

1139
00:53:21.883 --> 00:53:23.057
why can't you write it better?"

1140
00:53:23.057 --> 00:53:26.748
I say "you're right, I have to write it better."

1141
00:53:26.748 --> 00:53:28.497
I have great collaborators and mentors,

1142
00:53:28.497 --> 00:53:32.538
so professors here at Maryland, professors elsewhere,

1143
00:53:32.538 --> 00:53:37.251
researchers and of course my family, my wife and kids

1144
00:53:37.251 --> 00:53:41.114
who put up with sometimes long hours and me typing on my

1145
00:53:41.114 --> 00:53:43.519
laptop and them talking to me and asking me questions,

1146
00:53:43.519 --> 00:53:45.621
and me nodding, and then quizzing me at the end of the

1147
00:53:45.621 --> 00:53:47.362
conversation and me going "what?"

1148
00:53:47.362 --> 00:53:49.325
(soft laughter)

1149
00:53:49.325 --> 00:53:51.658
Somehow this is all for you.

1150
00:53:53.965 --> 00:53:56.822
We need to make building software like building bridges.

1151
00:53:56.822 --> 00:53:59.783
We need to make it so that those vulnerabilities that will

1152
00:53:59.783 --> 00:54:02.144
lead to exploits aren't there in the first place.

1153
00:54:02.144 --> 00:54:04.220
And we can do it.

1154
00:54:04.220 --> 00:54:06.577
We can get away from penetrate and patch and we can build

1155
00:54:06.577 --> 00:54:08.784
better technology and better methods

1156
00:54:08.784 --> 00:54:10.981
that put security in from day one.

1157
00:54:10.981 --> 00:54:14.196
You just need to get the word out and you need to keep on

1158
00:54:14.196 --> 00:54:17.782
making that technology more applicable, easier to use

1159
00:54:17.782 --> 00:54:21.282
and available to folks building something.

1160
00:54:24.718 --> 00:54:27.385
(loud applause)

1161
00:54:35.929 --> 00:54:38.679
<v Sameer>We have got questions.</v>

1162
00:54:40.051 --> 00:54:42.718
<v Attendee>Thanks a lot, Mike.</v>

1163
00:54:43.643 --> 00:54:46.040
You pictured a couple of reasons why best practices

1164
00:54:46.040 --> 00:54:48.526
are slow to take hold you said on the one hand there's

1165
00:54:48.526 --> 00:54:50.723
ignorance and on the other hand there's concerns

1166
00:54:50.723 --> 00:54:53.411
about cost, retraining.

1167
00:54:53.411 --> 00:54:58.172
I'd also argue that maybe another one is fear of liability.

1168
00:54:58.172 --> 00:55:00.290
Bridges don't fall down now because if a bridge does fall

1169
00:55:00.290 --> 00:55:03.373
down there's substantial cost to pay.

1170
00:55:05.740 --> 00:55:10.307
Do you foresee any time in the not-too-distant future

1171
00:55:10.307 --> 00:55:13.188
where maybe software feedbacks

1172
00:55:13.188 --> 00:55:16.105
will also lead to liability issues?

1173
00:55:18.545 --> 00:55:19.881
<v ->I think that what you're getting at is that</v>

1174
00:55:19.881 --> 00:55:24.028
if software developers were liable for their security hacks,

1175
00:55:24.028 --> 00:55:26.337
so if they could be sued or the company could be held

1176
00:55:26.337 --> 00:55:28.937
accountable for writing software that's later hacked,

1177
00:55:28.937 --> 00:55:32.783
maybe they would try harder to avoid that kind of thing.

1178
00:55:32.783 --> 00:55:34.113
I have two answers to that question.

1179
00:55:34.113 --> 00:55:38.280
One is, liability applies when software is in a product.

1180
00:55:39.283 --> 00:55:41.069
So software in your toaster.

1181
00:55:41.069 --> 00:55:43.072
If your toaster with some software has a bug in it

1182
00:55:43.072 --> 00:55:44.754
and causes your toaster to catch on fire

1183
00:55:44.754 --> 00:55:45.876
and bring your house down,

1184
00:55:45.876 --> 00:55:48.237
the toaster manufacturer is liable.

1185
00:55:48.237 --> 00:55:49.993
It was their choice to make that software

1186
00:55:49.993 --> 00:55:52.628
but product liability laws hold them accountable.

1187
00:55:52.628 --> 00:55:53.984
Software in my laptop,

1188
00:55:53.984 --> 00:55:57.040
I don't actually own any of that software, I lease it.

1189
00:55:57.040 --> 00:55:58.411
That's why you have to sign a license agreement,

1190
00:55:58.411 --> 00:55:59.507
and when you sign that agreement,

1191
00:55:59.507 --> 00:56:01.680
you're saying "I'm taking this software as is,"

1192
00:56:01.680 --> 00:56:03.796
so we're getting off liability by every time

1193
00:56:03.796 --> 00:56:05.817
we sign that license agreement.

1194
00:56:05.817 --> 00:56:08.981
So we could imagine passing laws or regulations

1195
00:56:08.981 --> 00:56:11.965
that set a minimum standard that companies had to meet

1196
00:56:11.965 --> 00:56:13.749
and they would be liable otherwise.

1197
00:56:13.749 --> 00:56:15.593
And I think the big impediment here is

1198
00:56:15.593 --> 00:56:17.453
what would those standards be?

1199
00:56:17.453 --> 00:56:19.353
And this gets back to we've only been building software

1200
00:56:19.353 --> 00:56:23.775
for 50 years and the landscape is changing so rapidly.

1201
00:56:23.775 --> 00:56:26.227
You saw how many different programming languages there are

1202
00:56:26.227 --> 00:56:28.510
just as one example.

1203
00:56:28.510 --> 00:56:31.295
So there are some efforts, NIST is running some efforts now.

1204
00:56:31.295 --> 00:56:33.637
Congress has attempted to on several occasions,

1205
00:56:33.637 --> 00:56:35.350
come up with regulations and standards

1206
00:56:35.350 --> 00:56:38.346
for at least government software.

1207
00:56:38.346 --> 00:56:41.762
But I think that's a real challenge and it needs to

1208
00:56:41.762 --> 00:56:43.757
be addressed but I think

1209
00:56:43.757 --> 00:56:45.632
it's gonna be really hard to address it.

1210
00:56:45.632 --> 00:56:47.677
People have a vested interest in keeping things

1211
00:56:47.677 --> 00:56:49.094
the way they are.

1212
00:56:50.882 --> 00:56:53.209
<v Sameer>Question.</v>

1213
00:56:53.209 --> 00:56:55.601
<v Attendee>One thing I've seen a lot of in the industry</v>

1214
00:56:55.601 --> 00:56:58.788
is that there's so much debate among programmers,

1215
00:56:58.788 --> 00:57:01.857
well lot of the software you see, sometimes you get a very

1216
00:57:01.857 --> 00:57:05.624
good programmer who will write the code properly,

1217
00:57:05.624 --> 00:57:08.924
test all the foreign cases, but very often

1218
00:57:08.924 --> 00:57:12.428
you'll get somebody who's mediocre and will just write

1219
00:57:12.428 --> 00:57:15.720
the program, test only cases that hits on,

1220
00:57:15.720 --> 00:57:18.009
and forget about everything else.

1221
00:57:18.009 --> 00:57:21.437
So the question that I ask is how do you deal

1222
00:57:21.437 --> 00:57:25.173
with those people, I think the biggest problem is trying

1223
00:57:25.173 --> 00:57:28.106
to educate all these mediocre programmers

1224
00:57:28.106 --> 00:57:30.584
to do the right thing.

1225
00:57:30.584 --> 00:57:32.462
<v ->So that's a great question, I guess I have</v>

1226
00:57:32.462 --> 00:57:33.454
two answers to it.

1227
00:57:33.454 --> 00:57:38.254
One is, even if you're not a mediocre programmer,

1228
00:57:38.254 --> 00:57:39.988
remembering all the details is hard.

1229
00:57:39.988 --> 00:57:42.230
So the way I think about writing a program in C

1230
00:57:42.230 --> 00:57:46.092
is you're having to make a 100 two-foot putts.

1231
00:57:46.092 --> 00:57:47.844
Two-foot putts are easy, you can just go out,

1232
00:57:47.844 --> 00:57:48.837
you could make a two-foot putt.

1233
00:57:48.837 --> 00:57:51.208
But try to make a 100 two-foot putts in a row,

1234
00:57:51.208 --> 00:57:52.941
chances are you're gonna miss one of those.

1235
00:57:52.941 --> 00:57:54.787
So even the best programmers will make those kinds

1236
00:57:54.787 --> 00:57:57.073
of mistakes, even Tiger Woods misses a two-foot putt,

1237
00:57:57.073 --> 00:57:59.951
probably more these days.

1238
00:57:59.951 --> 00:58:01.608
So when you have mediocre programmers, they're missing

1239
00:58:01.608 --> 00:58:04.243
two-foot putts 10 times out of a 100 instead of two

1240
00:58:04.243 --> 00:58:06.991
times out of a 100, so I think the answer is still the same.

1241
00:58:06.991 --> 00:58:08.765
Yes, we have to retrain them, we have to give the right

1242
00:58:08.765 --> 00:58:11.441
mentality but I think having the right methods,

1243
00:58:11.441 --> 00:58:15.608
the right tools, the right processes would really help

1244
00:58:17.045 --> 00:58:20.180
improve the situation even with those programmers.

1245
00:58:20.180 --> 00:58:21.877
If you didn't leave all of the testing

1246
00:58:21.877 --> 00:58:23.714
and all of the choices to mediocre programmers

1247
00:58:23.714 --> 00:58:26.206
but instead you said I'm gonna use this language,

1248
00:58:26.206 --> 00:58:28.183
I'm gonna use this analysis tool,

1249
00:58:28.183 --> 00:58:29.771
I'm gonna use this testing strategy,

1250
00:58:29.771 --> 00:58:31.710
and you have no way around it.

1251
00:58:31.710 --> 00:58:33.056
The best software houses, this is what they do.

1252
00:58:33.056 --> 00:58:34.986
At Microsoft, look there is a process,

1253
00:58:34.986 --> 00:58:37.581
you will follow it and that's it.

1254
00:58:37.581 --> 00:58:40.159
Maybe it limits some of the creativity of your best

1255
00:58:40.159 --> 00:58:43.644
programmers but it also limits the harm

1256
00:58:43.644 --> 00:58:45.644
by numerous programmers.

1257
00:58:50.841 --> 00:58:52.408
<v Sameer>Actually I have a question.</v>

1258
00:58:52.408 --> 00:58:54.541
One of the things that happens when you plug

1259
00:58:54.541 --> 00:58:56.471
in your computer to the internet,

1260
00:58:56.471 --> 00:58:58.464
it gives the challenge for anybody in the world

1261
00:58:58.464 --> 00:59:00.200
to hack it, to break it.

1262
00:59:00.200 --> 00:59:02.725
So you could also argue that they need more gateways,

1263
00:59:02.725 --> 00:59:04.570
at the network level to (mumbles)

1264
00:59:04.570 --> 00:59:06.771
if we're going to solve the problem,

1265
00:59:06.771 --> 00:59:08.248
just thought that I'd ask you.

1266
00:59:08.248 --> 00:59:10.729
<v ->That's a very good point.</v>

1267
00:59:10.729 --> 00:59:12.260
We've all heard of firewalls.

1268
00:59:12.260 --> 00:59:14.425
I briefly mentioned in the beginning,

1269
00:59:14.425 --> 00:59:17.175
and the idea with the firewall is

1270
00:59:18.970 --> 00:59:21.165
we're gonna limit the avenues of attack.

1271
00:59:21.165 --> 00:59:24.032
If I just connect to the network, now I have a general

1272
00:59:24.032 --> 00:59:26.274
network connection, if there's an attacker on the other side

1273
00:59:26.274 --> 00:59:28.908
of the network, maybe there's a way that they can connect

1274
00:59:28.908 --> 00:59:31.157
to my computer and perpetrate an attack.

1275
00:59:31.157 --> 00:59:32.963
So at one extreme, if I don't connect my computer

1276
00:59:32.963 --> 00:59:35.855
to the network, they can't attack me.

1277
00:59:35.855 --> 00:59:38.740
I need to have some way of getting outside information in,

1278
00:59:38.740 --> 00:59:40.631
maybe I could take that straight towards (mumbles)

1279
00:59:40.631 --> 00:59:42.177
and stick it in, and that's a mistake,

1280
00:59:42.177 --> 00:59:44.183
that would be a way to get outside information,

1281
00:59:44.183 --> 00:59:46.820
but if I have no connection, there is no attack.

1282
00:59:46.820 --> 00:59:49.241
Well the firewall tries to bring that idea into play

1283
00:59:49.241 --> 00:59:52.582
as well by saying "well I will limit the avenue by which

1284
00:59:52.582 --> 00:59:55.630
you can connect to my computer," but unfortunately,

1285
00:59:55.630 --> 00:59:58.313
this doesn't work because even if you have a hole

1286
00:59:58.313 --> 01:00:00.603
that's this small, attackers are very clever

1287
01:00:00.603 --> 01:00:03.128
and they will find a way to work through that tiny hole.

1288
01:00:03.128 --> 01:00:06.415
So for example, there is a protocol called SOAP,

1289
01:00:06.415 --> 01:00:09.677
which is used to communicate arbitrary data over the

1290
01:00:09.677 --> 01:00:13.015
protocol for the web, so every firewall at every company

1291
01:00:13.015 --> 01:00:15.289
has web traffic come in.

1292
01:00:15.289 --> 01:00:17.969
If I'm Amazon, I want people to connect to my web server

1293
01:00:17.969 --> 01:00:19.709
and buy my products.

1294
01:00:19.709 --> 01:00:22.160
Well I could use SOAP to go over the web protocol

1295
01:00:22.160 --> 01:00:24.398
which will certainly be allowed through my firewall

1296
01:00:24.398 --> 01:00:26.354
to try to perpetrate an attack by doing something

1297
01:00:26.354 --> 01:00:27.614
a little bit different.

1298
01:00:27.614 --> 01:00:29.837
And developers will work around these limits that are built

1299
01:00:29.837 --> 01:00:34.801
in by firewalls and attackers will work around them too.

1300
01:00:34.801 --> 01:00:36.376
I don't wanna give the impression that I think

1301
01:00:36.376 --> 01:00:38.347
what all these companies do is a waste of time.

1302
01:00:38.347 --> 01:00:40.273
They are stopping attacks if you haven't patched

1303
01:00:40.273 --> 01:00:43.840
your software yet, they will stop that.

1304
01:00:43.840 --> 01:00:45.548
It will limit the attack surface,

1305
01:00:45.548 --> 01:00:47.104
if people aren't smart enough to find a way through

1306
01:00:47.104 --> 01:00:49.698
the hole, so it is important to have these things.

1307
01:00:49.698 --> 01:00:51.865
But it's still backward-looking,

1308
01:00:51.865 --> 01:00:53.599
it's still not really solving the problem.

1309
01:00:53.599 --> 01:00:56.651
It's just stemming the tide of the attacks,

1310
01:00:56.651 --> 01:00:58.783
and unfortunately, that tide is rising.

1311
01:00:58.783 --> 01:01:03.308
So it's doing less good of a job that it used to.

1312
01:01:03.308 --> 01:01:04.859
<v Sameer>Elaf.</v>

1313
01:01:04.859 --> 01:01:09.694
<v Attendee>I'm wondering the educational value of the</v>

1314
01:01:09.694 --> 01:01:11.777
build-it, break-it thing.

1315
01:01:13.045 --> 01:01:16.196
You said people that knew C and chose not to use it,

1316
01:01:16.196 --> 01:01:20.712
they scored well but if you actually lost the contest,

1317
01:01:20.712 --> 01:01:23.641
if they were less likely to use C in the future (mumbles),

1318
01:01:23.641 --> 01:01:25.537
do you see a consequence of that so to speak?

1319
01:01:25.537 --> 01:01:27.391
<v ->That's an interesting question, so I had not thought</v>

1320
01:01:27.391 --> 01:01:29.450
to ask that question on the post-contest survey

1321
01:01:29.450 --> 01:01:31.740
so I will ask it this time.

1322
01:01:31.740 --> 01:01:35.569
The current contest starts Thursday so we'll have another

1323
01:01:35.569 --> 01:01:37.948
round of data and I'll make sure to ask that question,

1324
01:01:37.948 --> 01:01:40.116
that's a good question.

1325
01:01:40.116 --> 01:01:40.974
<v Sameer>Just to make a quick announcement,</v>

1326
01:01:40.974 --> 01:01:43.102
we have a reception in the room here.

1327
01:01:43.102 --> 01:01:44.618
There's food for about a hundred people,

1328
01:01:44.618 --> 01:01:47.121
so there's enough for you and a friend of yours,

1329
01:01:47.121 --> 01:01:49.291
please join us for the reception with Mike,

1330
01:01:49.291 --> 01:01:50.506
and let's thank him for a wonderful talk.

1331
01:01:50.506 --> 01:01:53.173
(loud applause)