WEBVTT

00:00:00.000 --> 00:00:01.360
So it happened.

00:00:01.820 --> 00:00:05.880
The world's first AI software engineer

00:00:05.880 --> 00:00:08.700
called Devin was introduced by

00:00:08.700 --> 00:00:11.080
Cognition AI, a company that, well,

00:00:11.500 --> 00:00:13.260
builds AI programs as it seems.

00:00:13.580 --> 00:00:16.239
For example, Devin, this AI software

00:00:16.239 --> 00:00:16.780
engineer.

00:00:17.640 --> 00:00:20.960
And that of course means that now the

00:00:20.960 --> 00:00:24.000
end is near for us software developers

00:00:24.000 --> 00:00:25.100
and engineers, right?

00:00:25.940 --> 00:00:27.680
Well, maybe not yet.

00:00:27.820 --> 00:00:32.040
And I'll share why I'm impressed or

00:00:32.040 --> 00:00:35.820
interested by and in Devin, but why I

00:00:35.820 --> 00:00:39.240
don't feel like it'll replace you or me

00:00:39.240 --> 00:00:40.060
anytime soon.

00:00:40.140 --> 00:00:40.800
And I'll explain.

00:00:41.420 --> 00:00:44.280
But first things first, what exactly is

00:00:44.280 --> 00:00:44.920
Devin?

00:00:45.020 --> 00:00:47.180
What is this thing that was introduced?

00:00:47.820 --> 00:00:50.960
Well, it's a program that in the end uses

00:00:50.960 --> 00:00:54.660
AI models under the hood to solve

00:00:54.660 --> 00:00:57.520
problems that are given to the program.

00:00:57.720 --> 00:01:00.120
And with problems, I don't mean, hey,

00:01:00.340 --> 00:01:03.800
give me the code for storing a file on

00:01:03.800 --> 00:01:06.420
AWSS free with Node.js.

00:01:07.180 --> 00:01:10.720
No, I mean, bigger problems, which then

00:01:10.720 --> 00:01:12.680
would require multiple steps to solve

00:01:12.560 --> 00:01:15.400
them and which would require logical

00:01:15.400 --> 00:01:16.840
thinking to solve them.

00:01:17.100 --> 00:01:19.420
I mean, these kinds of problems, like,

00:01:19.680 --> 00:01:23.220
hey, build me an online webshop

00:01:23.220 --> 00:01:26.540
website where users can authenticate,

00:01:27.180 --> 00:01:29.240
add products to their cart, and then

00:01:29.240 --> 00:01:30.760
proceed to checkout.

00:01:30.980 --> 00:01:32.440
I mean, problems like this.

00:01:33.160 --> 00:01:37.220
That's what Devin is meant to solve.

00:01:37.560 --> 00:01:38.460
These kinds of problems.

00:01:39.660 --> 00:01:41.020
And that, of course, sounds pretty

00:01:41.020 --> 00:01:44.260
impressive or intimidating, depending

00:01:44.260 --> 00:01:46.480
on where you come from, right?

00:01:46.780 --> 00:01:49.320
And indeed, Cognition AI, the company

00:01:49.320 --> 00:01:52.220
that built Devin, so that built this AI

00:01:52.220 --> 00:01:55.560
software engineer, they shared a video,

00:01:55.840 --> 00:01:57.800
an announcement video, and actually

00:01:57.800 --> 00:02:00.340
they shared multiple detailed videos as

00:02:00.340 --> 00:02:02.860
well, where they show what Devin can

00:02:02.860 --> 00:02:03.140
do.

00:02:05.020 --> 00:02:06.380
And they, for example, show that it was

00:02:06.380 --> 00:02:09.060
given the task of analyzing the

00:02:09.060 --> 00:02:11.620
performance of different AI models.

00:02:12.220 --> 00:02:14.420
And then Devin went ahead and wrote

00:02:14.420 --> 00:02:16.160
a program that would do that.

00:02:16.460 --> 00:02:18.240
And in that announcement video, they

00:02:18.240 --> 00:02:21.500
also showed which tools Devin is able to

00:02:21.500 --> 00:02:24.040
use at the moment to solve problems.

00:02:24.580 --> 00:02:26.960
Since it's meant to be an AI software

00:02:26.960 --> 00:02:28.820
engineer, it has access to a terminal

00:02:28.820 --> 00:02:30.880
where it can run commands to, for

00:02:30.880 --> 00:02:33.560
example, create a new project or run a

00:02:33.560 --> 00:02:34.020
project.

00:02:34.760 --> 00:02:37.560
It has an IDE, of course, to write code.

00:02:38.360 --> 00:02:41.140
And Devin has access to a browser to

00:02:41.140 --> 00:02:44.260
research, to take a look at official

00:02:44.260 --> 00:02:46.340
documentation pages, for example.

00:02:46.780 --> 00:02:49.040
And of course, also, if it's building a

00:02:49.040 --> 00:02:51.160
website, to test that website.

00:02:51.600 --> 00:02:53.480
And then Devin goes ahead and in the

00:02:53.480 --> 00:02:57.460
end, uses AI models or AI APIs behind

00:02:57.460 --> 00:03:00.480
the scenes to solve a problem step by

00:03:00.480 --> 00:03:00.980
step.

00:03:01.060 --> 00:03:03.840
So it's issuing multiple requests to those

00:03:03.840 --> 00:03:04.600
AI models.

00:03:04.760 --> 00:03:07.840
The first request might be, hey, how do I

00:03:07.840 --> 00:03:10.280
create a new web project?

00:03:10.580 --> 00:03:12.820
And then with the response given, it

00:03:12.820 --> 00:03:15.120
would execute the command that was

00:03:15.120 --> 00:03:15.620
given to it.

00:03:15.760 --> 00:03:17.840
And then it would continue asking the

00:03:17.840 --> 00:03:19.380
AI models behind the scenes.

00:03:20.080 --> 00:03:23.660
How do I use this framework or which

00:03:23.660 --> 00:03:25.360
framework should I use for this?

00:03:26.200 --> 00:03:28.060
What's the official documentation page?

00:03:28.140 --> 00:03:29.900
It would then maybe go to that page

00:03:29.900 --> 00:03:33.580
and pass the results it finds there back

00:03:33.580 --> 00:03:35.840
into the AI model to understand the

00:03:35.840 --> 00:03:39.260
results and deduct the next steps and so

00:03:39.260 --> 00:03:39.420
on.

00:03:39.460 --> 00:03:40.620
That's essentially how it works.

00:03:41.060 --> 00:03:44.600
And in their announcement post or in a

00:03:44.600 --> 00:03:47.100
video shared there, Cognition AI, for

00:03:47.100 --> 00:03:49.500
example, also showed that Devin was

00:03:49.500 --> 00:03:52.380
able to use these tools to, for example,

00:03:52.400 --> 00:03:54.240
solve jobs on Upwork.

00:03:54.940 --> 00:03:57.140
And in the announcement post on

00:03:57.140 --> 00:03:59.320
Twitter, for example, they also shared

00:03:59.320 --> 00:04:02.940
that they also used Devin to solve open

00:04:02.940 --> 00:04:04.140
GitHub issues.

00:04:04.380 --> 00:04:07.460
And there it was able to solve just a bit

00:04:08.100 --> 00:04:12.660
below 14% of all the issues it was given

00:04:12.660 --> 00:04:16.180
without assistance, which could be

00:04:16.180 --> 00:04:17.260
considered a low number.

00:04:17.519 --> 00:04:19.180
And I'll get back to that, but which is

00:04:19.180 --> 00:04:21.740
actually quite impressive, especially if

00:04:21.740 --> 00:04:25.060
you compare it to the numbers of similar

00:04:25.060 --> 00:04:28.080
projects or programs, which are far

00:04:28.080 --> 00:04:30.020
lower, as they also mentioned in their

00:04:30.020 --> 00:04:30.400
post.

00:04:30.640 --> 00:04:33.220
I think they mentioned that comparable

00:04:33.220 --> 00:04:36.720
programs only achieved around 2%

00:04:36.720 --> 00:04:39.960
without assistance compared to the 14%

00:04:39.960 --> 00:04:43.180
of Devin, so way less, and around 5%

00:04:43.180 --> 00:04:43.920
with assistance.

00:04:44.120 --> 00:04:46.980
So clearly Devin is pretty impressive

00:04:46.980 --> 00:04:49.320
regarding that, and it has some pretty

00:04:49.320 --> 00:04:50.900
interesting capabilities.

00:04:51.460 --> 00:04:52.920
So that's all great.

00:04:53.460 --> 00:04:56.920
But still, I'm not shaking in fear.

00:04:57.140 --> 00:05:00.080
I'm super interested in experiments like

00:05:00.080 --> 00:05:00.400
this.

00:05:00.660 --> 00:05:04.680
And I do believe that as developers, as

00:05:04.680 --> 00:05:07.280
software engineers, we should embrace

00:05:07.280 --> 00:05:09.100
technology like this.

00:05:09.220 --> 00:05:12.060
And I did already talk about that in

00:05:12.060 --> 00:05:13.880
another podcast episode.

00:05:14.220 --> 00:05:17.040
So I think we should really be open to

00:05:17.040 --> 00:05:17.320
that.

00:05:17.360 --> 00:05:20.480
And we should use these tools to make

00:05:20.480 --> 00:05:22.660
our job easier and to become more

00:05:22.660 --> 00:05:23.080
efficient.

00:05:23.400 --> 00:05:27.460
And I would gladly take an AI software

00:05:27.460 --> 00:05:30.880
engineer as a coworker and outsource

00:05:30.880 --> 00:05:34.640
tasks I don't want to do to that AI

00:05:34.640 --> 00:05:35.700
software engineer.

00:05:36.240 --> 00:05:39.900
But when it comes to Devin or other

00:05:39.900 --> 00:05:42.200
programs like Devin that might come

00:05:42.200 --> 00:05:47.640
up in the future, replacing me or you, I'm

00:05:47.640 --> 00:05:49.740
not too convinced that this will happen

00:05:49.740 --> 00:05:50.480
anytime soon.

00:05:50.740 --> 00:05:54.040
And the reason why I'm not afraid is that,

00:05:54.140 --> 00:05:56.740
of course, Devin does some impressive

00:05:56.740 --> 00:05:57.880
stuff here.

00:05:58.340 --> 00:06:03.420
But it's still far off the numbers, the

00:06:03.420 --> 00:06:05.300
numbers of issues solved, for example,

00:06:05.820 --> 00:06:07.800
that would be achieved by a human

00:06:07.800 --> 00:06:09.340
software engineer.

00:06:09.760 --> 00:06:12.300
And in addition, there also isn't too

00:06:12.300 --> 00:06:14.020
much information available regarding

00:06:14.020 --> 00:06:16.460
the exact issues it did solve.

00:06:16.820 --> 00:06:19.340
We don't know how well it would, for

00:06:19.340 --> 00:06:22.640
example, be able to build entire software

00:06:22.640 --> 00:06:24.820
projects that are maybe a bit more

00:06:24.820 --> 00:06:25.280
complex.

00:06:25.480 --> 00:06:27.900
So that's all unknown.

00:06:27.900 --> 00:06:30.100
And you would guess that if Devin

00:06:30.100 --> 00:06:33.360
would be able to build super complex

00:06:33.360 --> 00:06:35.700
applications, they would share an

00:06:35.700 --> 00:06:37.260
example of that as well, right?

00:06:37.520 --> 00:06:39.640
Now, you could say, it's just a matter of

00:06:39.640 --> 00:06:41.800
time, because these AI models will

00:06:41.800 --> 00:06:42.580
improve, right?

00:06:43.100 --> 00:06:45.320
And you would be right to some extent.

00:06:45.780 --> 00:06:49.240
But if we think about other areas where

00:06:49.240 --> 00:06:52.480
AI is used, think about self-driving cars,

00:06:52.900 --> 00:06:54.660
think about speech recognition.

00:06:55.280 --> 00:06:58.180
And if you then consider when we first

00:06:58.180 --> 00:07:01.100
heard about technologies like this, and

00:07:01.100 --> 00:07:03.100
when we first started using such

00:07:03.100 --> 00:07:05.340
technologies, which is pretty much 10

00:07:05.340 --> 00:07:08.020
years ago already, and how far we've

00:07:08.020 --> 00:07:10.380
come, then you would probably agree

00:07:10.380 --> 00:07:14.200
that, yes, there are impressive demos

00:07:14.200 --> 00:07:16.440
there and impressive use cases.

00:07:16.520 --> 00:07:18.280
And there are some valid use cases

00:07:18.280 --> 00:07:21.680
where AI might be used in our day-to

00:07:21.820 --> 00:07:22.180
lives.

00:07:22.360 --> 00:07:25.640
But you would probably also agree that

00:07:25.640 --> 00:07:29.580
AI hasn't really brought us self-driving

00:07:29.580 --> 00:07:32.640
cars yet, no matter if Elon Musk keeps

00:07:32.640 --> 00:07:36.000
repeating that it will be available for

00:07:36.000 --> 00:07:36.980
everyone soon.

00:07:37.660 --> 00:07:41.460
If we think about voice recognition and

00:07:41.460 --> 00:07:44.720
AI assistants that talk to us and that

00:07:44.720 --> 00:07:48.080
really understand us, also not too

00:07:48.080 --> 00:07:49.140
convincing in my opinion.

00:07:49.740 --> 00:07:52.860
And therefore, I feel like we got some

00:07:52.860 --> 00:07:54.060
interesting numbers here.

00:07:54.180 --> 00:07:56.000
We have a very interesting demo here.

00:07:56.200 --> 00:07:58.580
And Devin and similar tools might

00:07:58.580 --> 00:08:02.560
absolutely become helpful tools very

00:08:02.560 --> 00:08:02.980
soon.

00:08:03.260 --> 00:08:06.360
I mean, we have helpful tools like GitHub

00:08:06.360 --> 00:08:09.000
Copilot already, which also become

00:08:09.000 --> 00:08:10.420
better and better over time.

00:08:10.560 --> 00:08:13.360
So don't get me wrong, definitely, they

00:08:13.360 --> 00:08:15.160
will play an important role.

00:08:15.380 --> 00:08:18.600
They will do some of the work we need

00:08:18.600 --> 00:08:18.920
to do.

00:08:19.000 --> 00:08:21.820
But when it comes to replacing software

00:08:21.820 --> 00:08:25.760
developers or engineers altogether, I

00:08:25.760 --> 00:08:29.580
think we still have a very long way to go.

00:08:29.940 --> 00:08:32.539
Because, for example, if we stick to the

00:08:32.539 --> 00:08:36.159
14% of GitHub issues that's an interesting

00:08:36.159 --> 00:08:38.340
number, especially compared to the way

00:08:38.340 --> 00:08:41.179
lower numbers of comparable programs

00:08:41.179 --> 00:08:42.179
and models.

00:08:42.760 --> 00:08:45.980
But of course, it's still far away from what

00:08:45.980 --> 00:08:48.820
a human would be able to achieve here.

00:08:49.080 --> 00:08:51.000
Maybe it would take a human longer,

00:08:51.500 --> 00:08:53.980
but a human, a software engineer who

00:08:53.980 --> 00:08:56.700
knows what they're doing, would be able

00:08:56.700 --> 00:08:59.780
to solve all those issues essentially, and

00:08:59.780 --> 00:09:03.320
not just 14% if they wanted to.

00:09:03.600 --> 00:09:07.260
So that is an impressive number, but it's

00:09:07.260 --> 00:09:10.360
still far away from what we need for such

00:09:10.360 --> 00:09:13.860
a program to actually replace software

00:09:13.860 --> 00:09:15.620
engineers and developers.

00:09:16.500 --> 00:09:19.020
I also already mentioned in that other

00:09:19.020 --> 00:09:21.800
episode, I already mentioned earlier, that

00:09:21.800 --> 00:09:24.520
of course, the main thing developers do

00:09:24.520 --> 00:09:26.980
in the end is solve problems.

00:09:27.460 --> 00:09:29.820
Yes, we write code to solve problems,

00:09:29.920 --> 00:09:31.820
and that is a skill we have.

00:09:32.160 --> 00:09:35.980
But if you're honest, the problem solving

00:09:35.980 --> 00:09:39.360
part and coming up with solutions, and

00:09:39.360 --> 00:09:42.060
then writing code, refining and

00:09:42.060 --> 00:09:44.880
improving code, that's the main thing

00:09:44.880 --> 00:09:45.400
we do.

00:09:46.020 --> 00:09:48.920
And that's the thing an AI will need to do

00:09:48.920 --> 00:09:52.800
really well in order to replace software

00:09:52.800 --> 00:09:54.880
engineers or developers.

00:09:55.500 --> 00:09:59.240
And to me, it does not seem as if Devin

00:09:59.240 --> 00:10:03.500
or other AI models or programs we have

00:10:03.500 --> 00:10:07.180
out there are very close to replace

00:10:07.180 --> 00:10:10.400
human developers when it comes to

00:10:10.400 --> 00:10:10.780
that.

00:10:11.020 --> 00:10:14.020
Now, I'm fully aware that there are a lot

00:10:14.020 --> 00:10:16.180
of people out there who will absolutely

00:10:16.180 --> 00:10:17.720
tell you the opposite.

00:10:17.940 --> 00:10:21.300
And for them, it's very clear that AI will

00:10:21.300 --> 00:10:26.460
replace essentially all of the high skilled

00:10:26.460 --> 00:10:29.280
work humans do nowadays in the next

00:10:29.280 --> 00:10:30.820
five years or so.

00:10:30.940 --> 00:10:33.060
And of course, obviously, no one knows

00:10:33.060 --> 00:10:35.320
what will happen in the future, though

00:10:35.320 --> 00:10:39.820
they know at least if you read what some

00:10:39.820 --> 00:10:40.640
people are writing there.

00:10:40.720 --> 00:10:43.660
But anyways, obviously, no one knows.

00:10:43.800 --> 00:10:46.060
But for the reasons I mentioned, and if

00:10:46.060 --> 00:10:49.440
we take a look at other AI developments,

00:10:49.640 --> 00:10:53.760
like self driving cars and so on, I still am

00:10:53.760 --> 00:10:56.060
pretty confident and I feel pretty

00:10:56.060 --> 00:10:59.120
confident about being a software

00:10:59.120 --> 00:11:01.940
developer and about staying a software

00:11:01.940 --> 00:11:02.540
developer.

00:11:02.780 --> 00:11:05.420
But of course, I'm also interested in

00:11:05.420 --> 00:11:06.740
hearing your opinion.

00:11:07.040 --> 00:11:08.500
What did I forget here?

00:11:08.600 --> 00:11:09.520
What do you think?

00:11:09.680 --> 00:11:10.640
Where am I wrong?

00:11:10.780 --> 00:11:12.140
Or where do you agree?

00:11:12.520 --> 00:11:14.320
Definitely share your opinion.

00:11:14.720 --> 00:11:18.280
And well, I see and hear you in the next

00:11:18.280 --> 00:11:19.400
episode, hopefully.