[
  {
    "start": 0.001,
    "end": 2.42,
    "text": "That’s why it’s so surprising that actually,"
  },
  {
    "start": 3.32,
    "end": 4.4,
    "text": "when we have this tester mindset,"
  },
  {
    "start": 4.67,
    "end": 7.66,
    "text": "we still make the mistakes that we say other people make"
  },
  {
    "start": 8.14,
    "end": 8.92,
    "text": "because they’re not tested."
  },
  {
    "start": 9.76,
    "end": 12.14,
    "text": "I once had a team that was very proud and they said,"
  },
  {
    "start": 12.43,
    "end": 15.0,
    "text": "we have built 10,000 end-to-end test cases."
  },
  {
    "start": 15.92,
    "end": 16.52,
    "text": "And I’m like,"
  },
  {
    "start": 16.68,
    "end": 17.44,
    "text": "then you’re doing it wrong."
  },
  {
    "start": 18.64,
    "end": 19.8,
    "text": "So I found the first bugs"
  },
  {
    "start": 19.96,
    "end": 21.76,
    "text": "before they had written any line of code."
  },
  {
    "start": 22.54,
    "end": 24.62,
    "text": "I was always invited to those meetings"
  },
  {
    "start": 24.9,
    "end": 25.48,
    "text": "because they were like,"
  },
  {
    "start": 25.93,
    "end": 26.98,
    "text": "you have special skills."
  },
  {
    "start": 27.03,
    "end": 28.76,
    "text": "You can see errors before they happen."
  },
  {
    "start": 29.12,
    "end": 33.12,
    "text": "A lot is about communication and AI won’t fix that for us."
  },
  {
    "start": 33.74,
    "end": 36.18,
    "text": "Welcome to Software Testing Unleashed."
  },
  {
    "start": 36.2,
    "end": 41.54,
    "text": "My name is Richie and I have another episode from the Istanbul Software Testing Conference"
  },
  {
    "start": 41.66,
    "end": 43.24,
    "text": "2026 for you."
  },
  {
    "start": 43.5,
    "end": 47.66,
    "text": "My guest was Bart Knaack and we talked about the dark side of testing,"
  },
  {
    "start": 47.82,
    "end": 48.8,
    "text": "the failures we"
  },
  {
    "start": 48.84,
    "end": 50.26,
    "text": "cause by testing itself,"
  },
  {
    "start": 50.6,
    "end": 54.04,
    "text": "how test data leaks out of poorly protected environments,"
  },
  {
    "start": 54.4,
    "end": 59.1,
    "text": "why testing in production needs its own risk assessment and what helps before you hit"
  },
  {
    "start": 59.1,
    "end": 60.12,
    "text": "the start button."
  },
  {
    "start": 60.38,
    "end": 63.44,
    "text": "That’s all you find in this episode and now enjoy."
  },
  {
    "start": 73.74,
    "end": 77.56,
    "text": "Hi Bart, great to have you on the show here. Hi Richie, yeah it’s good to be here."
  },
  {
    "start": 78.2,
    "end": 81.7,
    "text": "We are at the Istanbul Software Technology Conference, oh no,"
  },
  {
    "start": 81.94,
    "end": 82.22,
    "text": "Testing"
  },
  {
    "start": 82.34,
    "end": 84.8,
    "text": "Conference and I like it."
  },
  {
    "start": 85.08,
    "end": 89.08,
    "text": "It’s a wonderful location, they have made a"
  },
  {
    "start": 89.08,
    "end": 91.36,
    "text": "And this is their third edition and I’m impressed."
  },
  {
    "start": 91.6,
    "end": 92.34,
    "text": "I’m really impressed."
  },
  {
    "start": 92.38,
    "end": 93.04,
    "text": "Yes, it’s always."
  },
  {
    "start": 93.1,
    "end": 95.92,
    "text": "I was here last year too and it was really,"
  },
  {
    "start": 96.1,
    "end": 96.52,
    "text": "really nice."
  },
  {
    "start": 96.52,
    "end": 101.3,
    "text": "And it’s great to come back and to see this active community and the professional setup."
  },
  {
    "start": 101.6,
    "end": 102.34,
    "text": "So it’s really,"
  },
  {
    "start": 102.82,
    "end": 103.56,
    "text": "really enjoyable."
  },
  {
    "start": 103.8,
    "end": 104.84,
    "text": "They really did a good job."
  },
  {
    "start": 105.04,
    "end": 105.28,
    "text": "Yeah."
  },
  {
    "start": 105.68,
    "end": 106.24,
    "text": "Kudos for them."
  },
  {
    "start": 106.48,
    "end": 106.62,
    "text": "Yeah."
  },
  {
    "start": 106.88,
    "end": 107.42,
    "text": "And of course,"
  },
  {
    "start": 107.8,
    "end": 110.26,
    "text": "being selected as a speaker gives me some bias,"
  },
  {
    "start": 110.42,
    "end": 110.96,
    "text": "but no,"
  },
  {
    "start": 111.14,
    "end": 112.54,
    "text": "it’s a good job."
  },
  {
    "start": 113.58,
    "end": 118.28,
    "text": "I want to talk about your topic you have here in the speech."
  },
  {
    "start": 118.96,
    "end": 121.12,
    "text": "about the darker side of testing."
  },
  {
    "start": 123.44,
    "end": 124.86,
    "text": "And when I read it,"
  },
  {
    "start": 125.1,
    "end": 125.44,
    "text": "I thought,"
  },
  {
    "start": 126.06,
    "end": 126.4,
    "text": "yeah,"
  },
  {
    "start": 126.9,
    "end": 127.78,
    "text": "you’re right."
  },
  {
    "start": 128.02,
    "end": 128.72,
    "text": "Yeah, we have to."
  },
  {
    "start": 128.92,
    "end": 130.32,
    "text": "Yeah, we have to think about that."
  },
  {
    "start": 131.24,
    "end": 132.36,
    "text": "So let’s get into it."
  },
  {
    "start": 132.36,
    "end": 134.08,
    "text": "What do you mean with the dark side?"
  },
  {
    "start": 134.88,
    "end": 137.34,
    "text": "Well, we do testing because we want to prevent stuff"
  },
  {
    "start": 137.36,
    "end": 138.48,
    "text": "from going wrong in production, right?"
  },
  {
    "start": 138.72,
    "end": 138.8,
    "text": "Yeah."
  },
  {
    "start": 138.96,
    "end": 139.72,
    "text": "So we test."
  },
  {
    "start": 140.12,
    "end": 140.58,
    "text": "Therefore,"
  },
  {
    "start": 141.12,
    "end": 142.6,
    "text": "it goes smoothly into production."
  },
  {
    "start": 142.64,
    "end": 143.86,
    "text": "We have no production failures."
  },
  {
    "start": 144.76,
    "end": 148.52,
    "text": "But sometimes the mere act of testing causes failures,"
  },
  {
    "start": 148.6,
    "end": 149.92,
    "text": "causes stuff to go wrong."
  },
  {
    "start": 150.94,
    "end": 152.24,
    "text": "It’s mostly because we don’t"
  },
  {
    "start": 152.42,
    "end": 154.0,
    "text": "think enough before we start testing."
  },
  {
    "start": 154.72,
    "end": 156.74,
    "text": "So what are the consequences of doing a test?"
  },
  {
    "start": 157.26,
    "end": 159.6,
    "text": "If you can’t bear the consequence of your test,"
  },
  {
    "start": 160.42,
    "end": 160.88,
    "text": "don’t test it."
  },
  {
    "start": 162.1,
    "end": 164.9,
    "text": "One of the questions that is always very interesting is,"
  },
  {
    "start": 165.74,
    "end": 166.66,
    "text": "should we actually build this?"
  },
  {
    "start": 167.1,
    "end": 168.14,
    "text": "That’s the first question."
  },
  {
    "start": 168.64,
    "end": 169.4,
    "text": "The next question is,"
  },
  {
    "start": 169.66,
    "end": 170.66,
    "text": "how are we going to test this?"
  },
  {
    "start": 171.3,
    "end": 174.42,
    "text": "Then it’s not just about setting up test cases,"
  },
  {
    "start": 174.42,
    "end": 176.8,
    "text": "but also what’s the environment going to be like?"
  },
  {
    "start": 177.02,
    "end": 178.56,
    "text": "How are we going to get the right data?"
  },
  {
    "start": 180.4,
    "end": 183.56,
    "text": "Can we build the environment good enough to actually do what we want to do?"
  },
  {
    "start": 183.64,
    "end": 184.82,
    "text": "For a performance test, for instance."
  },
  {
    "start": 185.4,
    "end": 189.84,
    "text": "If you don’t have a separate performance or a production-like environment,"
  },
  {
    "start": 190.48,
    "end": 191.84,
    "text": "are you going to do testing in production?"
  },
  {
    "start": 193.22,
    "end": 194.1,
    "text": "Questions like that."
  },
  {
    "start": 194.38,
    "end": 195.94,
    "text": "And then see where it goes wrong."
  },
  {
    "start": 196.0,
    "end": 199.42,
    "text": "And I have 10 beautiful examples from my own experience,"
  },
  {
    "start": 200.28,
    "end": 200.96,
    "text": "which I can tell about."
  },
  {
    "start": 201.24,
    "end": 201.38,
    "text": "Ah,"
  },
  {
    "start": 201.54,
    "end": 201.62,
    "text": "okay."
  },
  {
    "start": 201.9,
    "end": 203.22,
    "text": "Can you tell us what,"
  },
  {
    "start": 204.08,
    "end": 204.82,
    "text": "for some examples,"
  },
  {
    "start": 204.86,
    "end": 205.4,
    "text": "for this,"
  },
  {
    "start": 206.04,
    "end": 207.2,
    "text": "not about the consequences,"
  },
  {
    "start": 207.38,
    "end": 209.38,
    "text": "but about the failures we introduced with"
  },
  {
    "start": 209.66,
    "end": 210.18,
    "text": "testing?"
  },
  {
    "start": 210.4,
    "end": 210.66,
    "text": "Yeah, well,"
  },
  {
    "start": 211.1,
    "end": 213.0,
    "text": "one of them is test data leakage."
  },
  {
    "start": 213.6,
    "end": 217.26,
    "text": "If we have non-depersonalized test"
  },
  {
    "start": 217.48,
    "end": 222.6,
    "text": "data on our system and the system is not as well protected as production,"
  },
  {
    "start": 223.68,
    "end": 224.22,
    "text": "people can get"
  },
  {
    "start": 224.32,
    "end": 226.4,
    "text": "into a test environment and get the data from there."
  },
  {
    "start": 227.58,
    "end": 228.06,
    "text": "Or,"
  },
  {
    "start": 228.42,
    "end": 228.94,
    "text": "even worse,"
  },
  {
    "start": 229.64,
    "end": 230.38,
    "text": "at some point someone"
  },
  {
    "start": 230.38,
    "end": 232.54,
    "text": "someone walked into my office and said,"
  },
  {
    "start": 232.64,
    "end": 234.38,
    "text": "\"Can you depersonalize some data for me?\""
  },
  {
    "start": 234.38,
    "end": 235.0,
    "text": "I said, \"Yeah, sure."
  },
  {
    "start": 235.22,
    "end": 237.22,
    "text": "Put it in a safe environment.\""
  },
  {
    "start": 237.22,
    "end": 237.58,
    "text": "He said, \"No, no."
  },
  {
    "start": 237.84,
    "end": 238.96,
    "text": "I have it on the CD here.\""
  },
  {
    "start": 239.579,
    "end": 239.9,
    "text": "[LAUGHTER]"
  },
  {
    "start": 240.66,
    "end": 242.4,
    "text": "I took the CD and I just broke it."
  },
  {
    "start": 242.66,
    "end": 243.48,
    "text": "He said, \"What are you doing?"
  },
  {
    "start": 243.56,
    "end": 244.32,
    "text": "That’s my test data.\""
  },
  {
    "start": 244.32,
    "end": 247.46,
    "text": "I said, \"Yeah, but it’s production data on the CD,"
  },
  {
    "start": 247.94,
    "end": 248.9,
    "text": "which is non-governed."
  },
  {
    "start": 249.32,
    "end": 250.08,
    "text": "We’re not going to do that.\""
  },
  {
    "start": 250.24,
    "end": 250.34,
    "text": "Yeah."
  },
  {
    "start": 250.88,
    "end": 250.98,
    "text": "Right."
  },
  {
    "start": 251.06,
    "end": 252.74,
    "text": "So it’s also thinking about the process,"
  },
  {
    "start": 252.9,
    "end": 254.36,
    "text": "thinking about how can you set up"
  },
  {
    "start": 254.38,
    "end": 256.0,
    "text": "your tests in a safe and secure way."
  },
  {
    "start": 256.14,
    "end": 256.42,
    "text": "Yeah."
  },
  {
    "start": 257.14,
    "end": 260.56,
    "text": "Yeah, I think we have to think about that."
  },
  {
    "start": 261.14,
    "end": 262.88,
    "text": "So test data is the one thing."
  },
  {
    "start": 264.1,
    "end": 265.54,
    "text": "Test environments can be another thing."
  },
  {
    "start": 265.66,
    "end": 268.32,
    "text": "How is your test environment being protected against hackers?"
  },
  {
    "start": 268.68,
    "end": 271.96,
    "text": "Can hackers get into your test environment and see how the code is working over there?"
  },
  {
    "start": 272.52,
    "end": 276.16,
    "text": "Because if they can, they can use those failures to go into production."
  },
  {
    "start": 276.58,
    "end": 276.74,
    "text": "Yeah."
  },
  {
    "start": 277.08,
    "end": 282.74,
    "text": "So giving them a backseat or a look at your code"
  },
  {
    "start": 282.74,
    "end": 285.68,
    "text": "from an environment which is not as well protected"
  },
  {
    "start": 285.96,
    "end": 287.22,
    "text": "because it’s only a test environment."
  },
  {
    "start": 288.4,
    "end": 290.3,
    "text": "Think about how to protect your test environment."
  },
  {
    "start": 291.919,
    "end": 293.94,
    "text": "And I will give you one more example."
  },
  {
    "start": 294.78,
    "end": 295.7,
    "text": "Testing in production."
  },
  {
    "start": 297.22,
    "end": 300.08,
    "text": "Then be very sure that you clean up afterwards"
  },
  {
    "start": 300.48,
    "end": 302.06,
    "text": "or that you do the proper thing."
  },
  {
    "start": 302.18,
    "end": 304.02,
    "text": "Or if you have a performance test"
  },
  {
    "start": 304.02,
    "end": 306.04,
    "text": "and you have a separate performance environment,"
  },
  {
    "start": 306.54,
    "end": 309.52,
    "text": "is it really independent of your production environment?"
  },
  {
    "start": 309.74,
    "end": 313.1,
    "text": "Those questions you need to ask before you do stuff like that."
  },
  {
    "start": 313.42,
    "end": 315.22,
    "text": "And that’s what my presentation will be mostly about."
  },
  {
    "start": 315.44,
    "end": 315.58,
    "text": "Yeah."
  },
  {
    "start": 316.02,
    "end": 317.08,
    "text": "And when you say,"
  },
  {
    "start": 317.54,
    "end": 318.66,
    "text": "the examples you bring,"
  },
  {
    "start": 318.78,
    "end": 321.98,
    "text": "it looks to me it’s more on the non-functional side."
  },
  {
    "start": 322.16,
    "end": 322.38,
    "text": "So,"
  },
  {
    "start": 322.78,
    "end": 323.72,
    "text": "it’s security,"
  },
  {
    "start": 324.0,
    "end": 324.9,
    "text": "it’s performance,"
  },
  {
    "start": 326.26,
    "end": 327.76,
    "text": "governance and this stuff."
  },
  {
    "start": 327.94,
    "end": 328.84,
    "text": "Not only,"
  },
  {
    "start": 329.1,
    "end": 329.64,
    "text": "not only."
  },
  {
    "start": 329.86,
    "end": 332.52,
    "text": "But yeah, there’s a lot of the non-functional side."
  },
  {
    "start": 332.6,
    "end": 333.58,
    "text": "And why is that?"
  },
  {
    "start": 333.86,
    "end": 338.02,
    "text": "Because on the functional side, we mostly do okay."
  },
  {
    "start": 338.26,
    "end": 341.9,
    "text": "But like I said, the test data in a test environment that you can get out"
  },
  {
    "start": 342.3,
    "end": 343.38,
    "text": "could also be functional testing."
  },
  {
    "start": 343.52,
    "end": 347.92,
    "text": "So in that sense, but yeah, we mostly think quite well about functional testing"
  },
  {
    "start": 348.0,
    "end": 351.9,
    "text": "and then the non-functionals are just a side thing that we do on the side."
  },
  {
    "start": 352.06,
    "end": 353.0,
    "text": "And yeah."
  },
  {
    "start": 353.16,
    "end": 357.56,
    "text": "I think we are a normal tester when we think of functional testing"
  },
  {
    "start": 357.94,
    "end": 360.66,
    "text": "is often not aware of all this stuff."
  },
  {
    "start": 360.7,
    "end": 365.08,
    "text": "And I think maybe that’s also the problem that we don’t think about these consequences a lot."
  },
  {
    "start": 365.26,
    "end": 365.5,
    "text": "Yeah."
  },
  {
    "start": 366.54,
    "end": 369.52,
    "text": "So how can we deal with that?"
  },
  {
    "start": 369.6,
    "end": 374.4,
    "text": "How can we come into a mood to think about these consequences and to get rid of them?"
  },
  {
    "start": 375.38,
    "end": 375.78,
    "text": "Well,"
  },
  {
    "start": 376.06,
    "end": 380.7,
    "text": "first of all, I think also as a tester, you need to have some insight in the infrastructure of"
  },
  {
    "start": 381.58,
    "end": 382.72,
    "text": "your system that you’re working with."
  },
  {
    "start": 383.5,
    "end": 386.78,
    "text": "If you know the infrastructure, you also can see maybe some vulnerabilities in there."
  },
  {
    "start": 387.42,
    "end": 392.06,
    "text": "If you know that, then you also know what you want to do and what you do not want to do in your test environment."
  },
  {
    "start": 392.94,
    "end": 394.8,
    "text": "If you’re going to do testing in production,"
  },
  {
    "start": 396.52,
    "end": 397.56,
    "text": "if you’re going to do it,"
  },
  {
    "start": 398.76,
    "end": 401.12,
    "text": "make sure that you know what you’re doing in production,"
  },
  {
    "start": 401.38,
    "end": 402.7,
    "text": "because you are touching production."
  },
  {
    "start": 402.86,
    "end": 404.58,
    "text": "You’re actually doing something in the production system."
  },
  {
    "start": 404.96,
    "end": 407.04,
    "text": "And I do not advocate testing in production,"
  },
  {
    "start": 408.16,
    "end": 409.26,
    "text": "but sometimes you have no choice."
  },
  {
    "start": 409.56,
    "end": 411.08,
    "text": "Sometimes you have to do it there."
  },
  {
    "start": 411.72,
    "end": 416.26,
    "text": "But be very aware of the impact that a failing test can have on your production system."
  },
  {
    "start": 416.9,
    "end": 419.8,
    "text": "If you can bring down a production system with a failing test,"
  },
  {
    "start": 421.18,
    "end": 422.1,
    "text": "When are you going to do it?"
  },
  {
    "start": 422.28,
    "end": 422.4,
    "text": "Yeah."
  },
  {
    "start": 423.08,
    "end": 426.52,
    "text": "So it’s another risk assessment we have to do."
  },
  {
    "start": 426.72,
    "end": 430.02,
    "text": "So it’s like a risk assessment of the risk process itself."
  },
  {
    "start": 430.24,
    "end": 430.48,
    "text": "Yeah."
  },
  {
    "start": 431.16,
    "end": 438.12,
    "text": "And I think that’s it’s hard for us because a lot of testers don’t think about the normal product risks."
  },
  {
    "start": 438.66,
    "end": 442.8,
    "text": "So and not to think about in this meta perspective about the risks."
  },
  {
    "start": 443.24,
    "end": 444.54,
    "text": "And the funny thing is,"
  },
  {
    "start": 444.94,
    "end": 446.38,
    "text": "whenever I need to do something like this,"
  },
  {
    "start": 446.39,
    "end": 450.539,
    "text": "I take one of the network engineers and I invite him into the session"
  },
  {
    "start": 450.54,
    "end": 451.64,
    "text": "Because he knows, hey,"
  },
  {
    "start": 451.82,
    "end": 453.86,
    "text": "the test environment is being connected to that environment."
  },
  {
    "start": 454.96,
    "end": 455.94,
    "text": "What’s going wrong here?"
  },
  {
    "start": 456.18,
    "end": 458.32,
    "text": "Yeah, the test environment is being set up in such a way."
  },
  {
    "start": 459.16,
    "end": 460.34,
    "text": "Why are we using that database?"
  },
  {
    "start": 460.42,
    "end": 463.54,
    "text": "Right. So so so you have some from from the from the ops team"
  },
  {
    "start": 463.94,
    "end": 467.36,
    "text": "or the network team to sit with you to think about, OK, I’m setting this up."
  },
  {
    "start": 467.62,
    "end": 468.58,
    "text": "This test up like this."
  },
  {
    "start": 468.82,
    "end": 468.94,
    "text": "Yeah."
  },
  {
    "start": 469.3,
    "end": 470.64,
    "text": "What could possibly go wrong?"
  },
  {
    "start": 470.92,
    "end": 471.12,
    "text": "Yeah."
  },
  {
    "start": 471.32,
    "end": 471.52,
    "text": "Yeah."
  },
  {
    "start": 471.98,
    "end": 474.02,
    "text": "What could possibly go wrong?"
  },
  {
    "start": 474.14,
    "end": 474.34,
    "text": "Yeah."
  },
  {
    "start": 475.16,
    "end": 475.42,
    "text": "OK."
  },
  {
    "start": 475.54,
    "end": 479.0,
    "text": "Yeah, we we we often want to don’t want to look in this direction."
  },
  {
    "start": 479.06,
    "end": 481.12,
    "text": "No, we don’t, but we are testers, we should."
  },
  {
    "start": 481.3,
    "end": 481.78,
    "text": "Yeah, we should."
  },
  {
    "start": 481.9,
    "end": 483.62,
    "text": "That’s our job."
  },
  {
    "start": 483.62,
    "end": 488.18,
    "text": "And that’s why it’s so surprising that actually when we have this tester mindset,"
  },
  {
    "start": 488.47,
    "end": 488.82,
    "text": "we still"
  },
  {
    "start": 488.92,
    "end": 492.8,
    "text": "make the mistakes that we say other people make because they’re not tested."
  },
  {
    "start": 492.86,
    "end": 493.12,
    "text": "Yeah,"
  },
  {
    "start": 493.3,
    "end": 493.4,
    "text": "yeah,"
  },
  {
    "start": 493.62,
    "end": 493.72,
    "text": "yeah."
  },
  {
    "start": 495.08,
    "end": 495.34,
    "text": "Yeah,"
  },
  {
    "start": 495.56,
    "end": 495.92,
    "text": "that’s true."
  },
  {
    "start": 498.18,
    "end": 501.22,
    "text": "Do you have any idea for,"
  },
  {
    "start": 501.76,
    "end": 505.4,
    "text": "maybe you have some, for a checklist or something?"
  },
  {
    "start": 505.82,
    "end": 511.2,
    "text": "When it comes to my mind that a checklist would help a lot because I think the problems are often"
  },
  {
    "start": 511.88,
    "end": 515.58,
    "text": "the same or what you think about this data environments and all this stuff."
  },
  {
    "start": 515.86,
    "end": 515.96,
    "text": "Yeah,"
  },
  {
    "start": 516.12,
    "end": 518.479,
    "text": "so I have made some checklists in the past."
  },
  {
    "start": 519.87,
    "end": 522.38,
    "text": "The problem with checklists is a checklist"
  },
  {
    "start": 522.86,
    "end": 527.42,
    "text": "helps you remember stuff but the checklist also stops you from thinking for yourself."
  },
  {
    "start": 527.62,
    "end": 527.76,
    "text": "Yeah."
  },
  {
    "start": 528.16,
    "end": 531.18,
    "text": "Right, because you have the checklist so this must be right,"
  },
  {
    "start": 531.36,
    "end": 532.14,
    "text": "that cannot go wrong."
  },
  {
    "start": 532.38,
    "end": 534.02,
    "text": "We know we photographed it, by the way."
  },
  {
    "start": 534.819,
    "end": 536.46,
    "text": "So it cannot go wrong."
  },
  {
    "start": 537.08,
    "end": 537.78,
    "text": "We have the checklist."
  },
  {
    "start": 537.92,
    "end": 540.06,
    "text": "We’ve done the checklist and we stopped thinking for ourselves."
  },
  {
    "start": 540.54,
    "end": 540.94,
    "text": "So,"
  },
  {
    "start": 542.58,
    "end": 543.68,
    "text": "although I know what you’re saying,"
  },
  {
    "start": 543.84,
    "end": 545.36,
    "text": "and I kind of agree with it,"
  },
  {
    "start": 545.48,
    "end": 549.1,
    "text": "I just try to keep away from checklists as much as possible."
  },
  {
    "start": 549.62,
    "end": 551.7,
    "text": "I’d rather have some guidance or suggestions"
  },
  {
    "start": 551.94,
    "end": 554.2,
    "text": "or have some questions like you had in your talk."
  },
  {
    "start": 554.32,
    "end": 555.76,
    "text": "What are the questions you can ask?"
  },
  {
    "start": 556.6,
    "end": 559.9,
    "text": "Give some guidance on what you should think about"
  },
  {
    "start": 559.98,
    "end": 561.32,
    "text": "and not give them the checklist and say,"
  },
  {
    "start": 561.38,
    "end": 563.58,
    "text": "think, think, think, think, everything’s done,"
  },
  {
    "start": 563.98,
    "end": 564.6,
    "text": "now I’m safe."
  },
  {
    "start": 564.78,
    "end": 565.78,
    "text": "Because you’re never safe."
  },
  {
    "start": 565.96,
    "end": 566.08,
    "text": "Yeah."
  },
  {
    "start": 566.64,
    "end": 566.8,
    "text": "Right?"
  },
  {
    "start": 566.98,
    "end": 568.2,
    "text": "So that’s the big thing."
  },
  {
    "start": 568.38,
    "end": 568.5,
    "text": "Yeah."
  },
  {
    "start": 568.94,
    "end": 570.76,
    "text": "And how would you approach,"
  },
  {
    "start": 571.62,
    "end": 574.48,
    "text": "so you said you get some people from the networking staff,"
  },
  {
    "start": 574.62,
    "end": 578.46,
    "text": "and I think there are a lot of people who have value-added things."
  },
  {
    "start": 578.76,
    "end": 581.84,
    "text": "Would you suggest make a big meeting with everyone or a workshop"
  },
  {
    "start": 582.12,
    "end": 585.66,
    "text": "or going one-to-one with them or what’s your experience?"
  },
  {
    "start": 585.7,
    "end": 586.04,
    "text": "It depends."
  },
  {
    "start": 586.5,
    "end": 590.32,
    "text": "If you have a big end-to-end chain that you need to look at,"
  },
  {
    "start": 590.32,
    "end": 593.92,
    "text": "and sometimes having the group meeting makes sense"
  },
  {
    "start": 594.14,
    "end": 596.06,
    "text": "because people can inspire each other and stuff like that."
  },
  {
    "start": 596.12,
    "end": 596.2,
    "text": "Hey,"
  },
  {
    "start": 596.68,
    "end": 597.52,
    "text": "this is for your system,"
  },
  {
    "start": 597.53,
    "end": 599.72,
    "text": "but that’s not true for our system, stuff like that."
  },
  {
    "start": 600.7,
    "end": 602.24,
    "text": "If I do a smaller session,"
  },
  {
    "start": 602.36,
    "end": 604.14,
    "text": "then I mostly go one-on-one with people,"
  },
  {
    "start": 604.66,
    "end": 606.44,
    "text": "and I send them the end result and I say,"
  },
  {
    "start": 606.54,
    "end": 608.82,
    "text": "\"Hey, can you please check if what I said was right,"
  },
  {
    "start": 609.22,
    "end": 612.28,
    "text": "and if you get inspired by what other people have said.\""
  },
  {
    "start": 612.36,
    "end": 613.54,
    "text": "Because setting up big meetings"
  },
  {
    "start": 613.8,
    "end": 615.96,
    "text": "with agendas and stuff like that is always a hassle,"
  },
  {
    "start": 616.6,
    "end": 618.56,
    "text": "whereas one-on-ones are mostly very easy."
  },
  {
    "start": 618.73,
    "end": 619.78,
    "text": "Sometimes even at the coffee corner,"
  },
  {
    "start": 619.84,
    "end": 620.28,
    "text": "and just say, \"Hey,"
  },
  {
    "start": 620.5,
    "end": 621.14,
    "text": "can I talk to you?"
  },
  {
    "start": 621.58,
    "end": 622.84,
    "text": "\"Can you give me 10 minutes?\""
  },
  {
    "start": 623.26,
    "end": 626.16,
    "text": "And I normally do them in team sessions and I record them."
  },
  {
    "start": 626.18,
    "end": 627.48,
    "text": "I say, \"I’m gonna record this session"
  },
  {
    "start": 627.55,
    "end": 630.22,
    "text": "\"because I want to have the information afterwards.\""
  },
  {
    "start": 630.22,
    "end": 632.76,
    "text": "Do a transcribe and then use that transcribe already"
  },
  {
    "start": 633.08,
    "end": 634.4,
    "text": "as a first part of the report."
  },
  {
    "start": 634.92,
    "end": 635.06,
    "text": "Yeah,"
  },
  {
    "start": 635.24,
    "end": 635.34,
    "text": "yeah."
  },
  {
    "start": 636.2,
    "end": 639.28,
    "text": "How do you come with this argument"
  },
  {
    "start": 639.54,
    "end": 641.12,
    "text": "when then someone says,"
  },
  {
    "start": 641.3,
    "end": 641.44,
    "text": "\"Okay,"
  },
  {
    "start": 641.96,
    "end": 643.68,
    "text": "but it will never happen.\""
  },
  {
    "start": 643.68,
    "end": 645.42,
    "text": "So that will never happen that this"
  },
  {
    "start": 645.46,
    "end": 646.4,
    "text": "I have a nice example of that."
  },
  {
    "start": 648.24,
    "end": 650.26,
    "text": "I used to work at a firm and they did,"
  },
  {
    "start": 651.32,
    "end": 651.68,
    "text": "well,"
  },
  {
    "start": 652.44,
    "end": 653.42,
    "text": "it doesn’t matter what they did."
  },
  {
    "start": 653.63,
    "end": 655.3,
    "text": "They had two systems connected to each other."
  },
  {
    "start": 655.82,
    "end": 658.0,
    "text": "And I, as a tester, found out that if two messages"
  },
  {
    "start": 658.16,
    "end": 659.62,
    "text": "were overtaking,"
  },
  {
    "start": 660.21,
    "end": 660.86,
    "text": "it would go wrong."
  },
  {
    "start": 661.26,
    "end": 661.4,
    "text": "Oh,"
  },
  {
    "start": 661.58,
    "end": 661.68,
    "text": "okay."
  },
  {
    "start": 661.98,
    "end": 663.1,
    "text": "And then the guy said, yeah,"
  },
  {
    "start": 663.2,
    "end": 665.66,
    "text": "but this is a direct link between two systems."
  },
  {
    "start": 666.28,
    "end": 667.84,
    "text": "Messages will never overtake."
  },
  {
    "start": 668.1,
    "end": 669.54,
    "text": "So you don’t have to worry about this."
  },
  {
    "start": 669.62,
    "end": 670.58,
    "text": "That’s a networking issue."
  },
  {
    "start": 671.38,
    "end": 672.88,
    "text": "And then I said, if I can think about it,"
  },
  {
    "start": 673.02,
    "end": 674.1,
    "text": "it can happen in real life."
  },
  {
    "start": 675.14,
    "end": 675.92,
    "text": "And then he said, no, no, no, no."
  },
  {
    "start": 676.04,
    "end": 677.22,
    "text": "This is a one-on-one,"
  },
  {
    "start": 678.1,
    "end": 678.88,
    "text": "it will never happen."
  },
  {
    "start": 679.04,
    "end": 679.44,
    "text": "I said,"
  },
  {
    "start": 679.66,
    "end": 680.66,
    "text": "okay, give me this in writing,"
  },
  {
    "start": 680.9,
    "end": 681.62,
    "text": "put down your signature,"
  },
  {
    "start": 682.18,
    "end": 684.32,
    "text": "and I will put it at the defect that I found,"
  },
  {
    "start": 684.46,
    "end": 686.98,
    "text": "and if it’s gonna go wrong, it’s gonna be on you."
  },
  {
    "start": 687.23,
    "end": 688.44,
    "text": "He said, yeah, yeah, yeah, you can blame me."
  },
  {
    "start": 689.4,
    "end": 692.28,
    "text": "Three months later, network traffic came up so much"
  },
  {
    "start": 692.34,
    "end": 694.04,
    "text": "that this one cable wasn’t enough anymore,"
  },
  {
    "start": 694.84,
    "end": 697.22,
    "text": "so they put in a network in between."
  },
  {
    "start": 697.64,
    "end": 699.44,
    "text": "Now messages could overtake."
  },
  {
    "start": 700.12,
    "end": 701.3,
    "text": "And then they came to me and said,"
  },
  {
    "start": 701.46,
    "end": 702.44,
    "text": "why haven’t you tested this?"
  },
  {
    "start": 702.46,
    "end": 704.62,
    "text": "And I just slid in the report."
  },
  {
    "start": 705.08,
    "end": 706.78,
    "text": "So sometimes you cannot convince people,"
  },
  {
    "start": 707.2,
    "end": 708.8,
    "text": "but then at least if you cannot convince them,"
  },
  {
    "start": 708.88,
    "end": 712.76,
    "text": "make sure that you prove that you have done the right thing"
  },
  {
    "start": 712.82,
    "end": 714.02,
    "text": "and that you have tried to warn them."
  },
  {
    "start": 714.64,
    "end": 715.82,
    "text": "And in most cases,"
  },
  {
    "start": 716.0,
    "end": 717.12,
    "text": "people will listen, by the way."
  },
  {
    "start": 717.22,
    "end": 718.08,
    "text": "In most cases, they will say,"
  },
  {
    "start": 718.18,
    "end": 720.86,
    "text": "\"Oh, that might be something that we need to look at.\""
  },
  {
    "start": 720.92,
    "end": 723.6,
    "text": "\"Oh, you have a link with production."
  },
  {
    "start": 724.0,
    "end": 725.78,
    "text": "Maybe we should look at that.\""
  },
  {
    "start": 725.92,
    "end": 726.96,
    "text": "But yeah, in some cases,"
  },
  {
    "start": 727.33,
    "end": 728.28,
    "text": "you cannot convince people,"
  },
  {
    "start": 728.4,
    "end": 730.76,
    "text": "but then at least make sure that it’s being noted"
  },
  {
    "start": 731.08,
    "end": 731.96,
    "text": "and that you can say, \"Hey,"
  },
  {
    "start": 732.72,
    "end": 734.379,
    "text": "I tried to warn you.\""
  },
  {
    "start": 734.38,
    "end": 736.38,
    "text": "But what I heard out is,"
  },
  {
    "start": 736.8,
    "end": 738.1,
    "text": "I like this idea,"
  },
  {
    "start": 738.63,
    "end": 739.38,
    "text": "you don’t say,"
  },
  {
    "start": 739.66,
    "end": 740.22,
    "text": "don’t do it,"
  },
  {
    "start": 740.8,
    "end": 741.92,
    "text": "just to think about it."
  },
  {
    "start": 742.04,
    "end": 743.14,
    "text": "What are you doing there?"
  },
  {
    "start": 743.24,
    "end": 744.18,
    "text": "So when we think about,"
  },
  {
    "start": 744.62,
    "end": 745.52,
    "text": "especially about now,"
  },
  {
    "start": 745.8,
    "end": 748.56,
    "text": "going shift right and doing more in production and stuff,"
  },
  {
    "start": 748.66,
    "end": 748.82,
    "text": "because,"
  },
  {
    "start": 749.32,
    "end": 751.6,
    "text": "so to be aware of what we are doing there."
  },
  {
    "start": 752.02,
    "end": 752.74,
    "text": "Oh yes, oh yes."
  },
  {
    "start": 752.8,
    "end": 756.2,
    "text": "And always ask yourself the same question again,"
  },
  {
    "start": 757.12,
    "end": 759.26,
    "text": "why are we doing this and why are we doing it this way?"
  },
  {
    "start": 759.52,
    "end": 759.66,
    "text": "Yeah."
  },
  {
    "start": 760.44,
    "end": 761.52,
    "text": "Because it’s not only the why,"
  },
  {
    "start": 761.62,
    "end": 763.16,
    "text": "but also why are we doing it this way?"
  },
  {
    "start": 763.18,
    "end": 764.5,
    "text": "Think about the process as well."
  },
  {
    "start": 765.2,
    "end": 766.6,
    "text": "We’re going to do this test."
  },
  {
    "start": 767.08,
    "end": 767.98,
    "text": "Why are we doing that test?"
  },
  {
    "start": 768.5,
    "end": 769.04,
    "text": "Because blah, blah, blah."
  },
  {
    "start": 769.74,
    "end": 770.92,
    "text": "Why are we doing it this way?"
  },
  {
    "start": 771.22,
    "end": 771.36,
    "text": "Yeah."
  },
  {
    "start": 771.76,
    "end": 772.8,
    "text": "Because there is no other way."
  },
  {
    "start": 773.08,
    "end": 773.96,
    "text": "Is there no other way?"
  },
  {
    "start": 774.12,
    "end": 774.24,
    "text": "Yeah."
  },
  {
    "start": 774.32,
    "end": 775.32,
    "text": "No, there is no other way."
  },
  {
    "start": 775.52,
    "end": 777.76,
    "text": "OK, then what risks do we need to fix"
  },
  {
    "start": 777.96,
    "end": 779.06,
    "text": "before we can do it in this way?"
  },
  {
    "start": 779.2,
    "end": 779.34,
    "text": "Yeah."
  },
  {
    "start": 779.62,
    "end": 781.0,
    "text": "Or if we can find an alternative,"
  },
  {
    "start": 781.54,
    "end": 783.62,
    "text": "maybe we have a less risky alternative"
  },
  {
    "start": 783.88,
    "end": 785.5,
    "text": "that will prove the same thing"
  },
  {
    "start": 785.58,
    "end": 786.18,
    "text": "that we want to prove."
  },
  {
    "start": 786.32,
    "end": 787.64,
    "text": "Yeah, I think that’s a good idea"
  },
  {
    "start": 787.76,
    "end": 789.08,
    "text": "to think about an alternative."
  },
  {
    "start": 789.28,
    "end": 791.0,
    "text": "Do you have any example for that"
  },
  {
    "start": 791.04,
    "end": 792.139,
    "text": "or from your experience"
  },
  {
    "start": 792.14,
    "end": 792.48,
    "text": "where you switch to another?"
  },
  {
    "start": 792.48,
    "end": 792.64,
    "text": "Oh yeah,"
  },
  {
    "start": 792.84,
    "end": 792.92,
    "text": "well,"
  },
  {
    "start": 793.48,
    "end": 794.46,
    "text": "there’s a..."
  },
  {
    "start": 795.14,
    "end": 797.46,
    "text": "I once had a team that was very proud and they said,"
  },
  {
    "start": 797.84,
    "end": 800.4,
    "text": "\"We have built 10,000 end-to-end test cases.\""
  },
  {
    "start": 801.24,
    "end": 802.84,
    "text": "And I’m like, \"Then you’re doing it wrong.\""
  },
  {
    "start": 802.92,
    "end": 803.12,
    "text": "Yeah."
  },
  {
    "start": 803.64,
    "end": 804.72,
    "text": "And they looked at me and said, \"Why?\""
  },
  {
    "start": 804.72,
    "end": 805.1,
    "text": "I said, \"Well,"
  },
  {
    "start": 805.3,
    "end": 806.42,
    "text": "there must be alternatives"
  },
  {
    "start": 806.5,
    "end": 808.88,
    "text": "to do these tests at the earlier stages.\""
  },
  {
    "start": 808.92,
    "end": 809.1,
    "text": "Yeah."
  },
  {
    "start": 809.34,
    "end": 812.6,
    "text": "And I found out that they did the address check,"
  },
  {
    "start": 812.98,
    "end": 814.38,
    "text": "whether the address was valid or not,"
  },
  {
    "start": 814.9,
    "end": 815.86,
    "text": "in the end-to-end test."
  },
  {
    "start": 816.32,
    "end": 816.64,
    "text": "And I said,"
  },
  {
    "start": 816.8,
    "end": 818.52,
    "text": "\"So why don’t you do that during unit testing?\""
  },
  {
    "start": 818.54,
    "end": 818.7,
    "text": "Yeah."
  },
  {
    "start": 819.52,
    "end": 820.26,
    "text": "And they looked at me and they said,"
  },
  {
    "start": 820.34,
    "end": 821.98,
    "text": "\"Yeah, but if we do it during unit testing,"
  },
  {
    "start": 822.06,
    "end": 824.54,
    "text": "We don’t know if it’s working in end-to-end as well."
  },
  {
    "start": 824.54,
    "end": 827.18,
    "text": "I said, well, I’m pretty sure if it works at the unit test,"
  },
  {
    "start": 827.2,
    "end": 828.16,
    "text": "it will work at end-to-end."
  },
  {
    "start": 828.44,
    "end": 828.56,
    "text": "Yeah."
  },
  {
    "start": 829.04,
    "end": 831.26,
    "text": "So they actually did the unit test thing"
  },
  {
    "start": 831.32,
    "end": 834.74,
    "text": "and in the end they only needed 50 end-to-end test cases."
  },
  {
    "start": 834.76,
    "end": 834.88,
    "text": "Yeah."
  },
  {
    "start": 835.46,
    "end": 836.88,
    "text": "Just by thinking about the alternative,"
  },
  {
    "start": 837.16,
    "end": 838.08,
    "text": "what are you trying to prove?"
  },
  {
    "start": 838.8,
    "end": 839.72,
    "text": "Where can you prove it?"
  },
  {
    "start": 839.82,
    "end": 841.64,
    "text": "And do you have to prove it at the point"
  },
  {
    "start": 841.74,
    "end": 842.66,
    "text": "where you’re now proving it?"
  },
  {
    "start": 842.86,
    "end": 843.04,
    "text": "Yeah,"
  },
  {
    "start": 843.2,
    "end": 843.34,
    "text": "yeah."
  },
  {
    "start": 844.2,
    "end": 846.78,
    "text": "Yeah, it’s a clear, but it’s often not easy to start."
  },
  {
    "start": 847.14,
    "end": 852.82,
    "text": "So when now a tester hears that episode and says,"
  },
  {
    "start": 852.96,
    "end": 853.12,
    "text": "\"Okay,"
  },
  {
    "start": 853.72,
    "end": 854.5,
    "text": "we should do that,"
  },
  {
    "start": 855.32,
    "end": 856.48,
    "text": "but how to start?\""
  },
  {
    "start": 856.58,
    "end": 858.1,
    "text": "Yeah, well, that’s a very good question."
  },
  {
    "start": 858.24,
    "end": 861.06,
    "text": "And actually what I found out, and okay,"
  },
  {
    "start": 861.65,
    "end": 862.2,
    "text": "to be honest,"
  },
  {
    "start": 862.4,
    "end": 865.2,
    "text": "I don’t believe in the silos of testers and developers anymore."
  },
  {
    "start": 865.94,
    "end": 867.4,
    "text": "It’s a collaborative thing, right?"
  },
  {
    "start": 867.48,
    "end": 868.56,
    "text": "Quality is a collaborative thing."
  },
  {
    "start": 869.12,
    "end": 873.6,
    "text": "So sit with your developer as a tester and discuss this."
  },
  {
    "start": 873.9,
    "end": 874.46,
    "text": "Look at this."
  },
  {
    "start": 874.78,
    "end": 880.08,
    "text": "And I’ve actually found out that while we were doing joint workshops with previous testers"
  },
  {
    "start": 880.16,
    "end": 881.24,
    "text": "and previous developers,"
  },
  {
    "start": 881.5,
    "end": 884.02,
    "text": "we found out that the developers don’t know the testing techniques."
  },
  {
    "start": 884.44,
    "end": 885.82,
    "text": "They don’t know boundary value analysis,"
  },
  {
    "start": 885.98,
    "end": 887.26,
    "text": "they don’t know equivalence partitioning,"
  },
  {
    "start": 887.66,
    "end": 887.96,
    "text": "which are"
  },
  {
    "start": 888.24,
    "end": 890.4,
    "text": "extremely useful to set up your unit test cases."
  },
  {
    "start": 891.22,
    "end": 895.42,
    "text": "So we actually helped the developers in setting up proper unit test cases."
  },
  {
    "start": 895.8,
    "end": 900.1,
    "text": "We helped them to set the proper asserts because they were only doing code coverage and they"
  },
  {
    "start": 900.12,
    "end": 901.98,
    "text": "had assert is true at the bottom of the test."
  },
  {
    "start": 902.5,
    "end": 904.3,
    "text": "Then we said, \"Okay, what is this test proving?\""
  },
  {
    "start": 905.06,
    "end": 907.46,
    "text": "We don’t know, but it does increase the code coverage."
  },
  {
    "start": 908.52,
    "end": 909.5,
    "text": "Does it actually fail?"
  },
  {
    "start": 909.88,
    "end": 911.32,
    "text": "No, there’s an assert is true down there."
  },
  {
    "start": 911.76,
    "end": 914.58,
    "text": "So it actually knew that the test was flawed,"
  },
  {
    "start": 915.4,
    "end": 919.28,
    "text": "but it was needed for them to get the code coverage."
  },
  {
    "start": 919.88,
    "end": 921.66,
    "text": "Then I actually introduced them to mutation testing,"
  },
  {
    "start": 921.74,
    "end": 923.4,
    "text": "which is also one of the presentations here today."
  },
  {
    "start": 924.52,
    "end": 924.92,
    "text": "They were like,"
  },
  {
    "start": 926.22,
    "end": 926.48,
    "text": "\"Wow,"
  },
  {
    "start": 926.7,
    "end": 927.24,
    "text": "how?\""
  },
  {
    "start": 927.82,
    "end": 929.66,
    "text": "So that actually proved the value of"
  },
  {
    "start": 929.7,
    "end": 931.699,
    "text": "your unit test cases much better than"
  },
  {
    "start": 931.7,
    "end": 935.02,
    "text": "having this code coverage fallacy that people think will help them."
  },
  {
    "start": 935.2,
    "end": 935.34,
    "text": "Yeah."
  },
  {
    "start": 935.88,
    "end": 938.76,
    "text": "So to start with the nearest team members,"
  },
  {
    "start": 939.0,
    "end": 940.1,
    "text": "like developers or something,"
  },
  {
    "start": 940.26,
    "end": 943.22,
    "text": "is a good way to start discussing about it."
  },
  {
    "start": 943.5,
    "end": 945.66,
    "text": "And I think it’s also a process,"
  },
  {
    "start": 945.88,
    "end": 947.6,
    "text": "not just a one-time thing."
  },
  {
    "start": 947.8,
    "end": 951.0,
    "text": "Don’t think you’re going to solve it in one day, but start slowly,"
  },
  {
    "start": 951.52,
    "end": 952.24,
    "text": "give them examples,"
  },
  {
    "start": 952.78,
    "end": 955.38,
    "text": "let them experience what it is to actually do stuff like that."
  },
  {
    "start": 955.74,
    "end": 958.739,
    "text": "We now are introducing contract-based testing"
  },
  {
    "start": 958.74,
    "end": 961.44,
    "text": "just to get the interfaces between"
  },
  {
    "start": 961.6,
    "end": 963.88,
    "text": "the consumer and producer interface of"
  },
  {
    "start": 963.98,
    "end": 965.96,
    "text": "two APIs to actually work together in the proper way."
  },
  {
    "start": 967.0,
    "end": 968.9,
    "text": "In the early days, 20 years ago,"
  },
  {
    "start": 969.51,
    "end": 970.42,
    "text": "I did that by hand."
  },
  {
    "start": 970.59,
    "end": 970.8,
    "text": "Yeah."
  },
  {
    "start": 971.25,
    "end": 972.78,
    "text": "I had the consumer protocol,"
  },
  {
    "start": 972.95,
    "end": 974.28,
    "text": "I had the producer protocol,"
  },
  {
    "start": 974.78,
    "end": 977.16,
    "text": "and I would just match them and I would see the mismatches."
  },
  {
    "start": 977.52,
    "end": 980.64,
    "text": "So I found the first bugs before they had written any line of code."
  },
  {
    "start": 980.64,
    "end": 980.88,
    "text": "Yeah."
  },
  {
    "start": 981.35,
    "end": 982.72,
    "text": "I was always invited to"
  },
  {
    "start": 982.78,
    "end": 984.38,
    "text": "those meetings because they were like,"
  },
  {
    "start": 984.92,
    "end": 985.88,
    "text": "\"You have special skills."
  },
  {
    "start": 986.02,
    "end": 987.859,
    "text": "You can see errors before they happen.\""
  },
  {
    "start": 987.86,
    "end": 988.84,
    "text": "Yeah, the magic."
  },
  {
    "start": 989.12,
    "end": 990.62,
    "text": "All I did was just look at the protocol,"
  },
  {
    "start": 990.76,
    "end": 992.6,
    "text": "look at the other protocols, and this is not going to work."
  },
  {
    "start": 992.84,
    "end": 994.22,
    "text": "You have an integer over here,"
  },
  {
    "start": 994.24,
    "end": 995.28,
    "text": "it’s a string of what?"
  },
  {
    "start": 995.72,
    "end": 996.48,
    "text": "This is not going to work."
  },
  {
    "start": 997.06,
    "end": 998.66,
    "text": "Right? So stuff like that."
  },
  {
    "start": 1000.06,
    "end": 1000.42,
    "text": "Sometimes,"
  },
  {
    "start": 1001.44,
    "end": 1003.5,
    "text": "and this actually what it boils down to a lot,"
  },
  {
    "start": 1004.26,
    "end": 1006.74,
    "text": "is it’s almost always a lack of communication."
  },
  {
    "start": 1007.52,
    "end": 1008.8,
    "text": "People don’t talk to each other."
  },
  {
    "start": 1009.56,
    "end": 1011.02,
    "text": "People don’t listen to each other."
  },
  {
    "start": 1011.4,
    "end": 1012.44,
    "text": "People just do."
  },
  {
    "start": 1012.95,
    "end": 1013.7,
    "text": "They get an assignment,"
  },
  {
    "start": 1013.94,
    "end": 1014.88,
    "text": "they go their own way,"
  },
  {
    "start": 1015.02,
    "end": 1016.26,
    "text": "they do what they think is necessary,"
  },
  {
    "start": 1016.34,
    "end": 1019.08,
    "text": "and then it comes back together and it doesn’t work."
  },
  {
    "start": 1019.56,
    "end": 1019.82,
    "text": "Why?"
  },
  {
    "start": 1020.42,
    "end": 1020.82,
    "text": "(laughs)"
  },
  {
    "start": 1021.36,
    "end": 1022.56,
    "text": "It’s very, very strange."
  },
  {
    "start": 1022.96,
    "end": 1024.12,
    "text": "Yeah, it’s all about communication."
  },
  {
    "start": 1024.319,
    "end": 1024.8,
    "text": "It’s all,"
  },
  {
    "start": 1025.02,
    "end": 1026.14,
    "text": "a lot is about communication"
  },
  {
    "start": 1026.48,
    "end": 1027.98,
    "text": "and AI won’t fix that for us."
  },
  {
    "start": 1028.04,
    "end": 1028.14,
    "text": "Yeah,"
  },
  {
    "start": 1028.3,
    "end": 1028.819,
    "text": "yeah, that’s true."
  },
  {
    "start": 1029.5,
    "end": 1029.6,
    "text": "Yeah."
  },
  {
    "start": 1030.66,
    "end": 1033.68,
    "text": "Bart, thank you very much that you shared your experience"
  },
  {
    "start": 1033.72,
    "end": 1034.88,
    "text": "and give us this inspiration."
  },
  {
    "start": 1035.02,
    "end": 1036.959,
    "text": "I think we have to think about this stuff,"
  },
  {
    "start": 1037.02,
    "end": 1039.939,
    "text": "what can go wrong when we test it this way or that way."
  },
  {
    "start": 1040.28,
    "end": 1042.339,
    "text": "I think that’s a very good idea to think about"
  },
  {
    "start": 1042.4,
    "end": 1043.98,
    "text": "and a good takeaway from this episode."
  },
  {
    "start": 1044.6,
    "end": 1045.739,
    "text": "Thank you very much that you were here"
  },
  {
    "start": 1045.74,
    "end": 1047.64,
    "text": "and I hope you enjoy the rest of the conference."
  },
  {
    "start": 1047.85,
    "end": 1048.16,
    "text": "I will."
  },
  {
    "start": 1048.66,
    "end": 1049.88,
    "text": "I most certainly will."
  },
  {
    "start": 1050.24,
    "end": 1051.72,
    "text": "I give you one final thought."
  },
  {
    "start": 1051.72,
    "end": 1052.0,
    "text": "Yeah,"
  },
  {
    "start": 1052.18,
    "end": 1052.54,
    "text": "sure."
  },
  {
    "start": 1052.57,
    "end": 1052.7,
    "text": "Okay."
  },
  {
    "start": 1053.56,
    "end": 1055.08,
    "text": "So if you test an AI,"
  },
  {
    "start": 1057.14,
    "end": 1060.38,
    "text": "what are the things you should worry about when testing an AI?"
  },
  {
    "start": 1061.88,
    "end": 1063.5,
    "text": "Because that’s going to be the next level, right?"
  },
  {
    "start": 1063.5,
    "end": 1063.62,
    "text": "Yeah."
  },
  {
    "start": 1063.62,
    "end": 1065.32,
    "text": "We’re going to have to test AIs."
  },
  {
    "start": 1066.14,
    "end": 1069.04,
    "text": "There’s now been a documentary by the BBC where"
  },
  {
    "start": 1069.04,
    "end": 1072.18,
    "text": "they actually trained AIs to be malicious in a training environment."
  },
  {
    "start": 1072.82,
    "end": 1076.62,
    "text": "And those AIs were able to hide those true intentions"
  },
  {
    "start": 1077.91,
    "end": 1078.82,
    "text": "in the test environment"
  },
  {
    "start": 1079.9,
    "end": 1081.78,
    "text": "and only show them when they went into production."
  },
  {
    "start": 1081.96,
    "end": 1084.46,
    "text": "Now, a lot of people will say, \"Yeah, that’s fake news."
  },
  {
    "start": 1084.54,
    "end": 1087.26,
    "text": "That’s a hype that they made to kill AI.\""
  },
  {
    "start": 1088.2,
    "end": 1089.46,
    "text": "If it’s not happening now,"
  },
  {
    "start": 1090.21,
    "end": 1091.14,
    "text": "it will happen in the future."
  },
  {
    "start": 1091.78,
    "end": 1094.68,
    "text": "A great thing we have to talk about and think about."
  },
  {
    "start": 1095.28,
    "end": 1096.08,
    "text": "Thank you, Bart,"
  },
  {
    "start": 1096.46,
    "end": 1098.98,
    "text": "for this another idea at the end."
  },
  {
    "start": 1099.02,
    "end": 1099.78,
    "text": "Thank you very much."
  },
  {
    "start": 1100.18,
    "end": 1100.7,
    "text": "Have a good time."
  },
  {
    "start": 1100.82,
    "end": 1101.88,
    "text": "Thank you."
  }
]