Welcome to the Manufacturing Woes Show. I’m Sarah Scudder, CMO at SourceDay and the show host today. I am joined by Tom, Dave, Nelson, and Nikki. They have extensive manufacturing experience, and I’ve asked them to share their nightmare stories today. And I think all of them are some of the most well-traveled people I know as well. Dave’s in Alaska, Tom just got back from Italy, Eric’s always traveling at conferences, and Nikki is actually joining us from Chicago. So we’ve got some road warriors with us today. Our show sponsor is RapidRatings. I’ve been friends with their team for many years. I’ve actually been a customer and used their software myself. So I’ve asked Eric to join us and have him do a quick intro and explain specifically what their platform does for companies in the manufacturing space. Thanks, Sarah. Yeah, we’re great to be a sponsor and hope you guys enjoy today’s panel. So yeah, RapidRatings, we provide predictive analytics leveraging artificial intelligence, and it’s all about our models that really help identify those weak suppliers, especially for our manufacturing clients, early on before they really trend down and go bankrupt. And so we really hone in and focus on those issues. And particularly, we’ve been really busy, Sarah, with interest rates and inflation. And so we’ve been leveraging our FHR exchange where companies can share their ratings and scores back and forth and really help each other collaborate and lean in.

Where are you joining us from today, Eric?

Today I’m down in lovely hot and humid Orlando, and then head back to New York in a couple of days, and then Boston and Chicago next week. So yeah, yeah, you’re in what I call conference mode where it’s every single week you’re going to another supply chain conference. Yeah, pretty busy. Thank you for having me.

Well, thanks for joining us on the road, Eric, and again, highly recommend their company product being a former customer. So, would love to have you drop in the chat where you in the world you are joining us from. Always love to see what countries and cities and states people are in. So tell us where you’re joining us from in the chat throughout the conversation today. And then we’d also like to have you share a word or phrase to describe the biggest train wreck you are facing right now in your supply chain. So tell us who you are, where you’re joining us from, biggest train wreck.

And with that, we’re going to kick off our panel today. I’m going to have Tom start by doing a very brief introduction. Tom, why don’t you tell us a little bit about how you landed up in manufacturing, and then would also like to have you share a fun or personal fact about yourself.

Okay, well, Sarah, thank you. So my name is Tom Roddan. I am a long-time IT professional, most recently five years as CEO at Marion Medical Systems. We were just acquired about a year ago, and I was asked to move on as the new executive team from Germany, Siemens acquired us, came to town. So I got a parting gift and have been doing consulting ERP systems consulting since then. But prior to my life as an IT executive and professional, I spent 10 years in supply chain, doing manufacturing planning, transportation management, logistics management, inventory management. So I came to IT, in fact, as a customer initially in the supply chain space. So that’s been an interesting journey.

And a fun fact about myself, I always like to trot this one out, that I got married to the same woman twice. That was a fun fact. It was not actually as strange as it sounds, though. But it was a complex intercultural marriage. My wife is French, and we ended up getting married in a civil ceremony in London where neither of us were technically residents enough to get a kind of a church wedding. And then we ended up getting a more religious ceremony in France, where she was resident. So that was fun when we were both living in Europe at the time, and I was wearing that supply chain hat. So anyway, a little bit of background and a fun fact on me.

So, Tom, you have an example of where a cultural difference led to some serious issues in a warehouse operation. Would like to have you share this story with us.

Sure, yeah, I know that these are always kind of humorous in retrospect, their horror stories while you’re living it. But this was particularly amusing in retrospect. We had designed what we thought was a very sound, robust solution for the warehousing operations, supporting shipping and supporting manufacturing. And it involved RF guns and custom design on how to scan.

And today it’s not as groundbreaking as it was 15, 20 years ago, but we designed a solution that was pretty cutting-edge at the time, SAP-based platform, where the users were scanning the bin, scanning the product we were doing a. We were moving the organization from an order pick process to a batch pick process. And so, in order to do the batch pick after you know the distribution on the floor across all the orders, they had to then get to the floor with their batch picked items and scan again the cart that they were on, saying this is the batch they just picked, and then they would get instructions to the RF gun as to where to place the product. So it was basically a two-step process: do the batch pick and then break it down on the floor to either put it towards the manufacturing line or put it towards a customer order.

And we thought we had designed this, we’ve tested this, and I was responsible at the time for managing the implementation and go-live of this major Europe-wide distribution facility for this client that I was supporting. This is shortly after I joined, moving out of supply chain operations, and thought, ‘I know this stuff inside, now this is what I live, my bread and butter.’ So, we thought we had a great process, and what we found strangely was after day one, a bunch of product seemed to have disappeared. And we sorted that out, did an extra pick, got the orders, got the manufacturing lines running, you know, there was no issue other than we had some inventory discrepancies. Day two, same thing happened. Day three, same thing happened, and we were really scratching our heads, saying, ‘Why are all these discrepancies cropping up?’ I think it must have been the afternoon of day three where I and a couple of the locals started to do a tour, just a walk-around tour of how things looked in this big warehouse.

And we found in a corner of the warehouse a bunch of carts stacked with product. And we said, ‘This is odd, what’s all this stuff doing there?’ And so we asked one of the warehouse operatives, ‘You know, what’s with the carts here in the corner, stacked with product? That appeared to be all the missing product, all the discrepancies.’ They said, ‘Oh, well, you know, when we get to 2 pm or so, and it’s tea time in the UK warehouse, all of the operatives put their carts over here and then they go, you know, have their break. And then they, and I said, ‘Well, then they come back and pick up their carts and finish the work, right?’ And they’re like, ‘Well, yeah, that’s kind of strange, Tom, because what happens, it seems to be that, you know, they turn off their RF guns, which ends their session, and we didn’t really design for this kind of behavior. And then when they finish their break, they turn it back on, and, you know, they’re as if they’re a new user, and they’re getting instructions for a new pick. And what they had left unfinished is just gonna stay there in the corner forever.’ So we were like, ‘Yeah, we didn’t really anticipate in the middle of your shift that you turn off your gun and turn it back on.’ And so, that was simply a, you know, and all this driven by tea time in the UK warehouse.

So, we said, ‘Yeah, that was not part of our design,’ and we did make some corrections for that and did some additional training for people. But we were on the verge of, you know, running out of certain high-volume items for shipping and for manufacturing purposes because it was all stacked up in a corner in this warehouse, and you know, tea time. Yeah, so something, something just to be aware of, there are funny little cultural things that really can upset the best-laid plans. That was my experience, it’s pretty funny in retrospect. Note taken. Thank you for being with us, Tom.

Dave, my friend in Alaska, it’s on my bucket list, I have not yet been. My neighbor got back from a 14-day trip, absolutely loved it, she did like every single excursion possible and wants to go back. So, I’m even more motivated now to make the trek, and appreciate you rocking the SourceDay green, so love the love. Would like to have you introduce yourself, how do you fit into this crazy world of manufacturing, and we’d also like to have you share a fun personal fact. Absolutely, so everyone, hello, my name is Dave Griffith. I got into manufacturing, I like to joke because manufacturing continues to drag me back in, Sarah. I grew up with a family who, you know, for a couple of generations in various parts, worked in automation, worked in manufacturing and industrial verticals. And so, I, of course, as every young boy does, try to do something completely different than what your family does, and every time I did, it was like, ‘Hey, I’m going to go work for this company,’ but this company is a large OEM. And so, ‘Oh, look, there’s experience there,’ or, ‘Hey, I’m going to go found this startup,’ and that’s great, but someone has to figure out how to actually make the stuff. And, oh, Dave, you kind of sort of know how to make the stuff, so you’re now in charge of figuring out how to make the stuff. So, I like to joke that at the beginning, Sarah, manufacturing was that thing that was just pulling me back in until, I don’t know, maybe about 10 years ago where I just relented and have said, ‘Yes, this is what I’m going to do and spend my career doing,’ and it has been great, there’s been lots of opportunity. I love conversations like these. I spent some time kind of growing and building or rebuilding one or two supply chains, and so manufacturing, in many senses, is much broader than just the operators working the line. I would say a fun fact about myself: so, my wife Beth and I have been nomads for most of the last almost six years at this point. And, as you had mentioned earlier, we spent the summer in Alaska. We have about 48 hours left before we start running away before the snow falls.

So, Dave, I’d like to have you share a time in your career when globalization went wrong.

Absolutely, yes. So, as I had mentioned earlier, I spent some time going and rebuilding a couple of supply chains, kind of early to middle of my career. And I worked with a great manufacturer’s rep distributor in the Mid-Atlantic, and one of the companies that was kind of core on the line card was Honeywell Process Solutions. It was just generally a great company. When I got there, it was right after they had transferred all of the manufacturing from their location in Pennsylvania, right? So, they were manufacturing all their stuff in Southern Pennsylvania, it was a short drive from basically any of the offices to go pick the things up if there was really a critical customer need. And then they moved all of that down to Mexico. And so, as I was getting in, it was the kind of transition from, ‘Hey, we can get just about anything we need in, you know, a week or maybe two weeks,’ to, you know, you go send an email and two weeks later it comes back and they’re quoting 26 or 52 weeks. And I know we see a lot of that now, but this was, I don’t know, 10 or so years ago when most people were fully willing and expecting to go get a part in a week or a couple of weeks. And so, that completely changed the dynamic, right? It changed the, to the point of having to stock more items on the shelf, and it changed to the point of doing a lot of the things that we are doing now.

And you would have thought over the course of the first three or four years that it would have gotten better, and fun fact, Sarah, I can tell you it actually, at no point during the first three or four years that I was a part of it, did it ever get better. I hope it’s better now ten years later. It did not get better, and as a fun aside, there were a couple of their salespeople who apparently never read the internal emails that they were moving production down to Mexico. And at one point, they showed up to the old facility and called into the office and asked, ‘Hey, did we move buildings? Did our 2 million square foot manufacturing facility just, like, up and move a couple of miles away? Because we can’t find our offices anymore.’ And so, that, of course, caused lots of problems. I was, in fact, kind of the number one thing that I remember is how forgiving the first people were when I told them it was going to take a year to go get this thing that they told me that they needed. Like, I was the one that was nervous that I’m like, ‘No one will ever be okay with waiting a year for something.’ And I mean, I had customers wait a year or more sometimes. It showed up in two weeks, sometimes it showed up in 18 months. It was kind of when it showed up, you shipped it out and everyone is getting what they get. Sounds like communication gone bad, absolutely. Absolutely. There were a number of issues. I would say that someone somewhere should have written a case study, somewhere. Yes, someone should have written a case study on it, and it could be an entire thesis on things not to do.

Yeah, I love the premise of the show because I feel like a lot of times on panels, we’re sugar-coating and talking about all the good things. There is a lot of crap that happens in supply chain and in manufacturing, and I think it’s important that we talk about it and that we learn from each other. Absolutely. Tom, I’m going to throw it back over to you. When we were prepping for the show, you told me about a case where bad manufacturing planning data that was completely unnecessary led to a bad outcome. Would like to have you share this story. Sure, yeah, now that’s another one from my consulting days. I got to work a lot of different clients in quite a variety of these types of sad stories. So in this case, MPS and MRP are core processes for any manufacturing operation, and they require really good or sound data. For example, in this particular case where the operations were heavily dependent on a large number of suppliers around the globe for a variety of parts, the data on lead times from these vendors was poor, let’s say, and probably vendor behavior, vendor actual performance was a bit erratic. So it was kind of hard, I think, to be fair, to come up with good lead time information, consistently sound lead time information. In any case, I think the data that was input into the system was more or less best-case scenario, you know, expectations from the manufacturing operation that I was supporting. And so, you know, again, we went through the whole process of how is this going to work in the new system, how are the planners going to handle this, going through all the data conversions and data validation, it’s all exactly what the team wanted in the system. And then within weeks, the team was abandoning the system and moving back to their old spreadsheets.

And their explanations were, ‘Well, the system’s giving us all this inaccurate information. All the results from the MRP and MPS runs are all way off, we know it, and we just can’t rely on it. And it’s taking more time to fix all of the planned orders before converting them into firm production orders, so we’re kind of just saying the hell with it. We’re back to our spreadsheets and we’ll manually input production orders when as and when necessary. And we’re just going to do emails and phone contact and just, you know, kind of work with the vendors and manage it all on our spreadsheets, just like we used to do before this system was brought in.’ And that, you know, when we started to dig, what’s the driver, you know? They’re saying, ‘Well, it’s all the vendor master data, it’s all, it’s all bad, it’s all inaccurate.’ They’re saying, ‘Well, you know, we converted the data, you validated the data, you said it was right, now it’s not working.’ Like, ‘Oh yeah, but that was like, it was all kind of best-case stuff, it’s not really what happens, like, well, so then, then it’s not really the fault of the system, right?’ It’s, and so it took, I mean, it took probably a year to overcome what was perceived, not only at the planner level but up the hierarchy in the organization, to be a system problem where there was no trust, and everyone had abandoned it. Executive management heard this was, you know, the system was all messed up, you couldn’t rely on it, and it took a year to basically get everybody aligned again to say, you know, the data needs to be sound and then we can rely on these processes. But it was, it was a nightmare for I think the planning organization itself, even though they were going back to their old process, they were not getting any extra headcount. This was taking ever more time. It was causing downstream issues. The vendor management itself was problematic, it wasn’t really being done in the system the way they were placing the POs, you know, it was also manual, as well as creating the production orders to try and coordinate all this. I mean, they were back to the stone age when they were supposed to be, you know, at least in the 20th if not the 21st century at the time. So that was a huge nightmare for the planners, for the whole manufacturing and procurement operation, and for the implementation team and all down to expectations about the vendor master, you know, lead time data, you know, and just told me you really gotta think through what is it you’re putting into the system for these kinds of data sets and how reliable is it and what margin for error and and not just take it as, well, here’s the data, which is kind of how it happened at the time. Here’s the data, we’ve got good data, here it is, just load it, we’ll validate it, we’re good. And you know, the implementation wasn’t asking critical questions about the data, so that was a big lesson.

I have a girlfriend who has an entire company based around fixing dirty data and your exam your story is a great example of why that’s so important. Yeah, it wasn’t recognized as dirty either. I mean, it wasn’t screwed up in any step in the process. It just was never going to work. It was never good. It wasn’t what they relied on even in their old process, so yeah, it was quite a shock and a lesson.

Nikki, you are joining us from Chicago with convention internet, so I’m glad you were able to join the StreamYard today. Would like to have you introduce yourself, tell us a little bit about how you got into manufacturing, and then share a fun fact. I apologize if I drop out. It seems that my internet here is unfortunately not the greatest, so I hope that you guys can all hear what I have to say here. Yeah, thank you for having me, Sarah. I’ve, I’ve been in many… All right, about 15 years, maybe a little bit longer. Can you guys still hear me? Yeah, you’re cutting in and out. We’ll jump over to Nelson. Well, I started out as a sales… All right, Nelson, we’re going to shoot over to you. Welcome to the show. I would like to have you introduce yourself and tell us your path and journey about coming into this crazy world of manufacturing, and would also like to have you share a fun fact. Oh well, my name, obviously Nelson Birson. I live in Halifax, Nova Scotia, across the harbor, actually, in a twin city called Dartmouth. We are a port city. I came into this crazy world of manufacturing directly, indirectly. I started in hospitality, restaurant, and food service on the west coast of Canada. Summers around the year. The end of 1999, 2000, I entered the world of steel and steel distribution and steel purchasing and steel sourcing, and from there, I’ve been… wow. Based on whether or not the economy is up or down and whether steel’s a favored product in and out through construction manufacturing processes, obviously, multiple food service manufacturing processes. If my camera moves around a little bit, guys, it’s because I’m on my phone for whoever’s watching and trying to keep it stable and concentrate on speaking at the same time. I have… I’ve worked in machining, in steel fabrication, in ship repair. Early in my life, I was involved in stevedoring at the port, loading and unloading ships with all kinds of things. The best one was the cars. That was the easiest one. Other than that, that’s really kind of it in a nutshell. I… My fun fact, I guess really is I met my wife after coming up out of one of those ships in Halifax and seeing asbestos dust above our heads in the sunbeams for the first four hours of the day, and a week later, I left the site that day and decided I wasn’t going back. I hopped on a train to Banff, Alberta, and against all odds, I met my wife in a bar, and 37 years later, here we are. I’m on your show, Sarah. Online dating is overrated. It’s… I’m glad I’m too old for it. I’ll assure you of that.

So, Nelson’s got a few very unique kind of scary life and death stories. So, Nelson, I want to have you start out with the story that you shared with me about a young truck driver who was almost killed by a piece of steel. Yeah, it was a scary, scary day, working in a large steel distribution branch in the industrial park entirely in Nova Scotia, and we had a recessed loading bay, and the loading dock actually was only a step down from the top of a flat deck, step deck. It was pretty even, front and step down in the back. The real rule of safety in the facility at the time was you were not allowed to be in the loading well with your truck while something was being loaded overhead. It was followed most of the time, but we had this young crackerjack driver. He was amazing, actually. He had more energy than everybody else in the business, I think, and he was working on his straps and tightening everything down, and we had an odd-shaped piece of two-inch steel off the cutting table coming overhead, and it was a little over a ton. And luckily, he’s so aware, and he looked up and he heard the scraping, and it wasn’t the same as just the crane coming overhead, and he actually dove out of the way as a piece of steel let go and knocked himself out in the wheel well of his truck, but it caught his leg. He’s off work for a year and a half, and when he comes back, he has a permanent limp. I joined the safety committee and actually volunteered to be the head of the safety committee the next week. And I’d always been safety-conscious. I had my own restaurant business for many years, and it had to be, but this was the first time my official entry into safety and put my head into regulation, legislation, and practice, and yeah, kind of an eye-opener. Well, it’s a miracle he survived first of all. Yes, absolutely. I think it highlights an important part of manufacturing that doesn’t get talked about a lot is the danger factor. There’s a lot of injuries and things that happen on a manufacturing plant, and supply chain can play a very important role in that, so I wanted to make sure that we highlighted that story. Yeah, Sarah, it’s all through the supply chain, and it really is. It coincides with good operations, good safety, and good operations. It’s just part of the game, and it has to be more and more these days. We have bigger equipment, bigger things, more automation, more things to be aware of.

Dave, next question for you. You were involved in fruit production, so we’ll get some more detail on that. So, how has fruit production disrupted getting drinks to the shelf? Absolutely, Sarah. So, as you asked to go think about some of the biggest woes, some of the biggest issues, I have seen, I was thinking about a situation that is currently happening, and as I was thinking about that, I realized that I do a lot of work with drinks that have fruit in them. You know, a large number of people that order barrels of a variety of frozen fruits, and you can come up with five or ten times that people have wasted a hundred thousand dollars or more with fruit production. I, you know, you realize that you do a lot of work with a variety of fruit, and as I think about it, I find it very interesting how mother nature, if you will, is at the very core of many of the drinks and other things that we consume, and how a small, small change in nearly anything can cause large problems. And I say that as a precursor to I do a fair amount of work in the orange juice and in the orange juice industry. I have for a number of years with a variety of different facilities, and probably four or five months ago, I got a call or I started reading things online that orange juice, or that orange production was going to be down below at the lowest level since World War Two, down about 80% from the peak, which you don’t really think of until you realize that that means that many of these orange juice facilities aren’t going to be able to get a bunch of the raw materials that they have. And so, over the last couple of months, I have watched as a huge number of these facilities have struggled to one figure out what they’re going to do within the facility, as most of them bring in raw oranges, go extract the oils and the juices, store the… they run through a process, they store the juice, and then they bottle them, right? And so if we don’t have the majority of the oranges coming in, then we’re unable to do the front, the front half of that value proposition, which is where a lot of their money comes from. And so, I’ve watched as a number of these facilities have kind of run fire drills trying to figure out okay, which of the facilities within our groups are going to continue to extract the juice and juices and oils, and which of them are just going to bottle? And then from that, it’s where are we going to get hundreds of millions of gallons of orange juice because we, in Florida, are not currently able to produce it, so a lot of the orange juice is coming up from South America, which is where we kind of on and off again have gotten, uh, some of that. And so it’s been a huge disruption, both to the supply chain but also within the facilities in and of themselves, and have caused not insignificant issues for lots of the folks working there. There was a period of time over maybe a month where a number of people that I knew weren’t sure if they were going to continue to work at their facilities because they weren’t sure if their facilities were going to continue to run.

Yeah, crazy, crazy. All right, we’re gonna try Nikki one more time. She’s joining us from Chicago. She’s got some spotty internet with the convention center, so you want to try to do your intro again, Nikki? Yeah, say guys. And I apologize, so yeah, Nikki Gonzalez. I’ve been in manufacturing in the industrial automation side of things for most of my career. I started out as a machine vision systems sales engineer and then went into motion and handling and sensors and all kinds of stuff. And I eventually moved into supply chain analytics and data, working with software and retail supply chains. But kind of made my way back into industrial automation more recently with “QuoteBeam,” where we’re actually working to connect some of the supply chain when it comes to distributors and distribution of parts for industrial automation. And so it was really… I was tried by fire in factories of all kinds when I first started. I spent my first three years on the factory floor, mostly testing and implementing vision systems for quality control purposes, mostly for visual inspection, where operators just could not get a handle on, you know, inspecting millions of parts going by at super high speeds, or they would only be doing batch testing. I worked on a lot of R&D applications and things like that as well, but for the most part, trying to detect quality problems before shipping pallets and pallets of bad product to a customer or something to be, you know, found out much too late when you’ve wasted millions of dollars. A fun fact about me, I guess I am from Iceland. I have a daughter named Elsa, and I had never seen Frozen before we decided to give her that name. So, I was on a Southwest flight, and somebody’s… it was some little kid’s birthday, and the entire airline sang the song to her for her birthday. I have not seen the movie. I don’t have children, so it was my first… It was my first take at the Frozen music. Yeah, it was my first child, so I had also not seen any of these children’s movies.”

“So, Nikki, when we were prepping for the show, you have a story from early on in your career that still makes you cringe when you think about it. So, we’d like to have you share that story with us. Yeah, so it was a… It was a story of bad communication, I’ll say. I was working in Northern California. There was an injection molding facility in my territory that bought a robot arm to tend one of their CNC machines that included a vision system to inspect the parts. And they bought that from an integrator in Southern California. And for some reason, and I don’t know the backstory of this, they bought the vision system from a sales engineer in Chicago. And it was specified with some information that was given from the integrator as to what was necessary, how the parts were supposed to be inspected, and for what it was integrated in Southern California. And then this integrator brought the robot up with the vision system installed it in a facility in my territory. And my colleague from Chicago calls me up and says, ‘Hey, Nikki, we sold… You know, this system is coming into one of your customers. I’d like you to go do some training on it, show the operators how it works.’ I said, ‘Oh, oh, okay.’ Would have loved a heads up on this, but he’s like, ‘Oh, don’t worry about it. It’s really simple. You know, it’s just detecting if there’s these holes in this part or not.’ I’m like, ‘Okay, seems simple enough.’ So I show up for a one-day training on this vision system inspecting the… These are automotive parts for filters, and they’re injection molded, and there are four holes that are supposed to be very precise in the middle of this filter part. And I show up to train on this vision system, and number one, it has no monitor attached to it or any way to see the images or any of the logic that is going on. So, it was initially very hard for me to figure out, okay, what is it that I’m training on here, and how am I supposed to train someone? And then when I finally got a monitor hooked up to the thing and I… Well, just looking at the installation, first of all, before I got a monitor, you know, I just had a rock in the pit of my stomach because they installed this machine in a 24-hour facility right underneath the skylight, and the vision system had front lighting and no backlighting. And so it was… Immediately, I got scared.

Like, how is this supposed to work if it’s cloudy outside or in the middle of the night shift when there’s not enough light on the part and the conditions are continually changing? You know, I just saw kind of a cloud go over the skyline, and immediately, all the conditions of the image change. And then I was like, ‘Well, okay, still simple enough. We’re detecting the presence or absence of four rather large holes.’ But so that the cutting tool or the robot arm that pick up, they would pick out two parts out of them out of the injection molder at a time, and then it had these… The gripper would hold on to them, and it would clip the… I forget what they call it, the flash that sort of comes out of the molder. And so, there were scissors, tiny little scissors behind the holes on both parts, and on the top part, the scissors face like this, and on the bottom part, the scissors face like that. And then they were shining a bright red light on the front of it, and it was just, you know… Coming right off, there’s a lot of reflection off the scissors, again, no consistency, and there’s no way to mount a backlight to the back of this system because it’s basically… There’s nothing. There’s no way to put anything behind the part. So, I start to try to troubleshoot, okay, how am I gonna train people on this? This was not set up very well, but let me see what I can do. And I set it up, and I said, ‘Okay, this is, you know… I think this is gonna work, but you may have issues with the skylight situation. I don’t know how extreme the difference could… Could get.’ And they’re like, ‘Well, as long as you can measure it down to… I forget what mills back…’ They asked, and I said, ‘Measure what?’ And they said, ‘Well, we need to make sure that the holes are within spec down to, you know, some small, very small tolerance.’ And it turns out that that apparently those… Those requirements had not been communicated to the sales engineer that spec’d the system, and they had purchased an older model system that was cheaper that did not even have measurement capability at all. It didn’t have an algorithm in the software package to make a measurement of this kind. Well, either way to do it. And so I go speak with the manager of this facility, and I said, ‘Well, unfortunately, you have to buy a more expensive system for this to work.’ And they had no budget. They were like, ‘Well, this is what we told them that we wanted, and you need to make it work.’ And it was very stressful for me because I realized that and they were like, ‘And our customer is coming in for an audit of these parts in four weeks.’ And I said, ‘Okay, who’s your customer?’ Well, their customer was my customer down the road as well. And so, I just knew that if I didn’t get this to work, it would be, you know, bad reputation for us all around my company, you know, obviously, it wasn’t my fault, but something had to be done to make this work. And the customer was adamant that they had no budget to pay for another system, and this is… This has got to work. So, I went up there… I don’t know how many nights and weekends. This was about a two and a half hour drive for me up to Sacramento. I tried, we went up with the… With the forklift and tried shielding the lighting, and there was so much glare. I mean, I troubleshooted that thing for weeks. And I kid you not, I got it to work on the day of the audit, about two hours before the auditors from the customer showed up. And it was scrapping way too many parts, and I told them that they would have to purchase, you know, upgrade the system at some point. But it just… At least they were not letting bad parts pass, which was the main, you know, thing that they were getting audited for, quality, to make sure that they were not passing bad parts to their customer. But it… Man, every time I got a call from them, I got super scared, like, ‘What’s going on with this? This is a crazy system.’

I can’t handle it, and yeah, so it was definitely one of the learning experiences early on in my career. That, you know, when you’re speccing at something as critical as a vision system, you have to do it on-site. You have to know the details. You can’t pass information to another person, to another person, and play telephone with something like this. Nikki, the miracle worker. I gave them no guarantees that this would work past like, you know, a couple days after, but they’ve been… They ran it. Yeah, I don’t know how long, but that is apparently… I think it’s where I picked up my penchant for trying to fix unfixable problems. Yeah, all right. Tom, what works in theory or on paper does not always work in practice. You have a couple stories about good designs that turned out to be unworkable, but the real kicker to your stories is that it was only discovered after implementing those processes. So tell us the story.

Yeah, yeah, the learning, the late learning, kind of like Nikki was just describing, you know, incredibly painful, right? It’s one thing to discover things in the process of developing the solution, iterating, and it’s another thing to think, “This is rock solid,” and then discover, “Oh, expletive deleted, it doesn’t actually work. It’s rock solid but it doesn’t work.” Yeah, and again, I come at this mostly in my more recent career as an IT professional, so a lot of mine have an IT flavor, but they’re still in manufacturing workspaces. So in this case, it was MPS, MRP, and ATP, so available to promise as well. And we had a client when I was again back in my consulting days, a client who was trying to move from a weekly or even bi-weekly planning and promising process for customer orders at the end of this chain to something that was daily. And they said, “Yeah, we’re gonna have this new ERP system. It’s gonna be really whiz-bang, and we’re gonna be able to run through MPS, MRP, ATP every day and really manage everything so much more finely, and the commitments will be so much more reliable.” So that was their expectation, and they had about 60 different locations, probably a dozen manufacturing locations and another 40-plus distribution locations scattered around the globe. And there was a lot of inter-company shipping and supply in their network, so different locations were very dependent. They weren’t doing a lot of drop shipping directed customers. They were doing a lot of actual shipping to different facilities, and then who would consolidate and ship, say, complete orders to customers. And so they were doing, in their own system, this, as I said, once a week or even once every two weeks process of planning and allocating… Allocating, probably a little more frequent, but a lot of it was manual too. And they said we wanted to move to daily. And so we designed this whole process, this whole system. Said, you know, it’s gonna be… It’s gonna be great. We tested it, you know… Again, volume, volume, people, people sometimes forget, yeah, volume makes a big difference. So as we got to go live in the first couple days after go live, they realized that running MPS and then MRP and then ATP for, you know, this universe of locations globally actually was taking more than 24 hours to complete. So the idea of doing daily MPS, MRP, ATP, that was taking 36-plus hours, you know, doesn’t really work. It has to fit into 24 hours for it to be a daily process. So, you know, and then there were issues where, you know, there was a batch schedule and so, you know, they haven’t even finished the prior day and the new jobs are kicking off, right? So it was… It was really not pretty for a while, and one of the takeaways for me was, you know, the idea of these complex global processes in some cases, at least, really need to be tested out for… And I know, maybe I’ll clarify the acronyms. I see there might be a question there, but… But they really need to be tested out on a large volume, you know, production scale in order to be sure that the theory will work in practice. And so I’m talking about MPS, manufacturing planning, the… The planned orders that typically get created at a higher level, finished goods level, and then the MRP that blows that down into lower-level demand, the subassemblies and even the procurement phase of the manufacturing process. And then ATP is more related to customer orders and what is the… The available stock currently, plus maybe the planned orders and when they’re due to arrive, production orders and even the planned orders. You can look back up the supply chain and say, “Okay, for that customer order, you know, maybe it’s going to be available immediately or maybe it’s going to be available next week, maybe it’s going to take a month,” right? And so the idea of the MPS and the MRP creating supply, scheduled production orders, planned orders, and the procurement behind all that, that is a critical input to then promising customers when we can deliver, right? So, so that was the… The supply chain, you know, frankly, the one of the most critical pieces of the whole solution was this. So, so that was one example where… Theory, great design, which seemed like a great design and what worked in our quality system in testing didn’t actually work in production because it just took too long.

So anyway, I won’t go into how had to be redesigned, but that was the nightmare, and again, with the batch schedules, you can imagine, you know, jobs haven’t finished and new jobs are kicking off and then jobs are failing, and you know, it was… It was not good. And then the other one was kind of similar, although on a smaller scale, where we had planned for a much enhanced tracking approach to outbound shipments in the warehouse where we were introducing serial number capture where the customer hadn’t done that before. And again, it was all pretty straightforward in some respects, how do you… You know, boom, scan, boom, scan, boom, scan, all… All tracked, all put into the ERP system easily, easily reviewed with reports, you know, great design, reports, analytics, you know, ability to respond quickly to questions about, “Is this legit with this return?” So a very, very important improvement, and so everybody’s expecting this to be to work just… Perfectly. And then come the day of… Of go-live in the initial days after go-live, people find out that the process of boom, boom, boom, scan, scan, scan, it doesn’t quite work that quickly. And their volumes were pretty big in these operations, these warehouse distribution centers, and so they were… They were struggling to just get through that process which had never been a problem before.
The actual throughput had never been a problem, but suddenly now, the serial number capture was making this a much lengthier process, with product backing up at these packing stations, and to the point where they weren’t able to finish in time for air freight carriers who are coming daily to pick product up and ship it out to customers, to meet the customers’ you know, just in-time deliveries, keep their manufacturing lines running, maybe at customers that they were supplying. So, you know, they were missing the cut-offs, and then the shipments were late, and obviously, you know, impacts downstream with customers.

And finally, you know, the serial number capture had to be shut off initially for a period of time and then reintroduced for only the most critical parts. And so the whole process had to be redesigned. Again, it all went back to what works in theory if you don’t do significant volume testing, may turn out not to work in practice again. Technically it worked, but practically it didn’t. And so similar to the whole MRP, MPS, ATP schedule that was taking way too long, greater than 24 hours, here’s, you know, again a process where in the shift and with the delivery cutoff times, they couldn’t make it work, although everything technically seemed to be just fine. A couple of examples, big lessons about doing greater volume testing and understanding time motion, right, and not just technical execution.

Yeah, Nelson.

I would like to have you share your train wreck story when you were supervising welders in the rain on a massive dock in the harbor for six days straight.

Did you hear me, Nelson?

No, sorry. My mic had turned itself off. Okay, I said I’d like to have you share your train wreck story where you were supervising welders in the rain on a massive barrage docked in a harbor for six straight days.

Oh my heavens, yes.

It started, believe it or not, I’d taken a couple of days off. I was actually the Vice President of Business Development. I wasn’t really the project manager on the site, and I was five hours away from this site in another province and I was not familiar with the Canada province much. Like a big state geographically. Attending my daughter’s university graduation and I got a panic call from my boss, and I spent most of the graduation walking around the upper concourse on the phone trying to solve problems.

And we had taken this job which had was to take this arctic barge, clean off the entire surface of the barge, take all the tie-downs and all the anything that wasn’t completely flat with the surface of the barge, and this was an old bird, so it had a lot of rough territory on it. It was actually a resurfaced sunken oil tanker from years before that they rescued and repurposed as a barge. And it should have taken three, three and a half days for this job, as I said.

And I had to leave immediately following the graduation. I didn’t stay around for the celebrations, make the drive back. They put this young lady on, and she was a very talented young welder, and they just promoted her supervisor, and they put her into this mess. And to give you a little bit of the layout of it, the bow sections of the arctic patrol vessels that are being built in Halifax at the Halifax shipyard. They built the middle and the stern sections, two back to back in the big production facility in Halifax, but we were on the Dartmouth side, docked up. And it started to rain, and it never stopped raining for six days. We were wondering if we were in the great flood. And we were electric welding and electric grinding, and the equipment was failing.

And this poor girl was in a mess. They were going to fire her off the site. It wasn’t her fault. She didn’t, the main office was in another town, so she really didn’t have the connections to correct any of this with suppliers or rental companies in town. Anyway, I made the drive and stood beside her for a couple of days and got her reputation intact. So, it wasn’t her fault. But it was a challenge. There were welders blowing up and welding equipment was blowing up. Welders were getting shocked pretty severely, and the job took six days. Everything was tinted. We had to have dry clothes for these guys. We were through gloves like crazy. No one died. And all the brass of the shipbuilding outfit took pictures of us at the end and shook our hands. And we had to re-weld D-rings onto this thing at the end so they could tie down these big bow sections and ship them across the harbor to the main production facility and attach it to the other two sections.

And it was all for the first ship being built in the seven in a row that we were contracted for in this part of the country. And this was, you know, all the brass was there, the cameras were on. This had to go right. It had to go right on time. So yeah, a little bit of a nightmare, but we pulled it off, another scramble, much like Nikki’s. We just did what we had to. I had welders coming from everywhere.

Nelson is the dangerous one of the group here.

A little bit, not not by choice all the time.

So Dave, you had an experience in your career when supply chain disruptions actually caused a manufacturing line to shut down. Tell me about this story.

Absolutely. So much of the work that I do currently and for much of the last, you know, 10 years has been with operations on the plant floor, trying to find ways to keep facilities running or to make them run more profitably. And of all the facilities that I’ve ever worked with, I’m not sure any of them will ever say that they do a good job with preventative maintenance, right? So doing something to make sure that our machines and facilities stay up and running so that we don’t go down. I worked with one mid-sized co-packer within the last probably 18 months or so, and they’re the only facility that I will say has ever been allergic to preventative maintenance.

They had, you know, nearly 20 people on maintenance staff on this relatively small facility, and they all liked standing around. They all liked welding. They all liked disappearing at odd hours for extended periods of time. I’m not sure I saw a single person ever do preventative maintenance. They had a whole system in there to go help them do preventative maintenance. I’m not sure anyone ever checked up on that system. So, I was there working with them at the facility one Thursday morning, or I came in on the Thursday morning to check what’s going on, and they had shut the line down. And whenever you walk in and the line is shut down, that’s always a bad thing. They lose tens or hundreds of thousands of dollars, you know, every hour that the line is shut down.

So, I go check it out, and they’re telling me that it sounded like the gearbox of this machine sounded like a jet airplane taking off. And I’m like, well, that’s not how it should sound. So, we did a little bit of look and did a little bit of digging. Within the first hour or so, we realized that no one had any idea what type of oil or lubrication goes into it, which means that no one had checked it in the last, I don’t know, maybe 10 years, but certainly at least for the last 14 to 16 months.

And so long story slightly shorter, they nuked the gearbox, this I don’t know, ten to a hundred thousand dollar gearbox to this machine which was critical to the entirety of their facility. And they spent the next hour or they spent the next day or two kind of limping through at half speed while they were trying to go through the process to figure out if they could get another one. Fun fact, there were zero in the entire world because according to the manufacturer, no one had ever broken one before. Now, if we want to believe that no one had ever broken one before or not, I guess that’s a different story, because of a bunch of supply chain issues, it was all of the correct type of lubrication that they were supposed to put in the machine was out of stock for, you know, three weeks. And they had about 24 hours’ worth of this stuff left, and there were no spare parts to be had.

So, as we go through this process that they kind of got some part, they kind of got some parts in and kind of bubble gummed and duct taped it together in order to run at a significantly slower rate, and I’m not really particularly sure what sort of lubrication they put in at Sarah, because honestly, I don’t want to know, because I feel like that makes me more culpable if I were to know all of the all of these little details. But based on my calculation of the time that they were completely down, they lost somewhere between about two and five million dollars from the combination of the lack of kind of forethought and not having the right people in place and not being able to get the right parts.

I want to thank Dave, Nelson, and Tom for joining our tales of manufacturing supply chain woes. This is something we host every single month. Our next show is November 8th at 1 pm Eastern Time.