00:00

My name is Mark Rubin, and I'm a dream weaver. I hope these widgets help you weave your dreams into reality. This is a story about data conversion and naming standards are really it's a story about optimization, optimizing systems. The job I had, after I had a job at Pepco, the power company in DC was this job, I work for what used to be called Anderson Consulting, but is now called Accenture. I got the job, because I knew they had offices all over the world. And I wanted to be like an international consultant, and I live in cool places and have them pay for all my stuff and have fun of them early 20s I thought, why not? Now's the time. So I got this job. And immediately discovered that the first thing they want you to do in these kinds of consulting jobs back then was to be a programmer, mainframe COBOL. This was what they wanted me to do mainframe COBOL. And at that time, COBOL is in old computer language from like, I think either the 50s or the 60s. And it's used to run large back end systems for like, well used to be banking. And this was the state of Maryland, the Motor Vehicle Administration. And so the project, it was before the year 2000. So it was, I think it was like around 9596. The idea was, we had to fix all these dates on these computers. And I didn't want to learn COBOL because even at that time, it was a dying language. And when I say die, it's still in use. And it's still in operation. But it was built on old stuff. And I like new stuff. So first of all, I didn't want to invest the time in learning this language that I was only going to use on this one project for the rest of my life, and never use it again. And second of all, I have a hard time learning languages, especially computer languages, with all different syntax and all different like grammatical structures, object structure, different structures. That said, though, I can visualize an entire software architecture, which came later, as a result of the story I'm telling you now. So here's what happened. So they didn't know what to do with me on this project, because I wasn't gonna learn COBOL and I was the one that said no, I guess I was likable enough to say no, but I guess pushing the edge of the envelope. I'm like my utility at this at this company. And so they had a guy who was fixing computers. And he was doing basically he did want to fix computers anymore. And I can fix computers because I had a job in college where I looked at like a computer lab and stopped doing stuff, software stuff, hardware stuff. So he had a little cart, he now with some tools, little tool belt. So I became the computer repair guy on this project, perfect for me. Because I had a little cart, I did not have a direct boss. And I pushed my cart around. And just prior to that there was like 100 people on this project, two different buildings, big project, lots of developers. And basically, I got the push of this cart around to every corner of this project, and asked people what they were doing. What are you doing, like, what are you working on? And I would ask the developers who are like, fixing this, like mainframe code, like the year 2000 code, what they were doing, like, what part of the what part of like the MBA system were they working on? And what did it do? What was it called? What did it do? And I asked every person is that I was pushing my car around. So I had a model in my mind of everything that happened where they only had a model of whatever it was that they were doing. But I, I had a spatial model of at least the structure of all the pieces of this project from the from the code level. And then I also asked the managers what they managed, like, what were the functions that they managed and like, like what, like what were their teams doing and that could fit together the management layer on top of the operational layer. So structurally, I understood, like what people were doing, how I talked to other things, the relative importance of each thing. And the operating structure of the management team that was trying to get this thing done. Because we were running out of time. What was happening is back then the year 2000 For those that don't know, the century turned and there was a there was a time where to save memory. Back in the mainframe COBOL days. If you had a date, and let's say the year was 9099. You would just use 90 you would use to digits for the 99. But then when the year 2000 came along, if you had any math that involve that 99, which you always did always, you'd be subtract, you'd have negative numbers, click negative dates. So the date between like 2099 is negative 99 years in the old way. And that's a problem. Because this system not only ran the entire state of Maryland's Motor Vehicle Administration, it also ran their payroll system. For every like check, they got printed, and sent out to like anybody, either who worked in the state of Maryland, or like a vendor, or anybody. And if this thing broke, it would break like the state, like it would be bad. And so they were investing a ton of money in upgrading the software. At the same time, they were also upgrading the actual hardware, the mainframe itself, from like an old mainframe that was maybe like 1520 years old to like a brand shiny new one. And so this was the project. And I was just fixing stuff, just having fun messing around and low profile. But I decided that to get to Europe, I was gonna fix the hardest problem on a project, the hardest one. So based on my knowledge of what people were doing on this project, there was one room that I didn't had no idea what they were doing. And it was a room was by itself, it's like is a little quarter. And or two people working in it, sometimes three, they were working on this thing. And as I never understood what they were doing, and it turned out one day I asked them, and the reason I didn't know is because they were like, they were by themselves. They were like in the sequester room. And they weren't, like connected to the rest of the project and like in this functional way. And so I went in there and I looked around, and then there's room to Robbie's shelves all the way around the room. And there was 136, two inch binders on the shelf with words on them, like f l g 001, F LD 002. And I asked them what are what are in these binders? Like what is this room? They said those binders are all the programs that run that run this NBA. So all the programs that other people are like working on out there that all dates, those are in these binders, like the code is in these binders, like what the program is. And they said, we have like the hardest problem on this project. I'm like, What's that? I'm like, Okay, here we go. Okay, we, as part of the year 2000, upgrade, we're going to convert this data, take the old data, fix the dates, and then we're going to shove it on this new mainframe, the faster one. And they said, The problem is if we run these programs back to back to like, unload all that data and fix all the days, it's going to take three weeks. And we cannot shut down the state of Maryland for three weeks. While we're doing this. There's only two windows of the year that we have. And if I remember, right, it was Thanksgiving weekend, which is three days where everything's closed. And I think it was Memorial Day, there were only two chances left to do that. There was like a, there was a window for a test. And then there was the window to do it. And then it was like the year 2000. Okay, like that was that was it. Okay, it's already like, that's the way life worked out at this juncture of time. And there are huge penalties for missing the data, like millions and millions and millions of dollars, because it would it would break the system. And they were trying to figure out a way to take something that takes three weeks to happen in three days. That was what their job was this data conversion, migration, you know, a thing. And I asked them how they were doing it, they didn't work on it for like six months, in working a long time trying to figure out like even a way to do what they were trying to do. And they it seemed like they couldn't figure it out. And I opened up one of the binders, two inch binders, just to see. And I again, I can't write code or recode. But the first page in these binders had this information. It said the name of the program, actually, I think was FL g 00 3x. And first thing I noticed there were the naming standard, which meant each of those characters meant something and this made a lot of sense to me. And FLG meant that it was the flagging system. And the flagging system was used to flag licenses flag violations and create notices that later got mailed out like paper to people. So that makes sense. There was a name and then they meant something and also had a number at the end which was a sequence number. So one came before two scheme before three which came before four and sometimes it would skip some and sometimes that extra letters and stuff for whatever reasons it was and so I remember the flag that the first one that had this file called the flag master. But basically what this what this thing did is Isn't this piece of paper told me the name of the program. What Oh, Order was in what files came into it, what files went out of it. It also told me how many CPU seconds it costs, how many records in process, how many records per second, and some other some metadata about like, particular to the mainframe of like, where the file was located. And like what CPU cluster or whatever it was different things that were defined in something called JCL. So they told me enough to know the energy cost of the program, the number of records that it processed, and how fast it processed them. And it told me the dependencies of the order, what came first, what came before it, what fed into it. And what it said, after, I thought this is enough, because I can visualize the number of records spatially as a spinning gear. So if something is 1000 records, and something is 100 records, there's a 10 to one difference between those spatial representations of the number and the speed that is good or is spinning. So how many records per second, it is processing. And the CPU seconds is the energy cost to make that happen. The total number of records processed after a certain period of times is information, I map to a lie. And I was able to read these 136, binders, the front page, and I stat that weekend as a Friday, by the time I came in on Monday, I'd stack the binders floor to ceiling in the order that's the optimal order was string connecting them somehow numbers yarn on this order to twine on this. I found some string and some tape. And I was taping you know, these binders together but straight at different heights to show the branching, okay, of what was in series and what was in parallel. So if something was in series, it was stacked. And then if you could, if you could branch and you could run multiple programs at the same time, because they didn't have any dependencies. And they were in the in the right sequence in the flow, then I would stack them next to each other and then put a string to show that they were in parallel. So on the Monday there were all these binders on the floor and different piles with different strings and different pieces of tape. They came in and they said was this is it this is the optimal order. And they were like, you know, they're looking at me and my bottom chain advice came in. And the first thing she said I sincerely What's it sounds like, oh, it's the alma mater. And of course, she says, You don't even know how to write COBOL. See, because that's the thing is like, they hate what people think about like what it is, the details often don't matter. Okay, so they said, How long do you think it'll take and I said 12 hours. And that was based on the sequencing of everything and the number of CPU seconds I could consume, based on the flow that I had. And it took 12 hours. And later through some optimizations. We got it down to eight about the like things like where would the file be? And could the file be closer to like, where the work was happening. And like, there were like physical tapes back then that someone had like how to like, like, grab a tape and like, walk across the room and like, put it in a machine like this is the this is the era that this was. So it's like the closer if you understood the layout of the map of the floor, you could even optimize to have things pre sorted. So that was a long story. But that story really had a big impression on me. Because on this project, if you did something good, they gave you these little wooden tokens called tokens of inspiration. And you could trade them for like pizza or like, I don't know, whatever like a like a movie rental and like blockbuster movie theater, or Buffalo Bills or movie rental or like different things. And at leaving out some stuff, they gave me all of them. I got all the tokens of inspiration. And I could trade it for like a one week trip to the partners beach house, which I did. And you know, I guess the story was important in my life, because I realized these models that I visualized scale models of energy transfer are not just useful for marbles and ramps. I already knew they were useful for business system. But I see these models all around me all the time, energy transfer, 's energy transfer, same thing as the as the power plants about supply and demand. And same thing about modeling businesses. Same thing about the distance. A kid can run in one day. Time, work, energy distance, all related in your model.