Crunch Time

From TheCommandLineWiki

Part of The Inner Chapters Unbook.

Originally part of episode number fifty one.

Dedicated audio available from Podiobooks.

Contents 1 Original Notes 2 Transcript 3 8:00 4 15:10 5 21:36

Original Notes

• Compare to down time
• Deadlines
  • Sometimes beyond our control
  • Other times, we impose them to help inertia
    • Can mark progress
    • May facilitate prioritization, what makes the cut for working on right now
• Cost, features, date - pick two
  • Cost is always limited by available resources
  • There is a barrier beyond which adding resources won't help, anyway, see the Mythical Man Month
  • Some times the date can be moved
  • More reasonable is to re-negotiate features
  • Most customers, internal or external, would rather receive something, even if less than original promised, then receive anything late
• If you are a contributor
  • Keep your lead and/or manager up to date on any tasks that may take longer than expected
  • Never suffer in silence
    • The later you come forward, the less time there is for a manager to do something about it
    • Overexertion usually results in diminished quality
  • Remember the lesson of humility - we all make mistakes, we all need help at times
  • Re-visit the requirements continually to stay in scope
    • Scope creep is usually only though about during requirements negotiation
    • You may be doing more than what was asked during implementation
      • Building for future requirements
      • Over-engineering or over-delivering on a simpler feature
      • Keeps notes when you creep scope, these are good candidates for future enhancements, should go through the normal requirements process
• If you are a manager
  • When prioritizing features, tasks, always build a short list of items that can be pulled if time requires
    • Sets expectations early
    • Helps with the decision later, gets early buy-in
  • Know your resources and pad appropriately
  • Keep talking to your people
    • Learn of problems sooner rather than later
    • Also, identify opportunities, where one task is completed early the developer in question may have time to help someone else who is behind
  • Manage upwards
    • Insulate your team from outside interference
      • Sales
      • Customers
      • Filter this input and share where and how it would be appropriate
    • Always have options when discussing problems
      • Features to be pulled
      • New resources that could help
      • Alternate ways of meeting the requirements or business opportunities

Transcript

This time I wanted to talk about crunch time, and I wanted to contrast and compare that to downtime that I talked about in the last installment. Downtime, as we talked about, is when there is no imminent release pressure, when you don't have any scheduled work. It's when, as a contributor or a manager you're at loose end and left to your own devices as to how to utilize that time. And I had some good suggestions about prototyping, learning new things, doing some experimentation — making good use of that time, those nice opportunities, those pauses between the more heavy-lifting work that we have to do in order to support a business or support a project.

Crunch time, by contrast, is — well, I won't say it is the whole of implementation because I think there's a probably kind of a normal medium in between downtime and crunch time, but it's not remarkable in any way that we talk about it for having any particular characteristics, risks or opportunities. Crunch time is, as we're getting closer to the deadline and starting to feel the pressure, maybe we have to make some schedule-driven decisions about what it is that we do, how we go about it. Sometimes this is beyond our control. Other times we build deadlines into our release plans in order to help build some momentum or some inertia. I found in my experience working at a research and development company, when as a manager I didn't set forth some milestones, we kind of... well, we lacked focus. There was nothing really to shoot for, and then, on the remote side of the milestone there was really nothing to mark or to delineate our progress. So I found that it was very hard to get people kind of enthused about the work, to get people to concentrate and agree on the things we needed to do next. So working for a research and development company is not something I think I will find myself doing again. I enjoyed it at the outset because it was something so radically different, but I learned enough about it to now realize about myself that it's not one of the things I'm particularly good at. And so I was maybe nine months or a year into this job realizing that, you know, we weren't really moving forward along the lines that we needed to be, and so I had to fall back on the project management experience and skills that I had to try to get things moving forward. So I guess one of the positives that came out of that is that, even for workers and research scientists that are not used to arbitrary deadlines like we have in software engineering, the private industry, that agreeing on a milestone — whether there's a real customer driver around it or not, you know, so there's a real somebody waiting for delivery of something, or not — setting that deadline up can help get things moving, can overcome that static friction of starting out that you might not be able to do otherwise if you didn't have some arc to shoot for.

I think that it also may facilitate prioritization in that, if you're sitting there looking at like a research project and you don't know out of all the different avenues of research which ones are going to be most fruitful, and all things being equal you just can't make a decision. You set an arbitrary window to say, we want to achieve something by this date in time, then you can start to back off and say, "Well, what's going to fit there? Is this idea just going to take too long to pursue? Is some other idea more amenable or can we even, on a more long-range project, can we break things up to fit within that milestone?" So that was a trick that worked well.

And then, so if you find yourself in a scenario with more normative software development where you're churning a little bit for lack of focus, where you're not getting any of that kind of forward direction rolling, then that may be an indicator that you need to step in and say, well, you know, we either don't have a schedule, or it's so long that it's almost meaningless. So it's a 12- or 18-months schedule before you have to deliver something to market, that you may find that that's good trick to use on an individual basis if you're just a contributor to set out some interim milestones on your own, or if you're a manager to try to come up with some internal releases to kind of help get people focused in on planning out the work, doing the task breakdown, the estimations, the sort of natural things that go with planning and getting ready and moving forward. And I think you'll find that, at least for the engineers in your organization, that's going to get them moving a lot better than anything else that you might try.

One of my favorite sayings when it comes to schedules in software development is "Cost, features, date: pick two." And the idea is that you, out of those three, you really only can control two. And the third one then going to be a function of the two that you pick. This was something that came up that I first learned when I was doing consulting vs. product development, and it was something that we tried to convince the account directors and the project managers of from the engineering side, which is kind of strange. They should have known this on their own: that when going to a costumer, if the customer has a particular date in mind then we have to try to negotiate to settle on a features set that then results in a reasonable cost the customer is willing to pay, or if they have a preset cost in mind, being able to get down to a feature set that we can actually accomplish by the date.

So cost, unfortunately I think, is... you know it occurs to me I think I just misquoted this. It's "quality, features, or date". Huh, damn. That just blows out one of my other points. Well, but it still holds that if the customer has a particular quality in mind it must, you know, "It must meet my business needs. I must have it by a particular date" then you have to back the features out or the quality suffers or the date is going to slip. I think that you're always going to be constrained when approaching this. There's a fourth metric (that was the point I was going to make about cost, but I'm still going to make this and try to fit this into this model) is that you're always limited by available resources. And as much as we like to think that if we have an infinite amount of resources we could get an infinite amount of work done in a finite amount of time, there's a barrier beyond which adding resources just is not going to help. That's the whole McGuffin of the Mythical Man Month. You can go out and read that book. It's a great book, and the metaphor, the image that they use is, you know, you can't have nine women produce one baby in one month. There are just certain physical realities that you can't work around by throwing more resources at it. And that's going to hold true with this, the interrelation of these three choices that you have.

8:00

Sometimes the date can be moved, and I think that the earlier you can identify that, the better. I think, though, that it's more reasonable to renegotiate features, that moving the date around unfortunately can have negative consequences, unforeseen consequences with your customers, that I think most customers, in my experience, either internal or external, would rather receive something, even if it's less feature-wise than what was originally promised, than receive anything late. They'd rather get, you know a 70 or 80% release with the feature set followed up by another release a couple of months later that picks up the remainder than to have to wait a couple of months to get everything. You know, especially if you're in a market where there's a lot of time pressure because of competitors or limited opportunities or things of that nature. So I think even there, that there are some constraints on some of the choices we make, that you might like to set out and say "well, I can pick any of those two if I want to", or if you're talking to a customer you say, "Pick the two you want and we can figure out what the third one is going to be". I think in some cases you can't even pick the date. Or once you've picked it, you can't move it around.

So it really just drives a point that I'm going to get into as I talk through, first if you're a developer or if you're a contributor on a project, some of the experience that I've had, some of the advice that I can give you in dealing with this crunch time and how to manage it effectively, how to both prepare for it, how to deal with it when you're actually in the crunch time, really centers around how you manipulate the feature set. Because I think that's really what we have the most control over as technologists. I think that for a contributor, for an individual developer, you need to keep your leader or your manager as up-to-date and informed on any tasks that may be taking a little bit longer, on any features to develop taking a little bit more time than planned so that they have more time to deal with that in terms of renegotiating feature sets, redefining requirements, shuffling things around.

One of the things I've seen a lot of the developers that I've worked with do is suffer in silence. And it happens this way: they're having a problem, they'll talk to their peers about it, but they won't talk to their managers, I guess for fear of negative consequence, for fear of, well, maybe their job... I think that no one project is enough to lose you your job, but there's some sort of fear involved with speaking up. And I think unfortunately, if you come up earlier and talk about something, you come forward earlier and talk about a problem you're having, I think it actually speaks better of your responsibility. You realize that there's recognition there, that the earlier you talk about something, the earlier you discuss it as — however distasteful it may be — that you're just, you can't develop the feature that was requested, that was required. It's just, it's outside your abilities for whatever reason — you don't know the technologies that you need to know, there's too much of a learning curve to get on top of something new — it really doesn't matter what the real reason is, but if you hang back, you're just penalizing the organization, you're penalizing the team. If you come forward, you're giving everybody an opportunity to help you. And like I said, I think that's a positive thing. I think that speaks better of you, your maturity and your responsibility. And an organization that doesn't see it that way, well... I would argue that they probably have some other systemic problems that maybe don't make that a good place for you to be in overall.

I also think that, along with that suffering in silence, I think that a lot of developers try to just throw more effort into it. We're getting back to my point about, the point of the book The Mythical Man Month by Brooks, that sometimes you just can't achieve more by putting more effort in, by putting more resources in. And in my experience, the overexertion that that sort of leads to as you start doing 10-hour days, then 12-hour days, then 14-hour days, then 18-hour days because you're desperately trying to stay on top of your work and desperately trying to get it done just by working longer and longer hours, unfortunately results in diminished quality. That after about eight or ten hours you get tired, your blood sugar is going to start to drop at the end of the day if you're not taking breaks to eat well, to get some fresh air, to change scenery — you're just not working at your best at that point. And again, if you're not speaking to your leader or manager and they're not seeing that you're overexerting yourself, you're not giving someone else the opportunity to say, "You know what, don't worry about it. You know, let me go talk to the boss. It's going to be okay. Let me go run interference. You need to go take a walk, and then you need to go home and take a nap. Okay, get some rest, see your wife, see your kids, play with your dogs. Whatever, you need to take a break."

We're all human, and I think organizations that don't recognize that, like some of the big five consultancies during the height of the dot-com bubble, the only reason they got away with treating people as subhuman is because there were so many kids coming out of school willing to step in if you didn't do the work. I think that's still true of a lot of places that don't treat their employees with a bare minimum of respect and go, "Look, I can maybe get a 40- or a 50-hour week out of you, but then after that, you know, there are some penalties that are going to accrue over time and that are going to cost down the road. I mean I may not see the costs now, but I'm going to see them eventually". They're deluding themselves that they don't realize this. And they may be taking advantage of local market economies, like I said where if there is cheap labor, very available labor, where if you're not available to do it they'll get someone else that can do it. But how can you compete on that? That's not a sustainable model. However, this was not the point of this Inner Chapter, so I've gone off on a little bit of a tangent here.

Also remember the lesson of humility. If you get back to the point of talking about an individual developer and their decisions and actions and consequences during this crunch mode, this crunch time. That we all make mistakes and we all need help at times. So it's okay. It may be you this time pulling the whistle stop saying, "Geez, I can't get this feature done on time. I need help". Or, "I've bitten off more than I can chew" or "This is not what I thought it was and therefore it's going to take me longer". It's okay. It's going to be you this time; it's going to be somebody else next time. You know, unless you're grossly incompetent — and chances are you don't even realize you're grossly incompetent, and you don't understand anything I'm talking to you anyways — so unless you're grossly incompetent it's just going to happen. It's going to happen to somebody else. And when it happens to somebody else you're going to realize, Hey, I'm human, they're human, we're all human, and you're going to kick in, and you're going to feel better about helping them, pitching in and volunteering some time. Because they will have volunteered some time to help you out, whether it's a peer who has some spare cycles to help you out, or a particularly good manager, or a development leader that's going to step in to try to help you out.

15:10

I think that one of my experiences as a developer that's kind of counter-intuitive is, creeping up the scope as a developer. Normally we talk about scope creep in terms of requirements, analysis, and negotiation, and that the customer or the stakeholder that's taking receipt of the deliverables is always trying to push for more features and always trying to increase the scope without changing the date, you know, trying to get more for less. But I found that sometimes, especially as you get higher up as a developer, you get more experienced, you get more senior, you get more capable, that sometimes your pride gets in the way a little bit, and you might start to do a little bit more than what was strictly asked during the implementation phase. You may be thinking about the bigger picture, you may be thinking about some future requirements, and you may be doing a little more work than you need to to make your deliverable. And this can be problematic during crunch time. You may be over-engineering, over-delivering on a simpler feature when you have the least amount of time to really do more work than you should. I think that if you continually revisit the requirements and refresh yourself of what the core requirements actually are, the minimum you have to meet... this is not to say that that's all you'll ever do, but during crunch time you have to remind yourself of the core commitments you have to execute on, and you have to be willing to look at yourself critically and realize that you're doing more than what's strictly being asked of you.

I think what you can do is, you can keep notes when you identify those differences, when you identify those additional things that have started to creep in onto your plate, keep very good notes. Set them aside. Focus on the core requirements, get them done, get through that crunch time, and then revisit those notes during the next release planning to go, "Hey, you know what? When I was working on this, here's some really cool natural enhancements that I came across. Here's some areas of the architecture that I thought were weak, and I started to look at them, and I found myself going off on a tangent, kind of creeping out of the scope, creeping out of the focus of the release to go address these sort of weak areas in the architecture. Well, I held off so that we could meet our commitments, but I really think we need to revisit this. We need to make some time to pursue this." And so it's good, natural input into the next set of requirements discussion. So whether that's, if you have some traditionally or you can start to make some room for architectural or infrastructural improvements — you know, code cleanup, some of the plumbing we like to do up front — that that's good input into that. Or you could make a good argument as you're working on something, as I said, if you keep very detailed notes, you can point out exactly what you found and why that appears to be weak, why you felt like it needed some attention. But rather than putting your current deliverable at risk, defer that. Discuss that later.

If you're a manager or a development lead, there are some slightly different things to think about, some different opportunities that you have, some different realities you need to be aware of. When you're prioritizing features, and I think that's something you should continually be readdressing throughout release — and you're going to look at it differently early on in the project, in the middle of the project, and towards the end when you get into crunch time — always build in a short list of items that can be pulled if time requires. Always think about what's that list of things that can be deferred to the next release or pulled out altogether. I think that if you go in, even before crunch time, so when you're doing your release planning, when you're doing your requirements discussion. If you're talking about, "Okay, we have six weeks to get this done. We have eight weeks to get this done, and we've got a very aggressive set of features. I want to build in a set of things that we can all agree on right now, before anything else starts, that if we need to, we can pull it out to go in a future release, or we can pull it out indefinitely for further discussion later, so not our problem now." You can help set expectations really early if you do that, if you can get by and you can get other people to see it that way to say, "I'm not saying we won't do this. But what I'm saying is, what's of truly trivial importance, if we just don't get to it, it's okay if our customers don't get it. It's okay if it's missing from the deliverable. We can life without it". And then conversely, what that says is, "Everything else is important enough that we need to make sure we hit those marks." So, you know, if it helps you to cast it that way to say, "What's our core critical features set?", and then, by omission, anything else can be pushed out. But I think if you're up-front and honest about it and go, "Look, task estimation, project prediction is an inexact science at best. It's a guess. Our knowledge is constantly changing and improving over time, so whatever I'm saying now, I may be able to disprove that in three days and say, `Oh, this is going to take longer', `That's not going to take us long'."

So if up front you say, "I always want to have kind of that safety valve to be able to blow some features out if I have to", I think people will appreciate that if they've had any experience with software development. They're going to appreciate some of the inexactness of it, and they're going to appreciate that that's a sane way of managing risk. I think that — again, talking about your resource planning up front — know you resources. Know the engineers that your working with, whether they're your peers or they're your direct reports. And pad appropriately when you get their task estimates. Realize that somebody is good, but they're plodding, they're slow. They take a little more time. You know, add some padding in there if somebody's fast but risky, so they may get something done in three days, but it takes them four days to test it, and they only put the three days of initial development. And you know everything they do requires much more extensive testing. Leverage that knowledge. Leverage that experience. Build that in. I think that's probably very self-evident to a lot of the people I'm talking to right now. But it bears mentioning in this vein because you get into crunch time and that has to be taken into account as you start to make corrections, you start to readjust feature sets and alignments, you need to maintain that awareness of your resources and what impact those changes are going to have on them and on the work they're able to get done.

21:36

I think that you really need to keep talking to your people. You need to have conversations early and then consistently throughout the project. This is the flip side of the "suffering in silence" issue that I talked about from a contributor, from a developer's perspective. You need to find ways to make it very comfortable for people to talk to you about problems so that you can capture those problems as soon as possible. You want to try to address things when there's still room in the schedule, before you get into crunch mode ideally. But even when you get into crunch mode, the earlier you catch it, the more time you're going to have to deal with it, the more choices you're going to be able to make, the more alternatives, maybe more resources you'll be able to move around. I think also on the flip side, keeping up that conversation with your team very naturally, very continually rather than letting it drop off, especially during crunch time, allows you to identify opportunities on the plus side, where if somebody completes a task early, so they have a couple of spare days that were not anticipated in the schedule, you're going to be able to re-balance that. You're going to be able to use that person's time effectively to help somebody else that's having a little bit of trouble. So that's good too. So it's a plus and a minus thing, as it gets into kind of the black art of resource allocation and resource management over time — or resource balancing, I think, when we get to the midstream of a project we talk about in terms of resource balancing: who's ahead a little, who's behind a little, are there ways for me to reapportion the work to match, you know, who has bandwidth and who doesn't?

I think that also probably [unclear, 23:22] that you need to manage upwards very, very carefully especially during crunch time, that during crunch time that's the most important time that your team really needs to really focus, and they need to be kept free of distractions, so you need to insulate them from interference. So you need to keep sales off their back, you need to keep the customers away from them. I think that this is not to say that knowledge should not flow in from these sources, but you need to filter this input, especially at crunch time. Tempers are going to get very thin, people are going to get very sensitive, so you need to be the one that's willing to have the lengthy hallway with the sales guy and go, "Yes, you're right, you know. This feature seems a little smelly, and we need to do something to fix it up". And then keep anything that may feel overly critical away from a team that's feeling stressed out. So filter that input. Take good notes! That's good advice in any situation: take good notes. Think about it. Think about what, out of all of that conversation, out of all of that information, what does your team need to know about at that time, if anything? When is the appropriate time to share it? How would it be appropriate to discuss it, to frame that?

You don't want to get into a mode where you're holding information back, but you do need to be really, really careful, especially during crunch time, of just what you say and to whom you say it. Because you don't want your number one developer, who's feeling very put-upon — he's putting in 60-, 70-, 80-hour weeks, trying to get the deadline met, trying to help out the junior members of the team — and then you repeat an off-hand remark from sales, and it just pushes him over the edge. And he just takes off for a day. And then you're sunk. That's really the point I'm trying to drive at, is that sensitivity, kind of these soft skills that I think not a lot of the technical management books talk about quite as much, are that much more critical during crunch time.

Again, in that vein of upwards management: whenever you do have to interact with your direct [unclear, 25:29] to go up to the VP of engineering, the CTO, you want to have options when you're discussing problems. You don't want to come to them and just say, "There's a problem, and here it is", and leave it at that. That's not very useful. It's not constructive. It doesn't lead to a solution. It doesn't lead to a decision that they can make at their level that maybe you're not comfortable making that can move things forward, salvage the project and keep things on track.

So you not only need to identify the problems, but you need to identify options in solving the problem. And then you need to be prepared to present those options with their costs and their benefits, or their pros and their cons, however you want to frame it. You can put that in terms of features to be pulled, that it's not just a, you know, a "John's having problems with his features. We need to pull that off", but a "If we move off some of these lower-priority features it frees up developers to help John out", you know. So you need to really think about that, and you need to revisit your prioritization. As I said, prioritization is something that you should be visiting continually. It may be changing on a weekly basis, and that's not a bad thing. It's not something that you need to maybe expose to the rest of the team to say "We're constantly shifting things upwards and downwards". But that's what gives you the latitude to make those kinds of decisions. You may be able to group things together and say, "Here's a whole cluster now. All of the sudden I've shifted a couple of things down in the priorities. Now I have three low-priority features that I can pull out, when at the outset I thought there was only one". And that may allow you to do a lot more creative things to get through that crunch mode.

Also, be aware that new resources coul help, that that may be an option when you're doing your upward management, that somebody new may have hired on, somebody on another project team may have become available, and that's an option that you can discuss as well, up to that limiting point that we talked about, to say, "Well, hey, you know, Joe just came off this project, and wow, he's, whew! really good at message-oriented middleware systems, and we're having some trouble getting to deployment just so. If we could get a day or two of his time, that would really, whew! that would get us through this crunch mode". That's something else to be aware of as a leader, as a manager, you need to maintain that broader organizational knowledge and organizational perspective, I guess is the point there.

And then what it really boils down to is finding alternate ways of meeting the requirements of the business opportunities. And it may not be what you planned at the outset, but if you can be flexible and creative, and you can leverage a lot of things that I talked about in a role as a management or a leader, you're going to find that, at the end of the day, you're going to get through the crunch mode just fine. And that's what it's really all about. You want to keep the project from crashing and burning, and crunch time is the most critical time to stay on top of your game, to be executing really well — again, either as an individual contributor or as a leader or manager.

So hopefully some of the experiences that I've shared, some of the ideas that I've put forward, I'm sure some of these you've heard of before from other project and process experts out there. But these are driven mostly from my own experience, what I've found to work, what I continue to use and continues to work for me. Hopefully that's of some value to you as well.