Before we dive back into defining an approach that mixes Product and Custom focus, we should probably ask “How does one focus on the customer?”
In an ideal world there are no defects; but since there are, and customers expect them to be resolved, an ideal approach balances itself between being proactive and reactive; proactively resolve reported/discovered defects and appropriately manage escalations. On the surface, one focuses on the customer by making sure that the defects are fixed before a customer reports them and if a customer reports them, they are addressed promptly (top priority of course) and the customer is satisfied – below the surface, defect management and prioritization play an important role towards our customer focus as this tells us how soon we can (and will) actually resolve defects.
The list below doesn’t capture all possible approaches in resolving defects; it captures approaches that I have had some experience with (I recommend that you certainly do not try the first one):
Don’t resolve them
- Push the defect to the original developer, or the developer most familiar with the functionality for resolution.
- Allocate time for the first available developer to pull a defect in and resolve.
- Have a defect duty rotation where a person (or team) resolves only defects for a time period
- Have a dedicated team for defect resolution.
- … Some other create ways to resolve reported defects..
Always push to most familiar
Pushing the defect to the most familiar sounds like a great idea and in many cases it is because the one most familiar would be able to resolve quickest and positively impacts customer experience. The issue with this approach is that the most familiar person might be involved with something that has a higher priority than this defect, or is out on PTO and would not be able to resolve for another 2 weeks. Let’s say it takes someone familiar with the code to resolve in 1 hour but takes someone not familiar with the code 6 hours. If the defect goes to the person currently tied up and is most familiar, the customer will have to wait a minimum of 2 weeks + 1 hour; however, if it goes to the person not currently tied up and is least familiar, the customer will have to wait a minimum of 6 hours. Being collaborative and available for the team may improve the turnaround time on average, but a hard coded “always send to the one who created it” may not be the best approach.
Allocate time to pull
Allocating time towards the end of an iteration, development or whatever, towards “defect resolution” allows the team to first get the scheduled work out of the way, and “if” there is time, someone may pick something up from the backlog of defects. The issue with this approach is that if the development cycles are fully booked (maybe there is a hard date) and there is any risk or complexity that might lead to developers putting in all they have to meet the delivery; defect resolution gets thrown on the back burner. In most cases where the approach is “allocate time to pull defects”, the unspoken rule is that new products come first, defects second – unless; where the “unless” is for escalations and chaos/fire-fighting instances. For agile teams, if the time allocated towards defect resolution does not change from sprint to sprint, then there is no impact to velocity; however, if the time allocation is not fixed the velocity can get impacted depending on how much time is spent on defect resolutions (usually hours) vs. features (SP’s)…. You could estimate defects in story-points but this can lead to additional issues that will need to be worked out… i.e. do you really want to hold a sprint back? etc..
Defect resolution duty rotation
For a given time (usually the span of a sprint) a developer within a team, or a whole team themselves will be on defect resolution duty; once the time span is over, someone else (or a different team) takes on defect resolution duty and so on. This helps cross-train and helps make everyone familiar with the code base. It can also help improving code quality since everyone is learning from everyone’s mistakes and provides a great collaboration platform; while it does have some great benefits it does introduce some challenges. A significant issue is that developers and teams lose traction as they switch focus from “new product” to “old product”; the interruption can cause a delay since the developer(s) will need to get back to where they were after the rotation is over. For organizations that have many teams or larger teams this may be less of a concern since the rotation might happen every few months; but even then, when it does happen it does have a negative impact.
Dedicated team for defect resolution
The thought on this one is that if there is one team solely focused on supporting “released software” (defect/engineering sustaining team) and other team(s) focused on creating “new software” (Feature teams) that you end up with a two-tiered development approach where both the product and the customer can be focused on. The feature (new software) team is rarely impacted by defects from the “live” world and they can always focus on delivering new product; the defect/engineering sustaining team is dedicated to resolving defects and is not tied up with new features. The issue with this approach is that no one aspires to be a “defect fixer”, developers want to “develop” new and innovative “features” (or at least I did); It is possible to make this work if more attention is given to down-time cross-training, root cause analysis, collaboration, role rotations, etc… (I have seen teams evolve this approach into a “defect resolution duty rotation” approach)
In addition to the above, there can also be hybrid approaches that mix various approaches, i.e. defect resolution duty rotation with an added “pager duty” where someone (not on defect duty) is on-call but in general there is no “incorrect approach”; however:
Any approach can become incorrect when developers are forced to accept an approach that they do not agree with (or understand).
Any approach that is going to be implemented should be discussed with the teams that will be implementing it, focus on and explain the “why”. When an approach does not work, try to adjust it or try something else!
“if the code repository is an “elephant” and new code is peanuts being fed to this elephant by the other guys, then I am always cleaning up after the elephant; who wants to be a shit cleaner forever?”.
We are usually torn between building more “product” or satisfying the “customer”.
Prioritize Customer (maintaining product)
Fix and release to save the world
Prioritize Product (building more product)
Plan and build to a schedule
When you focus on product, you have a certain “amount” of product that needs to be built within a sprint (time box); you can plan for fixes by leaving some room for the defect backlog, but features come first – then come fixes, and the work is usually fixed where people know what they are working on for the sprint. In this scenario most will usually package after the sprint is over (test and release)
When you focus on customer, you have to fix and get the fix out the door ASAP. Priority is given to the most critical fix and/or customer and things will be packaged (test and release) as soon as appropriate and/or possible.
Some solve this by having a sustaining engineering team, but who really wants to only fix defects?; others use a mix of cross-functional or functional teams and adjust as needed, but this can be disruptive.
If you chose to “adjust as needed” (be agile) you will introduce some chaos every now and then but this can be minimized by having “plan of action”. How you adjust as needed and what type of “plan of action” you need also depends on what type of process you follow and what your focus is; is your culture focused on product? or is your culture focused on the customer? Depending on whom you ask (their role) within an organization, you may get different answers.
If you follow “Scrum” then your work queue is mostly “push” defined, where work is planned and pushed into a sprint/iteration before its worked on.
If you follow “Kanban” then your work queue is mostly “pull” defined, where work is pulled and worked on.
In an ideal world, one would have a process that allows the focus on both, the product and the customer. A process that allows you to “build what you say you will build” but also allow you to focus on “what is important right now”; such a process would mix Scrum and Kanban, giving you, ScumBan. The idea of a Scrumban is not new; there are books and many talks/posts regarding this. How Scrum and Kanban are merged together depend on the person who is cross-breeding the processes and what problem(s) they are trying to solve; I named my implementation “RAPID” for “Real-time Actionable Prioritized Individual Delivery”.
In my next post around RAPID, I will go over some examples that helped define the how and why behind RAPID and how it would be used in a team that wants to focus on both, the product and the customer.
So what does come after Scrum/Kanban/Agile?
The evolution of the SDLC continues as expected and if you look at the trend, the focus has been to get things done quicker – Rapid.
Andy @ Assembla calls it Scalable Agile and has great content explaining the concept behind it; I call it Rapid – Real-time Actionable Prioritized Individual Delivery, or Rapid Agile and the basics around the process are very similar. The focus or goal is to prioritize individual efforts rather than a team sprint, act upon real-time feedback and deploy much quicker; often deploying new features and bug fixes several times a day rather than ever other week or so.
Case Study: Blank Label
At Blank Label when we were much younger, we manually deployed whenever we wanted as we were trying to rapidly enhance and stabilize our offering. With every feature came a lot of bugs and it was usually the changes in usability that brought inconsistency in usability as we had limited resources and a lot of “to-do’s”. Eventually we settled on a 2 week delivery cycle but as we were attempting to find out who our customer was, we made some drastic changes only to see orders drop from several a day to almost none every week. We had no idea what of the 20 things we changed in the 2 weeks that killed it as A/B testing had told us that our sales would go up, not vanish.
Fast-forward many months – we saw that we had gone back to our old habits, but this time we had process and we were not disrupting things when we updated, we deployed to a staging server, tested things out there and then released to the live server. This still required manual builds and deployments, many times fixes just didn’t make it to the live system when issues were discovered because it required someone to build and upload…
Now fast-forward to yesterday – we now build the backlog in Jira and make use of green-hopper for their kanban board. We go through a to-do, doing, build and live workflow, where bamboo will automate code checkins and deploy with AWS ec2 to our staging server; once testing passes staging, the deployment to live is a simple click of a button and live is then refreshed.
For test, we have gone from 1-3 pushes to test per day to 5-8 pushes per day and for live will be going from one refresh per day to 2-3 refreshes. We still have a small team of developers, once this team grows we will probably see a large increase in pushes to test, but will probably maintain the 2-3 refreshes to live per day (depending on the functionality).
In the not so distant future, I will provide the development workflow process along with the tutorial to build all the Atlassian stuff that will get you to a rapid continuous integration, deployment and deliver model as I did not see a lot of solutions that focused on the .Net MVC3/4 Razor stack.
“Existing processes must be described before they can be well understood, managed, and improved. An organization must then define what these processes should be before attempting to support them with software engineering tools and methods; in fact, the definition forms the requirements for tool and method support. A variety of process description and definition technologies is available.” [*]
Ask a gardener how you can improve your software development practices and not only will you get a strange answer, you will also get a very confused look; unless of course the gardener use to be a software developer and knows your product (and current process) in an out. In other words… if one does not understand the inner working of things, such as the current chaos (or processes) or what goes on behind the scenes and how things work, then that person cannot provide you with useful information on how things can be improved.
A defined process is a prerequisite to process improvement; you cannot (well you could, but it would take quite a while) improve process without understanding what it is you are improving and how the change’s that are suggested and implemented will affect other processes.
As obvious as may sound; to begin improving process, you need to first draw it all out and identify all the bottle necks/issues one by one and strategize when and how to improve what. Many have wasted their time trying to implement something that may have worked elsewhere, or something that’s out of a book that’s there for the sake of “having a process” with no real benefit to the users.
When it comes to process – One size does not fit all.
[*] – Technical Report “Software Engineering Process Group Guide”. Flower, P & Rifkin, S, September 1990. CMU/SEI-90-TR-024, ESD-90-TR-255
“We are committed to being very agile in making sure that our waterfall process is iterative”
A team that does not work well not only is not a team; but is a blocker towards any success. It is a team (not) that does no work. – yes the not is from Borat.
We must be able to organize teams and networks to be ready for action as this helps the team get off the ground and moving. In order to help get things going as smooth as possible we can focus on a few key elements that can build teamwork and networks, these are leadership, teamwork attitude, common understanding and trust.
Leadership: In my experience, I have always enjoyed working on projects when I was able to be the leader, I think this is inherent to our nature that most of us want to be leaders instead of followers as it makes us feel important and think that we have make a significant contribution to the project. When working as a team, one of the great benefits (that most do not realize) is that the leadership can be shared equally among the team members, it can and should shift from person to person depending on what is required and the person’s strengths, this gives everyone a feeling of accomplishment and an equal and important say in the processes (even if it may not be). When working horizontally team members should be allowed to have debates, open discussions as they are key methods of identify opportunities and resolve conflicts that may arise….. so long as you can focus and move forward.
Teamwork: Rewarding team members to play nicely as a team always gets peoples motivations up. Teamwork makes a horizontal partnership cohesive so management should encourage early team building activities and open engagement that help develop a sense of collective ownership. This can be encouraged by giving incentives to work successfully as a team, such as recognizing members for their team efforts by giving awards and rewards.
Common Language: Recently I worked on a project where I was using the phrase “data entry” to describe an action where “any user enters data using a graphical user interface”. One of the team members was having a very hard time following the discussion because to him “data entry” was specific to “data entry personnel” (people hired to do data entry), who used a specific data entry user interface that was different than the user interfaces used by others, this was because they didn’t need the pretty features as all they did was repetitive data entry. This is why it becomes very important that the team members have a common mental model and have developed a vocabulary that is understood by all as this helps develop a working culture where misunderstandings or unclear terms will be kept to a minimum.
Trust: Trust is very important in maintaining relationships, it is the glue that holds a team together, if someone does not trust other members in a team, or does not fully place their trust in the team they will not be completely open and might not want to take part in discussions or coordinated efforts. The need for trust makes it important to invest in relationships and build credibility, this can be done by undertaking small tasks, being open and honest with others and delivering what was promised. When team members trust each other they will be willing to risk more together.
[Source: Moving from the Heroic to the everyday: Hopkins, Couture, Moore]
This isn’t anything new: Individual or a group of individuals (champions) see the benefits of process improvement, understand process improvement, and want the organization to benefit from process improvement by implementing process improvement. – What a novel idea.
In order to successfully implement and benefit from process improvement, its imperative that management sponsors the process improvement initiatives/activities. If management, or (in a smaller company) the owner does not see or understand the benefits of process improvement the truth is that it will never be successful.
Champions who try to implement process improvement by themselves will lose the motivation for (process) improvement along the road as they get busy with tasks and other responsibilities; this is especially true when they are going to be the only ones who are making the effort.
For example, an individual (software developer) trying to sponsor process improvement themselves to their own work, working for a company that has no form of formal requirement gathering or does not follow a software development model will soon get tired of putting in the effort (to gather requirements formally) as it will go unnoticed and unappreciated (maybe even criticized).
There might be even cases where the organization is so small (agile would work great here) that it would be impossible to place a “formal” process improvement methodology or cases where the organization feels that the processes they have right now are “perfect”; even though they might not be.
To determine if process improvement is required we can use the history of how well past projects went, under budget, over budget, issues that were faced and so on. We can also measure certain things and create actual results that tell us how good or bad our current process actually is.
It is always easier to explain your point to upper management when you have actual data and visuals such as charts that can be analyzed and used to substantiate your point; this is where measurements come in. Measurements themselves can be a complex and important tool that can be used in process improvement models such as CMMI, but it is not until level 2 and 3 (in CMMI) that you actually get to make use of those measurements. At some point (idea for another post) I will discuss how quick and dirty measurements can be put in place without too much of a hassle (and measure process without process). Not everyone is a “process” person and fail to understand the importance of process (and a formal SDLC).
Many will however understand results obtained from the measures when they are presented and used to talk about costs, budget, resources and timeline’s.