Category: SDLC
Beginning Process Improvement – What to do first
“Existing processes must be described before they can be well understood, managed, and improved. An organization must then define what these processes should be before attempting to support them with software engineering tools and methods; in fact, the definition forms the requirements for tool and method support. A variety of process description and definition technologies is available.” [*]
Ask a gardener how you can improve your software development practices and not only will you get a strange answer, you will also get a very confused look; unless of course the gardener use to be a software developer and knows your product (and current process) in an out. In other words… if one does not understand the inner working of things, such as the current chaos (or processes) or what goes on behind the scenes and how things work, then that person cannot provide you with useful information on how things can be improved.
A defined process is a prerequisite to process improvement; you cannot (well you could, but it would take quite a while) improve process without understanding what it is you are improving and how the change’s that are suggested and implemented will affect other processes.
As obvious as may sound; to begin improving process, you need to first draw it all out and identify all the bottle necks/issues one by one and strategize when and how to improve what. Many have wasted their time trying to implement something that may have worked elsewhere, or something that’s out of a book that’s there for the sake of “having a process” with no real benefit to the users.
When it comes to process – One size does not fit all.
[*] – Technical Report “Software Engineering Process Group Guide”. Flower, P & Rifkin, S, September 1990. CMU/SEI-90-TR-024, ESD-90-TR-255
The Waterfall-Iterative-Agile Process
“We are committed to being very agile in making sure that our waterfall process is iterative”
Horizontal Management – Maintaining momentum: Keep progress moving
While machines might give the same level of output everyday, the same doesn’t apply to people.
In a real environment, each member’s momentum and motivations will change without notice. This requires management to step in and make sure they are constantly evolving the processes so that members will remain motivated. Leadership is important to motivate key players, channel information to keep everyone engaged and make working horizontally routine. To accomplish this, management might bring in an outside champion, who will help wring in new ideas and “fresh” stream of motivation. Instead of looking at a project as a whole, it should be broken down into smaller goals that are realistic and achievable. Management should encourage and build on small success, to demonstrate that it was an important milestone and motivate members. Progress and results should be demonstrated so that members can all acknowledge the ongoing progress and success. Management can keep the members interests by ensuring that they are always focused on achieving the goals that were set, and if deviation is detected, corrective action is taken to bring things back on track.
A horizontal team is dynamic as it is always changing, management needs to account for its dynamic nature and make necessary adjustments to accommodate/adapt to the change. Money is probably the best motivator, but too much money too early in the process can hinder individual initiative and prevent people from innovating. Since horizontal structures focus on team collaboration, a downfall is that meetings can go on forever and schedules can be missed; this can unmotivated some members who feel that decisions are not being made in a timely manner, which is why management can implement deadlines which will help practitioners develop a common schedule and manage workloads effectively so that the projects momentum is kept at a steady pace.
[Source: Moving from the Heroic to the everyday: Hopkins, Couture, Moore]
Horizontal Management – Building support structures: To keep Horizontal team in check with Vertical roles
Even though a “horizontal structure” is flat in nature, it does still need a (vertical) support structure.
The main purpose of a support structure is to help managers build lasting relationships and achievements with teams. There are two basic type of support structures that managers can mix and match when building or working with teams, depending on the requirements as each has its own attributes.
- Informal structures don’t define rigid roles and responsibilities, they are open to interpretation and people can define their own roles and swap roles as needed, hence they are less resource intensive, more flexible and less binding on members; Informal structures help promote open discussion and communication among its team members; and
- Formal structures are more rigid and roles/titles might be defined where members have specific responsibilities; this makes them more resource intensive but less ambiguous; they require some logistical skill and expertise to implement. These structures are great when consistency and quality are important factors.
It’s important for management to recognize what type of structure needs to be in place and when its put in place; as “when” a structure is erected, it can play an important role in the success of the teamwork initiative. If too much formality is introduced early, people might feel there is no difference in the “horizontal structure” and the “vertical structure” where vertical structures are rigid, formal and bureaucratic and would be less motivated to work with the initiative. In contrast, waiting too long to build a support structure or the lack of a support structure can hinder the ability of the team to successfully work together as a team, especially when they need to be able to adapt when in tight spots.
Since the resources and effort required are greater when setting up formal structures, informal structures are great for short projects, however when the projects will go on for a long period of time or are of large scale, formal structures would be more suited as they are less ambiguous and will stay well formed longer (as they are more solid than informal structures).
Authority and roles in informal structures are more ambiguous and can be swapped around; In formal structures, the roles and responsibilities are more definitive, hence less ambiguous and in that regard, formal structures would work best in situations where important decisions need to be made.
[Source: Moving from the Heroic to the everyday: Hopkins, Couture, Moore]
Horizontal Management – Developing shared framework: Working for the same goals
When we work as a team, we usually work for the same goal(s). There will be times when we might stray away from the goals that were originally specified and we will not realize it. By developing a shared framework we can help ensure that we are all (still) working towards the same goals.
With open discussions, patience and a shared fact base (mental model and vocabulary) we can deal with conflicts internally (within the team) and develop a shared understanding of the key issues.
One major issue of working with a team is accountability, as many forget their individual accountability and it becomes unclear where the accountability lies (does it lie with the individual, or a team as a whole?). This is probably because the lines of individual accountability and the shared sense of purpose become unclear because of the focus and emphasis on the “shared” attitude towards everything and it is important to acknowledge the difference and maintain the separation. Clarifying roles, responsibility, shared goals and results upfront with open discussions and debate will help resolve accountability issues in advance. We might need to revisit and re clarify these elements as they might evolve over time due to changing circumstances and new opportunities. Having a shared and clear understanding of planning and reporting can help accomplish all these tasks (involved with developing a shared framework).
[Source: Moving from the Heroic to the everyday: Hopkins, Couture, Moore]
Horizontal Management – Mobilizing team and networks: Ready for action
A team that does not work well not only is not a team; but is a blocker towards any success. It is a team (not) that does no work. – yes the not is from Borat.
We must be able to organize teams and networks to be ready for action as this helps the team get off the ground and moving. In order to help get things going as smooth as possible we can focus on a few key elements that can build teamwork and networks, these are leadership, teamwork attitude, common understanding and trust.
Leadership: In my experience, I have always enjoyed working on projects when I was able to be the leader, I think this is inherent to our nature that most of us want to be leaders instead of followers as it makes us feel important and think that we have make a significant contribution to the project. When working as a team, one of the great benefits (that most do not realize) is that the leadership can be shared equally among the team members, it can and should shift from person to person depending on what is required and the person’s strengths, this gives everyone a feeling of accomplishment and an equal and important say in the processes (even if it may not be). When working horizontally team members should be allowed to have debates, open discussions as they are key methods of identify opportunities and resolve conflicts that may arise….. so long as you can focus and move forward.
Teamwork: Rewarding team members to play nicely as a team always gets peoples motivations up. Teamwork makes a horizontal partnership cohesive so management should encourage early team building activities and open engagement that help develop a sense of collective ownership. This can be encouraged by giving incentives to work successfully as a team, such as recognizing members for their team efforts by giving awards and rewards.
Common Language: Recently I worked on a project where I was using the phrase “data entry” to describe an action where “any user enters data using a graphical user interface”. One of the team members was having a very hard time following the discussion because to him “data entry” was specific to “data entry personnel” (people hired to do data entry), who used a specific data entry user interface that was different than the user interfaces used by others, this was because they didn’t need the pretty features as all they did was repetitive data entry. This is why it becomes very important that the team members have a common mental model and have developed a vocabulary that is understood by all as this helps develop a working culture where misunderstandings or unclear terms will be kept to a minimum.
Trust: Trust is very important in maintaining relationships, it is the glue that holds a team together, if someone does not trust other members in a team, or does not fully place their trust in the team they will not be completely open and might not want to take part in discussions or coordinated efforts. The need for trust makes it important to invest in relationships and build credibility, this can be done by undertaking small tasks, being open and honest with others and delivering what was promised. When team members trust each other they will be willing to risk more together.
[Source: Moving from the Heroic to the everyday: Hopkins, Couture, Moore]
Working Horizontally (agile) & understanding Horizontal Management (work/team structures) and its issues
In line with the content on SDLC and ALM, along with today’s needs and desires for teams to be more flat.. or vertically structured: My next few posts are going to be about working horizontally, or horizontal management (work/teams/structures), or. let me throw in a buzz word .. “working in a more agile manner” 🙂
Horizontal management is: about working collaboratively across organization boundaries. There are no hard and fast rules to horizontal management it is an art more than a science. This type of management is pervasive, occurring at entry level in an organization. It involves bringing people from diverse organizational and occupational backgrounds together into teams and networks with a common purpose and shared culture. Group thinking is encouraged, but this can be seen as a risk because it cuts lines of accountability and authority and pursuing consensus at the expense of serving the public interest. [*]
In a nutshell, working as a team/group is working horizontally. One might think that working as a team is an easy task as we all have done so at some point in time, but this is not the case when teamwork is the bases that can make or break the success of an initiative, especially when used in complex processes like software engineering and process improvement and we are in a teamwork state for long periods of time. There are various dynamics that come into play when working as a team, for example some might not get along or trust other members, some might not be motivated or good team players and others might forget their individual responsibilities. Working as a team requires that everyone get an equal part so that everyone has an equal interest in working together, towards a common goal, build long term relationships and be equally motivated for the long run. An interesting relationship between horizontal structures and software development is that working in a horizontal structure is viewed as an art, which is the same way that software development was viewed as before it was realized that it needs to be a manageable process, our focus will also be to help make horizontal management a manageable process.
Some of the issues that horizontal structures bring with them that need to be addressed are:
- Working with others as a team slows down fast paced, ready for action members
- Members are sometimes not motivated enough to work as a team, which damages team effort and progress
- Members usually trust their own judgment and find it hard to put that trust in the hands of others
- Each member may have their own vision and own motive, maybe a goal they want to accomplish as part of their job function and have their own priorities.
- With everyone on the same level, who provides direction and leadership? Who is held responsible?
- People need to be assured that their time and effort is resulting in progress.
Key Dimensions of horizontal management
Working as a team rather than working as an individual does bring with it a few issues that need to be addressed:
- When we work alone it is easier for us to switch gears and start making progress because we don’t have to wait for others to be ready, progress begins when oneself is ready, in a team progress begins when “everyone is ready”.
- When we work alone we know what our goal/vision is, we know what we want and know how we are going to get there if we start straying away its really easy to pull ourselves back; When we work as a team, its possible that not everyone knows that the common goal is, they might have their own goals or own ideas of how things need to be and what takes priority over the other, when working as a team its important to make progress by combining all efforts towards a “shared goal/vision”.
- Working alone means that you are responsible and accountable, you need to get the work done and you need to manage your own time, when working as a team, who does what gets blurred this is why a “support structure” must be in place (depending on the requirements) so that vertical roles and responsibilities are not forgotten.
- When you work alone, since you are in control (and responsible) of the progress, it is easier to maintain your momentum, you know how much progress you have made, you know how successful you have been and if needed can make changes to help maintain momentum; Teamwork introduces attributes that may effect momentum such as meetings that go on forever, goals that seem unrealistic or far fetched, not knowing if all the effort in working as a team is paying off or not and so on. When working as a team, we must be able to “maintain momentum” on a daily bases.
Hence there are four key dimensions for horizontal management that revolve around horizontal structures and keep the horizontal structure/movement intact: [*]
- Mobilizing team and networks: Ready for action
- Developing shared framework: Working for the same goals
- Building support structures: To keep Horizontal team in check with Vertical roles
- Maintaining momentum: Keep progress moving
Each section would detail a significant amount of content so I will break it up into 4 posts to cover this topic….. eventually.
[*] = Hopkins, M. Coture, C. Moore, E. Canadian Centre for Management Development, 2001. Lessons learnt from Leading Horizontal Projects
The SDLC evolution
- Software engineering is an art, there is no science
- we put what we are supposed to do on stickies, and the developers know how to work them
- why are they stuck to your monitor?
- so i know what requirements people tell me I need to create and complete
- I can never figure out you are working on, or does all this work keep coming from
- the cleaning crew lost my stickies
- we keep running late on our promised delivery
- we need project managers to manage your queue so we know what you developers are doing
- Looks like we need process and stages, so we know what the real requirements are, and have an official sign off
- Yea, no one should ever code unless we know what we want to build and how its going to get broken up
- Okay, it took 9 months to document everything we want, lets code
- ….another 9 months… code complete!
- hmmm, looks like we built something that would have been useful 15 months ago
- …and looks like no one really wants to use it the way we build it
- okay, so lets define a bit, code a bit, and deliver a bit
- great, that way we can deliver sooner and see if its useful
- hey, this is great, but can we do it faster and better? can we get rid of more processes?
- monthly products and feature sets released…
- ..wow, this is great, were pushing out really useful features based on customer needs
- yea, but all these sprints, and the time boxing, there’s too much structure
- there’s gotta be a way to be even faster, leaner
- yea, is there a way to make these sprints visual? and the work to be done visual?
- yea I’m very visual, i could just look at a board and understand what i need to work on next
- how about a big board with stickies?
- that would be great, we can have a to-do section with all the stickies that need to be done. and then move…
- ..yes, yes! and we can move them from one area to another., and we can write our name on the sticky
- and we can really focus on the stuff that we are working on… and…. and… and…
- …this will work,
- ……if we all understand and know how to work these stickies
- …wonder why no one thought of this sooner
Kill the (process) champion
This isn’t anything new: Individual or a group of individuals (champions) see the benefits of process improvement, understand process improvement, and want the organization to benefit from process improvement by implementing process improvement. – What a novel idea.
In order to successfully implement and benefit from process improvement, its imperative that management sponsors the process improvement initiatives/activities. If management, or (in a smaller company) the owner does not see or understand the benefits of process improvement the truth is that it will never be successful.
Champions who try to implement process improvement by themselves will lose the motivation for (process) improvement along the road as they get busy with tasks and other responsibilities; this is especially true when they are going to be the only ones who are making the effort.
For example, an individual (software developer) trying to sponsor process improvement themselves to their own work, working for a company that has no form of formal requirement gathering or does not follow a software development model will soon get tired of putting in the effort (to gather requirements formally) as it will go unnoticed and unappreciated (maybe even criticized).
There might be even cases where the organization is so small (agile would work great here) that it would be impossible to place a “formal” process improvement methodology or cases where the organization feels that the processes they have right now are “perfect”; even though they might not be.
To determine if process improvement is required we can use the history of how well past projects went, under budget, over budget, issues that were faced and so on. We can also measure certain things and create actual results that tell us how good or bad our current process actually is.
It is always easier to explain your point to upper management when you have actual data and visuals such as charts that can be analyzed and used to substantiate your point; this is where measurements come in. Measurements themselves can be a complex and important tool that can be used in process improvement models such as CMMI, but it is not until level 2 and 3 (in CMMI) that you actually get to make use of those measurements. At some point (idea for another post) I will discuss how quick and dirty measurements can be put in place without too much of a hassle (and measure process without process). Not everyone is a “process” person and fail to understand the importance of process (and a formal SDLC).
Many will however understand results obtained from the measures when they are presented and used to talk about costs, budget, resources and timeline’s.
Application Lifecycle
SDLC along with the words “Agile”, “Iterative”, “Kanban”, etc are, in my opinion buzz words. You can pick up a book and learn “Agile development”, you can take a few courses and get your “Scrum Certification”, but following the documented approach, as is, will only get you so far.
To really understand SDLC you have to get into the guts of how things really work; why do people do things the way they do and understand the “natural processes”.
For Start-ups, small companies, new companies, walking in one day, or starting of with something like “We are going to follow agile development”, works. It works real well because they choose to start with the textbook definition; it works real well because you have new developers who will be agile and flexible, and do things the way you choose to do. For such an environment, someone can walk in and make sure that things are being followed the way they are supposed to be; but unfortunately, this is not how it works when you try to take something that has been running for a (very) long time and you try to make it different.
As an example, look how some auto manufacturers have decided to implement and offer “high MPG vehicles”; some chose to restart and build a brand new vehicle; others attempted to re-purpose or salvage their offerings.
So talking about SDLC methodologies; Who really wants to follow a waterfall model? that makes you sound too outdated and boy, I wouldn’t want that on my resume. (Sarcasm). For many, waterfall is still the best methodology but they still want to move to something that’s more agile; and there is no harm in doing so, just as long as you understand and implement something that works well for you.
Now, to build something that works well for you, you really must understand YOUR current “Application Lifecycle” and see how it can be optimized. SDLC is just a very small part of the “Application Lifecycle” and once you understand, document and optimize your “Application Lifecycle” you can focus on modernizing your SDLC.
So, to build the example of understanding the “Application Lifecycle”, I will draw up a few diagrams. I will focus on the positive paths and not the negatives so that the model doesn’t get too complicated and I will also make some assumptions/skip extensive detailing, saving that for a separate post.
Starting the cycle.
- Once you have gone through all the (painful) steps of creating a product (in no order: Sales, Requirements, Dev, Qa, Test, Release, Etc) you end up with Product
- Customers use this product and they
- Report issues
- Reported issues go to your support team (hopefully)
- and the support team analyzes these issues.
Pretty simple right?
You have now documented the beginning of the cycle as after product deliver, you are getting things back that require work. So what does support do after they Analyze the issue?
- Confirm the issue and provide additional documentation (if needed), sending it to
- Some review board, or decision maker as to what to do with this supposed issue, and determine if its something that needs to be resolved now (defect) or something that needs to be done at a later time (enhancement), or maybe you do not want to do it. In any case,
- The review board makes a decision in regards to releasing it (now).
At this point, hopefully there is some queue setup that allows intake of things for development to work on as they will trickle things into their work queues (depends on what SDLC methodology you follow)
- Development picks up work and resolves the issue; after the issue is resolved, it is
- sent to QA for additional testing, and once
- QA approves the resolution, its
- given to Release Management, who then
- releases the resolution
- giving you a packaged resolution.
All this assumes that there was actually a code defect, and that it required immediate resolution as it was critically impact a live customer. Things get a bit more complicated when you bring enhancements, non-code issues, and other things into the picture.
Also, big “hand waves” are done at various points in my flow, for example, the following things need to be looked at as well
- How is the intake of work managed and prioritized, and by who?
- What is the regression/integration process?
- Who manages the movement of the issue from one area to another?
- How are issues that should not go to the review board handled?
- How does the information go back to the customer?
- How is information distributed to other customers?
- … and so on….
Understanding the Application Lifecycle, really does take “process improvement” to the next level as it allows you to follow several different best practices when it comes to SDLC or even the Organizational Lifecycle; For some organizations, they have a dedicated Business Process Office (or process office) that is responsible for looking at the big picture and ensuring that process is continuously optimized and improved.
So the next time you find yourself, wanting to “improve things” ask your self, “how big” of an impact you plan to make.
Being involved in such efforts right now, I wish you good luck.
Zeeshan Sheikh 