The Startup and the 3 P’s: Product, Process and People
I will not pretend to know everything about startups and startup culture, but I will list the reasons why startup culture is exciting, at least for me:
You meet great people, people who have ideas and want to try things, people who have passion and want to make an impact, people who will challenge you to do better. There is passion for working together as a team, passion for building trust within the team and passion for collectively making an impact in other people’s lives; or sometimes, passion for just making something happen – to create. There is passion for possibly creating something that could go big – disrupt everything, all built from the ground up with the teams sweat, blood and tears where everyone is high on adrenaline. Suits? Offices? As long as you are connected with your team and are working well together, those things don’t matter. There is no red-tape, or big top-down structures, everyone and anyone has access to all. Anyone can start working on anything, there are many hats to choose from; wear all. You don’t get bored as things are evolving and stay fresh, there are new ideas, old ideas, odd ideas; anything can change anytime.
At the end of the day, a startup is defined by its growth; when a startup doesn’t grow, it dies; it stops.
There can be several growth stages for a startup, and startups evolve; once they start growing they are now “startedup” and will hopefully grow exponentially. In a perfect world, the cultural values that made the startup fun would remain and in some cases they do (depending on where the growth has lead the startup) but there are times where the culture itself that helped the startup grow and evolve starts conflicting with what is needed to grow to the next level.
Let’s say you follow agile and you end up with iterations, planned work, release schedules and a clear pipeline of what needs to be built. This all worked great when you had 2 products and a team of 10; since you have grown, the expectations of what you can or will deliver have also grown. Some brilliant folks in your team have discovered 2 more products that should be added to your portfolio; how do you grow your current 2 products (since they have a feature and defect backlog) and also work on these 2 new products without increasing your team size, changing delivery for current products or burning out resources? Before you grew, you may have had your own expectations of when and how you would bring on these two new products; now that you’ve grown, others may have different expectations from you and your team(s). Maybe you say “we need more people”, which brings me to the next point
With the growth of the startup, either through sales, funding or more investment and the need to create more product it is decided that you bring on more people, and you do. You end up facing the same issue, how do you grow people with the same 10 resources you had who are busy working the two existing products; some of the people you bring on may be self-starters and will figure everything out by themselves but what about the ones who don’t? So now you say “we need some process and automation to free up some of the manual work so that we can do more with the same resources”, which brings us to…
How do you focus on process and automation to free up time when the people you have are busy with supporting the existing two products, or are supporting the existing two products and are also trying to bring the new hires on-board?
A part of me says that the above three growth challenges are not really challenges and that they are part of what it means to be a startup culture and are expected. However; there are a few by-products that the 3 P’s create that can become toxic, stop growth and hurt the culture if they are not accounted for when trying to grow.
The Frat party & the first team
The first team consists of the people that built the startup; it was their teamwork and effort that made the startup grow; anyone who comes later is an outsider and “we need to be careful about who we let into our frat party” (once upon a time I lived on frat row). This one is not intentional, but when you work closely in teams and blur the line between friendship and co-workers, you end up creating an inner circle and make it challenging for an outsider to easily integrate and feel welcomed. This by-product is a blocker for People growth.
The golden simple process
At some point there was predictability and little chaos in what all needed to be done (smaller team, less products) so everyone starts expecting things to always be perfect. Even though you have grown, you have kept your process simple and did not optimize for KPI’s and other metrics that can help with predictability, complexity, risk and estimation. There will be times where things change, dates get reset and/or product scope creeps. If you had built a roadmap of what releases when, had committed the teams to that and put all these releases with their iterations back-to-back (because of all the product that had to get pushed out to show growth and maturity) and dates or requirements change on you (usually not for the better) the team and its happy culture will get disrupted as it will take effort to get things back on track; when/if this happens all the time, it gets hard to get away from the domino effect and people burn out, get disengaged and/or leave. This by-product is a blocker for Process growth.
When you were small, everyone knew what everyone else was doing, everyone shared and individuals had their skillsets. Now you have grown, 2 months ago you were 10 people, today you are 75, the 65 newer ones don’t understand the code base or the original design, there is some good documentation but they need more information and there are 3 key people who know different things about the original products; original products that you want the new 65 people to work on so that the first team can work on the two new ones; how do you distribute the knowledge known by the 3 key people, make them available to the 65 and allow the 3 key people to focus on their new projects? If they are constantly being pinged by others and cannot get their work done; their sense of accomplishment doesn’t scale much; especially if you did not plan for them to set time aside and help others. This by-product is a blocker for Product growth.
Each blocker is situation (just like leadership) and can be solved; we will examine and solve for each, before we move onto other “StartedUp” culture challenges. The next post goes into process KPI’s and metrics – addressing the golden simple process blocker.
Success is always welcomed; we always work towards success, and preferably in most cases we can track towards it and see it coming…
But what do you do when success comes and you are the least bit prepared?
In the eCommerce world, your ability to evolve will make or break you. If you don’t rush to ensure you are prepared, you will end up with failure. I learnt an important lesson in eCommerce and startup almost a year back, may 2010.
Most tech based startups, start lean, and thats the appropriate approach. You do not want to go out and get dedicated hosted, or even get on a cloud host that costs a few hundred $’s a month when your revenue is $0 or already in the negative.. on that train of thought…
We were hosting our Blanklabel website on a shared host. Our provider was discountasp.com and I had (and still do) used them for years for many other projects/websites. I have always known the benefits and drawbacks of using a shared host…. and at 60k visitors per month and no set bandwidth limits, it was a great home for a “start up”….
That was until the New Your Times published an article on us.
To be fair, I did know that we were getting an article published in NTY; but it was a day or two before and there wasn’t much we would have done in preparation for it mainly because we did not know to expect. Our website was functioning, the order and payment system was fine, everything looked good…
Within a few minutes of the article showing up… we went from “functioning perfectly fine” to the brink of failure! How? Why?…. I live in CA so below is a time-log in PST of the sequence of events..
3:00 AM – I was coding Away… decided I needed to get some sleep so I closed shop…
3:30 AM – My phone rang, went to voice mail, woke me up,
3:35 AM – Checked my email, saw some questions about performance, looked at the website, seemed fine, sent an email in response, sent an email to discountasp went back to sleep
4:30 AM – Phone ran, answered, heard something like “It seems to be fine on my end, but customers are saying that the images are not loading”., I said “How many? just a couple? Call me back if its more than couple, could just be their connection, its working fine for me as well”… checked email, discountasp responded with the same “looks good here”, emailed them again “We are getting more complaints”.. back to sleep
6:00 AM – Phone rang again, Panicking person (you know who you are) on the other end “I have 22 chat sessions open with customers who are trying to buy, there is something definitely going on”.. This time, I got out of bed, I cleared cache, refreshed the pages a few times, tried to place and order, and there it was, random images were missing, email went out to discountasp again..
6:15 AM – I decided it was time to start looking at the logs… and there it was… As people started their morning on the East coat and were looking at their NYT print and web… our hits were rising… we had gone from a few hundred per day… to 35,000 per second… yes PER SECOND.
6:15 AM – Emailed discountasp again! asking if we could increase shared resource or get some sort of load balancing and pay for multiple servers… and also decided it was time to look for a Plan B
6:16 AM – Started looking for my Rackspace contact….
6:17 AM – Started to chat with Rackspace
7:00 AM – Signed up for Rackspace Cloud
7:19 AM – Started to download the current discountasp FTP snapshot
7:20 AM – 6 month old daughter woke up, so I was entertaining her while i did the rest below…
7:30 AM – Response from discountasp “Yea, we cant help you”…..
7:35 AM – Started to prepare a mental migration plan.. aka Plan B…
7:40 AM – Plan B was ready! Created SSL request at RackSpace, MS Sql server provisioned
7:45 AM – Download database backup file
8:00 AM – Restored Database file at RackSpace… Decided I should account for the orders that will continue to go over to the discount house so that no orders are lost and that I can migrate them over later… So I put in a 1500 order jump in the identity seed
8:XX AM – The rest of this hour went towards uploading the website to Rackspace,updating code to ensure that things worked fine on Rackspace, doing a few test runs…..updating DNS, generating and applying SSL,
9:XX AM – Started seeing orders come in on the new Database…. not just 1 or 2 here and there., but 15-20 every few minutes minutes. Was still chatting with Rackspace support to see if we could get some sort of stats, but since we had just signed up, it was not yet setup.
10:00 AM – Started to see discountasp log show traffic had reduced, server was now able to process requests that it got while the DNS updated globally.
10:XX AM – Declared Victory, while all the above was going on, My daughter was entertaining me.. or was it I who was entertaining her?…
11:00 AM – Continued to see things progress… orders still coming in… in hundreds… Called it a day for now… needed a break from this..
11:30 AM – Some more emails came in from discountasp ….. I wasn’t too happy with their lack of customer service and commitment… I guess the name does fit it well… its a discount store.. nothing compared to Rackspace’s fanatical customer support… I would not say they are really that “Fanatic”.. but… its better than discountasp and they are usually helpful.
6:XX PM – Due to the large # of orders, outgoing emails got blocked because the Rackspace servers saw it as SPAM/Flood. after an hour or so of going back and forth, Rackspace unblocked the email and thousands of emails went out to customers.
2:00 AM – Discountasp hosted site was at 0 hits; Did a global DNS check, found all routes leading to Rackspace host.
2:XX AM – Started coding scripts to bring in the orders that ended up going to discountasp servers while DNS was still updating
3:XX AM – Executed the scripts after verification.. was impressed to see that my buffer of 1500 served me well., we had 1393 orders that ended up saving on the old database… once these were broight over with the same Order Id we had a gap of just 107 orders!
4:XX AM – Realized that I forgot to put in a buffer for the User Id, had to resolve this as we now had two users with the same User ID’s because of the split servers
5:00 AM – Emailed out the #’s of shirts we sold to the team!…
7:00 AM – “Shut down the website, we cannot process any more, we are over capacity”…. I will leave this one for another day..
In the time-log above, I have highlighted take-aways in bold that will help you prepare and execute a “Live Migration” from one host to another host AND ensure that all your data comes with you, with no additional negative impact to your customers. Today, we are still with Rackspace and are content with their service… We have however grown and believe we are ready to take the next big step in cloud hosting.
This blog post is just one of the many examples where potential success can take you down if you do not know how to prepare, execute and evolve in real time.