Agile: Coaching vs. Training

Meet Tom. Tom is your neighbor. His twelve year old knows more about computers than he does. Despite this fact, Tom is the best Coach in the business.

Meet Steve. Steve has worked at six major software shops before he started his consulting company. Steve knows all the ins and outs of TDD, Retrospectives, IPMs, RPMs, Velocity, and fourteen programming languages and their contextual stacks. Steve is well equipped to be a Trainer, but he's no good at coaching.

At this point, you may have a "wha?" face. There's a difference between coaching and training, a picture I intend to paint clearly in the next few paragraphs. Then I'll talk about how Pivots are often both Toms and Steves. Lastly, we'll get in to how the two can interfere with each other.

Coach Tom is the guy who listens to your less tangible challenges. He not the "I need an algorithm..." guy, and notice I said "listens". Tom is important because he recognized that for these types of problems, you, not Tom, is best equipped to solve them. He knows that you have a lot more context around this challenge, you know the boundaries, and you know what your own capabilities are. Tom facilitates by asking very open ended questions that get you to examine the problem with your own eye. Tom respects you as a high functioning, creative, intelligent, adult. If you were not those things, Tom would do his job very differently. Coach Tom in a lot of ways might resemble a [good] psychologist in the way he gets you to self-assess the various qualities of a situation.

You also need Trainer Steve at your company. Trainer Steve is the guy you call in when you aren't the best person to solve your own problems. Steve does a lot more showing and telling, and a lot less listening than Tom. Steve has expertise that you or your organization does not possess. Steve knows frameworks and implementations. Steve's experience, not yours, is why you brought in a Trainer. Steve should feel like a teacher in the way he imparts his specialized skills unto your group.

Pivots, of Pivotal Labs fame, in almost all cases, are both Tom and Steve at once. A Pivot is a developer first, he has a wealth of knowledge about your platform coming in to the project. He also has a bank of experience about what works well and what doesn't when it comes to running a project. A Pivot can train you how to make your product awesome if you ask him to.

Pivots are also great coaches. We pair nearly 100% of the time, which means that at least one of us is listening half the time. Pivots understand that while we have a lot of technical and process knowledge, you have the domain knowledge, and that you are the best person to solve challenges in that problem spaces. Over time, we get really good at asking questions that lead you to the answers - almost like a modern day Socratic method. Most of which isn't even a conscience effort. I suspect the same drive that makes us want to learn and understand new programming languages or frameworks, and the rewards that has, also shows itself as a desire to understand our clients because we intrinsically know that our output will be better if we can take the time to listen.

Pivots, despite their ability to span this Coach-Trainer domain, are still just mere mortals, and we get frustrated at times. It probably strikes hardest when we want to train, but a client has asked to be trained, and maybe hasn't even acknowledged the gap that the training would fill. Maybe some of the features are in a document, and the bugs are in Bugzilla, and there's a list of stuff "we should do eventually" in the corner of a white board in one of the conference rooms.  As a Pivot my instincts would scream at me to get you on Tracker, our award-winning software that helps you keep all your product-related work in one place and gives you the added bonus of velocity to help you predict when things will be finished. For you, the current system works. You haven't felt the pain, so you have no interest in change. As a Pivot, I want to spare you from that pain, but I can't.

You have to feel it. Then you have to want the coaching, where someone will listen to you, and ask you open ended questions that bring you to the realization that "Hey, I have no idea when this set of features will be complete", or "Mike in Boston can't see what's written on the white board, but he'd be great for knocking off a lot of those tasks". "But what's the right solution? I've only every used Excel to make lists." Now you are ready for the training because you've identified a problem, and a lack of expertise.

You can see now that there are two clear roles, a Trainer, and a Coach. The diference between the two is who has the expertise to solve the problem. In the case of the Trainer, the trainer has the expertise. In the case of the Coach, you the coachee has the expertise. Pivots, because of the nature of our business, are capable of being both Trainer and Coach. It is this duality that sometimes causes us to pull our hair out, but is ultimately responsible for your sustained success.

The Goal

I participated in a meeting recently that was designed to create a document that would help govern the actions of the organization. I quickly felt that the goal of making a document was lost when we started debating the order of words. I wondered how we were going to get to talking about behavior when we were getting hung up on sentence structure. As I sat there thinking about the situation, I was reminded that the same kind of thing happens in Agile all too often.

People try to implement the details of Agile without thinking about the goal at hand. With Agile, or any software business methodology, the goal should be to deliver value to the customers. The customers define what is valuable, so all you have to do is behave in a way that gets the value from an idea to tangible.

I was reminded today that my writings about some of the details can sound absolute - a "My way, or the highway" sort of ring to them. Sometimes this is one hundred percent true, because the goal of a specific piece may be to evoke a response, to challenge one's current thinking. Other times my writing is about a specific situation in my work that I felt went particularly well or particularly poorly.

At the end of the day, I really have no preference for how a company operates, so long as it does operate in a way that the good employees are happy. My thinking here is that the "good employees" I mention and I can work any number of places, for any range of salaries, and any range of responsibility levels. Yet, they are going to pick to work for one company over another based on their job satisfaction.

My argument, and central theme for my writings in this blog, is that happy employees are good for customers and delivering value. I also submit for your review that Agile has a lot of practices which have a high success rate of leading to happy employees. Thank you for your readership, I look forward to using this new feedback to increase the clarity of what is important when I write in the future.

Generalist Myth

I consider myself a generalist. I like to think I have a layered understanding of a lot of technology and how those pieces connect to each other. I am the kind of guy when an interviewer says "So I see you know ASP.Net, how's your HTML?", I kinda cock my head to the side like a dog and say "Well, it's really good.", because to me, one lends itself to the other. I'm a better C# web app developer because I'm also skilled at HTML. Similarly, I also write better HTML because of my experience working with OOP.

Now the question "How does that play into a team?" comes into play. David Christiansen says that he wants a team of Will Reads (a scary thought, especially if you're Matt) when starting out, then progress to specialists. And this completely makes sense. Up until 2004 all of my work was essentially solitary. To compensate for not having a DB guy, a UI guy, and a Java expert gal at my ready, I did my best to become all of those things. I naturally created a team of sub-specialists within me. It's the same way with a small company that's finding new legs. Do you want the guys that really know about the details of Oracle's implementation of the currency type, or do you want the guy who can flip from pretty good architecture design, to dealing with a customer bug, to filling the coffee pot when it's empty? Me? I don't like coffee very much.

No problem, it's half a year in the future of the dot com boom and you now have more developers than you can shake a stick at. Along the way you needed some specialists because you needed to know how to scale well, and you wanted some great design on the next product line, and you really needed someone who could make UI that blew the pants off your users (which is a really nice feature if you happen to be making a porn site). Now you're wondering if you can make a team of specialists, or do you need a healthy dose of generalists to mellow out the silo of knowledge? This is where Scott Ambler talks about generalizing specialists. These are people who have a specialty, but are also inclined to learn about the domains that neighbor their own.

In the case of a highly functional Agile team, you really can't prevent specialists from learning about other areas, not that you would want to. There's just too much communication for that knowledge to not spread amongst team members. You can facilitate it, encourage it, though various mechanisms like pair programming, but you can't stop it if the team works to its fullest potential.

What really should be going through your head is this: Why am I even interested in this topic? The answer is undoubtedly the #1 concern of Agile: FEAR. Project managers are afraid that somehow their company will hire 600 people who love to work with databases, only four guys who know how to write the application layer, and they'll contract a guy to do the UI "as needed". Their fear is warranted because a team of database experts is going to have a hard time making the jump to the application layer on their own. However, a team of five database experts and one application layer lady stands a much better chance of becoming a cross-functional team that can deliver on features; and at the end of the day, that's really what matters most. If Susan decides that in order for her team to deliver on time, she has to crank out some Java instead of SQL, then it really shouldn't matter to anyone but the team. It also shouldn't matter to anyone outside of the team that the other five developers aren't "well rounded" if they are meeting expectations. If the team is not meeting expectations, then the team needs to be made aware, but beyond that, it is up to them to fix the problem. That's the beauty of a self-organizing team.

Agile is built for generalists like me. It is also built for specialists. Fear is what Agile eliminates. Agile relies on the team to react to changing job requirements, and empowers the team to make the changes needed. This gets the team to invest themselves in what they produce. I believe for these reasons that it does not matter if you hire generalists or specialists, only that you hire people who are willing to adapt, people who are willing to be Agile.

Selling Agile to Developers?

One question I got recently was "How do you convince developers to adopt Agile methodologies?"

And I scratched my head.

And I gave my inquisitor a funny look, like maybe he said "sea gulls" instead of "developers", but his face was steady, he really said "developers".

So I gave him some answer, indicating that "Yes, I understand what it's like to have to go above and beyond explaining what Agile is to really sell it to a dev." But I don't think I can relate at all. How can a developer see that he gets to control his work load because he (and the team) get to provide the estimates, and not be in love immediately? If that wasn't enough, a dev now has:

• a Scrum Master defending his estimates with empirical data from things like burn down charts
  
• increased communication and involvement with his team, meaning that he is going to grow professionally from it, and so are his team mates
  
• more contact with the person or people who have input on a feature, and
  
• less management that only impedes his process
• a well defined list of features to work on (though the individual features may still remain vague, at least the target is not moving anymore)

The dev will also find that he gets to deliver working software numerous times throughout the year. I can't imagine what kind of person you can hire that would say, "Really? That's what I get from Agile? No thank you." I would probably fire such a person because either a) that person had a perfect job before working for me, and he's a moron for leaving, or b) that person is not actually a developer, but is in fact a joy-hating monster, or c) that person has never actually worked as a developer before.

If you put a microphone in a non-Agile developer's home I bet his top complaints would be:

• Of the things I'm working on, I don't know which one has the highest priority so I don't know what to do first!
• My team mates have no clue... about anything!
• My boss just set a random deadline for next week after I told him this project would take a month!
• I thought I did what was asked for, but it turns out they wanted something completely different!
• I can't get any work done because my boss always asks me for status updates!

With Agile, all that crap is fixed. To me, the hard sell is to the business side. Suddenly the Product Owner has to be available for questions a lot more, and he has to maintain a prioritized list of features and bugs, rather than just a list and pretending that the entire list is filled with high priority features, all of which will go in the next release. The project manager has to change the way he reports on progress, and has to participate in stand up meetings. Sales has to reign in what it promises to customers because it can't inject a feature into the product at will (instead it can introduce a feature via the PO at the start of a sprint and should expect to get it at the end of a release instead of "by the time the customer signs"). Customers have to start really dealing with trade-offs, "Do I want feature Z, or do I want Bug Y fixed?". And there's more visibility across the board, no one can hide behind lies or shift blame any more.

The business side of the system has been working pretty well for a while, but at the expense of the developers. The developers should want to fix it if they have their head on straight. The rest of a company could justifiably take a stance of, "If it ain't broke, don't fix it", and that's a hard perspective to change. Your developers should want to adopt Agile if they're worth their weight in salt. If they're opposed to Agile, make sure to check yourself and that you are presenting it so that the benefits to developers are as clear as a bell. Agile it should sell itself if you've done that correctly.

iPhone Web App in Ruby on Rails

This app has been made before, but not by me, not for the iPhone via the web. It's basically an overgrown score pad for fencing referees. When done by hand, a ref has to have a stop watch, a piece of paper, and a writing device, and he's not supposed to use a clip board because he has to make hand gestures to indicate the action and who is awarded the point.

My former fencer, Josh, came to me talking about an idea he and Ed had to incorporate iPhones and iPod Touches with scoring boxes and the bout committee. Up till then, I had been reading Rails for .NET Developers, and I had been hungry for a project to try out my new found knowledge.

In all honesty, there's no need for any server-side scripting in this app...yet. Once it ties into data from the BC, then it'll make more sense - so for the moment it really could just be straight HTML, CSS, and JavaScript.

What I like about this little app, this one page view into the status of a fencing bout, is that I finally did all of the suggested best practices that I had been avoiding. I used table-less design, so the presentation is all in the CSS, and not inline in the HTML. I did the same kind of thing with my behavior. All of the onclick event handlers are assigned in a script block using Prototype's Event.observe method. From there I even separated the code into separate blocks, one for dealing with the score, one for the timer, one for priority, one for swapping, and one for resetting. Once I started to make that kind of separation, my dependencies were clear as a bell, and easily refactored.

The resulting HTML, CSS, and JavaScript are all chunks of code I feel comfortable with. I feel like this is maintainable client-side code - code that I could easily have someone else work on with little explanation. I have never felt that way about client-side code before. I have known that the nagging feeling about client side code was the same feeling that others had to be experiencing, but I didn't know it could be addressed in the current state of the web.

Sadly, this only takes care of my code. It will not stop other people from writing bad code. Event the widely accepted method in the iPhone community for detecting if the browser is in portrait or landscape mode had a lot of room for improvement. Originally it was polling ever half second, when it could be tied into the resize event. Then I also removed the check to see if the width was a magic number of 320, and instead just did a test to see if the screen width was less than the screen height. Now if Apple comes out with a new phone with higher resolution, I'm all gravy baby.

Developing for the iPhone also made me think about the UI a lot more. In other apps like this, you'll see separate buttons for starting and stopping the timer. You won't see buttons like that on my app because I realized that a user might be ok just tapping on the timer itself to make it change state. The font color changes to reflect that the clock is ticking. A similar idea is employed with the score. Taping on the score itself adds one to the count. I wanted to use double-tap to decrement the score, but sadly the double-tap is reserved by the system for other functionality. At the end of my development, I found documentation on making objects wiggle to indicate that they are draggable, access to the multi-touch events which are handy for rotating, scaling, and just interacting with all kinds of things on the screen, and also a little lock-slider clone.

Big Org, Small Bandwidth

You're using Scrum? Excellent! But now you're adding a new team and you want to stay connected so you're now going to two stand up meetings each day. Okay, that works. But what if you add another team? Two more teams? Five? Now you're either spending your whole day in stand up meetings (and that's a lot of standing up), or you're only getting part of the picture each day.

You do some research and come upon this Scrum of Scrums thing. A representative from each team now makes up a new team and they have a stand up meeting. And this works pretty well until you've reached about fifteen teams. That's fifteen people giving updates about the progress of 100 employees. It starts to become an information overload. So you do what worked last time, and add another layer, a Scrum of Scrums... of Scrums (aka SoSoS or S3 because we're quickly running into the land of ridiculousness). You have successfully formed a tree network (see the diagram above from the Wikipedia entry on Network Topologies) and you have also managed to alienate yourself from all the people doing the work. *screeching brake sound*

You think to yourself, "But this is how business has always worked! Yet.. shouldn't the head know what the feet are up to?" You scratch your head because the tree network isn't making sense, and usually when a thing has this "smell" to it, it means you could be doing something better. "But I don't have the bandwidth to stand up with 15 teams, how can I possibly stand up with 50?" For that, I have a suggestion.

The real impediment is you as the department head. You don't have to know everything. There are some things you need to know, like projected release dates, and product backlog priorities, but these are things that you should be able ot easily get from your teams either electronically or by walking by their team space where they have their burn down charts posted. The people who need to know everything, are the people doing the work. Jim needs to know about the new Logging API so he doesn't have to rewrite one of his own. Bob needs to know that Sue has a problem with the parameters being passed to her function because she wasn't expecting NULL to be allowed. Dave needs to know that he needs to add upgrade the Linux servers to the latest Java VM to fix a bug Mark found, and the new employee, Mindy, needs to figure out where to download the third party DLLs from. If you're doing SoS or S3, somehow that knowledge has to not be omitted from being presented at the SoS, and then brought back to the team, to someone who might have a solution, and then that brough back to the SoS, and back to the original team.

You don't need to know all that, but they do.

And yet, those team members don't have the bandwidth to convey that information directly to everyone else any more than you do. We've tackled this problem of bandwidth in software plenty of times. When we can't push enough data in a cost-effective way between a server and the clients, we either a) get more servers and a domain controller to route traffic along the tree, or we make a peer-to-peer application like BitTorrent. We create a bunch of partially connected networks, using a part of the available badwidth of each user to create a very scaleable network that is more resistant to single points of failure, is cost effective, and still gets the job done.

I propose, that people, Scrum teams, can communicate the same way. If Team A always talks to Team B and Team C, and Team C talks to Team D, then all four teams are either directly or indirectly connected. Eventually, knowledge from Team A would propagate to Team D through Team C. This means that instead of a Tree network, we're looking at a Mesh network.

Goals:

• Allow the network to be self-managed. Teams should naturally work with other teams where there is overlap. But do give them the time and tools to form those networks. Maybe instead of having a cross-team standup every day, you provide those teams with lunch at the same time in the lunch room.
  
• Help reduce the degrees of separation. If Team A works with Team B, who works with Team C, who then works with Team D, A and D are very separate, and the chance of knowledge being passed from one to the other is very small. Do this only when it makes sense by fostering cross-team communication (assign them related feature sets, have an off0site with the two teams, etc). Ideally you'd want a Fully Connected network, but then you're out of bandwidth all over again.
  
• Identify "hubs", and participate in those stand up meetings. Some teams will naturally be more "social" because of the nature of their feature set, or by the makeup of the team membership. Hook into these teams to get a feel for what is going on day-to-day. Think of it as standing in Grand Central Station, you're much more likly to see someone from San Francisco pass through there than you are if you waited in Boonyville, USA.
  
• Don't let the connections stagnate. Some connections will always need to be there, but by forming new connections, now there's a chance for all of the knowledge to transfer to a new area within the company. Transfering members across teams who have strong bonds with their old team could serve you well in this area.
  

Through this Mesh network, a company can produce a much more adaptive communication structure that diseminates knowledge more reliably, is at less risk of a communication failure, and addresses the issue of limited bandwidth in a way that scales to any sized company. Best of all, it fits with Agile principles of self-organizing, and removing waste in a way that enhances your business.

The New Commodity

When you watch TV, you get to enjoy your show because they fill it with commercials. This monetizing model can be seen in radio and even some areas of the Internet too. Ads! But I submit to you that while money will still make the world go around, knowledge, structured knowledge, will make the money change hands. All the technology is heading that way, and if you aren't on board, you're not going to get into the money flood that's coming.

The demand is out there for knowledge - this broad "I don't know" type of idea, where people know they'll want to know 'things about stuff' but they're not sure to what end just yet. Google is poised very well for when the dam breaks. If you strip out the Google Ad Words revenue stream, you might think that Google has no way to pay the bills. But the truth is that they probably house more knowledge about people, demographics, chatting, email, images, video, and everything else, than any other library, corporation, or other KB group out there today. That is to say, that if knowledge is power, Google is will be the world's new king.

What is interesting, is how they do it. They give their software away. It's easy to use, there's no barrier to "buy" because you don't have to buy it, you don't even have to install it, and if you already have a Google account, you don't have to sign up for anything extra. Just point your browser at a URL and away you go. So now you're using their product giving them plenty of knowledge nuggets with each click and keystroke to be a data-miner's wet dream. And because of that, they can release new features or rearrange their UI to match what people want.

The really brilliant part is the platform and the price. Since it's all SaaS, they can push out an update whenever they want. And Google knows exactly what version you're running (hint: it's the same version as all of their other users). The price ($0) means that you're going to be less likely to complain if the server goes down, or the behavior is a little quirky. Sure, you might file a bug, but you're not going to be able to threaten to take you business else where, because you're just one person who has zero cash bargaining power. Yes, you have some value to Google, they want to know what you know, and they want to know what you don't know you know. But that knowledge you have is just a tiny slice of the millions of other users they have.

Other companies are starting to realize what's going on. JobScore is one such company that gives away the applicant tracking system they make, so that they can sell the knowledge that recruiters input into their system to other recruiters. They make the same old software that anyone else can make, but they sell the knowledge that only they had the foresight to see as a commodity. Now recruiters get better data thanks to the work that other recruiters were doing anyway. Here in the Bay Area, I see start ups all over the place with the same kinds of business models, though some don't have a market to sell their knowledge back to, they hope to be gobbled up by a knowledge giant like Google.

This idea that knowledge is a commodity is really what sets the big G apart from Microsoft and Yahoo who really seem to only fill a direct need with software, but neglect that they can get something extra from a user's experience. Ultimately, this leads us to write software that lets us solve hard problems. Things that fall into a category of 'dynamic pattern recognition' are great problems for humans to solve, but hard for machines to recognize. "Are these two faces the same?", "Does this MP3 contain profanity?", "Would a mother be upset if her child saw this picture?". One solution is to bring in all kinds of math-minded people to set up all kinds of algorithms that run in O(n*log n) time and gets it right only slightly more than if it just randomly guessed. Or, you can write software that asks a human "Hey, what do you think?" You have to work with humans though, humans who are sometimes lazy, who lie, who game the system for their own benefit.

Google has a happy set up for solving just such a problem. Actually they have millions of solutions - their users. By asking a lot of people the same question, they quickly can create some easy math to work with "What does the majority say, and who are the outliers?" You can ask your 15 year old son to compute the standard deviation of most any data set, but it is the kind of work that machines were built to handle.

Knowledge gathering is really quite simple. Step one: Write software that is easy to use and gathers knowledge. Step two: Get users to use it, thereby gathering knowledge. Step three: Profit! It's the bread and butter of Google, and that business scheme is rapidly being adopted in the start up world as well.