Monday, June 13, 2011

Can Big (Companies) Be Beautiful?

As companies grow, they slow down. Once they get past a certain size, they usually experience what’s called organizational sclerosis--a culture change that makes effective adaptation almost impossible. They cling to old business models, become less responsive to their customers, and make big costly mistakes. Their shares drop.

People have various theories as to why this happens. Most of them relate to organizational complexity or conflict from within. The strategies for fixing it usually involve hiring expensive consultants or laying off hundreds of employees. These strategies are painful and often simply don’t fix the core problem. But what if there was a way to make large companies as dynamic as small ones? I think I might have found one.

Part of the Tokenomics research that I’ve been doing of late has involved building computer simulations of organizational cultures. (See previous post.) The simulations are still very simple and there’s lots of work still to be done before any of the results find their way into a science paper. However, some of the results I've discovered are pretty compelling. This is one of them. 

I’ve been exploring the idea that people look for predictable, repeatable self-validation in their interactions with others, or ‘token collection’--an idea that you can find out more about on the SF Behavior Lab website, or in the in-depth documents stored at Almost all the token collection that we do is driven by non-conscious pattern-seeking. This is very useful because it means that we don’t have to model complete human minds in order to get a rough picture of how cultures work. We can simply create populations of agents that interact by seeking out repeatable patterns of dialog, and modifying their behavior if their conversations don’t run as expected.

In the examples I described in my earlier post, the agents building cultures had a pretty limited set of options available to them and so the communities they formed didn’t show much variation. However, for the experiments I want to share with you this time, the interactions are rather different. Agents speak in turns, rather than at the same time, and have a choice of 128 different words they can say. (128 openings x 128 responses). This means that the agents have a much harder time of it finding familiar subjects to talk about with their peers, and so have to be a little creative (random) in their choices.

Most of the interactions that the agents try out don’t meet with much success. They produce fairly short-lived patterns shared only by a few robotic acquaintances. However, some interaction patterns manage to spread and dominate, just like fads in real life. Once these patterns get established, they become incredibly stable. They become ‘cultural interactions’ and end up being common to all the agents in the simulation. Removing or adding agents incrementally doesn’t change the culture one bit.

Also, interestingly, changing the size of the population of agents has a serious effect on how often new patterns get incorporated into the culture. Bigger populations have a lot more trouble accepting new ideas than small ones. At some level, though, this shouldn’t surprise us. In order for an idea to develop critical mass and spread rapidly through the group, a certain percentage of the agents have to have already heard it. That’s going to get steadily less likely as the system scales up.

For these simple agents, a critical drop-off happens before you get to a population size of forty.  So far as I know, there’s no equivalent change in performance for human groups in this size range. The literature suggests that real organizations start having problems at around a hundred and fifty people. However, in the context of agent simulations, this makes sense. The agents have much smaller memories than people and thus less intrinsic flexibility. So, while the drop off in effectiveness may not be exactly the same phenomenon as organizational sclerosis, there seems to be a close enough fit here to make exploring the effect worthwhile.

The question I asked was ‘what do I have to do to a set of agents in order to keep their rate of cultural adaptation high while the population increases in size’. Specifically, what did I need to do to make a population of forty agents as imaginative as one of size twenty.

The answer, for agents, at least, is simple. You divide them into two groups of size twenty and let them run independently. Then after a while, you reshuffle them into two new groups, and leave them alone again. Every time the two groups are shuffled, learning from each community gets transferred into the total population. However, because the groups stay small, the rate of creativity stays at what you’d expect for a much smaller population. 

This process is rather like the genetic recombination that happens in sexual selection. We keep the rate of change high by randomly mixing our ingredients and then making sure that any useful results get shared back into the population.

Also, it turns out that this isn’t the only method that will work. Any solution that involves keeping group sizes small most of the time, combined with random mixing, will do the job. And there appears to be an optimum group size for a given setting of the simulation parameters at which creativity can spread fastest.

The lesson seems clear: turning big organizations into a cross-polinating swarms of smaller ones might make them a lot more agile. While this approach probably isn’t a fit for all kinds of business, I can think of plenty where it might work: finance and software being the first two to spring to mind.

Once I’d managed to keep a big culture as adaptable as a small one, I wanted to know if I could do better. After all, large organizations have more people and therefore more ideas per day than small ones. In an ideal world, shouldn’t they be able to capitalize on that?

The answer is yes if you bend the rules a little. I couldn’t find a way of reorganizing the teams that led to further improvements in creativity beyond what I’d already tried. However, when I started messing with the parameters of the simulation, I noticed something interesting: increasing the number of possible witnesses to each interaction by one caused the rate of creativity to jump up rapidly. One way to say this is that by raising the number of people in a team who get to listen in on important interactions increases the rate at which new ideas can spread 

This provides some simulation evidence that the people in the Agile Software Development movement are on the right track. They’ve been saying for years that creating small, tightly-knit teams where information is regularly shared creates the right environment for creative work. Furthermore, the agents in this simulation are so simple and general that we can expect the same logic to hold for a large number of possible situations.

The important next step will be to see whether the same tricks work as I add in more of the complexity of real organizations. For instance, trust relationships, hierarchies, and differences in effective value between the habits being propagated all have yet to be added. If the splitting/merging effect still works when all these features are added in, it might be time to start rethinking how we grow companies. Organized packs of collaborative mammals could turn out to be a lot more adaptable than a few big dinosaurs.

1 comment:

  1. Hey Alex - this is really interesting stuff! I am especially intrigued by the idea of how fads spread, and of the size of population needed for creativity.

    The 150-people size limit reminds me of something I heard that people tend to have on average 150 friends - more than that, and we can't keep them all straight in our head. So it's not that ideas can't spread beyond 150 people, it's that individual people can't know more than 150 other individuals.

    Interesting. Keep up the good work! :)