Many of the high growth companies in our portfolio have run into scaling issues. There is a lot of information out there on various technical approaches to scaling. What most of those leave out is the interaction between the choice of architecture and organizational scaling. Some architectures lend themselves much better to organizational scaling than others. A horizontal approach with a Data Access Layer, a Business Logic Layer, and a Presentation Layer suffers from a lot of organizational coordination overhead. To implement new functionality the various horizontal teams need to coordinate so that anything can get done.
I am therefore a big fan of a services based architecture, which takes more of a vertical approach to dividing up systems. For instance, most web sites and services have a concept of a user profile. In a services based architecture everything having to do with user profiles might be encapsulated in one service (create a profile, retrieve a profile, etc). Organizationally it now becomes possible to have a team that’s in charge of the profile service. That team can make changes to the service implementation as long as the changes don’t break the service API. In fact, the team can even enhance the functionality of the service by adding new methods to the API. This allows for much better organizational scaling as innovation no longer requires nearly as much coordination. In addition to innovation by each service team, it’s also possible to innovate by combining the existing services in novel ways to deliver end user functionality.
