For a very long time people have been making money on the Internet by taking other people's websites and jamming a bunch of them into the same server and branding it. First it was GeoCities with their bundle of servers running your mom's baking website. Then it was every other person with a cPanel license standing up servers for their friends. Now it's companies building Platforms as a Service (PaaS). Each time the goal is the same -- get as many people to sign up and squeeze them into as few servers as possible.
The problem with these services is that both parties in the hosting arrangement (the provider and the customer) have different goals and incentives. The hosting provider wants to reduce their expenses by cramming as many customers into the smallest resource pool possible while the customer wants to have as many resources available while reducing their costs. These two goals are mutually exclusive.
A good example of this has popped up recently regarding Heroku's platform. James Somers over at Rap Genius wrote a wonderful piece about the dirty secrets he discovered about Heroku's platform.
In mid-2010, Heroku redesigned its routing mesh so that new requests would be routed, not to the first available dyno, but randomly, regardless of whether a request was in progress at the destination.
James Somers, 2012
Heroku sells dynos which act as application servers -- a single threaded application processing one request at a time. When you have two dynos running, you can run two instances of your application concurrently. Adding more dynos adds additional concurrency as long as new requests are sent to idle dynos.
As James discovered, at scale, this model became too expensive for Heroku to maintain. Keeping the routing queue intelligent meant more cost for Heroku to maintain the infrastructure which isn't billable to one customer. By making the routers dumb (random), they were able to foist the cost onto the customers by forcing them to purchase additional dynos to improve performance. This is where the mutually exclusive business goals came into play.
Heroku didn't have an incentive to disclose their routing changes to their customers -- the changes don't help customers in any way, it only costs them more money. Customers who want to reduce their costs were looking at inaccurate reporting trying to make sense of numbers that really didn't have any indication of the true problem.
Now that Heroku has been outed, they're pleading dumb -- in their follow-up blog post they do a good job of explaining the problem but fall short of actually promising to solve anything for existing customers.
To reiterate, here is what we are doing now:
- Improving our documentation so that it accurately reflects how our service works across both Bamboo and Cedar stacks
- Removing incorrect and confusing metrics reported by Heroku or partner services like New Relic
- Adding metrics that let customers determine queuing impact on application response times
- Providing additional tools that developers can use to augment our latency and queuing metrics
- Working to better support concurrent-request Rails apps on Cedar
Jesper, Heroku Inc., 2012
Notice how they aren't promising to go back to the old ways of routing intelligently -- that would require them to spend money on infrastructure. Updating documentation is cheap and doesn't solve anything. Fixing the reporting is only going to show Heroku users that they need to spend more money (possibly exponentially) on dynos to get additional performance increases. The only measurable improvement to the Heroku service is to parts which directly affect the cost of a dyno -- more concurrency in a single application requires larger, more expensive dynos.
This whole situation just highlights the fact that hosting providers are not looking to save their customers money. They're in the business of making money and anything that isn't directly billable to a customer isn't worth investing in.
If you really want to get your moneys worth from your hosting costs learn to be your own sysadmin. Don't let someone else run part of your application, own everything. Take back your webstack. Make sure you know exactly how each part of your environment works so you can identify problems before they arise. If you're relying on someone else to run it for you, you're paying them to give you the cheapest service they can.
A hosting provider's interests lie in pushing as many customers into the same resource pool as possible. This means taking control away from their customers and insisting that they know best. Take back control of your environment by refusing to accept anything less than full disclosure. Spend the time to learn best practices or ask someone knowledgeable to teach you.
Spending the money to become your own SysAdmin will give you peace of mind knowing you have control over your own systems.