Alex Weinstein
On Growth, Product, and Leadership
Leaky Clouds

Joel Spolsky has two timeless pieces – Fire and Motion and The Law of Leaky Abstractions that are cornerstones of what I’m about to preach in this post. Please take a moment to read those articles by Joel – they’re almost 10 years old now, but are as relevant as ever.

My field, software engineering, is young and filled with changing winds. Client-server! No, web-based!.. Win32! No, PHP! No, Cocoa! No, HTML5 / JavaScript!.. Mainframe! No, rich client! No, thin client! No, mobile client!..

Entire industry moves like the trees under a powerful wind. I’ll stipulate that there are two factors in play here:

  1. Psychology. The nerds that are in IT leadership positions are still very much gadget nerds. They love new stuff. They love playing with new stuff. They get the fanciest gadgets available on the market today. It’s not the Patek Phillippe crowd – they won’t go after a 100-year-old watch. They’ll go after a released-1-day-ago iPad v7, stand in line for it for hours, and pay through their nose. Just because it’s new – and new stuff is really exciting.
  2. Incentives of market movers. Large technology vendors – particularly those in the leadership positions in their markets – want to keep customers in the spot where they want to buy again and again. Yes, we sold you a solution to your problems yesterday; but wait, now we have a NEW, MUCH BETTER way to solve that same problem! Yeah, you haven’t yet figured out half of the benefits of the old thing, but the new thing – it’s SO MUCH BETTER!..

Abstractions are an outstanding concept in science: instead of dealing with a bunch of low-level concepts, we create building blocks – and juggle those instead. It’s much easier to build a house if you have bricks; it’s much easier to perform a surgery if you don’t have to make a scalpel from the iron ore yourself.

Perfect abstractions require zero knowledge of how the building block was created. Such abstractions don’t exist – even the surgeon needs to know a little bit about the manufacturing process and chemical composition of the scalpel. For example, the scalpel has certain melting temperature and certain chemical characteristics (hmmm, I guess I shouldn’t put it into hydrocloric acid..).

In software, most abstractions are actually pretty bad. We like to think that the operating system hides away most of the complexities from the users – but it doesn’t. With Android, unless you know to kill apps that are running in the background, your battery life will be abysmal. Even with the famed iPad, you have strange artifacts like “zoomed mode” for old iPhone apps – total weirdness from the end-user point of view.

ORM (object-relational mapping), one of the most famous abstractions, is probably one of the worst (most leaky) – it will make you *think* that you don’t need to know SQL, but my god, that’s a horrible lie.

And yet, for system designers, these abstractions are frequently natural. As software engineers that build those abstractions, we are tempted to design for ourselves. In the end, we’re developers that build on top of those frameworks, too! “How am I not the target customer?..” That, of course, is a horrible mistake that causes unusable abstractions and frameworks that only their creators can understand.

Enter cloud computing. The latest excitement of our industry. The premise is really simple – instead of investing into technology upfront, pay as you go. Have someone else host that technology for you. That someone has vast economies of scale. Increase your investment easily in response to demand.

Sounds like a dream come true to the enterprise, right? I have 50 servers that are each utilized at 3%; I have IT staff of 10 to maintain these technology solutions; that’s really expensive. Instead, I come to a cloud vendor and pay a fraction of the cost.

Also sounds like a dream come true to an independent software shop, right? With my Find Touch project, I used to need a server in my garage – maintaining hardware, network connectivity, software stack, everything. Instead, the cloud promises to solve all of the same issues at a fraction of the cost.

With cloud computing, the situation is even worse. The field is in the “new and super hot” realm – just like those pesky iPads – and early adopters are discovering that the promise of “just move your existing application into the cloud” isn’t quite working.

The lower the level of the service that you’re moving to the cloud, the easier it is to map to what’s in your datacenter. The higher you go up the value chain, the harder the migration becomes.

If all you’re getting from your cloud vendor is hardware, everything is pretty easy from the technology standpoint. The problem, of course, is that the value add of such a service is pretty low. Yeah, I can spin up more machines and I don’t need to wait for DELL to bring those machines to my doorstep.

If, on the other hand, you want to move a VB6 application into the cloud.. And gain the benefits of deployment, scalability, billing, and all the other goodness of SaaS.. Good luck. You’re in for a treat. Get ready to essentially rewrite your software. Oh, you no longer are in touch with the vendor that wrote the source code?.. Too bad…

Moreover, the further you go from technology towards business aspects of cloud computing, the harder the cloud becomes to understand.

Recently, I worked on moving my web-based application from a standard hoster (1&1) to a cloud platform. I chose Amazon EC2 for my experiment. MY GOD, even as a technologist, I was at loss to understand what the costs for my migration are going to be. My old platform offered a pretty large VPS; they screwed up with customer service, so I was looking for a new home. All I wanted was an apples to apples comparison of price and major features. I didn’t expect major demand spikes – even though the ability to respond to them is nice. All I saw was myriads of instance types and datacenter options and pre-pay versus pay-as-you-go per minute of usage and my head was spinning after an hour.

My god, can’t you just tell me how much I’ll pay for the load I *know* I will have? A stupid little calculator for idiots like me that will tell me how much I’ll pay a month?.. To see if your service is worth it?..

Hosting vendors have this down to an art: show just three options, with a clear feature matrix; highlight the middle one with a tad more features than most people want; help upsell that one.

Cloud vendors? We have EC2 and S3 and fancy queues and monitoring with a cool name and you need to learn all of those proprietary acronyms and oh my god am I going to have to have my IT staff relearn everything?.. Cuz my CEO sure isn’t going to like this..

Seriously?.. You call that an easy switch to the cloud?..

I’m not saying the cloud is a fad. I am, though, saying that the cloud is in a state where it’s “hot and sexy because it’s new” and it’s primarily driven by technologists – not the business leaders. I want to see cloud vendors lead with value proposition – and better abstractions – and not with the newfangled technology “fire and motion.”

I want to see someone who’ll take my PHP app and will put it into the cloud – for the same price for a single VPS as my dumb old hoster. Using the paradigms I’m used to. I want my app to *not change at all* in order to adapt a cloud-based billing service. I want my storage to work just like I’m used to. I don’t want my IT staff to learn anything new – I want the framework to adapt to THEM.

Let’s greet the cloud – and hope that it grows up from the infancy it is in now.