We need a consistent OpenStack

The following is a table of some basic implementation details in OpenStack’s Mitaka API projects. It isn’t intended to shame anyone; it is intended to highlight tool and framework fragmentation in OpenStack. Here’s the data, the article follows below.

Integrated Release APIs

Project
ceilometer PasteDeploy pecan,wsme py27,py34 global-requirements oslo-generate-config
cinder PasteDeploy routes py27 global-requirements oslo-generate-config
glance PasteDeploy routes,wsme py27,py34 global-requirements oslo-generate-config
heat PasteDeploy routes py27,py34 global-requirements oslo-generate-config
ironic pecan,wsme py27,py34 global-requirements
keystone PasteDeploy routes py27,py34 global-requirements oslo-generate-config
neutron PasteDeploy pecan py27,py34 global-requirements oslo-generate-config
nova PasteDeploy routes py27,py34 global-requirements oslo-generate-config
sahara flask py27,py34 global-requirements oslo-generate-config
swift ? py27,py34
trove PasteDeploy routes py27,py34 global-requirements

Supporting API Projects

Project
aodh PasteDeploy pecan,wsme py27,py34 oslo-generate-config
barbican PasteDeploy pecan py27,py34 global-requirements
cloudkitty PasteDeploy pecan,wsme py27,py34 oslo-generate-config
congress PasteDeploy routes py27,py34 global-requirements oslo-generate-config
cue pecan,wsme py27,py34 global-requirements oslo-generate-config
designate PasteDeploy flask,pecan py27,py34 global-requirements
freezer falcon py27,py34 global-requirements
fuel web.py py27,py34
kite pecan,wsme py27,py34
magnum pecan,wsme py27,py34 global-requirements oslo-generate-config
manila PasteDeploy routes py27,py34 global-requirements oslo-generate-config
mistral pecan,wsme py27,py34 global-requirements oslo-generate-config
monasca-api falcon py27
monasca-log-api falcon py27
murano PasteDeploy routes py27 global-requirements oslo-generate-config
searchlight PasteDeploy routes,wsme py27,py34 global-requirements oslo-generate-config
senlin PasteDeploy routes py27,py34 global-requirements oslo-generate-config
solum pecan,wsme py27,py34 oslo-generate-config
tacker PasteDeploy routes py27,py34 global-requirements
zaqar falcon py27,py34 global-requirements oslo-generate-config

Just scratching the surface

The table above only scratches the surface of OpenStack’s tool fragmentation, as it only focuses on frameworks and configuration in API projects. It does not address other inconsistencies, such as supported image types, preferred messaging layers, testing harnesses, oslo library adoption, or a variety of other items.

I’ve already spoken about Cognitive Load and OpenStack, how the variability in our projects can trick your brain and make you less effective. Furthermore, we’ve seen a lot of discussion on how we should Choose Boring Technology, as well as hallway discussions about how OpenStack should be more opinionated in its deployments. In fact, things have gotten so bad that the shade project was created – a library whose express intent is to hide all the differences in deployed OpenStack clouds.

Variability is bad for OpenStack

The lack of consistency across OpenStack is harming us, in very specific ways.

Contributing to multiple projects is hard

Nobody wants to climb multiple learning curves. Knowledge from one project directly transfers to another if the frameworks are similar enough, reducing this learning curve. In short, differences in projects create barriers to cross-project fertilization and contribution, and one way to chip away at those differences is to keep them as similar as possible.

Supporting OpenStack’s dependencies is hard

As an open source community, there is a strong ethos of helping support any projects that we depend on. Yet, how do we pick which upstream project to help fix? If all projects were consistent in their use of, say, WSME, there would be a far larger pool of talent invested in success, and discussions like this one would not happen as frequently (Note: I’m not necessarily advocating WSME here – it merely provides a very useful example).

Maintaining feature consistency is hard

There are many features which our various projects should all support. Simple things, like consistent search query parameters, consistent API version negotiation, consistent custom HTTP Header names – basically anything cross-project or from the API working group.

I have personal experience with this: In Mitaka I was able to land CORS support in most of OpenStack’s API’s. Of the 23 projects that I contributed to, most required that I learn project-specific approaches to governance, launchpad usage, testing harnesses, commit-message constraints, folder structure, and more. The entire experience only taught me one thing: Trying to bring a common feature to OpenStack is something I never want to do again.

Deploying/Running OpenStack is hard

Without an opinionated OpenStack install (down to the supporting services), the chances that someone has run into the same problem as you, drops significantly. Features which rely on service abstraction (messaging for instance) depend on layers of driver abstractions which add more points of failure, and often have to provide workarounds for features supported in one service, but not in another (assuming they don’t give up on that feature entirely).

Portability between clouds is hard

To quote Monty Taylor: “The existence of shade is a bug”. It turns out that OpenStack’s implied portability promise is pretty much a lie, and you will spend a significant amount of effort figuring out how this OpenStack happens to differ from That OpenStack.

We need a consistent OpenStack

We have seen the consequences of inconsistency first hand. In some cases, a complete lack of developer mobility has resulted in echo-chambers, entire projects that are completely convinced that their approach is superior to others’. Other projects are effectively code deserts, unable to recruit contributors. Deployments are difficult, feature support is inconsistent, and rather than address the problem and simplify our projects, we’ve instead built layers of abstraction so we can avoid our most grievous mistakes.

We need a consistent OpenStack. If each project takes a slightly different approach to something, it makes subsequent management and support very difficult. To that end, all of our projects should use a consistent set of tools, frameworks, and libraries.

It’s the Experience, Stupid [Advice to Mobile Providers]

Two years ago, my colleague Isaac went to SXSW, and came back with a presentation on Mobile Development. In it he said that one of the greatest challenges is getting a mobile application “on deck”. “On Deck” is the term used for an application that’s available on a provider’s mobile platform, that place you goto online when you browse applications, ringtones and such, and to get something on there used to take an Act of God. Why? Because all billing had to be handled through the provider, all sales had to be done though your phone bill, and payments to third party companies had to be set up through their system (and usually required a hefty premium). In short- more trouble than it’s worth. Fact is, this is largely still the case. Yes, with greater adoption of mobile web browsers these things are becoming a lot easier, yet getting an application onto a phone remains problematic, especially if the consumer isn’t aware that you have it. The best option these days seems to be building a Mobile website, which is a far cry from a good user experience.

So lets talk about the consumer for a bit, in particular the consumer described in The Long Tail. Here is an individual who knows his/her own lifestyle and needs, who like particular applications over others, whose preferred experience with one service might be different from his/her neighbor’s experience for the same. This is an individual who, when presented with a Smart Phone, wants to have features and applications behaving a particular way, and will only put up with the provided options of the telco because there’s no other choice. In short, they have a craving for personalization, and will gladly move to any platform that has applications that meet their needs.

Deploying a mobile application to meet these needs is, in its simplest form, the following: First, you must have a developer able to access your SDK, platform and testbed with minimal fuss. Then, that developer must be able to deploy his application to a location where a potential user may download it onto their device. Lastly, the device must be able to run said application. These three tenets of Develop, Deploy, Consume can be easily seen on the regular web. Developers have their SDK’s, Deployment happens on a web server, and browsers allow consumption of the offered services.

Mobile-1.png

In the mobile world this isn’t quite as simple. Yes, there are many websites now that have mobile components, yet lets face it: The form factor and memory constraints of mobile computing devices make it so that few applications can offer full functionality and experience while constrained by regular browser controls. Additionally, we are back to (for the time being) the computing world of the early 90’s, where applications are restricted by processing and memory constraints. Back then, however, we could load applications from Floppy disks, something that’s not so easy on a mobile device. The development platforms exist, but deployment (as noted above) is extremely difficult, and the broad diversity of devices makes your target installable base problematic at best.

So let us take a look at the various players in the mobile market right now. First of all, let us look at the telecommunication companies, those that have the aforementioned problem with getting something On Deck. From their perspective, they have control over their delivery platform, and getting something from their platform to a mobile device is simple enough, however their process is such that they have completely lost their developer base. There are too many devices, too many different platforms and form factors, and deploying an application is simply too difficult for any developer to bother building something. Imagine the computing world in the mid-80’s, and you’ll understand what I mean.

Mobile-5.png

Next let’s take a look at Microsoft, which in our case will also double as any company that has a mobile application which they want to deploy. Microsoft has historically been brilliant in its developer support tools, so much so that it’s stupidly easy to develop an application for their operating system. Similarly, the fact that they are in the software and not the hardware business lets them sell a platform rather than a device, which allows mobile manufacturers to create devices that can run any application built for it. The place where they have failed, however, is in securing and simplifying the delivery platform, which remains under the control of the telco’s.

Mobile-4.png

Third, let us look at Google and Android. Here, the development platform is easy to access and has some brilliant development tools. And… then what? Well, frankly, we don’t know a whole lot about how Android will be delivered, because while many manufacturers and providers have jumped on the bandwagon, we as of yet have no idea whether Android will use an open application deployment platform, or even whether phones built by their manufacturing partners will come with Android pre-installed. The suggested promise of Android is Build Once, Deploy To Any Phone, but if it means that a developer still has to get something On Deck with a Telco, you will once again be in the same boat as Microsoft- a common platform, but no control over application distribution.

I should note here that I doubt Google will not have some central application service, it simply hasn’t been announced yet.

Mobile-3.png

Lastly, let us come to Apple. In this case I have to make a special note, in that while I recognize the SDK is publicly available, it is unfortunately not available for Windows (No, Aptana junkies, that’s just a web browser). Thus while they have alienated a good percentage of computer users overall, the additional cost of an OSX development platform is comparatively minor, and won’t detract a company interested in building an iPhone application. Taking that into consideration, they have everything buttoned up. Developers can access the SDK, they have an established and largely ubiquitous deployment platform not tied to the telco, and they have a device that will run any software built. This end-to-end solution is exactly the same thing that made the iPod so successful, because it got a user from Purchase to Play in one smooth experience.

Mobile-2.png

And no, I don’t own an iPhone.

Adobe Announces Open Screen Project

This morning (well, at 12:01 AM) Adobe announced a large cross-industry collaborative effort called the Open Screen Project. According to the marketing boilerplate, it is dedicated to driving consistent rich Internet experiences across televisions, personal computers, mobile devices, and consumer electronics.

What it really means is the following:

Removing restrictions on use of the SWF and FLV/F4V specifications

This was more of a licensing restriction if I’m not mistaken, meaning any device manufacturer can now make use of these formats. In practice you might see native (or embedded) implementations of .swf or .flv, including, say, your Tivo, Streaming Video on your XBox, PSP, Nokia, etc etc etc.

Publishing the device porting layer APIs for Adobe Flash Player

Flash Player has been annoying to push to other devices, largely because of the ridiculous number of handhelds that are out there. Getting it working for so many devices has been, quite simply, impractical, so instead they’ve opened up the lower level device calls of the flash player so anyone can port their own.

This particular one I find the most compelling, because of the names that are notably not on the list of partners: Google, Apple and Microsoft. Whether they are still in talks with these companies or not I honestly don’t know, but given that between the iPhone, Google Android and Windows Mobile they have significant sway in the device industry, not having them on the list seems a little odd. Mind you, everyone else is on that list, so I’m wondering if Adobe’s trying to use the community to strongarm those three into adopting their runtime. Consider consumer A, who sees all kind of neat Flash/AIR apps on other devices, but doesn’t see them on his iPhone or his Windows Mobile device. Is he more likely to move? Would someone else be less likely to leave? I don’t know, but the strategic play by play will be very interesting to watch.

Publishing the Adobe Flash® Cast™ protocol and the AMF protocol for robust data services

We’ve heard rumors about this, but having these protocols public and formally supported by Adobe (rather than reverse engineered by the community) is a pretty big deal. Open projects like BlazeDS, AMFPHP and WebOrb can now support the entire protocol layers, and I think we’ll soon see live video streaming options for all of those platforms.

Removing licensing fees – making next major releases of Adobe Flash Player and Adobe AIR for devices free

Up until recently, Flash for a handheld device cost money. Soon, it won’t. There’s not much to this other than removal of the barriers to adoption for the consumer, and many handheld devices may come with the flash player pre-installed.

Quotes from the Boardroom

A bit of background first: The presentation I delivered yesterday was on Confluence, a Mortgage Regulation Compliance Solution provider (I can see you falling asleep) that’s competing in the space with Charles River Development who dominates their market. Essentially we had to write a paper and give a presentation on it, and since my teammate Brian (who actually works at Confluence) wasn’t going to be there for the presentation, he was kind enough to write the paper so I could base the presentation on it.

Now, after reading the paper… well, take a look at this quote from their Sales Manual- Their Proprietary Sales Model is apparently so good and amazing because it is “based on the process by which customers in our market decide if they want to give a vendor hundreds of thousands of dollars”…. so how, exactly, is this different from any other B2B Sales Organization model out there?

That’s just one quote, and man, lemme tell you, I had fun with it. To be completely honest, Brian made the mistake of letting me give this presentation without supervision… because I ripped into Confluence. Seriously ripped into them, to the point where the Professor was laughing at some of the processes and methods I was pointing out.

On one level, I feel bad. On the other, he left me hanging.

Idle strategic insight into Subway’s business model

I used to wonder why subway seems to be doing so well with competitors like Wendy’s and McDonalds, but then I really thought about it- Subway, really, doesn’t have any competitors. The company is in the low-cal fast food market, bordering on health food. Yes, other burger joints are trying to muscle in on that territory, but when you think McDonalds, the first thing that comes to mind is ‘Burger’. Subway has managed to position itself as equivalent to ‘healthy’, even though some of their sandwiches really aren’t.

Given that,… Subway really has absolutely no competition, and is dominating a market that is, quite frankly, ready for some variety. Anyone with a good business plan and an interesting idea could start chipping away at their empire. Though… how do you make health food fast, tasty, cheap, and appealing to the average American?