We need a consistent OpenStack


The following is a table of some basic implementation details in OpenStack’s Mitaka API projects. It isn’t intended to shame anyone; it is intended to highlight tool and framework fragmentation in OpenStack. Here’s the data, the article follows below.

Integrated Release APIs

Project
ceilometer PasteDeploy pecan,wsme py27,py34 global-requirements oslo-generate-config
cinder PasteDeploy routes py27 global-requirements oslo-generate-config
glance PasteDeploy routes,wsme py27,py34 global-requirements oslo-generate-config
heat PasteDeploy routes py27,py34 global-requirements oslo-generate-config
ironic pecan,wsme py27,py34 global-requirements
keystone PasteDeploy routes py27,py34 global-requirements oslo-generate-config
neutron PasteDeploy pecan py27,py34 global-requirements oslo-generate-config
nova PasteDeploy routes py27,py34 global-requirements oslo-generate-config
sahara flask py27,py34 global-requirements oslo-generate-config
swift ? py27,py34
trove PasteDeploy routes py27,py34 global-requirements

Supporting API Projects

Project
aodh PasteDeploy pecan,wsme py27,py34 oslo-generate-config
barbican PasteDeploy pecan py27,py34 global-requirements
cloudkitty PasteDeploy pecan,wsme py27,py34 oslo-generate-config
congress PasteDeploy routes py27,py34 global-requirements oslo-generate-config
cue pecan,wsme py27,py34 global-requirements oslo-generate-config
designate PasteDeploy flask,pecan py27,py34 global-requirements
freezer falcon py27,py34 global-requirements
fuel web.py py27,py34
kite pecan,wsme py27,py34
magnum pecan,wsme py27,py34 global-requirements oslo-generate-config
manila PasteDeploy routes py27,py34 global-requirements oslo-generate-config
mistral pecan,wsme py27,py34 global-requirements oslo-generate-config
monasca-api falcon py27
monasca-log-api falcon py27
murano PasteDeploy routes py27 global-requirements oslo-generate-config
searchlight PasteDeploy routes,wsme py27,py34 global-requirements oslo-generate-config
senlin PasteDeploy routes py27,py34 global-requirements oslo-generate-config
solum pecan,wsme py27,py34 oslo-generate-config
tacker PasteDeploy routes py27,py34 global-requirements
zaqar falcon py27,py34 global-requirements oslo-generate-config

Just scratching the surface

The table above only scratches the surface of OpenStack’s tool fragmentation, as it only focuses on frameworks and configuration in API projects. It does not address other inconsistencies, such as supported image types, preferred messaging layers, testing harnesses, oslo library adoption, or a variety of other items.

I’ve already spoken about Cognitive Load and OpenStack, how the variability in our projects can trick your brain and make you less effective. Furthermore, we’ve seen a lot of discussion on how we should Choose Boring Technology, as well as hallway discussions about how OpenStack should be more opinionated in its deployments. In fact, things have gotten so bad that the shade project was created – a library whose express intent is to hide all the differences in deployed OpenStack clouds.

Variability is bad for OpenStack

The lack of consistency across OpenStack is harming us, in very specific ways.

Contributing to multiple projects is hard

Nobody wants to climb multiple learning curves. Knowledge from one project directly transfers to another if the frameworks are similar enough, reducing this learning curve. In short, differences in projects create barriers to cross-project fertilization and contribution, and one way to chip away at those differences is to keep them as similar as possible.

Supporting OpenStack’s dependencies is hard

As an open source community, there is a strong ethos of helping support any projects that we depend on. Yet, how do we pick which upstream project to help fix? If all projects were consistent in their use of, say, WSME, there would be a far larger pool of talent invested in success, and discussions like this one would not happen as frequently (Note: I’m not necessarily advocating WSME here – it merely provides a very useful example).

Maintaining feature consistency is hard

There are many features which our various projects should all support. Simple things, like consistent search query parameters, consistent API version negotiation, consistent custom HTTP Header names – basically anything cross-project or from the API working group.

I have personal experience with this: In Mitaka I was able to land CORS support in most of OpenStack’s API’s. Of the 23 projects that I contributed to, most required that I learn project-specific approaches to governance, launchpad usage, testing harnesses, commit-message constraints, folder structure, and more. The entire experience only taught me one thing: Trying to bring a common feature to OpenStack is something I never want to do again.

Deploying/Running OpenStack is hard

Without an opinionated OpenStack install (down to the supporting services), the chances that someone has run into the same problem as you, drops significantly. Features which rely on service abstraction (messaging for instance) depend on layers of driver abstractions which add more points of failure, and often have to provide workarounds for features supported in one service, but not in another (assuming they don’t give up on that feature entirely).

Portability between clouds is hard

To quote Monty Taylor: “The existence of shade is a bug”. It turns out that OpenStack’s implied portability promise is pretty much a lie, and you will spend a significant amount of effort figuring out how this OpenStack happens to differ from That OpenStack.

We need a consistent OpenStack

We have seen the consequences of inconsistency first hand. In some cases, a complete lack of developer mobility has resulted in echo-chambers, entire projects that are completely convinced that their approach is superior to others’. Other projects are effectively code deserts, unable to recruit contributors. Deployments are difficult, feature support is inconsistent, and rather than address the problem and simplify our projects, we’ve instead built layers of abstraction so we can avoid our most grievous mistakes.

We need a consistent OpenStack. If each project takes a slightly different approach to something, it makes subsequent management and support very difficult. To that end, all of our projects should use a consistent set of tools, frameworks, and libraries.

3 Comments

  • […] DaoCloud 完成千万美元 A 轮融资,Docker 梯队继续前进 观点 We need a consistent OpenStack  […]

  • You make some excellent comments about the lack of consistency in OpenStack. It is disappointing there is not a better architectural overview and then an enforcement of a given policy (e.g. using global requirements and oslo config generator are easy ones), or to ensure that not yet another product is used when existing products are in use. Developers that work in isolated silos need to "live a day in an operators/deployers shoes" to appreciate that these decisions have real-world time and cost ramifications after the lifespan of the use of the project, not just the milestone in a development cycle. As I like to say "The world would be a better place if we all worked on making the world a better place". For developers of software (not just OpenStack), considering the long term impact of a decision is unfortunately a lost art in the agile way of doing things.

  • […] a big list, and we need help to land it during the Newton cycle. Overall themes for this cycle are Consistency, Interoperability, and engaging with the JavaScript community at large, all topics which I’ve […]

Leave a Reply