atom feed6 messages in net.launchpad.lists.openstack[Openstack] [Swift] swift news and plans
FromSent OnAttachments
John DickinsonMay 4, 2012 11:56 am 
Soren HansenMay 9, 2012 5:50 pm 
Suchi Sinha (susinha)May 14, 2012 11:19 am 
Chmouel BoudjnahMay 14, 2012 2:21 pm 
Andy EdmondsMay 15, 2012 12:21 am 
Soren HansenMay 15, 2012 6:49 am 
Subject:[Openstack] [Swift] swift news and plans
From:John Dickinson (me@not.mn)
Date:May 4, 2012 11:56:15 am
List:net.launchpad.lists.openstack

TL;DR: removing code from swift, associated projects doc, swift 1.5.0

I want to let the openstack community know of some recent changes within swift and how those changes will affect the next version of swift. Swift has a growing developer community and a rapidly expanding deployed base. While this growth is fantastic, it does come with new challenges, especially for the swift core developers. As more and varied use cases are presented to swift, patches are submitted that enhance swift's functionality either by offering optional features or alternative APIs.

The challenge with this growth is that the core developers become responsible for understanding and maintaining an ever-increasing codebase. This responsibility becomes a timesink, both for reviews and for fixing regression bugs as new core features are added. For non-core developers, the review process for new code becomes slower, and changes that don't affect swift's core functionality often fall to the bottom of the pile--sometimes even to the point of expiring due to inactivity.

Our solution for these problems is to limit the scope of swift. Swift's core functionality is to provide cheap, durable, and scalable object storage exposed through its own API. Other functionality and alternative APIs should be maintained separately from the swift codebase.

As a result of this focus in scope, we have begun removing some of the optional parts of swift. Initially, this will include the tempurl, formpost, staticweb, rate limiting, swift3, domain remap, and cname lookup middleware modules. Proposed patches that offer alternative APIs (like CDMI) or include optional functionality that can be implemented external to swift will be encouraged to be developed separately from swift.

We have already begun the process of removing many of these pieces of middleware from swift and moving them into their own respective repos.

However, all of this functionality is quite valuable and beneficial to swift. There is a real need for most of these modules. Separating them from swift introduces the problem of discoverability. As a result, we have added a new page to our swift docs that lists associated projects and added links to that page on swift.openstack.org.

http://swift.openstack.org/associated_projects.html

This page is fairly limited right now, but the basic structure is there. As things are removed from swift and as new associated projects are created, they will be added to the list. This doc page is maintained in the swift codebase, so updating it is subject to the same requirements of any other patch to swift.

An important note is that this list offers no distinction or references to "official" or "approved" associated projects. This list is independent of any openstack CI integration that may or may not happen in the future.

Once we finish the process of migrating the optional pieces of swift away from the swift codebase, we will cut our next release: swift 1.5.0. There is no date set for this yet, but I hope the migration process can be finished in the next several weeks. Swift 1.5.0, therefore, will be somewhat larger than most of our other swift releases. Existing deployers will need to be careful about upgrading to ensure that new dependencies are met.

If you have any questions, please feel free to email me. This whole effort is a work-in-progress. I know that there are several similar discussions going on within the openstack community, and swift's solution is not necessarily intended to replace any more general solution that may eventually arise. If there is a better solution at some point, we will do what we can to integrate with it.

--John