As an Evergreen developer, I believe our project has a few significant gaps that projects like RSCEL might be able to help address for the overall good of the community by bringing in outside resources to the project. Or perhaps there are skills within the community that don't feel like they've been called on yet; when I say that we lack skills, I'm basing that on the lack of patches and offers of assistance that I've seen in these areas. I would be delighted to be proven wrong! Either way, I submit this for the community's consideration.
- 3rd party security audit: Before Conifer adopted Evergreen, I had hoped that we would be able to fund a security audit of the code by a trusted and competent 3rd party like OmniTI (from a previous life, I believe that OmniTI employs some of the best people in the business, thus the plug - but there are certainly other options out there). As developers, we try our best to avoid vulnerabilities, but as the recently disclosed vulnerability in open-ils.pcrud attests, we're not experts in security. An audit of the public-facing interfaces (the catalogue, feeds, etc) would be a great help to the project. I would expect a prioritized list of areas that need to be addressed, along with recommendations on how to address those problems (whether they be cross-site scripting, session fixation attacks, authentication encryption attacks, etc). Our community's process (or lack thereof) for reporting and addressing security vulnerabilities might be an appropriate subject for an audit as well.
- Testing framework: Our project is woefully short on tests, either human-powered or automated, for determining the state of the code at any given point in a release cycle. Thus, we have put out release after release that either won't install cleanly, or won't upgrade successfully from a previous release. The trunk version of the code had a error that meant that Evergreen couldn't be compiled; that problem existed for three weeks before somebody noticed and fixed it. I'm not pointing fingers, here; if I did that, I wouldn't have enough fingers to point back at myself for all of the problems I've introduced that other people have had to fix. Johnathan Nightingale in The Most Important Thing … or How Mozilla Does Security and What You Can Steal provides a great overview of Mozilla's philosophy about and approach to testing. There is all kinds of goodness in this presentation, but one of the most interesting points is that "money can be exchanged for services" -- that is to say, if your existing development team doesn't have the skills or time to implement a testing infrastructure, there are companies that do have the ability to put together a test infrastructure for a given project. Once that infrastructure is in place, it tends to get extended and used by the existing development team because it makes their lives easier; they don't need to manually test a given code path every time in the future, or deal with regressions that aren't noticed until months in the future when the changes they were making are no longer fresh in their minds. It sometimes requires a culture change, though.
- Continuous integration: Hand in hand with a testing framework is a continuous integration server that provides testing feedback on every commit to the Evergreen repository for a given set of branches. Even without a testing framework, it is possible to have a continuous integration server run through the process of installing all prerequisites, configuring the code, building and installing the code, and creating the database schema to at least determine whether the basics can be accomplished successfully to confirm that a branch is ready for release. This arguably also goes hand in hand with a team's process for addressing a security vulnerability: if you have a continuous integration server that can tell you if a given fix does not introduce basic build and install errors, then you can get a new release out with much more confidence that you're not going to be encouraging your users to jump to a broken package. Note that Equinox ran a continuous integration server for OpenSRF and Evergreen trunk for a while, but that was killed and replaced by a call for volunteers to build a new continuous integration service (I can't find a more to-the-point call for volunteers, so perhaps it just hasn't been advertised widely enough - or again, perhaps we lack the skills in the community to get a standard CI service like Hudson running.)
- Packaging: To decrease the difficulty of installing and configuring Evergreen, we need more investment in packaging Evergreen and all of its unpackaged dependencies. The idea is that a user should be able to run "aptitude install evergreen" or "yum install evergreen" and have the entire system installed and configured, and then run "aptitude upgrade" or "yum upgrade" to have newer versions installed. Right now the process is still rather onerous and requires a great deal of manual effort, although it has improved significantly since the early days of 2007. Again, this requires a particular set of skills that the Evergreen community does not appear to possess in depth: autoconf, automake, APT and RPM packaging - and perhaps some redesign of elements like skins to make local customizations easier to incorporate and keep up to date. This would be a natural complement to a continuous integration service, but much of the effort could also be done on its own.