Open Source Evaluation

 

Introduction

This document provides a means for evaluating software projects for release via Open Source. The paper is divided in to three sections: In Section 2, it presents the factors that lead to a successful Open Source project. Projects that meet these factors are candidates for open sourcing – after that, the decision is a matter of the assessing business case for open sourcing. This is addressed in sections 2 and 3. Section 2 discusses the benefits that accrue to a successful open source project. These should be evaluated for resonance with the business objectives of the software project. Section 3 discusses the costs. Open Source projects cost money and effort to maintain. It also may require sacrificing on some of your business objectives.

Success Factors

This section describes the factors that determine whether a software project is a candidate for a successful open source project. For the purpose of the section, a successful open source project is one that attracts and retains the critical mass of users and developers required to drive (or at least facilitate) on going development of the software. This is distinct from whether or not the project is a business success; success open source projects do not always is result in business success.

Broad Appeal

The single most important factor is that the project must have a broad appeal: The breadth of the appeal sets the upper bounds on the size of the community that can be built around a software project. To have broad appeal, the software should solve a common problem, either for the general public or for an identifiable community. A pre-existing community helps by providing effective means for locating and communicating with potential users/developers.

Problem Type

The type of problem solved is another important consideration. First, the problem solved should be mission critical, a problem that must be solved by the potential users. This is important because, although open-source users don’t usually buy the software, they nevertheless incur costs associated with adopting (and often supporting) the software. The benefits of the software must justify these costs.

Second, it’s important that the problem solved be well-defined. Potential users of the software must be able identify themselves as having the problem solved by the software and agree on that software constitutes a reasonable solution to the problem. In a way, the market for the software has to be able to self-organize and head in a mutually agreeable direction, since there are usually no marketing campaigns to do it for them.

Finally, it helps if the software benefits from network effects: Adopting the software is more attractive when it has been adopted by others. For example, peer-to-peer software has great potential for open-sourcing. Infrastructure software also usually meets this criteria (operating systems, server software, etc), though not as directly.

Cost-Constrained End-user

Initially, it is important that the potential users are cost-constrained. Further, it helps if there are no affordable commercial solutions available. There is a widely held perception that open-source software cannot compete with commercial software in the area of support. However, commercial software vendors are notoriously bad about supporting their own software. Open-source communities are often quite effective at answering questions, identifying bugs, and resolving them. Users also have the option of helping themselves since they have the source code to work with. On the other, there are rarely warranties or service level agreements to accompany open source software. There is also no one else to blame if something goes wrong. End users that have commercial options or that are not cost constrained will usually choose to purchase software. Users without cost constraint also have the option of developing custom solutions that directly meet their needs. Linux and Apache have begun to make progress on this front, but it has been years in coming.

Complete Solution

To attract the users and developers, the software project will need to be a complete solution to the problem. Users must be able to address their problem with the software without extensive reconfiguration or custom development. As an example, most of the software in the initial Linux distributions had existed several years prior as a result of the GNU project. But it wasn’t until Linus Torvald completed the solution by providing a Unix-like kernel for Intel platforms that the software the free Unix communities really blossomed. Prior to the Linux kernel, the cost of installing and using the GNU tools wasn’t justified for most users (particularly on Intel platforms).

This means the core of the software must be owned by those who intend to release it open source. This has proved less obvious than it seems: Netscape’s open source release was delayed because they had to write code to replace the commercial graphics libraries they were using and the calls to the library’s APIs.

It is also important that the software be compatible with existing infrastructure. This is minimizes the cost of adopting the software.

It also helps if the software, its architecture, and its source code are of reasonable quality. The software should be reasonably stable; its architecture should be scalable, modular, and extensible; its source code should be well-commented and documented.

Generous Licensing

A key factor in the success of many open source projects has been the licensing. Several projects have failed to attract communities because the potential users were put off by various restrictions included in the licensing. Eric Raymond gives the following three requirements for open source licenses [ESR99]: First, the licensing must be reasonably symmetric. People don’t mind other profiting from their contributions to the project as long as no is uniquely privileged to profit from them. Second, the users should have the option to split with the project, form a community, and take the software in an alternate direction. Raymond calls this the option to fork and identifies it as the primary reason that Sun’s Community License failed to form an open-source community for the development of Java. This is important to users because it allows them to protect their investment in open source software should the project leaders take the software in a direction that is undesirable. Finally, the license should be restriction free. For example, early projects tried to limit free use of their software to non-commercial uses. These projects suffered as a result of these restrictions because it created a barrier to entry for commercial users who are often the best contributors to open source projects; it also made potential distributors reluctant to include the software in their software anthologies (an undesirable effect).

Project Leadership

To succeed as an Open Source initiative, a project should have well-identified leadership to provide direction and coordination for the project. This leadership includes providing design guidance for the overall project as well as for code submissions. It is said that Linus Torvald’s biggest contribution to Linux wasn’t in generating code, but in selecting just the right things to include and the correct directions for the architecture. It also helps for the project leadership to have strong people skills: The ability to recruit and motivate volunteer contributors is essential. It helps if the project leaders have community respect, but it isn’t essential (in part, because it can earned with time).

To succeed as project, the software needs to show clear progress, i.e., reported bugs are fixed, contributed patches are incorporated, and new features are added, all in a timely manner. Frequent source releases are also seen as important. Most projects make the up-to-the-minute source code available for the adventurous and provide official version releases no less than once a quarter.

Opportunities for Innovation

Developers are attracted to projects where they see opportunities to make significant contributions. The most successful Open Source projects are those where there opportunities for innovation. For example, Linux has attracted many developers who saw opportunities to improve on old ideas (e.g. by optimizing the network stack) or to try new ones (e.g. adding clustering capabilities to Linux). The same is true for Apache, where the ability to add modules of functionality has spawn many sub projects. If the opportunities to contribute are simply to debug and provide bug fixes, many potential contributors will lose interest.

Supporting Service Availability

Availability of supporting services can help adoption of Open Source software. Supporting services are particularly helpful when the target community is adopts new technology conservatively or when the software has matured to the point where it is a viable candidate for mainstream and late adopters. Linux adoption has been broadened by the availability of services from companies like Redhat (user support), Cygnus (customization), and VA Linux (certified hardware).

Benefits

This section describes the benefits of Open Sourcing. For projects that meet the criteria described in the previous section, there are many potential benefits. But Open Sourcing is not a pancea for all software business issues. Projects should evaluate the benefits against their business objectives to decide if they are aligned.

Demand for Complements

A free product can often drive demand for complementary products. Loosely speaking, complementary products are products that can be used together – so that when the demand for one increases, the demand for the other increases too. Joel on Software offers an excellent analysis. In a nutshell, if you have two products that are complementary and you make one a commodity or, better, free, you will strengthen demand for your other product. This is how IBM is using open source software. They sell hardware, services, and software, which are all complements of each other. When they participate in free software projects, they are inevitably contributing to a free project that complements one of there three businesses -- Apache’s Web Server complements their hardware business by making software available their mainframe platforms; it also increases demand for services by lowering costs for data centers and for web application development. Linux does the same. Even where they have commercial software products, they work on complementary componenets – e.g. the Eclipse IDE and the Apache Axis web service stack are complements to their WebSphere product.

Increased Adoption

Almost by definition, successful Open Source projects have strong market appeal (if they didn’t they probably wouldn’t succeed as an open source project). But releasing software open source can also lower barriers to entry that exist for commercial products. First, users have a lower cost of adoption since they are not required to pay for the software itself. If the software as it is available provides a complete solution, users may incur only the cost of installing and configuring the software. Second, it eliminates the threat of lock-in to a proprietary architecture. Since the software can be modified by the users, they have the ability to control the future evolution of the software. Third, it decreases the threat of stranding, as can happen to customers when a company (or it’s product) fails. Because the source code is available, the user always has an opportunity to provide support to themself or to look to the software’s community for support. This can be a significant barrier for smaller companies, particularly when trying to compete against established players.

Successful open source projects can also establish de-facto open standards without going through a contentious open standard process. Gnutella relies on a protocol developed by the authors to communicate with peers. Once it was released, several projects sprang up to develop Gnutella-compatible clients (aka servents) for different platforms and environments. The protocol became a standard overnight, without discussion or debate (n.b., subsequent development of the protocol has been stymied because the original authors were not able to carry the Gnutella project forward). Software projects that are fortunate enough to define de-facto standards also benefit from reinforcing network effects.

Software Quality

The size of a project’s community and the community’s participation in developing and refining the code gives open source software well-verified code. This has significant benefits for the quality (e.g. bug free), integrity (e.g. correct logic), and security (e.g. free of malicious code) of the software developed by open source. Also, the number of code reviewers and the diversity of their perspectives and backgrounds enables open source projects converge towards superior designs. It can also provide reduced testing costs, as long as community isn’t abused routinely with non-functioning software. Finally, community members often contribute both general optimization, and platform-specific optimization.

Maintenance

Through activity community contribution, open sourcing a software projects can reduce software maintenance costs. The wide availability of source code allows the those who are most affected by a bug to identify bugs themselves. They are also often the source of bug fixes. It also allows the project team to focus on new functionality rather than maintaining the old code.

Development

A significant benefit of open source is the project community’s contribution to on-going development. A successful open source project can also rely on their community as a significant source of new development. Members will take advantage of their access to the source code to add new functionality to address their specific needs. Community participation is also a significant source of innovation: because users have the capability to modify and extend the software, they will often try to apply the software to problems not considered by the originators. Open Source communities also do a fair amount of porting to bring the existing software to new platforms, extending the reach of the software.

Customer Intimacy

Companies that release software products through open source necessarily build a more intimate relationship with their customers. First, access to the source code promotes open and honest communication between a software company and their clients. Second, the company and its customers collaborate on each new release. So, customers also have greater insight into the day-to-day development and can participate directly in setting the direction for future development. The dialog between the company and its customers happens in a community setting, so customers can collaborate with each other to solve common problems. Finally, customers of open sourced products feel a greater sense of ownership and generally benefit from increased adoption of the software (more hands to help with the work), so they often provide evangelism for the product.

Valuation

A company that runs a successful open source project will almost certainly see the benefits of their community reflected in their valuation if the project is complementary to their sources of revenue.

Business Opportunities

Open Source software can also create or enhance other business opportunities. First, a successful open source project creates often creates opportunities for supporting services (e.g. certifications, professional services, distribution, support, etc.). They also can be a value add for existing products or services (e.g. hardware vendors might open source drivers).

Costs

Intellectual Property

The main cost of open sourcing is the sacrifice of the intellectual property of the software itself. In some cases the intellectual property might be a potential source of differentiation. In choosing to release the source code, companies also sacrifice what ever competitive advantage the invested effort might have accrued. For example, it eliminates any time to market advantage or any advantage from novel business practices or engineering knowledge in the software. Given the typical open source license, the company is also sacrificing any exclusive right to profit from the software and will likely also have to forego enforcing any patents.

Valuation

Existing code, patents, and trade secrets are often used by investors to assess the value of a company. As a result of sacrificing the intellectual property in the software, companies also potentially sacrifice a significant part of their valuation to investors.

Project Maintenance

Open Source projects require investments both in the in the initial setup of the project and in the on-going maintenance. The main costs are in providing the initial release, which entails defining license terms, preparing code for public release (e.g. removing confidential code and documenting the source code), and building project infrastructure (i.e. code repository, site, mailing lists, etc). On an on-going basis, a company will need to provide at a minimum project leadership and release management.

Scope Expansion

Taking a project Open Source can expand the scope of development for the project. In many cases, it will force build rather than buy for software components that are commercially available. The project may need to build the components in order to be able to field a complete offering to their community. This was Netscape’s biggest hurdle in going open source. On an on-going basis, it will may make it a challenge to maintain strategic focus. In the interest of maintaining the community, companies may spend more time than is desirable incorporating fixes and new functionality from users that are not strategic to their business.

Common Denominator Platform

In order to achieve critical mass in their community, open source projects often end up coding to a common denominator platform – whatever everyone has. This may restrict the choice of hardware and operating system platform for the project and preclude the use of any commercially available components.

 

Further Reading