On Plug-ins and Extensible Architecture

In a world of increasingly complex computing requirements, we as software developers are continually searching for that ultimate, universal architecture that allows us to productively develop high-quality applications. This quest has led to the adoption of many new abstractions and tools. Some of the most promising recent developments are the new pure plug-in architectures.

What began as a callback mechanism to extend an application has become the very foundation of applications themselves. Plug-ins are no longer just add-ons to applications; today’s applications are made entirely of plug-ins. This field has matured quite a bit in the past few years, with significant contributions from a number of successful projects.

This article identifies some of the concepts around the foundation of pure plug-in architectures and how they affect various stakeholders when taking a plug-in approach. The intention is to discuss the general issues involved, but the article also presents some lessons learned from Eclipse (www.eclipse.org) to better illustrate some of the concepts.

MOTIVATION FOR PLUG-INS

Customers’ requirements control the creation and deployment of software. Customers demand more and better functionality, they want it tailored to their needs, and they want it “yesterday.” Very often, large shops prefer to develop their own in-house add-ons, or tweak and replace existing functions. Nobody wants to reinvent the wheel, but rather to integrate and build on existing work, by writing only the specialized code that differentiates them from their competition. Newer enterprise-class application suites consist of smaller stand-alone products that must be integrated to produce the expected higher-level functions and, at the same time, offer a consistent user experience. The ability to respond quickly to rapid changes in requirements, upgradeability, and support for integrating other vendors’ components at any time all create an additional push for flexible and extensible applications.

TRADITIONAL PLUG-INS VERSUS PURE PLUG-INS

Most people are familiar with traditional plug-ins, downloadable software bundles that extend the functionality of hosting applications such as Web browsers or text and graphical editors. They are not compiled into the application, but linked via well-defined interfaces and extension mechanisms. Most often, building a plug-in does not require access to the source code of the application. Plug-ins implement functions that the host application can recognize and activate when needed.

In the new pure plug-in architectures, everything is a plug-in. The role of the hosting application is reduced to a runtime engine for running plug-ins, with no inherent end-user functionality. Without a directive hosting application, what is left is a universe of federated plug-ins, all playing by the rules of engagement defined by the framework and/or by the plug-ins themselves.

To support composing a larger system that is not prestructured, or to extend it in ways you don’t foresee, the architecture must support the extensibility of plug-ins by plug-ins. In the absence of a hosting application, plug-ins themselves become hosts to other plug-ins by providing well-defined hook points where other plug-ins can add functionality. These hook points are commonly known as extension points. When a plug-in contributes an implementation for an extension point, we say that it adds an extension. Much like defining any contractual obligation, an extension model provides a structured way for plug-ins to describe the ways they can be extended and for client plug-ins to describe the extensions they supply.

In a simplified way, figure 1 illustrates the main structural difference between traditional and pure plug-in architectures.

Click for figure

PLUG-IN RUNTIME CONSIDERATIONS

As already mentioned, plug-ins are not stand-alone programs; their execution is orchestrated by a runtime engine. The plug-in runtime engine—let’s call it the kernel—is the core engine that is started when a user launches an application. At a minimum, the kernel must provide runtime support for the basic plug-in infrastructure by:

• Finding, loading, and running the right plug-in code.
• Maintaining a registry of installed plug-ins and the functions they provide.
• Managing the plug-in extension model and inter-plug-in dependencies.

Optionally, the kernel can supply other utility services, such as logging, tracing, or security. The user should also be able to turn off or remove parts of the application, or dynamically install new functions. For obvious reasons, it is desirable for the kernel to be small and simple, yet robust enough to build industrial-strength functionality on top of it, without many (or any) hacks.

To illustrate the basic workings of a plug-in environment, let’s take a behind-the-scenes look at the Eclipse plug-in runtime structure and control flow. While mostly known as a powerful Java integrated development environment, Eclipse is actually a universal plug-in architecture for creating “anything, but nothing in particular.” The runtime engine itself is implemented as a number of core plug-ins, except for a tiny bootstrap code. This code starts the core plug-ins to initialize the plug-in registry and the extension model and to resolve plug-in dependencies. Other than the core plug-ins, no other plug-in code is run at this time. All the needed plug-in metadata is read from the plug-in manifest files (plugin.xml and/or manifest.mf).

Once the initialization is complete, the runtime kernel is ready to run applications. The extension mechanism, shown in figure 2, kicks in and an entire application is built progressively, from inside out, as follows:

1. The runtime core plug-in defines an applications extension point so that any plug-in can declare itself an application by contributing an extension to it.
2. An applications extension must provide a concrete implementation for the extension point’s callback interface, IPlatformRunnable.
3. Eclipse’s default applications extension is contributed by the IDE plug-in, by providing a concrete implementation class that, when called by the extension point handling code, creates the workbench graphical user interface and runs the event loop until it exits.

Click for figure

This workbench window looks like any standard application: it has menus, toolbars, views, and editors, and you can launch wizards, open preference dialogs, and so on. The basic workbench by itself, however, is just a frame for displaying visual parts and other user interface elements. Its power comes from the extension points (e.g., views, wizards, editors, action sets, help support) it defines, to which other plug-ins, or the workbench plug-in itself, can contribute extensions.

In general, the plug-in that defines an extension point is responsible for looking up its extenders (using the extension registry), instantiating the associated callback class, and invoking the appropriate interface methods. Under the covers, the runtime kernel is responsible for providing the extension registry lookup functionality, creating a class loader for the extender plug-in, loading it, and starting it. This ensures the extender plug-in is initialized and active before its classes are created and run.

Finally, it is worth noting that some extension points may require more than one callback interface and others do not require any. For example, the help system extension point toc is used for contributing documentation tables of contents and does not require any callbacks. When help is invoked, the help plug-in processes the toc extensions by creating a larger, integrated table of contents from all contributing plug-ins.

CONFIGURING AND DISCOVERING PLUG-INS

An important consideration when deploying a plug-in system is configuring and discovering the plug-ins. Unlike monolithic applications that are typically installed in one folder on a user’s machine, plug-in-based applications may have a higher degree of freedom for installation layout and plug-in discovery. This flexibility can also be a source of major headaches for installers and plug-in management.

Consider a multiuser shared installation where the entire product is installed by the central administrator on behalf of a community of users. The individual users do not need to perform any installation. They are simply given a local “shortcut” to invoke the shared installation (the shortcut can be in the form of a command-line invocation or a true desktop shortcut encapsulating the command-line invocation). If you’re like most users, you will want to install some other cool new features, especially when the third-party plug-in offerings are plentiful (and often free). Obviously, the new plug-ins, private to that user, cannot be installed in the read-only, shared install location, so the product should allow users to install and configure extra plug-ins in a location where they have more privileges.

Configuring only a subset of plug-ins for a particular user based on access criteria, such as user roles, adds more challenges. Other configuration issues arise in scenarios where a common set of plug-ins is shared by many applications running on the same machine, or where some plug-ins are not installed locally but run from a remote location (e.g., an application server provider).

In theory, the platform must find the available plug-ins on its own and gracefully cope with missing plug-ins or those that are dynamically coming and going. In practice, many solutions for these problems involve predefined files and directory locations for key configuration files or for the plug-ins themselves. More advanced plug-in systems offer pluggable configuration and plug-in discovery mechanisms. Usually some basic bootstrap code is run to start a configurator plug-in, which will then discover the other plug-ins based on its own strategy. For example, Eclipse provides an Update Manager configurator plug-in that picks up plug-ins from the eclipse/plugins folder, as well as from other local plug-in folders linked from the eclipse/links folder or dynamically added when users install new plug-ins to locations of their choice. Because the configurator is pluggable, anyone can plug in another configurator, which can provision and configure the plug-ins from a remote server.

CHALLENGING ISSUES

Pressured by time, budgets, or slick marketing claims, you may be tempted to adopt a pure plug-in architecture without being aware of its potential pitfalls. Alternatively, you might truly believe in Murphy’s law and therefore won’t use plug-ins because they’re doomed to fail, anyway. I tend to follow those who walk the middle road: if something can fail, then it first should be understood and then fixed. The rest of the article presents issues that are likely to pose some challenges when you employ plug-ins, and that will help you ask questions or provide answers when evaluating a plug-in architecture.

Installing and updating.Many modern products automatically detect when they are out of date with respect to available service or product version. Either on start-up, or as a result of an explicit update action, the products compare the current installation level against some network-based baseline. The product then automatically downloads required fixes or upgrades and applies them, often as part of the product execution. Additionally, users can, and most often will, install additional plug-ins from various sources to extend the functionality provided by their current application.

For plug-in-based applications, the installation and update process can be a real nightmare: on the one hand, there are the traditional installation issues that arise in any application—ability to roll back changes, migrate existing program data and preferences, or ensure the installation is not corrupted. On the other hand, because plug-ins may originate from various providers that are not related to each other, the resulting configuration has likely never been tested. This poses a number of interesting challenges that we address in the context of the other issues discussed in the next sections.

Security.Systems can never be too secure, and you need to pay particular attention to securing a system based on plug-ins. Since arbitrary plug-ins can be installed—for example, by downloading them from the Web—and are allowed unlimited access to the system they plug into, security in a plug-in environment must be carefully planned. On top of this, some plug-ins require support for executing custom install code during installation, so they can have control over some parts of their installation. To prevent software security accidents or failures, the plug-in framework must address the issues of downloading from third parties and controlling a plug-in’s access to other code and data. Supporting digitally signed plug-ins or secure connections helps, but it still relies on trusting the download source or the plug-in provider. Some programming environments, such as Java, offer built-in runtime security mechanisms that can be used effectively to close some of the security gaps.

The cold reality is that, unless you are careful about what you install, it is almost impossible to be confident that the installed plug-ins are not ill-intentioned. Of course, a pure plug-in architecture also means that even a well-intentioned plug-in with serious bugs can do as much damage as an ill-intentioned one when installed.

Concurrent plug-in version support. Without a doubt, managing concurrent plug-in versions and dependencies is one of those problems that can keep architects, developers, and installation folks awake at night. Most of you have probably experienced “DLL hell” at some point and will look with suspicion on something that has the potential of being a “plug-in hell.”

Any serious plug-in model defines some versioning scheme, but allowing multiple versions of a plug-in to be concurrently installed and executed is an issue that must be considered early on in the design, and it must be properly enforced by the install and update processes.

To illustrate the complexity of this area, let’s start by considering the scenario of an application suite that integrates two stand-alone plug-in products, each installed in a separate location. It is possible to have the same plug-in, at the same version or at different versions, installed in two locations. The runtime kernel must deal with the two plug-in instances by either running them both or dropping one. When both versions are running, and the plug-ins contribute user interface elements such as menus or preference pages, the result may be a very confusing interface, with duplicate menu entries or preference pages. There is more trouble: would the user need to be exposed to this anomaly? How would you update or uninstall these plug-ins?

The difficulty of the problem is amplified by the other important role plug-in versions play here: they are part of plug-in dependency specifications. Typically, a plug-in requires the functionality provided by other plug-ins, and very often there are strict criteria about what versions are required. For example, a plug-in may require a certain level of an XML parser contributed by another plug-in, or it may need to contribute an extension to a particular version of an extension point only. Version dependency can be in the form of a fixed version number such as 3.0.1, or as a range of versions—say, 3.0.1 or newer. Properly resolving plug-in dependencies at runtime is critical to the correct functioning of the application. Managing plug-in dependency graphs is also critical to ensure a consistent configuration state after installing or updating plug-ins, as well as for properly rolling back changes. The install/update process may need to abort installation of plug-ins that leads to an unresolved dependency, or search for and install the missing plug-ins. Either way, it is possible to end up with multiple versions of the same plug-in, so we’re back to the problem of managing concurrent versions.

There is no simple, general solution to managing concurrent versions of plug-ins. Eclipse has adopted a reasonable trade-off convention for concurrent plug-in versions: only versions of plug-ins that contribute code libraries but no plug-in extensions (no user interface contributions, no documentation, and so on) are allowed to coexist in the same runtime instance. For all the other plug-ins, the latest version is usually picked up, unless a configuration file precisely defines what to run. The main advantage of this approach is that it still allows various levels of the same code to coexist at runtime, but hidden to the end users, who will get a consistent user interface. The downside is that this will not cover all user scenarios, and that plug-ins are not treated uniformly as elsewhere in Eclipse. This special treatment of plug-ins is not just a runtime/install issue, but also something that developers and product packagers must consider.

Scalability, up and down. Another challenge of working with plug-in architectures is scalability. Just as the problems associated with multiple versions and dependencies can quickly escalate, so too can the sheer number of interacting plug-ins quickly become a problem. For example, when Eclipse was first designed, it was thought that a product, a large one for that matter, would consist of a few hundred plug-ins. A few releases later, some enterprise-class products built on Eclipse are known to have passed the thousand plug-in mark, so the platform goal has been revisited to support scaling between 5,000 and10,000 plug-ins.

When designing a plug-in system for scalability, developers must consider various mechanisms that make start-up faster and have a smaller memory footprint. A general principle of runtime is that the end user should not pay a memory or performance penalty for plug-ins that are installed but not used. A plug-in can be installed and added to the registry, but the plug-in will not be activated unless a function provided by the plug-in has been requested according to the user’s activity. In general, this requires support for plug-in declarative functionality. It is often realized in practice via plug-in manifest files, so no code has to be loaded for obtaining the function contributed by plug-ins. Caching of the plug-in registry and the plug-in manifests/declarations can reduce processing during start-up in subsequent application launches, improving response time. What is gained in performance and memory footprint, however, is lost in code complexity: more code needs to be written to cache the data and to synchronize the cache with changes in the installation configuration.

Scalability problems almost always surface during plug-in install/update operations. The larger the product, the larger the download size of patches and upgrades. For product upgrades, the download time can be improved by simply downloading only the plug-ins that changed. Another installation scalability problem can arise during an interactive install/update operation. Quite often plug-in boundaries are established for development reasons (such as function reuse) and present the wrong level of granularity in terms of what the user sees as the unit of function, and therefore as an installation unit. Considering that enterprise-level applications can scale up to hundreds and thousands of plug-ins, with complex dependencies, the user can be overwhelmed by various installation choices. A possible solution is to introduce a packaging and installation component that groups a number of plug-ins to offer a higher level of function. For example, in Eclipse, Update Manager does not install plug-ins directly; it processes features. Features are bundles of plug-ins and are considered deployment units with install/update semantics.

The other side of scalability is scaling down, so products can run on devices with limited resources, such as cellphones and PDAs. This usually means rethinking the core framework plug-ins to refactor them into smaller plug-ins, some of them optionally deployable, so that one can run with a minimal configuration.

OVERCOMING THE BUMPS

General discussion about plug-in architecture and design issues may not mean much for you or your company’s bottom line unless you’re a participant or plan to be one—either by creating a plug-in framework or developing plug-ins for fun and profit, or by using a plug-in-based application. Employing a plug-in framework is not without bumps, but they can be overcome with proper preparation.

While traditional plug-ins have been around for some time, the pure plug-in models have only recently emerged as robust, enterprise-level quality application development environments. As many major industry players are rapidly adopting plug-in technologies for their software lines, we can expect further research and development into improving the current architectures. Some of the areas that will receive increased focus will be:

• Security at all levels, including installation, update, and runtime.

• Performance: speed and low resource usage.

• Improved tools for development, testing, packaging, and deployment of plug-ins.

• Improved installation and updates, especially by merging changes without taking the system down.

• Remote management of various processes (provisioning, configuration, and so on).

• Convergence, compatibility, or interoperability of various plug-in frameworks.

• Wider range of deployment platforms (from desktop PCs and high-end servers to digital mobile phones and embedded devices).

Increasingly, much of this effort will be happening in the open source domain or standards bodies.

Back to On Plug-ins and Extensible Architectures

Discussion

Overview

The cost impact to a company of a failed project can be severe indeed. The impact on the reputation of the project manager can be disastrous.

Software project management is not easy, and it requires considerable skill to successfully manage the many different risks that conspire to de-rail a project:

Numerous methodologies are available for mitigating these risks – PRINCE2, RUP, DSDN, eXtreme programming – and these have helped to some extent.

This document introduces the 3D™ methodology – a set of best practices and quality tools developed by BuildMonkey, which can be summarised as.

De-risk. Deploy. Deliver.

In any software project, deployment is a milestone on the project plan that is usually associated with payment – or staged payment. Through the course of development, any number of problems can arise to derail efforts to reach this milestone.

The 3D™ methodology and supporting toolset is based on many years of experience at the sharp end of projects, understanding what has worked and what has not, and the lessons learned from each.

Competent practitioners, and experienced project staff, will find resonance with many of the contents of this document and may find themselves saying “this is just common sense”. This is certainly true, but the main problem with common sense is that it is not as common as people think it is.

This document, and the 3D™ methodology, is an attempt to bring that common sense together in a single location, as a coherent set of best practices supported by proven tools to help you to release on-time, on-budget, with no defects.

The Problem To Be Solved

No methodology has yet focused on the component that all development projects share – the build.

One of the reasons for this is that the term “build” is interpreted differently by different people:

• The development team sees it as compilation and assembly;
• The integration team see it as the bringing together of all of the components in the application in a format suitable for release;
• The deployment team see it as something which produces the artifacts that they have to install and configure;
• The testing team see it as something which produces the artifacts that they have to test;
• The Project Manager sees it as an opaque step that nobody is entirely responsible for;;
• The end customer should not see it at all;

The BuildMonkey view is that the build is the combination of processes and technology that take software from design to deployment – _where the return on investment starts to be seen.

It is clear that a methodology is required to de-risk development projects and to standardise use of the term “Build Management”.

Best Practice: “Build Management” encompasses everything from compilation, all the way through to release to the customer.

No Release, No Revenue

Any Finance Director knows that development is just an activity that needs to be tolerated in order to produce something that will deliver a return on investment.

It may sound strange, but a large number of software developers do not appreciate and embrace this basic truth. This is in part due to their closeness to the application being constructed.

A common problem faced by development projects is therefore that it is the software developers who manage the build. This creates a situation where the build is focused on the needs of development, and is not geared towards releasing the output of coding such that business value can be realised.

Build Management should therefore focus on the end result of development – a return on investment – and ensure that all of the inputs to the process are incorporated in pursuit of this goal:

Best Practice: Focus on the end, and accommodate the inputs

Software Tools Are Only Part of the Answer

Software projects are a complex set of interdependent people and teams and can be likened to a convoy of ships. A convoy has to move at the rate of the slowest ship. Increasing the speed of a single ship in the convoy will not increase the speed of the convoy – it will simply increase the amount of wasted capacity in the speeded-up ship.

Speeding up the slowest ship will, however, have a positive effect since the whole convoy can now move faster.

Many Project Managers try to improve productivity by implementing some degree of automation in development projects – particularly in the area of the build – and often purchase “magic bullet” build software that provides this.

Simply using automated build software does not improve productivity any more than the example above improves convoy speed - as it only increases the speed of a single ship in the convoy.

There is no point in speeding up development, if the target production infrastructure cannot keep pace – this just increases the inefficiency. A lot of organisations make this mistake – highly agile development processes trying to feed into considerably less agile deployment processes. The result is inefficiency, waste and over-run.

Before considering using an automated build tool it is essential to ensure that the inputs to, and outputs from, the build can cope with the improved build speed. It is imperative to ensure that the processes and technology employed are geared towards taking the project to a successful conclusion – on-time and on-budget.

Best Practice: Don’t rely on software tools alone, they may solve symptoms whilst creating problems elsewhere

Configuration Management Best Practices

Software Configuration Management (SCM) is a relatively mature discipline with much written about methodologies and techniques, and these will not be recreated here.

We will focus instead on leveraging the SCM repository, and the facilities that it offers, to further the goals of the project rather than to consider SCM in its own right.

It All Starts Here

The SCM repository - how it is managed and used – is the keystone of good build management and successful delivery of projects.

The SCM repository is the slave of the project, not the other way round. It should be solid and reliable, yet flexible enough to accommodate the needs of new projects. Project Managers should not have to retrofit their planning to accommodate an inflexible SCM setup.

If used correctly, the SCM repository will enhance productivity, and minimize risk, through being able to provide exactly what the project – and project management – require. If used incorrectly, it can cause delay and slippage through having to do things inefficiently further down the chain.

Best Practice: The SCM repository is the slave of the project, not the other way round.

Configuration Management is not Version Control

Most software developers regard the SCM repository as a massive storage area where they simply check versions in and out – a common cause of problems.

Simply checking things into an SCM repository is not Configuration Management any more than karaoke is opera.

A well-maintained SCM repository is so much more than version control, and should provide:

• The ability to recreate any identified baseline, at any time;
• Meaningful statistics on what is changing, when and by whom;
• Management information, such as “how many new defects were introduced by the refactoring of component ‘X’?”

In order to be truly effective in a project, the SCM repository should store all of the artifacts that form part of a baseline or a release.

Most development projects simply store the code that is being developed and their use of the SCM repository is no more sophisticated than this.

Most applications nowadays are not just source code. Take the example of a modern computer game – the vast majority of the code base is made up of artifacts other than code such as audio clips, pictures and movie clips.

Where an application relies on a database this database will have a schema and structure that may change from release to release – this schema must be captured.

There will normally also be seed data for the database which should be considered as part of the baseline.

In a large distributed application, the software being developed will sit atop a number of pieces of software (e.g. application servers, web servers and message queues).

The configuration of these underlying applications have an effect on the quality – or otherwise – of the software being developed and should, therefore, be considered part of the baseline for a release.

The underlying environment and infrastructure is a key component of the project, particularly in the case of distributed applications.

Such banal considerations as DNS zone files, user account information and system parameters have to be considered as some of the moving parts which affect the application and therefore be placed under configuration control.

This is of particular importance when there is more than one environment involved in the project (e.g. a development environment and a separate test environment) since the question of “are these environments the same?” crops up again and again.

Best Practice: Everything that can be changed, and affect the behaviour of the application, is a candidate for configuration control

The Point of Labelling

It is a common misconception that simply applying labels, or tags, to the SCM repository creates a baseline but this is only partly true without corresponding records of:

• What label has been applied;
• When that label has been applied;
• Why it has been applied (i.e. what milestone, or other external event, the label is associated with);

The use of a naming convention for labels can deceive even further. For example, a project that uses a date-based labeling convention (dd_mm_yyyy) will have several labels of the form (03_05_2004, or 09_06_2004) and will reasonably assume that they have some kind of record of the baseline on those dates.

But what was happening in the project on those dates? Was the 03_05_2004 label applied immediately before a release to test, or immediately after?

Best Practice: Labels should be used to identify and inform about events in the project

This may seem like stating the obvious, but there should be a reason for a label being applied – the whole purpose of labeling is to identify some event in the development cycle that may need to be re-visited.

To this end, labels can be divided into two categories:

• Point-in-time
  Associates particular versions with a particular calendar date, or other event that is fixed in time (e.g. MONTH_END_JAN_2004, or RELEASE_1_0_1);
• Point-in-process
  Associates particular versions with events in the project that may recur at a later stage (e.g. LATEST_RELEASE, or CURRENTLY_IN_UAT);

Best Practice: Every label should have a point, whether point-in-time or point-in-process

Management Information

The job of the Project Manager, ultimately, is to bring the project to a successful conclusion. If this were an easy task that happened by default, then there would be no need for a Project Manager.

In order to be able to do this job well, a Project Manager needs information. He needs to know what is going on in the project – who is doing what, who is working on which components, and a wealth of information can be obtained from a well-managed SCM repository:

• What is changing in the environment – what has changed since a given point in time or how often particular elements are changing;
• Who is changing things in the environment;
• Why things are changing in the environment;

Of course, the final item in the list requires that committers are using descriptive comments to indicate why they are marking a particular change. A well-managed SCM repository should enforce this.

Best Practice: The SCM repository should provide meaningful, and accessible, management information

Build Best Practices

As explained at the beginning of this document, the term “build” means different things to different people. The most common interpretation is the one used by developers, where the term “build” describes the compilation and assembly step of their development activities but this narrow interpretation is a common cause of problems and over-runs, on development projects.

Building is not Compiling

At the outset of the project, the Project Manager will ask the question “how long to set up the build?” and a developer – thinking of compilation and assembly – will answer something like “1 day” – a task and duration which is then duly marked on the project plan and development begins.

Later in the project, when it is time to start deploying and testing the application, this “build” needs to be refactored to accommodate the deployment and test tasks. In doing so, it turns out that the way the application is being assembled is not conducive to it being deployed or tested correctly – so the compilation and assembly staged need to be refactored as well.

In the meantime, the development team sits on its hands whilst the “build” is refactored to accommodate the needs of the project – valuable time is lost whilst the deadline continues to advance.

Best Practice: Know what will be required of the build before starting to write the scripts

Don’t Throw Out The Baby With The Bathwater

From a build perspective, projects with similar architecture (both in terms of the team and the application) will have similar attributes. There will obviously be some changes required, but these will tend to follow the 80/20 rule to a large degree.

For example, a web application that is being developed by an in-house team and that will be deployed to a Tomcat servlet container and Oracle database will follow largely the same steps and require largely the same deployable artifacts.

A good SCM repository will enable the latest versions of boiler-plate build scripts for such an application to be found. These can be used almost off-the-shelf – meaning that development can start apace without having to wait on the build to be constructed for similar applications .

Best Practice: Well-crafted builds are re-usable and should be re-used

The Architecture Drives The Build<

Following on from the previous section, it should be clear that the architecture of what is being developed – and the structure of the team(s) developing it – will dictate how the build should look.

There is little value to be gained in trying to retrofit build scripts for a computer game (developed by 12 people all in the same room) into a project to produce a large J2EE application with development occurring at six different sites around the world.

Best Practice: Well-crafted builds are flexible, but a “one-size-fits-all” approach can be costly

Management Information

There are a number of people who need information that the build can provide:

• The Project Manager needs to track overall progress against defined milestones – number of outstanding defects, whether a committed release date will be met etc;
• Development leads need to be sure that the code is of the quality that they require – test reports, bug analysis patterns, code metrics, document and UML generation etc;
• The deployment team need to know that the artifacts will work in their target environment, and that the environment is as the application expects it to be. They also need to know whether two (or more) environments are “the same”;
• The test team need to have confidence that they are testing against a known baseline, and whether defects that they see have been rectified in development (or whether they are re-appearing after already being fixed);
• Everybody needs to be able to communicate effectively using the same language, and have a common terminology for release versions – particularly if there are multiple threads of development;

A good build infrastructure will provide all of the above information, and more besides.

Best Practice: The build should tell all project participants what they need to know

Deployment Best Practices

Considering that it is generally an important milestone on a project plan, normally resulting in payment or a staged payment, deployment is one of the most overlooked areas of software development.

The normal course of events is:

1. Release artifacts are created;
2. Some installation and release notes are cobbled together in the form of a README;
3. The deployment team work frantically to install and configure the application – the testing team (or, worse still, the customer) are idle and unproductive in the meantime;
4. Some symptoms are found which are suspected to be application defects;
5. The development team blame the environment;
6. The deployment team blame the application;
7. Repeat (5) and (6) ad nauseam.

When a documentation team are also considered - responsible for creating documentation that the end user will need to install, configure and use the application – the situation becomes even more difficult.

This situation can be avoided by planning for deployment from the beginning. Deployment is an inevitable part of software development, yet it always seems to take people by surprise.

Best Practice: Know that deployment is inevitable, and incorporate it into the automated processes

Deployment Is Not Installation

As part of normal development activities, artifacts are installed into sandbox environments – and test environments – many times. But this is not deployment, this is installation.

In order to get an application into its production environment, be that an internal environment or on hosted-infrastructure, a number of hurdles must be overcome:

• The application must pass UAT;
• The application must be installed and configured correctly:
• All pre-requisites for the application must be satisfied;
• The end customer must accept the handover;

Deployment is that point in the life of an application where it starts to produce a return on investment. “Launch”, “Go-live”, “Release”, “First Customer Shipment” are all phrases which describe the same event.

Best Practice: Deployment is the point where an application starts to provide a return on the development investment.

The Environment Is a Refactorable Component

This point cannot be stressed enough, particularly in large distributed applications.

Every application, large or small, has a runtime environment in which it operates. In a simple desktop application, this is a standalone machine (such as a PC). In larger applications, this will be a combination of machines (e.g. an application server and a database) operating together to provide the runtime environment.

In either case, the application expects certain facilities to be available from the runtime environment and will function incorrectly – or cease to function – if these are not present .

The environment itself, whether standalone or a network, contains many moving parts that can be independently configured. IP addresses, or hostnames, can be changed. User privileges can be modified or revoked. Files and directories can be removed. Each of these can have an effect on the way that the application behaves.

In an environment that is owned and managed internally this can be bad enough. In an environment that is owned and managed by an external third party, and where project success is contingent upon successful UAT in that environment, this can be disastrous.

Best Practice: Be able to identify whether the deployment environment is as prescribed, and “fit for deployment”

One of the most common questions that arises in development projects containing more than one environment is, simply, “are these environments the same?” and its answer can be elusive.

It is essential to be able to answer that question – quickly and accurately – so that any perceived defects in the application can be categorised as “defect” or “environmental”.

This ability becomes particularly poignant where on or more of the environments are owned by different teams, or organisations.

Best Practice: Be able to prescribe what the deployment environment should look like, and have a capability to test it quickly.

The environment, as explained earlier, is a refactorable component. It can be changed, and parts can be moved or deleted. However, unlike application code, changes may need to be made to the environment in response to external events (e.g. hardware failure, or security policies).

Applications, particularly complex ones, use regression tests to ensure that observed behaviour after a change is exactly as it was before the change was made. The same should be true of the environment.

Best Practice: Automated regression tests for the environment that will compare observed behaviour both before and after changes are made.

For example, suppose that a number of operating system patches or service packs are applied to an environment where the application has been, or will be, deployed. How are these tested? Do you wait for users, or testers, to start calling to say that there are problems?

Or do you make sure that you know what problems have been introduced before your users do?

As stated earlier, the SCM repository should be used to store any artifact that can be changed and that may have an effect on the environment.

It may not seem obvious, but some of the most obscure environmental changes can cause an application to fail:

• Hostname resolution;
• Non-existent user or group accounts;
• IP and network connectivity;
• Existence, or otherwise, of files and directories ;
• Application or operating system configuration files;

It is essential that these environmental variables be placed under configuration control and able to be identified as part of a baseline.

Best Practice: Environmental artifacts that are not part of the application should be part of the baseline

Automate, Automate, Automate

Every single task that is performed as part of a development project – throughout the entire lifecycle – can be placed into one of two categories:

1. Tasks which require some form of human judgment;
2. Tasks which do not;

Tasks which fall into the first category can use some degree of automation, but should stop and wait for human intervention wherever judgment is required.

Tasks in the second category should be automated. There is no value in having expensive resources employed to do mechanical or repetitive tasks that a computer could do more quickly, accurately and consistently.

Best Practice: Automate anything that does not require human judgment

A note of caution - it may be tempting to think that automation will increase productivity on its own, but this is not necessarily the case. Automating an inefficient process will simply magnify its inefficiency – as explained in the section on Software Tools are Only Part of the Answer.

This, and it is worth repeating, is a common error – to assume that automated build software alone will improve productivity.

Best Practice: Do not automate inefficient processes, or you will only maximize the inefficiency

About BuildMonkey

BuildMonkey is a division of Nemean Technology Limited - a leading technical innovator specialising in Agile Deployment and build management.

We have over a decade of experience in helping clients increase productivity in their build, integration and test cycles. We have a track record of massively reducing costs and defects through process automation and proven methodologies.

Formed in 1999, and profitable every quarter, we invented the discipline of BuildMaster - we are the original and the best.

We provide specialist Build Management products and services - when all you do is Build Management, you get pretty good at Build Management. Our world-class Professional Services staff are the best in their field, complemented by original software products of the finest quality.

We aim to be the leading provider of Build Management products and services in the UK by providing excellent service to our customers, and by empowering Software Project Managers to aim for release on-time, on-budget, with no defects.

About the Author

John Birtley has been a specialist in Configuration and Build Management since 1994 with a focus on deployment - particularly the packaging and installation of developed applications. He has worked for a number of blue-chip organisations including Sun Microsystems, Hewlett Packard, Sony and Ericsson in the transition from development to deployment. BuildMonkey was launched in January 2004, to package and commoditise this cumulative experience under the title of "Agile Deployment". He lives in Southampton, where dogs and children conspire to take what little spare time he has.

PRINTER FRIENDLY VERSION

Flier Lu问到CLR Loader和Java Class Loader有什么不一样。要回答这个问题不容易，因为我对Java一窍不通。但既然问题提出来了，打肿脸来充胖子也得回答啊。

于是今天下午我在google上逛了一圈，找了些关于Java Class Loader的文章看了看。我的结论是，和Java Class Loader类似的东东，在CLR里是不存在的。

简单的说，CLR里不存在一个类似于Java.Lang.ClassLoader的东西。所以你无法实现你自己的CLR Loader。CLR里只有一个Loader，那就是CLR Loader。当然，CLR提供了自己独特的方式让你做动态装载。你所拥有的自由度是远比Java要大得多。

下面是具体分析。

我对Java Class Loader的理解主要是从这两篇文章里来的。

“Inside Class Loader” by Andeas Schaefer (http://www.onjava.com/lpt/a/4337) 和 “The basics of Java Class Loaders” by Chuck McManis (http://www.javaworld.com/javaworld/jw-10-1996/jw-10-indepth_p.html)。我看了看JVM Spec。但是我觉得不如上面两篇文章清楚。

先说Java里的Type吧。Java里的Type就是Class 。Class就是Namespace + class name.  Class通过Class Loader来装载的。System Class Loader缺省的话只在CLASSPATH中寻找Class。如果你要在CLASSPATH之外转载Class的话，你就需要自己的Class Loader。如果两个Class有相同的名字，并且在同一个Class Loader里，那么它们就被JVM认为是相同的，可以互相赋值。如果有一个不一样的话，它们就被认为是不一样的。互相赋值会发生ClassCaseException。换句话说，Class name + Class Loader是一个Type的独特的ID。

Java的这个模型有很多问题。首先它把Class name做为Type的ID。两个Class如果有同样的名字，但是实际内容不一样的话，如果它们被同一个Class Loader装载，JVM会认为它们是同一个 Type。这个太搞笑了。结果就是Class Loader必须要有一些特别的办法来管理Class name。一般的办法是加一个前缀，比如说Class的URL。如果Namespace管理不好的话，很容易就是安全漏洞。这是为什么JVM Spec里提到Class Loader必须要让Parent Class Loader先搜索Class，再自己寻找。而且Class loader必须要保存Class resolution的结果，这样下次Class resolution的时候，Class loader会返回同样的结果。Java Class Loader有太多的限制，同时又有太多的责任。

另一个问题是共享问题。如果同一个Class被两个不同的Class Loader装载的话，JVM认为这两个Class不是同一个Type，不能互相赋值。结果就是Class无法共享。被不同Class Loader装载的Class无法直接对话。直接对话会产生linkage错误。它们只能间接对话，比如通过Interface，或者共同的Base Class。

还有一个问题是Versioning。因为Java的Type里只有名字，所以当你看到一个Type的时候，你无法知道它是第一版，还是第二版。所以如果你需要第二版，但是Class loader给你装载了第一版的时候，祈祷吧。也许上帝能救你。

现在回过头来看看CLR的模型。CLR的Type包括两部分：Class name + Assembly Name。Class name是和 Java的Type类似的东西，就是namespace。Assembly是Java没有的东西。Assembly Name的属性包括Name, Version, Culture, PublicKey(Token)。 如果两个Assembly有一个属性不一样，那么它们就认为是不一样的。两个Type如果有同样的Class name，但是Assembly Name不一样的话，它们也认为是不一样的。

CLR的Type引用总是包括Class name 和 Assembly Name的。所以CLR在寻找一个Type的时候，主要是寻找Assembly。找到了Assembly之后，CLR看看这个Assembly里有没有这个Class。有的话就万事大吉，没有的话就是TypeLoadException。

CLR区别Type主要靠Assembly。Assembly有PublicKey(Token)。别人无法冒充你。所以象Java里的namespace的问题就不存在了。Assembly有自己的Version。你要第二版的时候CLR不会装载第一版给你。所以象Java Class Loader那样的限制，责任和问题都不存在了。从这个角度上讲，这正是为什么CLR里没有象Java Class Loader那样的东西。

但并不是说你就不能动态装载了，而是说你动态装载的时候就不用考虑那些垃圾了。

关于共享的问题，因为没有了你自己的Class loader，所以任何Type都可以直接对话。同样的Type也不会因为Class Loader不一样而被认为不一样。

CLR Loader有自己的规则怎么寻找Assembly。它先看GAC，再看你的程序目录。都没有的话它还有一个AssemblyResolveEvent去问你，看你能不能提供。如果你要动态装载的话，你有Assembly.Load, Assembly.LoadFrom, Assembly.LoadFile, Assembly.Load(byte[])。你可以提供privateBin，你也可以提供codebase hint。你还能有policy。总之，你想从哪找，就可以去哪找。细节问题我就不多说了。你可以去看MSDN，Suzanne的Blog，Alan的Blog，和我的英文Blog.

结论，CLR Loader远比Java Class Loader要Secure, Powerful and Flexible。

The Eclipse Development Process calls for the creation of three Councils: Requirements, Planning and Architecture. Together, these Councils are responsible for the creation and maintenance of the Eclipse Roadmap. When approved by the Board, the intent of the Eclipse Roadmap is to communicate the directions in which the Eclipse community will be taking its projects over the next year. New projects started during the life of a Roadmap are expected to be consistent and complementary to the Roadmap.

This document is the first draft of the Eclipse Architecture Plan. We welcome your feedback on the Eclipse Foundation Newsgroup.

The Eclipse community is organized into six top-level projects, each of which has provided its detailed architecture using the links below. Alternatively, you may use the image map below to navigate the Eclipse architecture.

• Platform Project
• Tools Project
• Test and Performance Tools Platform Project
• Web Tools Platform Project
• Business Intelligence and Reporting Tools Project
• Technology Project

Other projects may be added over time.

鼠标快捷键

下面是 Mozilla Firefox里面最常用的鼠标快捷键, 相对应的是 Internet Explorer 和 Opera 中的快捷键. 这些快捷键是 Windows 下的， 但大部分在 linux 下也可以使用.

¹: 默认情况下，鼠标中键或 Ctrl+Left-click 是在后台标签页打开链接 (打开后仍定位于你原先的页面 )  Shift+鼠标中键或 Shift+Ctrl+Left-click 是在前台标签页打开链接. 想得到相反的效果, 选择“工具” > 选项... > 基本信息 > 窗口 不选“ 在后台打开链接选项.

下面列出的是那些没有被包含在增强版中，但是仍然推荐安装的插件：

下面是 Mozilla Firefox最常用的键盘快捷键的列表以及它们在,  Internet Explorer 和 Opera 中相对应的快捷键.

¹: 这个快捷键只在windows中起作用.