Ubiquitous Build System

Research projects with available code are useful, Open Source is awesome, Free Software is even more, but ultimately building any piece of software has never been as easy as one wish.

The fact is that there is such a miscellaneous of tools to build software that it becomes confusing to build. It wastes developers time and decreases code reuse. Some companies and projects keep a repository with all needed code, external binaries and create build scripts and configurations so that any developer can easily rebuild any part of the project. If you ever worked on such company or project, you know how useful those can be when compared to a miscellaneous of tools available on the wild.

As most developers I am not specially interested in building details, I just wish for a building system to work. Though as an engineer I do wonder what properties such system should have. On the rest of this post I try to specify what requirements I foresee:

Time Persistent

Languages, compilers and other tools disappear and lose support with time. Nonetheless if once a given target could be build it should be possible to build it again [given access to sources has not been lost].

Additionally the build specification language should itself be time-proof, last thing you want to hear is that the target you wanted to build is not available for your updated build system.

Detached

It is not to expect that all projects will use the same build description language or that they will keep two or more specifications in sync. As so it should be possible to have a build description detached from the project source itself. In such a scenario, even if the build language is never adopted wildly it will still be useful.

With time, several people would start to write build specifications for projects that do not have one and community maintained specifications would become incredibly useful.

Building is More than Compiling

Any sufficiently advanced project has needs beyond compiling and linking source code. In a certain way building is a flowing data through a set of processes to produce a certain target.

This probably means that the build specification should be fairly flexible, e.g. build on top of programming language such as scons is built on python.

Distributed

By distributed I don't mean the specific capability to build in a distributed system, but the build specification to be distributed leading (for example) to the following interesting properties: avoiding one to keep a local copy, allowing anyone to easily re-use and share any target.

A possibility would be to build on top of the web, and allow people to do things like: "build http://project1/foobar", which could then import dependencies from http://project2/libXYZ

Disclaimer: This post has been on the draft list since September 2011, but I just cleaned and published it now. I did wonder about many others issues such as security, exact reproducible builds, ability to inject modified components in a build target (e.g. you might simply want to use a library that gives API x... not necessarily have to build one from source or even have to build exactly the version imported by the original specification), which language it should be based on, what could be cached and shared, and so on... Although I then realized I should just keep this post clean and small. So I just cut text out of it and maybe will I blog more or chat about those if the opportunity arrives.