The muddle release mechanism¶

Summary¶

The muddle release mechanism is intended to be useful in preparing some form of meaningful binary release for a customer. It is more specific than can be obtained using muddle distribute, and also has more mechanism associated with it.

The normal pattern of usage is expected to be:

Produce a build that needs releasing to the customer.
Define what is to be released, in the build description (write a release_from() method).

Verify the build tree is up-to-date and builds the entities to be released:

$ cd build-dir
$ muddle pull _all
$ muddle veryclean
$ muddle build _release

Produce a release file, and ideally archive it:

$ muddle stamp release simple v1.0
$ pushd versions
$ git commit simple_v1.0.release -m 'Release file for simple v1.0'
$ muddle stamp push

(a more detailed commit message would doubtless be a good thing.)

Test the release mechanism in the current build tree (this doesn’t prove the build works, because it doesn’t check the whole tree out anew and try building it from scratch, so it mustn’t be used as the final release):
```
$ muddle release -test versions/simple_v1.0.release
```
If that shows problems, repeat from an earlier stage.

In order to produce an actual release, create a clean working directory and prepare the release there, using the same release file:
```
$ cd ..
$ mkdir release-dir
$ cd release-dir
$ muddle release ../build-dir/versions/simple_v1.0.release
```
Send the customer the resultant archive file, which will be called something like:
```
simple_v1.0_40c60888d187c4e639820e77bd9532d007f74f92.tgz
```

Muddle release files¶

Muddle releases are defined in release files, which are muddle stamp files (specifically, a stamp version files) with extra fields (at the start):

[RELEASE]
version = 1.2
name = simple
archive = tar
compression = gzip

See the DESCRIPTION section of muddle doc version_stamp for details on the content of stamp files.

Release files are created using muddle release, and put into the “versions/” directory. muddle stamp release will automatically add them to the version control system in that directory, just as muddle stamp version does for its stamp files. They can then be committed, and then pushed with muddle stamp push.

Explicit version numbering¶

The command:

$ muddle stamp release Project99 1.2

creates a release stamp file called:

versions/Project99_1.2.release

which looks like a normal version 2 stamp file with an extra section after the [STAMP] section:

[RELEASE]
name = Project99
version = 1.2
archive = tar
compression = gzip

Guessing the next version number¶

Since release files are named consistently, and are version controlled in the “versions/” directory, muddle should be able to make a sensible guess at the next version number to use.

Thus:

$ muddle stamp release Project99 -next

will look through the “versions/” directory for Project99 release files, which are called:

Project99_v<major>.<minor>.release

where <major> and <minor> are release numbers.

Note

Leading zeroes are ignored, so 1.01 is the same as 1.1, and indeed 001.001 is also the same as 1.1.

The current “guessing” mechanism only supports two-part version numbers.

So if the “versions/” directory contains:

Project88.v28.1.release
Project99_v1.1.release
Project99_v13.2.release
Project99_v22.3.release
Project99_v27.1.release
Project99_v28.release

then the next version for Project99 will be 27.2.

The Project88 file will be ignored, as it has the wrong name, and the Project99_v28.release file is ignored because its release number is not of the form <major>.<minor>.

Archive and compression switches¶

You can use the -archive and -compression switches to specify the release archive mechanism and release compression - for instance:

$ muddle stamp release Project99 1.2.3 -archive tar -compression bzip2

Template release files¶

It is also possible to create a “template” release file, with unspecified release name and version:

$ muddle stamp release -template

which creates a file called:

versions/this-is-not-a-file-name.release

with the [RELEASE] section set to:

[RELEASE]
name = <REPLACE THIS>
version = <REPLACE THIS>
archive = tar
compression = gzip

The user is expected to rename the file and edit the <REPLACE THIS> values to something sensible. The muddle release command will not accept release files where these fields have not been edited.

Note

Both release name and release version follow the same rules:

the first character must be an ASCI alphanumeric (‘A’-‘Z’, ‘a’-‘z’, ‘0’-‘9’)
any following characters must be ASCII alphanumeric, ‘-‘, ‘_’ or ‘.’

The release archive must currently be “tar”.

The release compression must currently be one of “gzip” or “bzip2”. The default is “gzip”.

More information¶

See muddle help stamp release and muddle help stamp push for more information.

The `_release` build target¶

This is a build target specifically for use in muddle build _release, indicating what is to be built for a release.

Its meaning is defined in the build description, using builder.add_to_release_build().

Items can be added to the _release entity as labels, individually:

builder.add_to_release_build(package('fred', 'x86')

or as sequences:

builder.add_to_release_build([package('fred', 'x86'),
                              Label.from_string('package:(infrastructure)jim{*}/*')])

Wildcarded labels can be used, if necessary.

The “special” values that start with an underscore may also be used, for instance:

builder.add_to_release_build('_default_deployments')

Warning

The values _all, release and just_pulled may not be used:

The interpretation of _all depends on the muddle command being used - use _all_checkouts, _all_packages or _all_deployments instead.
Using _release inside _release would, of course, just be asking for trouble.
And just_pulled is too transient to be useful as a release target.

The _release value is interpreted and expanded lazily, when the muddle command line is interpreted. This means one is allowed to specify:

builder.add_to_release_build('_default_deployments')

in the build description before the deployments have all been described, which is useful.

Warning

This also means that add_to_release_build() itself cannot complain if you add a non-existent label. Such warnings will have to wait until _release is actually used in a “muddle” command.

muddle query release can be used to find out what _release is set to (see muddle help query release for details).

Warning

If your build includes subdomains, then you need to know that only calls to add_to_release_build() in the top-level build will be effective. Calls in subdomain build descriptions will be ignored. It is up to the top-level build to explicitly include anything it wishes to release from the subdomain.

This decision is made to avoid confusion over what is meant by adding special names (such as _all_checkouts) to the release specification in a subdomain. It is possible that a future version of muddle might instead expand a subdomain’s _release value as the subdomain was “promoted”, but I am not sure that this is worth the added complication.

There is also a strong argument that only the top-level build description can have full awareness of what should actually be released.

The `release_from` function in the build description¶

The _release special name describes what is to be built for a release, but another mechanism is needed to tell muddle what to do with the results.

The release_from() function in the build description is responsible for copying files into the release directory (which will become the release tarball). For instance:

import shutil

def release_from(builder, release_dir):

    f0 = package('first_pkg', 'x86')
    m0 = package('main_pkg', 'x86')
    s1 = package('second_pkg', 'x86', domain='subdomain1')

    install_path = builder.package_install_path

    # Copy our executables, with permissions intact
    shutil.copy(os.path.join(install_path(f0), 'first'), release_dir)
    shutil.copy(os.path.join(install_path(m0), 'main0'), release_dir)
    shutil.copy(os.path.join(install_path(s1), 'second'), release_dir)

The normal build description will have been loaded before this function is called, so it is safe to assume that builder has its normal content.

Also, muddle will have added the build description’s checkout directory to the Python path before calling it (so other Python files therein can be imported), just as it does before calling describe_to.

Muddle will already have copied the release file into the release directory, but otherwise it will be empty.

The following methods may be particularly useful:

builder.checkout_path
builder.package_obj_path
builder.package_install_path
builder.deploy_path

Use muddle doc <method> to confirm what they do.

Note

muddle bootstrap puts an empty release_from function into its template build description.

Warning

If your build includes subdomains, then you need to know that only the release_from() function in the top-level build will be executed. Any release_from() functions in subdomain build descriptions will not be called by muddle itself.

The `muddle release` command¶

The muddle release command actually makes a release.

For example:

$ muddle release project99-1.2.release

This:

Checks that the current directory is empty, and refuses to proceed if it is not.

We always recommend doing muddle init or muddle bootstrap in an empty directory, but muddle insists that muddle release must be done in an empty directory.
Does muddle unstamp <release-file>,
Copies the release file to .muddle/Release, and the release specification to .muddle/ReleaseSpec. The existence of the former indicates that this is a release build tree, and “normal” muddle will refuse to build in it.

The ReleaseSpec contains the basic information describing a release (the name, version, archive and compression mechanisms, and the SHA1 hash of the release file).
Sets some extra environment variables, which can be used in the normal manner in muddle Makefiles:
- MUDDLE_RELEASE_NAME is the release name, from the release file.
- MUDDLE_RELEASE_VERSION is the release version, from the release file.
- MUDDLE_RELEASE_HASH is the SHA1 hash of the release file
“Normal” muddle will also create those environment variables, but they will be set to (unset).
Does mudddle build _release.
Creates the release directory, which will be called <release-name>_<release-version>_<release-sha1>. It copies the release file therein.
Calls the release_from(builder, release_dir) function in the build description, which is responsible for copying across whatever else needs to be put into the release directory.

(Obviously it is an error if the build description does not have such a function.)
Creates a compressed tarball of the release directory, using the compression mechanism specified in the release file. It will have the same basename as the release directory.

The existence of this latter file indicates that this is a release build tree, and some muddle commands will thus refuse to work in it (notably, anything to do with pushing to or pulling from a VCS).

(Of course, the user can delete the file, but if they do then that’s their responsibility.)

The -test variant (muddle release -test) omits the first two stages, and may thus be done in the working build tree. It still copies the Release and ReleaseSpec files into the .muddle directory, and thus marks the build as a release build - this can be undone by deleting .muddle/Release.

Warning

Do not release a test release. Always prepare a proper release created in an empty directory. Test releases cannot be trusted as the build tree may still contain artefacts from earlier development, and a clean build may or may not work.

Other useful commands¶

Whether a build is a release build, the content of the release specification, and what _release is defined as, can be found using:

$ muddle query release

You can find out exactly what labels will be built in the normal manner, using:

$ muddle -n build _release

Since a release file is an extended stamp file, it is perfectly legitimate to use it as such:

$ muddle unstamp simple_v1.0.release

This will tell you that it is ignoring the RELEASE section, which it has no use for.

Version include files¶

If you want to produce a version.h file, so that C or C++ programs can report the build version, then you can use muddle subst and the release specific environment variables.

For instance, create a template version.h.in file for “project99”:

#ifndef PROJECT99_VERSION_FILE
#define PROJECT99_VERSION_FILE
#define BUILD_VERSION "${MUDDLE_RELEASE_NAME}: $(MUDDLE_RELEASE_VERSION}"
#endif

(of course, a string constant might be more appropriate, depending on the language and situation).

In the Makefile.muddle, then, for instance, alter:

.PHONY: config
 config:

to be:

.PHONY: config
 config: $(MUDDLE_OBJ_INCLUDE)/version.h

and add a new rule:

$(MUDDLE_OBJ_INCLUDE)/version.h: version.h.in
    -mkdir -p $(MUDDLE_OBJ_INCLUDE)
    $(MUDDLE) subst $< $@

Remember that the MUDDLE_RELEASE_ environment variables will be set to (unset) when the build is not a release build.

The muddle release mechanism¶

Summary¶

Muddle release files¶

Explicit version numbering¶

Guessing the next version number¶

Archive and compression switches¶

Template release files¶

More information¶

The _release build target¶

The release_from function in the build description¶

The muddle release command¶

Other useful commands¶

Version include files¶

The `_release` build target¶

The `release_from` function in the build description¶

The `muddle release` command¶