Script helpers

releng-tool provides a series of helper functions which can be used in script-based packages, post-processing and more. Helper functions provided are listed below:

Available functions

releng_tool.debug(msg, *args)

Logs a debug message to standard out with a trailing new line. By default, debug messages will not be output to standard out unless the instance is configured with debugging enabled.

debug('this is a debug message')
Parameters:

  • msg – the message

  • *args – an arbitrary set of positional and keyword arguments used when generating a formatted message

releng_tool.err(msg, *args)

Logs an error message to standard error with a trailing new line and (if enabled) a red colorization.

err('this is an error message')
Parameters:

  • msg – the message

  • *args – an arbitrary set of positional and keyword arguments used when generating a formatted message

releng_tool.hint(msg, *args)

Added in version 0.13.

Changed in version 1.4: Ensure availability in script helpers.

Logs a hint message to standard out with a trailing new line and (if enabled) a cyan colorization.

hint('this is a hint message')
Parameters:

  • msg – the message

  • *args – an arbitrary set of positional and keyword arguments used when generating a formatted message

releng_tool.log(msg, *args)

Logs a (normal) message to standard out with a trailing new line.

log('this is a message')
Parameters:

  • msg – the message

  • *args – an arbitrary set of positional and keyword arguments used when generating a formatted message

releng_tool.note(msg, *args)

Logs a notification message to standard out with a trailing new line and (if enabled) an inverted colorization.

note('this is a note message')
Parameters:

  • msg – the message

  • *args – an arbitrary set of positional and keyword arguments used when generating a formatted message

releng_tool.releng_cat(file, *args)

Added in version 0.11.

Attempts to read one or more files provided to this call. For each file, it will be read and printed out to the standard output.

An example when using in the context of script helpers is as follows:

releng_cat('my-file')
Parameters:

  • file – the file

  • *args (optional) – additional files to include

Returns:

True if all the files exists and were printed to the standard output; False if one or more files could not be read

releng_tool.releng_copy(src, dst, quiet=False, critical=True, dst_dir=None, nested=False)

Changed in version 0.12: Add support for dst_dir.

Changed in version 1.4: Add support for nested.

This call will attempt to copy a provided file, directory’s contents or directory itself (if nested is True); defined by src into a destination file or directory defined by dst. If src is a file, then dst is considered to be a file or directory; if src is a directory, dst is considered a target directory. If a target directory or target file’s directory does not exist, it will be automatically created. In the event that a file or directory could not be copied, an error message will be output to standard error (unless quiet is set to True). If critical is set to True and the specified file/directory could not be copied for any reason, this call will issue a system exit (SystemExit).

An example when using in the context of script helpers is as follows:

# (stage)
# my-file
releng_copy('my-file', 'my-file2')
# (stage)
# my-file
# my-file2
releng_copy('my-file', 'my-directory/')
# (stage)
# my-directory/my-file
# my-file
# my-file2
releng_copy('my-directory/', 'my-directory2/')
# (stage)
# my-directory/my-file
# my-directory2/my-file
# my-file
# my-file2
Parameters:

  • src – the source directory or file

  • dst – the destination directory or file* (*if src is a file)

  • quiet (optional) – whether or not to suppress output

  • critical (optional) – whether or not to stop execution on failure

  • dst_dir (optional) – force hint that the destination is a directory

  • nested (optional) – nest a source directory into the destination

Returns:

True if the copy has completed with no error; False if the copy has failed

Raises:

SystemExit – if the copy operation fails with critical=True

releng_tool.releng_copy_into(src, dst, quiet=False, critical=True, nested=False)

Added in version 0.13.

Changed in version 1.4: Add support for nested.

This call will attempt to copy a provided file, directory’s contents or directory itself (if nested is True); defined by src into a destination directory defined by dst. If a target directory does not exist, it will be automatically created. In the event that a file or directory could not be copied, an error message will be output to standard error (unless quiet is set to True). If critical is set to True and the specified file/directory could not be copied for any reason, this call will issue a system exit (SystemExit).

An example when using in the context of script helpers is as follows:

# (stage)
# my-file
releng_copy_into('my-file', 'my-directory')
# (stage)
# my-directory/my-file
# my-file
releng_copy_into('my-directory', 'my-directory2')
# (stage)
# my-directory/my-file
# my-directory2/my-file
# my-file
releng_copy_into('my-directory', 'my-directory3', nested=True)
# (stage)
# my-directory/my-file
# my-directory2/my-file
# my-directory3/directory/my-file
# my-file
Parameters:

  • src – the source directory or file

  • dst – the destination directory

  • quiet (optional) – whether or not to suppress output

  • critical (optional) – whether or not to stop execution on failure

  • nested (optional) – nest a source directory into the destination

Returns:

True if the copy has completed with no error; False if the copy has failed

Raises:

SystemExit – if the copy operation fails with critical=True

releng_tool.releng_env(key, value=<object object>)

Added in version 0.3.

Provides a caller a simple method to fetch or configure an environment variable for the current context. This call is the same as if one directly fetched from or managed a key-value with os.environ. If value is not provided, the environment variable’s value (if set) will be returned. If value is set to a value of None, any set environment variable will be removed.

An example when using in the context of script helpers is as follows:

# get an environment variable
value = releng_env('KEY')

# set an environment variable
releng_env('KEY', 'VALUE')
Parameters:

  • key – the environment key

  • value (optional) – the environment value to set

Returns:

the value of the environment variable

releng_tool.releng_execute(args, cwd=None, env=None, env_update=None, quiet=None, critical=True, poll=False, capture=None, expand=True, args_str=False)

Changed in version 1.13: Add support for expand.

Changed in version 1.14: Add support for args_str.

Runs the command described by args until completion. A caller can adjust the working directory of the executed command by explicitly setting the directory in cwd. The execution request will return True on a successful execution; False if an issue has been detected (e.g. bad options or called process returns a non-zero value). In the event that the execution fails, an error message will be output to standard error unless quiet is set to True.

The environment variables used on execution can be manipulated in two ways. First, the environment can be explicitly controlled by applying a new environment content using the env dictionary. Key of the dictionary will be used as environment variable names, whereas the respective values will be the respective environment variable’s value. If env is not provided, the existing environment of the executing context will be used. Second, a caller can instead update the existing environment by using the env_update option. Like env, the key-value pairs match to respective environment key-value pairs. The difference with this option is that the call will use the original environment values and update select values which match in the updated environment request. When env and env_update are both provided, env_update will be updated the options based off of env instead of the original environment of the caller.

If critical is set to True and the execution fails for any reason, this call will issue a system exit (SystemExit). By default, the critical flag is enabled (i.e. critical=True).

In special cases, an executing process may not provide carriage returns/new lines to simple output processing. This can lead the output of a process to be undesirably buffered. To workaround this issue, the execution call can instead poll for output results by using the poll option with a value of True. By default, polling is disabled with a value of False.

A caller may wish to capture the provided output from a process for examination. If a list is provided in the call argument capture, the list will be populated with the output provided from an invoked process.

An example when using in the context of script helpers is as follows:

releng_execute(['echo', '$TEST'], env={'TEST': 'this is a test'})
Parameters:

  • args – the list of arguments to execute

  • cwd (optional) – working directory to use

  • env (optional) – environment variables to use for the process

  • env_update (optional) – environment variables to append for the process

  • quiet (optional) – whether or not to suppress output (defaults to False)

  • critical (optional) – whether or not to stop execution on failure (defaults to True)

  • poll (optional) – force polling stdin/stdout for output data (defaults to False)

  • capture (optional) – list to capture output into

  • expand (optional) – perform variable expansion on arguments

  • args_str (optional) – invoke arguments as a single string

Returns:

True if the execution has completed with no error; False if the execution has failed

Raises:

SystemExit – if the execution operation fails with critical=True

releng_tool.releng_execute_rv(command, *args, **kwargs)

Added in version 0.8.

Runs the command command with provided args until completion. A caller can adjust the working directory of the executed command by explicitly setting the directory in cwd. The execution request will return the command’s return code as well as any captured output.

The environment variables used on execution can be manipulated in two ways. First, the environment can be explicitly controlled by applying a new environment content using the env dictionary. Key of the dictionary will be used as environment variable names, whereas the respective values will be the respective environment variable’s value. If env is not provided, the existing environment of the executing context will be used. Second, a caller can instead update the existing environment by using the env_update option. Like env, the key-value pairs match to respective environment key-value pairs. The difference with this option is that the call will use the original environment values and update select values which match in the updated environment request. When env and env_update are both provided, env_update will be updated the options based off of env instead of the original environment of the caller.

An example when using in the context of script helpers is as follows:

rv, out = releng_execute_rv('echo', '$TEST', env={'TEST': 'env-test'})
Parameters:

  • command – the command to invoke

  • *args (optional) – arguments to add to the command

  • **cwd – working directory to use

  • **env – environment variables to use for the process

  • **env_update – environment variables to append for the process

  • **expand – perform variable expansion on arguments

  • **args_str – invoke arguments as a single string

Returns:

the return code and output of the execution request

releng_tool.releng_exists(path, *args)

Allows a caller to verify the existence of a file on the file system. This call will return True if the path exists; False otherwise.

An example when using in the context of script helpers is as follows:

if releng_exists('my-file'):
    print('the file exists')
else:
    print('the file does not exist')
Parameters:

  • path – the path

  • *args – additional path parts

Returns:

True if the path exists; False otherwise

releng_tool.releng_exit(msg=None, code=None)

Provides a convenience method to help invoke a system exit call without needing to explicitly use sys. A caller can provide a message to indicate the reason for the exit. The provide message will output to standard error. The exit code, if not explicit set, will vary on other arguments. If a message is provided to this call, the default exit code will be 1. If no message is provided, the default exit code will be 0. In any case, if the caller explicitly sets a code value, the provided code value will be used.

An example when using in the context of script helpers is as follows:

releng_exit('there was an error performing this task')
Parameters:

  • msg (optional) – error message to print

  • code (optional) – exit code; defaults to 0 if no message or defaults to 1 if a message is set

Raises:

SystemExit – always raised

releng_tool.releng_expand(obj, kv=None)

This expand utility method will attempt to expand variables in detected string types. For a detected string which contains substrings in the form of $value or ${value}, these substrings will be replaced with their respective key-value (if provided) or environment variable value. For substrings which do not have a matching variable value, the substrings will be replaced with an empty value. If a dictionary is provided, keys and values will be checked if they can be expanded on. If a list/set is provided, each value which be checked if it can be expanded on. If a dictionary key is expanded to match another key, a key-value pair can be dropped. If a set may result in a smaller set if expanded values result in duplicate entries.

An example when using in the context of script helpers is as follows:

import os
...

os.environ['MY_ENV'] = 'my-environment-variable'
value = releng_expand('$MY_ENV')
print(value)
# will output: my-environment-variable
Parameters:

  • obj – the object

  • kv (optional) – key-values pairs to use

Returns:

the expanded object

releng_tool.releng_include(file_path)

Added in version 0.12.

The provided call will execute code at the provided file path. The path will be relative to the caller’s script, unless an absolute path is provided. The executed script will be initialized with globals matching the caller’s script.

An example when using in the context of script helpers is as follows:

# load "my-other-script" found alongside the current script
releng_include('my-other-script')
Parameters:

file_path – the script to invoke

releng_tool.releng_join(path, *paths)

An alias for os.path.join. See also https://docs.python.org/library/os.path.html#os.path.join.

releng_tool.releng_ls(dir_)

Added in version 0.11.

Attempts to read a directory for its contents and prints this information to the configured standard output stream.

An example when using in the context of script helpers is as follows:

releng_ls('my-dir/')
Parameters:

dir – the directory

Returns:

True if the directory could be read and its contents have been printed to the standard output; False if the directory could not be read

releng_tool.releng_mkdir(dir_, *args, **kwargs)

Added in version 0.3.

Changed in version 0.13: Add support for critical.

Changed in version 1.3: Accepts multiple paths components.

Changed in version 1.3: Returns the created path.

Attempts to create the provided directory. If the directory already exists, this method has no effect. If the directory does not exist and could not be created, this method will return None. Also, if an error has been detected, an error message will be output to standard error (unless quiet is set to True).

An example when using in the context of script helpers is as follows:

if releng_mkdir('my-directory'):
    print('directory was created')
else:
    print('directory was not created')

target_dir = releng_mkdir(TARGET_DIR, 'sub-folder')
if target_dir:
    # output] target directory: <target>/sub-folder
    print('target directory:', target_dir)
else:
    print('directory was not created')
Parameters:

  • dir – the directory

  • *args (optional) – additional components of the directory

  • **quiet (optional) – whether or not to suppress output (defaults to False)

  • **critical (optional) – whether or not to stop execution on failure (defaults to False)

Returns:

the directory that exists; None if the directory could not be created

releng_tool.releng_move(src, dst, quiet=False, critical=True, dst_dir=None, nested=False)

Changed in version 0.14: Add support for dst_dir.

Changed in version 1.4: Add support for nested.

This call will attempt to move a provided file, directory’s contents or directory itself (if nested is True); defined by src into a destination file or directory defined by dst. If src is a file, then dst is considered to be a file or directory; if src is a directory, dst is considered a target directory. If a target directory or target file’s directory does not exist, it will be automatically created.

In the event that a file or directory could not be moved, an error message will be output to standard error (unless quiet is set to True). If critical is set to True and the specified file/directory could not be moved for any reason, this call will issue a system exit (SystemExit).

An example when using in the context of script helpers is as follows:

# (input)
# my-directory/another-file
# my-file
# my-file2
releng_move('my-file', 'my-file3')
releng_move('my-directory/', 'my-directory2/')
releng_move('my-file2', 'my-directory2/')
# (output)
# my-directory2/another-file
# my-directory2/my-file2
# my-file3
Parameters:

  • src – the source directory or file

  • dst – the destination directory or file* (*if src is a file)

  • quiet (optional) – whether or not to suppress output

  • critical (optional) – whether or not to stop execution on failure

  • dst_dir (optional) – force hint that the destination is a directory

  • nested (optional) – nest a source directory into the destination

Returns:

True if the move has completed with no error; False if the move has failed

Raises:

SystemExit – if the copy operation fails with critical=True

releng_tool.releng_move_into(src, dst, quiet=False, critical=True, nested=False)

Added in version 0.14.

Changed in version 1.4: Add support for nested.

This call will attempt to move a provided file, directory’s contents or directory itself (if nested is True); defined by src into a destination directory defined by dst. If a target directory directory does not exist, it will be automatically created.

In the event that a file or directory could not be moved, an error message will be output to standard error (unless quiet is set to True). If critical is set to True and the specified file/directory could not be moved for any reason, this call will issue a system exit (SystemExit).

An example when using in the context of script helpers is as follows:

# (input)
# my-directory/another-file
# my-directory2/another-file2
# my-file
# my-file2
releng_move('my-file', 'my-file3')
releng_move('my-directory', 'my-directory3')
releng_move('my-file2', 'my-directory3')
releng_move('my-directory2', 'my-directory3', nested=True)
# (output)
# my-directory3/my-directory2/another-file2
# my-directory3/another-file
# my-directory3/my-file2
# my-file3
Parameters:

  • src – the source directory or file

  • dst – the destination directory

  • quiet (optional) – whether or not to suppress output

  • critical (optional) – whether or not to stop execution on failure

  • nested (optional) – nest a source directory into the destination

Returns:

True if the move has completed with no error; False if the move has failed

Raises:

SystemExit – if the copy operation fails with critical=True

releng_tool.releng_remove(path, quiet=False)

Attempts to remove the provided path if it exists. The path value can either be a directory or a specific file. If the provided path does not exist, this method has no effect. In the event that a file or directory could not be removed due to an error other than unable to be found, an error message will be output to standard error (unless quiet is set to True).

An example when using in the context of script helpers is as follows:

releng_remove('my-file')
# (or)
releng_remove('my-directory/')
Parameters:

  • path – the path to remove

  • quiet (optional) – whether or not to suppress output

Returns:

True if the path was removed or does not exist; False if the path could not be removed from the system

releng_tool.releng_require_version(version, quiet=False, critical=True)

Added in version 0.11.

Enables a caller to explicitly check for a required releng-tool version. Invoking this function with a dotted-separated version string, the string will be parsed and compared with the running releng-tool version. If the required version is met, this method will have no effect. In the event that the required version is not met, the exception SystemExit will be raised if the critical flag is set; otherwise this call will return False.

An example when using in the context of script helpers is as follows:

# ensure we are using releng-tool v1
releng_require_version('1.0.0')
Parameters:

  • version – dotted-separated version string

  • quiet (optional) – whether or not to suppress output

  • critical (optional) – whether or not to stop execution on failure

Returns:

True if the version check is met; False if the version check has failed

Raises:

SystemExit – if the version check fails with critical=True

releng_tool.releng_tmpdir(dir_=None)

Creates a temporary directory in the provided directory dir_ (or system default, is not provided). This is a context-supported call and will automatically remove the directory when completed. If the provided directory does not exist, it will created. If the directory could not be created, an FailedToPrepareBaseDirectoryError exception will be thrown.

An example when using in the context of script helpers is as follows:

with releng_tmpdir() as dir_:
    print(dir_)
Parameters:

dir (optional) – the directory to create the temporary directory in

Raises:

FailedToPrepareBaseDirectoryError – the base directory does not exist and could not be created

releng_tool.releng_touch(file)

Attempts to update the access/modifications times on a file. If the file does not exist, it will be created. This utility call operates in the same fashion as the touch system command.

An example when using in the context of script helpers is as follows:

if releng_touch('my-file'):
    print('file was created')
else:
    print('file was not created')
Parameters:

file – the file

Returns:

True if the file was created/updated; False if the file could not be created/updated

releng_tool.releng_wd(dir_)

Moves the current context into the provided working directory dir. When returned, the original working directory will be restored. If the provided directory does not exist, it will created. If the directory could not be created, an FailedToPrepareWorkingDirectoryError exception will be thrown.

An example when using in the context of script helpers is as follows:

with releng_wd('my-directory/'):
    # invoked in 'my-directory'

# invoked in original working directory
Parameters:

dir – the target working directory

Raises:

FailedToPrepareWorkingDirectoryError – the working directory does not exist and could not be created

releng_tool.success(msg, *args)

Logs a success message to standard error with a trailing new line and (if enabled) a green colorization.

success('this is a success message')
Parameters:

  • msg – the message

  • *args – an arbitrary set of positional and keyword arguments used when generating a formatted message

releng_tool.verbose(msg, *args)

Logs a verbose message to standard out with a trailing new line and (if enabled) an inverted colorization. By default, verbose messages will not be output to standard out unless the instance is configured with verbosity.

verbose('this is a verbose message')
Parameters:

  • msg – the message

  • *args – an arbitrary set of positional and keyword arguments used when generating a formatted message

releng_tool.warn(msg, *args)

Logs a warning message to standard error with a trailing new line and (if enabled) a purple colorization.

warn('this is a warning message')
Parameters:

  • msg – the message

  • *args – an arbitrary set of positional and keyword arguments used when generating a formatted message

Raises:

RelengToolWarningAsError – when warnings-are-errors is configured

Importing helpers

Scripts directly invoked by releng-tool will automatically have these helpers registered in the script’s globals module (i.e. no import is necessary). If a project defines custom Python modules in their project and wishes to take advantage of these helper functions, the following import can be used to, for example, import a specific function:

from releng_tool import releng_execute

Or, if desired, all helper methods can be imported at once:

from releng_tool import *