runperf.utils package

Submodules

runperf.utils.cloud_image_providers module

class runperf.utils.cloud_image_providers.BaseProvider(distro, arch, pub_key, base_path, session, setup_script)

Bases: object

Base provider to fetch and prepare a cloudinit image

get_url()

Return url to the base cloud image

static is_for(distro, arch)

Check whether this provider is valid for given combination

is_up_to_date()

Check whether base_path contains up-to-date distro of given arch

Parameters:

• distro – Version, eg. (RHEL-8.0.0-20200408.n.0)
• arch – Guest architecture (x86_64)
• base_path – Basic path to store images (/var/lib/libvirt/images)

Returns:

None when up to date, explanation why not otherwise

prepare(default_password)

Prepare the image for use

class runperf.utils.cloud_image_providers.Fedora(distro, arch, pub_key, base_path, session, setup_script)

Bases: runperf.utils.cloud_image_providers.BaseProvider

Fedora image provider

get_url()

Return url to the base cloud image

static is_for(distro, arch)

Check whether this provider is valid for given combination

runperf.utils.pbench module

class runperf.utils.pbench.Dnf(session, extra=None, test=None)

Bases: object

Install pbench via “dnf” (Fedora/RHEL)

install()

Make sure pbench is installed and the default toolset is registered

runperf.utils.pbench.install_on(session, extra=None, test=None)

Try available providers to install pbench

runperf.utils.pbench.register_tools(session, tools, clients)

Unregister all tools and then register the provided ones

Module contents

class runperf.utils.ContextManager(log, root=None)

Bases: object

Object to keep track of the current path

get()

Get the current path

msg(msg)

Log a context message

profile = 0

set(level, name, msg=None)

Set the path to the $root + path

Parameters:

• level – current level, adds/removes the previous levels if needed use “-1” to only replace the last item of the current level
• name – name of the current level

set_level(level, msg=None)

Set the current level

set_root(root, msg=None)

Set new root location

store(path, content)

Append $content to a file inside a dir structure

test = 1

class runperf.utils.LogFetcher(paths=None, cmds=None, params=None)

Bases: object

Object that handles fetching logs or command outputs

check_errors(path)

Check for issues in the fetched files

Parameters:path – Path where outputs are stored

collect(path, host)

Collect all paths and command outputs

Parameters:

• path – Path to store the outpus locally
• host – machine.BaseMachine-like object to collect from

collect_cmds(out_path, host, cmds)

Collect the commands output from host

static collect_files(out_path, host, paths)

Fetch files from host

class runperf.utils.MutableShellSession(name, *args, **kwargs)

Bases: aexpect.client.ShellSession

Mute-able aexpect.ShellSession

runperf_stage(stage)

Mark a next stage

send(cont='')

Send a string to the child process.

Parameters:cont – String to send to the child process.

class runperf.utils.ThreadWithStatus(group=None, target=None, name=None, args=(), kwargs=None, *, daemon=None)

Bases: threading.Thread

Thread class that sets “self.completed” to True after execution

This constructor should always be called with keyword arguments. Arguments are:

group should be None; reserved for future extension when a ThreadGroup class is implemented.

target is the callable object to be invoked by the run() method. Defaults to None, meaning nothing is called.

name is the thread name. By default, a unique name is constructed of the form “Thread-N” where N is a small decimal number.

args is the argument tuple for the target invocation. Defaults to ().

kwargs is a dictionary of keyword arguments for the target invocation. Defaults to {}.

If a subclass overrides the constructor, it must make sure to invoke the base class constructor (Thread.__init__()) before doing anything else to the thread.

completed = False

exc = None

run()

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

runperf.utils.check_output(*args, **kwargs)

Execute command while masking stdin and providing better errors.

Parameters:

• args – args to be passed to subprocess.check_output
• kwargs – kwargs to be passed to subprocess.check_output * when stdin is not present, devnull is used * when stderr is not present, subprocess.STDOUT is used

Raises:

RuntimeError – In case of subprocess.CalledProcessError

runperf.utils.comma_separated_ranges_to_list(text)

Provides a list from comma separated ranges

Parameters:text – string of comma separated range Return list:list of integer values in comma separated range

runperf.utils.human_to_bool(value)

Accepts multiple human values and turns it into a boolean

runperf.utils.iter_tabular_output(matrix, header=None)

Generator for a pretty, aligned string representation of a nxm matrix.

This representation can be used to print any tabular data, such as database results. It works by scanning the lengths of each element in each column, and determining the format string dynamically.

Parameters:

• matrix – Matrix representation (list with n rows of m elements).
• header – Optional tuple or list with header elements to be displayed.

runperf.utils.list_dir_hashes(path)

Recursively hashes all files inside the path and reports them as a dict

Parameters:path – Path to be processed

runperf.utils.list_of_threads(cpus)

How many threads to use depending on no cpus

runperf.utils.named_entry_point(group, loaded_name)

Return first matching plugin for a given group based on loaded name

Parameters:

• group – entry-point group
• name – plugin.name of the loaded entry point

runperf.utils.random_string(length)

Generates string of random characters

Parameters:length – number or characters to generate

runperf.utils.read_file(path)

Read a file and return it’s content or -1 in case the file does not exists

runperf.utils.record_failure(path, exc, paths=None, details=None)

Record details about exception in a dir structure

Parameters:

• path – Path to create the ‘__error%d__’ dir with details in
• exc – Forwarded exception
• details – Extra human readable details
• paths – Paths to attach to this failure

runperf.utils.shell_find_command(session, command)

Helper to detect path to a command

Parameters:

• session – aexpect.ShellSession session
• command – command we are looking for

Returns:

path or empty string when not found

runperf.utils.shell_write_content_cmd(path, content, append=False)

Generate shell cmd to safely write/append content to file

runperf.utils.sorted_entry_points(group)

Return alphabetically sorted entry points for a given group

Parameters:group – entry-point group

runperf.utils.ssh_copy_id(log, addr, passwords, hop=None)

Use “ssh-copy-id” to copy ssh id, try passwords if asked for.

runperf.utils.string_to_safe_path(input_str)

Convert string to a valid file/dir name.

This takes a string that may contain characters that are not allowed on FAT (Windows) filesystems and/or ext3 (Linux) filesystems, and replaces them for safe (boring) underlines.

It limits the size of the path to be under 255 chars, and make hidden paths (starting with “.”) non-hidden by making them start with “_”.

Parameters:input_str – String to be converted Returns:String which is safe to pass as a file/dir name (on recent fs)

runperf.utils.tabular_output(matrix, header=None)

Pretty, aligned string representation of a nxm matrix.

This representation can be used to print any tabular data, such as database results. It works by scanning the lengths of each element in each column, and determining the format string dynamically.

Parameters:

• matrix – Matrix representation (list with n rows of m elements)
• header – Optional tuple or list with header elements to be displayed

Returns:

String with the tabular output, lines separated by unix line feeds

runperf.utils.wait_for(func, timeout, step=1.0, args=None, kwargs=None)

Wait until func() evaluates to True.

If func() evaluates to True before timeout expires, return the value of func(). Otherwise return None.

Parameters:

• timeout – Timeout in seconds
• first – Time to sleep before first attempt
• step – Time to sleep between attempts in seconds
• text – Text to print while waiting, for debug purposes
• args – Positional arguments to func
• kwargs – Keyword arguments to func

runperf.utils.wait_for_machine_calms_down(session, timeout=600)

Wait until 1m system load calms below 1.0

Parameters:

• session – session
• timeout – timeout

Returns:

True on success, False when it’s still busy

runperf.utils.write_file(path, content, mode='w')

Write content to path, create the necessary upper dirs