runperf.utils package

Submodules

runperf.utils.cloud_image_providers module

class runperf.utils.cloud_image_providers.BaseProvider(distro, arch, pub_key, base_path, session, setup_script)

Bases: object

Base provider to fetch and prepare a cloudinit image

get_url()

Return url to the base cloud image

static is_for(distro, arch)

Check whether this provider is valid for given combination

is_up_to_date()

Check whether base_path contains up-to-date distro of given arch

Parameters:

  • distro – Version, eg. (RHEL-8.0.0-20200408.n.0)

  • arch – Guest architecture (x86_64)

  • base_path – Basic path to store images (/var/lib/libvirt/images)

Returns:

None when up to date, explanation why not otherwise

prepare(default_password)

Prepare the image for use

class runperf.utils.cloud_image_providers.Fedora(distro, arch, pub_key, base_path, session, setup_script)

Bases: BaseProvider

Fedora image provider

get_url()

Return url to the base cloud image

static is_for(distro, arch)

Check whether this provider is valid for given combination

runperf.utils.pbench module

class runperf.utils.pbench.Dnf(session, extra=None, test=None)

Bases: object

Install pbench via “dnf” (Fedora/RHEL)

install()

Make sure pbench is installed and the default toolset is registered

runperf.utils.pbench.install_on(session, extra=None, test=None)

Try available providers to install pbench

runperf.utils.pbench.register_tools(session, tools, clients)

Unregister all tools and then register the provided ones

Module contents

class runperf.utils.ContextManager(log, root=None)

Bases: object

Object to keep track of the current path

get()

Get the current path

msg(msg)

Log a context message

profile = 0
set(level, name, msg=None)

Set the path to the $root + path

Parameters:

  • level – current level, adds/removes the previous levels if needed use “-1” to only replace the last item of the current level

  • name – name of the current level

set_level(level, msg=None)

Set the current level

set_root(root, msg=None)

Set new root location

store(path, content)

Append $content to a file inside a dir structure

test = 1
class runperf.utils.LogFetcher(paths=None, cmds=None, params=None)

Bases: object

Object that handles fetching logs or command outputs

check_errors(path)

Check for issues in the fetched files

Parameters:

path – Path where outputs are stored

collect(path, host)

Collect all paths and command outputs

Parameters:

  • path – Path to store the outpus locally

  • host – machine.BaseMachine-like object to collect from

collect_cmds(out_path, host, cmds)

Collect the commands output from host

static collect_files(out_path, host, paths)

Fetch files from host

class runperf.utils.MutableShellSession(name, *args, **kwargs)

Bases: ShellSession

Mute-able aexpect.ShellSession

Initialize the class and run command as a child process.

Parameters:

  • command – Command to run, or None if accessing an already running server.

  • a_id – ID of an already running server, if accessing a running server, or None if starting a new one.

  • auto_close – If True, close() the instance automatically when its reference count drops to zero (default True).

  • echo – Boolean indicating whether echo should be initially enabled for the pseudo terminal running the subprocess. This parameter has an effect only when starting a new server.

  • linesep – Line separator to be appended to strings sent to the child process by sendline().

  • termination_func – Function to call when the process exits. The function must accept a single exit status parameter.

  • termination_params – Parameters to send to termination_func before the exit status.

  • output_func – Function to call whenever a line of output is available from the STDOUT or STDERR streams of the process. The function must accept a single string parameter. The string does not include the final newline.

  • output_params – Parameters to send to output_func before the output line.

  • output_prefix – String to prepend to lines sent to output_func.

  • prompt – Regular expression describing the shell’s prompt line.

  • status_test_command – Command to be used for getting the last exit status of commands run inside the shell (used by cmd_status_output() and friends).

  • pass_fds – Optional sequence of file descriptors to keep open between the parent and child.

  • encoding – Override text encoding (by default: autodetect by locale.getpreferredencoding())

runperf_stage(stage)

Mark a next stage

send(cont='')

Send a string to the child process.

Parameters:

cont – String to send to the child process.

class runperf.utils.ThreadWithStatus(group=None, target=None, name=None, args=(), kwargs=None, *, daemon=None)

Bases: Thread

Thread class that sets “self.completed” to True after execution

This constructor should always be called with keyword arguments. Arguments are:

group should be None; reserved for future extension when a ThreadGroup class is implemented.

target is the callable object to be invoked by the run() method. Defaults to None, meaning nothing is called.

name is the thread name. By default, a unique name is constructed of the form “Thread-N” where N is a small decimal number.

args is a list or tuple of arguments for the target invocation. Defaults to ().

kwargs is a dictionary of keyword arguments for the target invocation. Defaults to {}.

If a subclass overrides the constructor, it must make sure to invoke the base class constructor (Thread.__init__()) before doing anything else to the thread.

completed = False
exc = None
run()

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

runperf.utils.check_output(*args, **kwargs)

Execute command while masking stdin and providing better errors.

Parameters:

  • args – args to be passed to subprocess.check_output

  • kwargs – kwargs to be passed to subprocess.check_output * when stdin is not present, devnull is used * when stderr is not present, subprocess.STDOUT is used

Raises:

RuntimeError – In case of subprocess.CalledProcessError

runperf.utils.comma_separated_ranges_to_list(text)

Provides a list from comma separated ranges

Parameters:

text – string of comma separated range

Return list:

list of integer values in comma separated range

runperf.utils.human_to_bool(value)

Accepts multiple human values and turns it into a boolean

runperf.utils.iter_tabular_output(matrix, header=None)

Generator for a pretty, aligned string representation of a nxm matrix.

This representation can be used to print any tabular data, such as database results. It works by scanning the lengths of each element in each column, and determining the format string dynamically.

Parameters:

  • matrix – Matrix representation (list with n rows of m elements).

  • header – Optional tuple or list with header elements to be displayed.

runperf.utils.list_dir_hashes(path)

Recursively hashes all files inside the path and reports them as a dict

Parameters:

path – Path to be processed

runperf.utils.list_of_threads(cpus)

How many threads to use depending on no cpus

runperf.utils.named_entry_point(group, loaded_name)

Return first matching plugin for a given group based on loaded name

Parameters:

  • group – entry-point group

  • name – plugin.name of the loaded entry point

runperf.utils.random_string(length)

Generates string of random characters

Parameters:

length – number or characters to generate

runperf.utils.read_file(path)

Read a file and return it’s content or -1 in case the file does not exists

runperf.utils.record_failure(path, exc, paths=None, details=None)

Record details about exception in a dir structure

Parameters:

  • path – Path to create the ‘__error%d__’ dir with details in

  • exc – Forwarded exception

  • details – Extra human readable details

  • paths – Paths to attach to this failure

runperf.utils.shell_dnf_install_cmd(pkgs)

Helper to generate dnf command to install rpms via dnf

Parameters:

pkgs – List of pkg names to be installed

Returns:

Command to install all packages

runperf.utils.shell_find_command(session, command)

Helper to detect path to a command

Parameters:

  • session – aexpect.ShellSession session

  • command – command we are looking for

Returns:

path or empty string when not found

runperf.utils.shell_write_content(run, path, content, append=False)

Write bigger files per 4096 chunks

Parameters:

  • run – Command that accepts cmd as first argument

  • path – Path to which we want to write to

  • content – New content

  • append – Whether to append or overwrite the file

runperf.utils.shell_write_content_cmd(path, content, append=False)

Generate shell cmd to safely write/append content to file

With big files it might fail, use shell_write_content in such cases.

runperf.utils.sorted_entry_points(group)

Return alphabetically sorted entry points for a given group

Parameters:

group – entry-point group

runperf.utils.ssh_copy_id(log, addr, passwords, hop=None)

Use “ssh-copy-id” to copy ssh id, try passwords if asked for.

runperf.utils.string_to_safe_path(input_str)

Convert string to a valid file/dir name.

This takes a string that may contain characters that are not allowed on FAT (Windows) filesystems and/or ext3 (Linux) filesystems, and replaces them for safe (boring) underlines.

It limits the size of the path to be under 255 chars, and make hidden paths (starting with “.”) non-hidden by making them start with “_”.

Parameters:

input_str – String to be converted

Returns:

String which is safe to pass as a file/dir name (on recent fs)

runperf.utils.tabular_output(matrix, header=None)

Pretty, aligned string representation of a nxm matrix.

This representation can be used to print any tabular data, such as database results. It works by scanning the lengths of each element in each column, and determining the format string dynamically.

Parameters:

  • matrix – Matrix representation (list with n rows of m elements)

  • header – Optional tuple or list with header elements to be displayed

Returns:

String with the tabular output, lines separated by unix line feeds

runperf.utils.wait_for(func, timeout, step=1.0, args=None, kwargs=None)

Wait until func() evaluates to True.

If func() evaluates to True before timeout expires, return the value of func(). Otherwise return None.

Parameters:

  • timeout – Timeout in seconds

  • first – Time to sleep before first attempt

  • step – Time to sleep between attempts in seconds

  • text – Text to print while waiting, for debug purposes

  • args – Positional arguments to func

  • kwargs – Keyword arguments to func

runperf.utils.wait_for_machine_calms_down(session, timeout=600)

Wait until 1m system load calms below 1.0

Parameters:

  • session – session

  • timeout – timeout

Returns:

True on success, False when it’s still busy

runperf.utils.write_file(path, content, mode='w')

Write content to path, create the necessary upper dirs