CLIEngine – a base for Command Line Interfaces

The CLIEngine is designed with “Convention over Configuration” in mind. That is, it aims to set everything up so that it will work out -of-the-box, and the user is responsbile for adjusting undesireable behavior. Basically, all you have to do is write a subclass of CLIEngine with only a do() method (no need for __init__()!). See CLIEngine Examples for examples.

Writing your own Subclass

This class should be subclassed by the user, who should re-implement the following methods:

• do() - Does the ‘real work’.
• kill() - Called if the engine tries to exit abnormally.

These funcitons are used in the normal operation of the command line engine.

The user should also override the desired instance variables on the class. Useful instance variables to consider overriding are defaultcfg and description

There are some methods that the user can optionally override to alter the behavior of the CLIEngine on startup. These are:

• init() should add arguments to the parser which are fully formed before configuration.
• before_configure() should perform actions on the configuration which will be overwritten by any configuration file values.
• after_configure() should perform actions on the configuration which must occur before command line arguments are parsed by parse()

Other methods are used to control the engine during normal operation. These methods should in general not be overwritten, but can be modified in subclasses, so long as they are called using the super(ClassName, self).method() construct.

Running your subclass

To run the engine, use run(). To run the engine without instantiating the class, use script(), a class method that instantiates a new object, and runs the tool. Both methods accomplish the same thing at the end of the day.

Using __main__ or just running the file

To run your command as a simple python file, write your command class, then call script(). You can use script() to start the engine with the “if main” python convention:

if __name__ == '__main__':
    Engine.script()

You can also call script() outside of an “if main” block. Be aware, however, that script() will use the command line arguments passed to the python interpreter as a whole!

Using distutils

Using script() allows the developer to set this as an entry point in their setup.py file, and so provide the command line enegine as a command line tool in a distutils supported python package. A basic entry point for this tool would appear in the console_scripts key like

["PyScript = mymodule.cli:Engine.script"]

These console scripts can be installed using python setup.py install. A helper will be placed in your python script path (watch the setup.py output to see where that is!). The helper script manages the call to your class.

Running without a command-line

If you want run the CLIEngine without gobbling the command line arguments passed to python, you’ll have to instantiate the CLIEngine object yourself. By default, the CLIEngine constructor takes no arguments. You should pass any pseudo-command-line-arguments to arguments(). Each arguemnt item should be a separate iterable member, so "--config somefile.yml" should be passed in as ["--config","somefile.yml"]. A minimal CLIEngine call would look like:

ENG = CLIEngine()
ENG.arguments('-foo','bar')
ENG.run()

Note

To parse no command line arguments, pass something in like tuple()

How CLIEngine works

The engine is set up to use a configuration file system, provide basic parsing attributes, and an interruptable command line interaction flow. The configuration is designed to provide dynamic output and to configure the system before it completes the parsing process.

1. Initialization sets up the object, and sets up the argument parser. Method init() is called at this point. The parser should only understand arguments that will not be impacted by any configuration values. At this stage, by default, the parser is aware of the --config argument. This allows the CLIEngine to load a user-specified configuration file, and to use that configuration file to affect the command line arguments presented and parsed.
2. Preliminary Arguments are parsed by arguments().
3. Configuration is handled by the configure() function. This function loads the following configuration files in order (such that the last one loaded is the one that takes precedence):
   • The defaultcfg file from engine.__module__. This allows the developer to provide a base configuration for the engine.
   • The requested configuration file from the user’s home directory.
   • The requested configuration file form the current directory.
   • If no configuration file is requested, the filename for defaultcfg will be used. As well, if the engine cannot find the requested configureation file in the current directory (i.e. the user asks for a file, and it isn’t there) a warning will be issued.
4. Help message is inserted into parsing, and remaining arguments parsed by parse(). At this point, the entire configuration process is complete.
5. The function do() is called, This should do the bulk of the engine’s work.
6. If the user interrupts the operation of the program, kill() will be called. If python is in __debug__ mode, this will raise a full traceback. If not, the traceback will be suppressed.

Configuration

CLIEngine uses StructuredConfiguration objects. These objects represent rich nested mapping types with fast accessors and quick access to configuration file services using PyYAML.

To entirely disable configuration, and remove the configuration object, set CLIEngine.defaultcfg to False. This will prevent the CLIEngine from adding the --config and --configure command line options, and will not load any data from YAML files.

Configuration is a powerful, heirarchical system. To understand the load order, see pyshell.config.StructuredConfiguration.configure(). However, there are several levels where you can customize the order of loaded configurations. Configurations loaded earlier in the system will be overwritten by those loaded later.

• CLIEngine.supercfg allows the developer to set confguration files which should be loaded before all others.
• CLIEngine.defaultcfg allows the developer to set both the configuration file name which should be loaded from the CLIEngine-subclass’s directory (think os.path.dirname(__file__)), and the configuration file which will be searched for in the current directory.
• --config can change the name of the configuration file to look for in the current directory.
• --configure can write new configuration values into the configuration at the end of the configuration loading process.

Command Line Help

Reference for CLIEngine, the Command Line Interface Engine

class pyshell.base.CLIEngine(prefix_chars='-', conflict_handler=u'error')

A base class for Command Line Inteface facing tools. CLIEnigne provides the basic structure to set up a simple command-line interface, based on the argparse framework.

Parameters:prefix_chars (str) – Sets the prefix characters to command line arguments. by default this is set to “-”, reqiuring that all command line argumetns begin with “-”.

At the end of initialization, the configuration (config) and parser will be availabe for use.

description

The textual description of the command line interface, set as the description of the parser from argparse.

debug = True

Whether this tool will show stack traces.

epilog = u''

The text that comes at the end of the argparse help text.

defaultcfg = u'Config.yml'

The name of the default configuration file to be loaded. If set to False, no configuration will occur.

supercfg = []

This is a list of tuples which represent configurations that should be loaded before the default configuration is loaded. Each tuple contains the module name and filename pair that should be passed to resource_filename(). To specify a super-configuration in the current directory, use __main__ as the module name.

parser

argparse.ArgumentParser instance for this engine.

config

pyshell.config.Configuration object for this engine.

opts

Command Line Options, as paresed, for this engine

log

This engine’s logger

init()

Initialization after the parser has been created.

arguments(*args)

Parse the given arguments. If no arguments are given, parses the known arguments on the command line. Generally this should parse all arguments except -h for help, which should be parsed last after the help text has been fully assembled. The full help text can be set to the self.parser.epilog attribute.

Parameters:*args – The arguments to be parsed.

Similar to taking the command line components and doing " -h --config test.yml".split(). Same signature as would be used for argparse.ArgumentParser.parse_args()

before_configure()

Actions to be run before configuration. This method can be overwritten to provide custom actions to be taken before the engine gets configured.

configure()

Configure the command line engine from a series of YAML files.

The configuration loads (starting with a blank configuration):

1. The module configuration file named for defaultcfg 2. The command line specified file from the user’s home folder ~/config.yml 3. The command line specified file from the working directory.

If the third file is not found, and the user specified a new name for the configuration file, then the user is warned that no configuration file could be found. This way the usre is only warned about a missing configuration file if they requested a file specifically (and so intended to use a customized file).

after_configure()

Actions to be run after configuration. This method can be overwritten to provide custom actions to be taken before the engine gets configured.

parse()

Parse the command line arguments.

This function uses the arguments passed in through arguments(), adds the -h option, and calls the parser to understand and act on the arguments. Arguments are then stored in the opts attribute.

This method also calls configure_logging() to set up the logger if it is ready to go.

Note

Calling configure_logging() allows configure() to change the logging configuration values before the logger is configured.

do()

This method should handle the main operations for the command line tool. The user should overwrite this method in Engine subclasses for thier own use. The KeyboardInterrupt or SystemExit errors will be caught by kill()

kill()

This function should forcibly kill any subprocesses. It is called when do() raises a KeyboardInterrupt or SystemExit to ensure that any tasks can be finalized before the system exits. Errors raised here are not caught.

run()

This method is used to run the command line engine in the expected order. This method should be called to run the engine from another program.

classmethod script()

The class method for using this module as a script entry point. This method ensures that the engine runs correctly on the command line, and is cleaned up at the end.

_remove_help()

Remove the -h, --help argument from the parser.

Warning

This method uses a swizzle to access protected parser attributes and remove the argument as best as possible. It may break!

_add_help()

Add the -h, --help argument.

_add_configfile_args(*args)

Add a parser command line argument for changing configuration files.

Arguments :The set of arguments to be passed to add_argument().

_add_configure_args(*args)

Add a parser command line argument for literal configuration items.

See parse_literals().

Arguments :The set of arguments to be passed to add_argument().

configure_logging()

Configure the logging system using the configuration underneath config["logging"] as a dictionary configuration for the logging module.

Call structure of run()

The call structure of the method run(), the main script driver. This is provided as a reference. Note that the init() is called during arguments() when arguments() is called outside of run(), allowing you to use the following sequence to run a subclass of CLIEngine:

CLIInstance = CLIEngine()
CLIInstance.arguments("--faked CLI --arguments".split())
CLIInstance.run()

    def run(self):
        """This method is used to run the command line engine in the expected 
        order. This method should be called to run the engine from another 
        program."""
        if not self._hasinit:
            self.init()
        if not self._hasargs:
            warn("Implied Command-line mode", UserWarning)
            self.arguments()
        self.before_configure()
        self.configure()
        self.after_configure()
        self.parse()
        try:
            self.do()
        except SystemExit as exc:
            if not getattr(exc, 'code', 0):
                self.kill()
            if self.debug:
                raise
            self.exitcode = getattr(exc, 'code', self.exitcode)
        except KeyboardInterrupt as exc:
            self.kill()
            if self.debug:
                raise
        return self.exitcode