optmatch - Python command line parsing made easy

Quick introduction

optmatch is a python library that allows parsing command line options in a simple way. For example, a tool supporting two options called mode and verbose, and requiring two arguments, called file and where, would be coded as:

class Example(OptionMatcher):

	@optmatcher
	def handle(self, file, verboseFlag=False, modeOption='simple', where=None):
		...	

This tool would support a syntax such as:

[--verbose] [--mode MODE] file [where]

In the previous example, the suffix of the method's parameters define the role of each parameter. Alternatively, the decorator can embed this information, like in:

class Example(OptionMatcher):

	@optmatcher(flags='verbose', options='mode')
	def handle(self, file, verbose=False, mode='simple', where=None):
		...	

Which is equivalent to the initial example. A more complicated case would be:

class Example(OptionMatcher):

	@optmatcher
	def handleHelp(self, helpFlag):
		...	

	@optmatcher
	def handleCompression(self, file, compressFlag=False):
		...

	@optmatcher(flags='verbose', options='mode')
	def handle(self, file, verbose=False, mode='simple', where=None):
		...	

In this case, the tool supports three possible alternatives:

[--help]
[--compress] file
[--verbose] [--mode MODE] file [where]

And the library issues usage error messages if the user enters invalid arguments, like --compress and --mode--, which are mutually incompatible.

The previous examples show the simplest ways to access this library, which contains quite a lof of functionality to cover most aspects related to command line parsing, including printing usage messages, handling aliases, etc.

Purpose

optmatch defines the command line parsing by setting the handler or handlers than will process the command line. The library then matches the received input with these handlers, invoking the most convenient one, or issuing error messages if the input does not match the expected syntax.

In addition to these handlers, optmatch supports a specific interface to define aliases, documentation associated to each argument/option, etc, and it is able to automatize the generation of usage messages.

Its initial purpose was to extend the existing related functionality. There are two existing libraries to handle command line options parsing in python: getopt and optparse.

  • getopt usage is rather simple, and it is almost just limited to split the received arguments into a list of expected arguments: the programmer must still make sense of the received input, checking that all arguments are there, that they are not incompatible, and then, invoking the handler or handlers to process them.
  • optparse provides a much richer interface, although it mostly makes the usage of getopt convenient -and it also provides printing usage messages-. Using optparse requires a set of well defined steps: defining the flags/options, etc, its aliases and documentation. For each option is possible to associate some actions, like storing a variable or invoking a function, all defined in a completely procedural way.

The initial purpose behind optmatch was to extend optparse to handle some usual operations: defining incompatibilities between arguments, or whether an option would require the presence of some other options. Eventually, it came the idea of just matching command line options to the signature of the parameters, which allows for most of the initial planned checks, and makes handling the command line options very simple.

Concepts

optmatch is not limited to GNU style command line options. In this style, options can be specified in short format or long format. For example, an alias can be defined between the short option v and the long one verbose. The user can then enter -v or --verbose, indistinctly.

Other valid styles also supported in optmatch include the typical Windows format, like in /help or using - as prefix for short and long options, like in -help. In this document, the GNU style is refered as getopt mode.

In any case, some terms apply to both styles:

  • option: is an argument prefixed with the option prefix -normally - or --, and with a value associated. For example:
    --mode=optimized
  • flag: is an option without associated value. If present, it is assumed to have the boolean value True. For example:
    --verbose
  • prefix: is an option that can be specified multiple times. For example:
    -I/usr/include -I/opt/include
  • parameter: any argument in the command line that does not include the option prefix.

Note that, in this document, it is normally used the word option to cover not only options, but also flags and prefixes.

In getopt mode, there are two option prefixes:

  • -: short prefix. Only short options apply, containing a single letter. The user can enter multiple short options together, like in:
    -iof here
    which stands for the short flags i, o, and the short option f, with value here
  • --: long prefix. Only long options apply, containing two or more letters. The user cannot enter multiple long options together. If a value is specified, it can be done on a separate argument, like in:
    --mode optimized
    Or, alternatively, in the same argument, separated by = (or any other pre-specified character):
    --mode=optimized

In non-getopt mode, only one option prefix applies, normally - or /. In this case, there is no distinction between short and long options, and the user must enter each option on a separate argument, like in:

/mode:optimized

Finally, optmatch uses the concept 'gnu mode'. If specifically defined, it implies that all option arguments must be specified at the beginning of the command line. Otherwise (the default), options and parameters can be freely intermixed.

Tutorial

The basics

There are three main elements to import from optmatch:

  • OptionMatcher: the main class; users must implement the methods that handle the command line options as part of a subclass of OptionMatcher.
  • optmatcher: a decorator that specifies that a method in a class is a command line handler.
  • optset: a decorator that specifies that a method in a class handles common options to one or more handlers.

The following code defines two such handlers, and processes the command line arguments:

from optmatch import OptionMatcher, optmatcher, optset

class Example(OptionMatcher):

	@optmatcher
	def handleCompression(self, file, compressFlag):
		'''Compress the specified file'''
		...

	@optmatcher
	def handleMove(self, file, where=None, verboseFlag=False):
		'''Moves the file to the specified directory'''
		...	

Example().process(sys.argv)

This code allows the tool to handle command line arguments where the user specifies one of:

[--compress] file
[--verbose] file [where]

The suffix for each parameter on the handlers define the parameter role. The valid suffixes are:

  • Flag
  • Option
  • Prefix
  • OptionInt: the associated value is converted to an integer
  • OptionFloat: the associated value is converted to a float number

Please note that optmatch automatically expands all shell/user variables in the options.

Basic help

In the previous example, in addition to the two specified cases, the user can enter --help to receive some basic usage information. It would look like:

Usage: [common options] file where

options:
  --compress
  -h, --help            shows this help message
  --verbose

alternatives:

* --compress file       Compress the specified file

* [--verbose (False)] file [where]
                        Moves the file to the specified directory

* -h                    shows the help message

Note that the documentation for each handlers is used to document the alternatives, but the options are not documented. To document them, it is needed to supply it as:

OptionMatcher.setUsageInfo(optionsHelp={'compress':'compress the specified file'})

The information for all the options must be provided at once, in a dictionary.

Aliases

Aliases are the way to connect short and long options, like specifying that -v is equivalent to --verbose

In getopt mode, an alias must always match a short option (one letter) to a long one, or viceversa. There are two ways ot specify aliases: on the OptionMatcher constructor, or using its setAliases method, which expects a dictionary, such as:

OptionMatcher.setAliases({'v':'verbose'})

The help system automatically displays the option's aliases.

Public names

As specifed so far, optmatch would not be able to handle options that are not valid variable names. For example, how to define the flag dry-run or the option email@?

As it happens, options are quite well behaven, defined usually as valid Python variable names: it is not usual to have an option starting with a digit, or having a character such as $ or @ inside. In any case, optmatch allows such possibilities, although it must be explicitly programmed.

The most usual case envolves using dashes, like in dry-run. optmatch automatically translates camel case into dashes, so the following example:

@optmatcher
	def handle(self, dryRunFlag):
		...	

expects the flag dry-run, instead of the flag dryRun.

This behaviour can be embedded in the decorator, using the as specification. The previous example is equivalent to:

@optmatcher(flags='d as dry-run')
	def handle(self, d):
		...	

Obviously, this possibility allows defining any kind of option, using all kind of illegal variable names:

@optmatcher(flags='d as @f***ing!')
	def handle(self, d):
		...	

Yes, this handler expects a flag --@f***ing!

Although the previous two possibilites should cover all the bases, it is still possible to define a mapping for these public names on the OptionMatcher constructor, or using its setPublicNames method, which expects a dictionary, such as:

OptionMatcher.setPublicNames({'d':'@f***ing!'})

optset

There are cases where one or more options apply to multiple handlers. A typical example would be the verbose flag. Instead of defining it on all the matchers, it is possible to use the decorator optset, like in:

@optset
	def handleHelp(self, helpFlag):
		...	

It is quite equivalent to the optmatcher decorator, so it is possible to specify its behaviour through the decorator:

@optset(flags='help')
	def handleHelp(self, help):
		...	

These handlers are called, for convenience, when possible. That it, if, in the previous example does not include the flag --help, the method handleHelp is not invoked. However, if could have been defined with default values, like in:

@optset(flags='help')
	def handleHelp(self, help=False):
		...	

In this case, the method will be always called, which simplifies setting some common variables

Advanced optset

optset is also useful to define mandatory options. For example, a tool could require that the flag --test is provided with an option --file=FILENAME where the file to test is specified.

This could be defined as:

@optset
	def handleTest(self, testFlag, fileOption):
		...	

If the user specifies --test but not --file= an exception is automatically raised.

Now, there could be multiple matchers, but this option --test could only apply to one of the matchers. It is possible to limit the scope of a optset handler to one or several matchers:

@optset(applies='zip')
	def handleTest(self, testFlag, fileOption):
		...	

@optmatcher
	def zip(self, file):
		...	
		

It is possible to define multiple matchers, separated by commas or using limited regular expressions:

@optset(applies='zip, test*')
	def handleTest(self, testFlag, fileOption):
		...	

In this case, it would apply to the method zip, and to all methods starting with test.

If a optset handler has no applies specification, it would apply to all defined matchers, unless a matcher specifies exclusive=True, such as:

@optmatcher(exclusive=True)
	def handle(self, testFlag, fileOption):
		...	

Usage mode

By default, OptionMatcher works on getopt mode. In other words, it is compatible with getopt and optparse: there are short options, prefixed with -, and long options, prefixed with --.

By default also, the gnu mode is disabled: option arguments can be freely intermixed with required arguments. This mode can be disabled on the OptionMatcher.process method, specifying the argument gnu=False.

This mode can be overriden by specifying a different option prefix. As usual, this can be done on the contructor, or using a specific method, in this case: OptionMatcher.setMode. For example,

OptionMatcher.setMode(optionPrefix='-')

In this example, the distinction between short and long arguments dissapear, and all options are expected with the simple prefix -.

It is also possible to define the character that specifies the assignment, which is by default =. For example,

OptionMatcher.setMode(optionPrefix='/', assigner=':')

enables Windows typical mode:

/mode:optimized

Handling incorrect usage

When the user's input does not match the expected input, an exception is raised.

This exception is the UsageException, defined in the optmatcher library. However, by default, it is automatically handled, so that a message is printed on the standard error stream.

To disable this behaviour, allowing the library's client to process it at will, it is needed to invoke the OptionMatcher.process like in:

.process(handleUsageProblems=False)

Handling incorrect syntax

UsageException is a class that inherits from OptionMatcherException; this exception is raised when the syntax or aliases are incorrectly defined. It does not depend on the user's input.

More on help

By default, optmatcher adds a matcher to handle help requests. That is, -h or --help, or even /help it the correct prefix was setup, will automatically display the normal usage message.

There are a few ways to override this behaviour:

  • Disabling the default help: this can be done on the OptionMatcher constructor, or invoking its OptionMatcher.enableDefaultHelp method.
  • Overriding the OptionMatcher.printHelp method, that will be automatically invoked by the default help.
  • Defining an explicit help matcher, such as:
    optmatcher
    	def handleHelp(self, helpFlag):
    		...	

In these two last cases, the OptionMatcher class provides some functionality to display the required information. The method OptionMatcher.getUsage returns a OptionMatcher.printHelp UsageAccessor instance, that can be used to format the usage message and to retrieve the required information, related to defined options, paraeters, etc.

Var names

varnames is a concept related to the help system. If it is defined an option called filename, the default usage for this option will print something like:

filename = FILENAME

It is possible to redefine the associated variable (hende the var name), by setting the var names on the constructor:

OptionMatcher(optionVarNames{'filename':'ORIGIN'})

This would print a usage message such as:

filename = ORIGIN

Decorators

The two decorators in this module, optmatcher and optset, allows defining the behaviour of the underlying matcher via their parameters. Both decorators share most of the parameters:

  • options: defines which of the parameters are considered options. This parameter is a string, where the defined options are separated by commas, like in:
    @optmatcher(options='mode, file')
    def matcher(self, mode, file):
       ...
    
    Each of the parameters can be defined like:
    parameter as public_name
    For example:
    @optmatcher(options='mode as verbose-mode, file as target.file')
    def matcher(self, mode, file):
       ...
    
    In this example, the matcher expects two options, named verbose-mode and target-file
  • intOptions: defines which of the parameters are considered options associated to integer values. The remarks given to the normal options also apply for integer options.
  • floatOptions: defines which of the parameters are considered options associated to floatvalues. The remarks given to the normal options also apply for float options.
  • prefixes: defines which of the parameters are considered prefixes. The remarks given to the normal options also apply for prefixes.
  • flags: defines which of the parameters are considered flags. The remarks given to the normal options also apply for flags. As additional feature, it is possible to define orphan flags, which are specified in the decorator, but have no associated matching parameter. For example:
    @optset(flags='quiet')
    def setQuiet(self):
       ...
    
    In this case, only when the user enters --quiet, this matcher is invoked. Note that it would be possible to introduce a parameter quiet in this matcher, but, when invoked, it would always be set to True. Hereby, it is possible to define it on the decorator only.
  • renamePars: defines the public name for the parameter variables. For example:
    @optmatcher(renamePars='d as dir')
    def handle(self, d):
       ...
    
    The help associated to this matcher would show the parameter d with the name dir.
  • priority. Matchers are tried in order, being the order defined by alphabetical sorting on the matcher method names. This order can be observed when the help lists all the alternatived for the current OptionMatcher. It is possible to alter this order by defining the priorities of each matcher. Higher priorities are invoked first.

In addition, optset can define the matchers to which it applies, using the applies parameter in the decorator, as described in the advanced optset tutorial. Its counterpart is the exclusive parameter in the optmatcher decorator.

History

  • Version 0.8.7, 13th June 2009. Minor leftover changes
  • Version 0.8.6, 13th June 2009. Added version for Python 3.0
  • Version 0.8.5, 2nd June 2009.
    • Improves help format
    • All provided options are automatically expanded (shell and user variables)
  • Version 0.8.4, 29th May 2009. First downloadable version.
  • Version 0.8, 14th May 2009. API totally simplified, including minor refactoring.
  • Version 0.7, 1st May 2009. Introduction of varnames in help system, help support vastly improved.
  • Version 0.6, 2nd April 2009. Introduction of functionality to help checking the syntax (this was removed on 0.8).
  • Version 0.5, 15th January 2009. Introduction of decorators.
  • Version 0.4, 10th December 2008. Basic help system.
  • Version 0.3, 3rd November 2008. Support for non-getopt mode.
  • Version 0.2, 25th September 2008. Introduction of common handler concept.
  • Version 0.1, 12th September 2008. First working version, totally functional.

Download

optmatcher is open source, with the following generic python license:

Copyright (c) Luis M. Pena <lu@coderazzi.net>  All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:

Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
Redistributions in bytecode form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in
the documentation and/or other materials provided with the
distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.