Custom Parsing Function For Any Number Of Arguments In Python Argparse

I have a script that gets named parameters via command-line. One of arguments may be supplied multiple times. For example I want to run a script: ./script.py --add-net=user1:10.0.0

Solution 1:

As it is clear that argparse internally puts the default as initial value of the resulting object, you should not directly set the default in the add_argument call but do some extra processing:

parser.add_argument('--add-net',
                    action=ParseIPNets,
                    dest='user_nets',
                    help='Nets subnets for users. Can be used multiple times',
                    default = {})

args = parser.parse_args()
if len(args.user_nets) == 0:
    args.user_nets['user1'] = "198.51.100.0/24"

Alternatively, if you want a better user experience, you could make use of the way Python processes mutable default arguments:

class ParseIPNets(argparse.Action):
    """docstring for ParseIPNets"""
    def __init__(self, option_strings, dest, nargs=None, **kwargs):
        super(ParseIPNets, self).__init__(option_strings, dest, **kwargs)
    def __call__(self, parser, namespace, values, option_string=None, first=[True]):
        if first[0]:
            namespace.user_nets.clear()
            first[0] = False
        location, subnet = values.split(':')
        namespace.user_nets[location] = subnet

parser.add_argument('--add-net',
                    action=ParseIPNets,
                    dest='user_nets',
                    help='Nets subnets for users. Can be used multiple times',
                    default={"user1": "198.51.100.0/24"})

args = parser.parse_args()

That way, the optional default will be cleared if the option is present.

But BEWARE: this will work only at first call on the script. It is acceptable here because parser.parse_args() should only be called once in a script.

Ancilliary remark: I removed nargs='*' because I find it more dangerous than useful here if you call it that way, and also removed the erroneous loop over values always using values:

test.py --add-net=a:10.0.0.0/24 --add-net=b:10.1.0.0/24

nargs='*' would make sense for following syntax:

test.py --add-net a:10.0.0.0/24 b:10.1.0.0/24

and the code would be:

    def __call__(self, parser, namespace, values, option_string=None, first=[True]):
        if first[0]:
            namespace.user_nets.clear()
            first[0] = False
        for value in values:
            location, subnet = value.split(':')
            namespace.user_nets[location] = subnet

Solution 2:

It's usually not a good idea to use a mutable default argument (a dict in your case), see here for an explanation:

Create a new object each time the function is called, by using a default arg to signal that no argument was provided (None is often a good choice).

Solution 3:

My first approach to this problem would be to use action='append', and turn the resulting list into a dictionary after parsing. The amount of code would be similar.

'append' does have this same issue with defaults. If default=['defaultstring'], then the list will start with that value as well. I'd get around this by using the default default ([] see below), and add the default in post processing (if the list is still empty or None).

A note on defaults. At the start of parse_args, all action defaults are added to the namespace (unless a namespace was given as a parameter to parse_args). Commandline is then parsed, with each action doing its own thing to the namespace. At the end, any remaining string defaults are converted with the type function.

In your case namespace.user_nets[location] = subnet finds the user_nets attribute, and adds the new entry. That attribute was initialized as a dictionary by the default, so the default appears in the final dictionary. In fact your code would not work if you left the default to be None or some string.

The call for the _AppendAction class may be instructive:

def __call__(self, parser, namespace, values, option_string=None):
    items = _copy.copy(_ensure_value(namespace, self.dest, []))
    items.append(values)
    setattr(namespace, self.dest, items)

_ensure_value is a function defined in argparse. _copy is the standard copy module that it imported.

_ensure_value acts like a dictionary get(key, value, default), except with a namespace object. In this case it returns an empty list if there isn't already a value for self.dest (or the value is None). So it ensure that the append starts with a list.

_copy.copy ensures that it appends values to a copy. That way, parse_args will not modify the default. It avoids the problem noted by @miles82.

So 'append action' defines the initial empty list in the call itself. And uses copy to avoid modifying any other default.

Did you want values as opposed to value in?

location, subnet = values.split(':')

I'd be inclined to put this conversion in a type function, e.g.

def dict_type(astring):
   key, value = astring.split(':')
   return {key:value}

This would also be a good place to do error checking.

In the action, or post parsing these could added to the existing dictionay with update.