plac current status

I have just released version 0.7.4 of plac. The tool has grown considerably in its three months of life, both in size - now it is over a thousand lines of code - and capabilities. The core is still two hundred lines long, and the basic usage of plac as a command-line arguments parser can still be explained in 10 minutes, but there is much more now.

Judging from the downloads, I have a couple hundred of potential users. Since I have got a few people sending me emails with questions and/or feedback, I assume I have at least a few real users ;) I suspect most people use plac simply as a command-line argument parser. There is nothing wrong with that, it is its intended usage after all: in release 0.7 I have even improved its capabilities in such sense. Still, I would like to encourage people to start using the full capabilities of plac.

Officially plac is still in alpha status, i.e. I do not guarantee backward compatibility between releases. In practice, the basic API for parsing the command-line arguments (plac.call(callable, arglist)) has stayed unchanged from the beginning and it will never change. Possibly, some optional arguments may be added or may change, but not the basic API. Actually in its three months of life plac has seen only minor incompatible changes, on non fundamental features or experimental features. So, while formally plac is in alpha status, in practice you can pretty much rely on its basic features. Apparently they work fine too, since I never had any bug report. In other terms, there is no excuse not use plac, even if it is a still young project ;)

The support for subcommands has improved

argparse - the library underlying plac - has support for subcommands, but writing a parser with subcommands is not as simple as it could be. plac makes it trivial thanks to the Interpreter.call classmethod (new in release 0.7). For instance, here is how you could implement a SVN-like tool with two commands checkout and commit. The trick is to write a class with two methods checkout and commit and an attribute .commands listing them, and to call the class with plac.Interpreter.call:

$ cat vcs.py
class VCS(object):
    "A fictitious version control tool"
    commands = ['checkout', 'commit']
    def checkout(self, url):
        return 'ok'
    def commit(self):
        return 'ok'

if __name__ == '__main__':
    import plac; plac.Interpreter.call(VCS)

The line plac.Interpreter.call instantiates the VCS class by passing to it the arguments in the command line and then calls the appropriate method.

You can use the script as follows:

$ python vcs.py -h
usage: vcs.py [-h] [args [args ...]]

positional arguments:
  args

optional arguments:
  -h, --help  show this help message and exit

$ python vcs.py checkout url
ok
$ python vcs.py commit
ok

plac takes care of parsing the command line, giving the correct error message if you pass wrong arguments or not enough arguments:

$ python vcs.py checkout
usage:  checkout url
 checkout: error: too few arguments

You should realize that there is no real difference between a command-line argument parser featuring subcommands and an command interpreter, therefore the previous script also works as an interactive interpreter:

$ python vcs.py -i
i> .help

special commands
================
.help  .last_tb

custom commands
===============
checkout  commit

i> checkout url
ok
i> commit
ok

There is full help support, i.e. you can ask for .help <command> for any command, including the special ones such as .help and .last_tb. There is full support for autocompletion and command history too, provided you have the readline library installed (on Unices) or the pyreadline library (on Windows).

plac also support a batch mode; you can write a set of commands on a file and have them executed by the plac runner:

$ echo vcs-commands.plac
#!vcs.py:VCS
checkout url
# nontrivial commands here
commit

$ plac_runner.py --batch vcs-commands.plac
ok
skip #<lines with comments are skipped>
ok

Notice the shebang line (#!vcs.py:VCS) which specifies the name of the file where the interpreter is defined and the name of the plac factory to use (in this case it is the class VCS).

There is also a doctest mode to check that the execution returns the expected output; for an explanation, you should check out the full documentation of plac.

There is a plac server now

Perhaps you have thought in the past that it would be nice if plac could execute commands on a remote machine: the good news is that starting for release 0.7 it can.

The trick is to start the plac interpreter in server mode: then any client can connect to the server. Here is an example working on localhost:

# plac_runner --serve 2199 vcs.py:VCS
$ telnet localhost 2199
Trying ::1...
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
i> checkout url
ok
i> EOF
Connection closed by foreign host.

It also works on a real remote host, if you use a port which is not firewalled. Notice that the plac server is in an early stage of development, so for the moment is does not support any authentication/authorization feature, nor any kind of security (that mean that it should be used on a protected LAN). Also, in order to avoid dependencies, it is implemented on top of the asyncore/asynchat modules in the standard library. Nothing stops you for implementing a better server using Twisted or any other technology, anyway: after all, implementing a line-oriented protocol is pretty trivial. Finally, at the moment there is no plac client: you can use telnet, but that means loosing the autocompletion features. There is still room for improvement there (patches are welcome!).

Parallel programming with plac

plac was not born as a tool for parallel programming and you should not expect too much from it in that area; still, it has some support to running commands in background, and it can be used to solve simple parallel tasks. Let me give a real life use case which happened to me a few weeks ago. I needed to perform a rather long query on our production database and on our testing database and to compare the results. The two databases are on two different machines and in order to compare the results the best way is to save the query results in a local database, say in two tables 'prod' and 'test': then one can compare the two tables with ease. In the past I have always solved such problem sequentially, by performing the two queries one after the other. One query takes nearly 5 minutes to run, so I had to wait 10 minutes to compare the results. However, this is a perfectly parallel job and since I had plac at my disposal I decided to use it to perform the queries together and to halve my waiting time. To this aim plac provides an utility function plac.runp(genseq, mode='p', start=True) which takes a sequence of generators and executes them in parallel, by using processes if mode='p' or threads if mode='t'. The functions returns a list of task objects; each task object has a .result property which returs the result of the task, i.e. the last yielded value. Then the problem was solved with code like the following:

def run_query_and_save(sourcedb, targetdb):
    output = run_query(sourcedv)
    save(output, targetdb)
    yield 'saved %d rows on %s' % (len(output), targetdb)

tasks = plac.runp(run_query_and_save(prod_db, localdb_prod),
                  run_query_and_save(test_db, localdb_test))
for t in tasks:
    print r.result

You can look at plac documentation for more details about task objects. The .result property was inspired by the concept of futures: it blocks the main thread until the running task has finished its job.

plac future

Speaking of futures, it is time to ask myself and my users where we want to go from here.

First of all I would like to reorganize the documentation, since now it has become quite large (around 45 pages in PDF form).

Then there is the long standing issue of multiline support: I would like to extend plac to support commands spanning multiple lines. The problem is that Python readline module does not expose the multiline support of the underline C-level readline module so that I cannot use such futures; moreover, I am also limited by the capabilities of pyreadline on Windows.

Multiple line commands are best entered in an editor anyway, so I would like to support some integration between plac and at least Emacs, which is the editor I use. In other words, it should be possible to run plac scripts in an inferior mode inside Emacs. Some help from Emacs experts would be welcome on that idea.

Other unplanned features may enter in plac as I see the need for them in my day-to-day work: for instance plac.Interpreter.call was not designed in advance, but it emerged from my practical needs.

What do you think? What users want for plac? Speak your mind, I am here to hear from you!