Computing Thoughts
More on Remote Procedure Calls
by Bruce Eckel
September 5, 2011

Summary
The feedback from my previous article has been very helpful, and it has raised more questions about RPC technology. I've also included notes about converting Python libraries from Python 2 to Python 3.

I've used XML-RPC in the past. In the article, I used JSON-RPC because that's the only thing that Go implements in the standard library (someone on the Go team clearly thought JSON-RPC was valuable). But in my researches, there seems to be a "lost civilization" feel to JSON-RPC. Lots of activity in the 2006-2008 timeframe, then a plague seems to have wiped everyone out. For example, the last entry by Roland Koebler, the proposer for JSON-RPC 2.0 reads: "But the (in my opinion) much improved JSON-RPC 2.0 is stable and will probably be officially released soon." The last update on that page was 2008, so "soon" seems not to have happend. I emailed Koebler asking him about this, and after a protracted delay, he replied:

"My Python JSON-RPC implementation currently supports something "like" JSON-RPC 2.0. I created it according to a JSON-RPC 2.0 draft (which I wrote, by the way ;)) before JSON-RPC 2.0 was officially released. It works flawlessly on several servers out there.

"Unfortunately I did not have the time since then to check if it really meets the JSON-RPC 2.0 specification or add the missing functionality. But I'm planning to release an updated version of my library soon."

The various other JSON-RPC libraries I've come across don't seem to have any updates or corrections after around this time, either.

I recently learned about Thrift. There is Go support (which I haven't yet tested). The bug fixes seem to be very active, and the appearance of Thrift seems to coincide with the apparent loss of interest in JSON-RPC. Is that in fact what happened? Did Thrift come along and, promoted by Facebook, effectively wipe out the other RPC strategies like XML-RPC and JSON-RPC? (Or is there something else I should be looking at?).

There are a number of other RPC technologies; Google's "protocol buffers" has been suggested, however that is only a data format and doesn't include a way to make remote calls.

Then there were the heady days of CORBA (Common Object Request Broker Architecture), which began when the original C++ standards committee was meeting. There may have been earlier attempts to "connect everything to everything else" but this was my first exposure and it seemed promising indeed: We'd be able to save legacy code by connecting it to modern code, and we'd be able to get the best of all worlds by combining languages. CORBA was shaping up to be the ultimate answer.

A committee was formed. The Object Management Group (OMG, and the irony is not lost). The OMG then wanted to be an organization on its own, so it needed funding. It was decided that to be on the committee you needed to be (A) a corporation and (B) pay an annual fee that only a fairly successful corporation could afford (the C++ committee, in contrast, was accessible enough that independents like me could participate). The rest you can imagine: it took a very long time and all the obscure problems of the corporations on the committee had to be solved. The result was huge and complex, and didn't get much use (but surprisingly, some aspects of CORBA are still in use if you search in enough dark corners). The Tower of Babel was a project that got waylaid when everyone started speaking different languages (management issues were not so different back then, it seems) and the CORBA project lost its way because no one could agree on what the "common language" was, so they just threw in everything they could think of.

It would have been interesting if they had started with small prototypes and tried them out to see what worked, what was too complicated, etc. But in those days you designed something and then waited for awhile (often years, in the case of C++ features) to see how it came out.

Thanks for any insights. I'd like to figure out the best (current) approach to RPC.

Converting Python 2 to 2 & 3 Code

I don't know if this is always possible, but so far I've been able to take several pieces of Python 2 code (I'm running 2.6) and modify it so that it will also work with Python 3 (I've got 3.2).

As an example, we can convert Roland Koebler's JSON-RPC library by going to his page and downloading from his link. Make a copy of this file in a local directory to experiment on.

To see whether it works as-is, you can try "running library module as a script" like this:

python -m jsonrpc

With Python 3, this will produce errors. Fortunately, Python 3 includes the 2to3 utility. If you run this on the file without using flags, you'll get the diffs that it wants to apply. But if you use the -w flag, it will apply the diffs:

2to3 -w jsonrpc.py

Now python -m jsonrpc will discover that the simplejson module is missing. This was required because Koebler wrote his module for an earlier version of Python that did not contain a json module, but now (since 2.6) there's one in the standard library. So we can edit the file and change the line:

import simplejson

to:

import json as simplejson

Now python -m jsonrpc completes successfully -- but we're not done yet, because for some reason 2to3 misses the buffer translations (perhaps it is unable to parse for them -- a place where static type checking wins). When we try this program:

import jsonrpc

rpc = jsonrpc.ServerProxy(jsonrpc.JsonRpc10(),
                  jsonrpc.TransportTcpIp(addr=("127.0.0.1", 1234)))

for i in range(10):
    print(rpc.RPCFunc.Echo("hello " + str(i)))

(With the server from the previous post running) we get the complaint:

'str' does not support the buffer interface

This means we have to follow the line numbers and change line 404 from:

data = self.loads(string)

to:

data = self.loads(string.decode())

and line 780 from:

self.s.sendall( string )

to:

self.s.sendall( string.encode() )

And now it works, for both versions of Python. Note that Koebler's code is posted with an open-source license, so you can republish the modified code if you want.

One thing that was different between jsonrpc.py and the jsonclient.py code from the previous article:: jsonrpc.py opens and closes a connection for each call, while jsonclient.py keeps the connection open for all the calls. You'll notice a speed difference from the overhead of opening and closing all those connections (Perhaps jsonrpc.py has a "keep open" option; I didn't look).

Talk Back!

Have an opinion? Readers have already posted 2 comments about this weblog entry. Why not add yours?

RSS Feed

If you'd like to be notified whenever Bruce Eckel adds a new entry to his weblog, subscribe to his RSS feed.

Digg |

del.icio.us |

About the Blogger

Bruce Eckel (www.BruceEckel.com) provides development assistance in Python with user interfaces in Flex. He is the author of Thinking in Java (Prentice-Hall, 1998, 2nd Edition, 2000, 3rd Edition, 2003, 4th Edition, 2005), the Hands-On Java Seminar CD ROM (available on the Web site), Thinking in C++ (PH 1995; 2nd edition 2000, Volume 2 with Chuck Allison, 2003), C++ Inside & Out (Osborne/McGraw-Hill 1993), among others. He's given hundreds of presentations throughout the world, published over 150 articles in numerous magazines, was a founding member of the ANSI/ISO C++ committee and speaks regularly at conferences.


	Web Artima.com