How To Serialize Python Objects In A Human-readable Format?

I need to store Python structures made of lists / dictionaries, tuples into a human-readable format. The idea is like using something similar to pickle, but pickle is not human-fri

Solution 1:

For simple cases pprint() and eval() come to mind.

Using your example:

>>>d = {'age': 27,...'name': 'Joe',...'numbers': [1, ...2, ...3,...4,...5],...'subdict': {...'first': 1, ...'second': 2,...'third': 3...             }...}>>>>>>from pprint import pprint>>>pprint(d)
{'age': 27,
 'name': 'Joe',
 'numbers': [1, 2, 3, 4, 5],
 'subdict': {'first': 1, 'second': 2, 'third': 3}}
>>>

I would think twice about fixing two requirements with the same tool. Have you considered using pickle for the serializing and then pprint() (or a more fancy object viewer) for humans looking at the objects?

Solution 2:

If its just Python list, dictionary and tuple object. - JSON is the way to go. Its human readable, very easy to handle and language independent too.

Caution: Tuples will be converted to lists in simplejson.

In [109]: simplejson.loads(simplejson.dumps({'d':(12,3,4,4,5)}))
Out[109]: {u'd': [12, 3, 4, 4, 5]}

Solution 3:

You should check out jsonpickle (https://github.com/jsonpickle/jsonpickle). It will write out any python object into a json file. You can then read that file back into a python object. The nice thing is the inbetween file is very readable because it's json.

Solution 4:

If you're after more representations than are covered by JSON, I highly recommend checking out PyON (Python Object Notation)...although I believe it's restricted to 2.6/3.0 and above, as it relies on the ast module. It handles custom class instances and recursive data types, amongst other features, which is more than is provided by JSON.

Solution 5:

What do you mean this is not human-readable??? ;)

>>> d = {'age': 27, 
...   'name': 'Joe',
...   'numbers': [1,2,3,4,5],
...   'subdict': {'first':1, 'second':2, 'third':3}
... }
>>>>>>import pickle
>>> p = pickle.dumps(d)      
>>> p
"(dp0\nS'age'\np1\nI27\nsS'subdict'\np2\n(dp3\nS'second'\np4\nI2\nsS'third'\np5\nI3\nsS'first'\np6\nI1\nssS'name'\np7\nS'Joe'\np8\nsS'numbers'\np9\n(lp10\nI1\naI2\naI3\naI4\naI5\nas."

Ok, well, maybe it just takes some practice… or you could cheat...

>>>importpickletools>>>pickletools.dis(p)0:(MARK1:dDICT(MARKat0)2:pPUT05:SSTRING'age'12:pPUT115:IINT2719:sSETITEM20:SSTRING'subdict'31:pPUT234:(MARK35:dDICT(MARKat34)36:pPUT339:SSTRING'second'49:pPUT452:IINT255:sSETITEM56:SSTRING'third'65:pPUT568:IINT371:sSETITEM72:SSTRING'first'81:pPUT684:IINT187:sSETITEM88:sSETITEM89:SSTRING'name'97:pPUT7100:SSTRING'Joe'107:pPUT8110:sSETITEM111:SSTRING'numbers'122:pPUT9125:(MARK126:lLIST(MARKat125)127:pPUT10131:IINT1134:aAPPEND135:IINT2138:aAPPEND139:IINT3142:aAPPEND143:IINT4146:aAPPEND147:IINT5150:aAPPEND151:sSETITEM152:.STOPhighestprotocolamongopcodes=0>>>

You'd still have to read the pickled object from a file, however you wouldn't need to load it. So, if it's a "dangerous" object, you still might be able to figure that out before doing the load. If you are stuck with a pickle, it might be a good option for deciphering what you have.