Mandalika's scratchpad [ Work blog @Oracle | My Music Compositions ]

Old Posts: 09.04  10.04  11.04  12.04  01.05  02.05  03.05  04.05  05.05  06.05  07.05  08.05  09.05  10.05  11.05  12.05  01.06  02.06  03.06  04.06  05.06  06.06  07.06  08.06  09.06  10.06  11.06  12.06  01.07  02.07  03.07  04.07  05.07  06.07  08.07  09.07  10.07  11.07  12.07  01.08  02.08  03.08  04.08  05.08  06.08  07.08  08.08  09.08  10.08  11.08  12.08  01.09  02.09  03.09  04.09  05.09  06.09  07.09  08.09  09.09  10.09  11.09  12.09  01.10  02.10  03.10  04.10  05.10  06.10  07.10  08.10  09.10  10.10  11.10  12.10  01.11  02.11  03.11  04.11  05.11  07.11  08.11  09.11  10.11  11.11  12.11  01.12  02.12  03.12  04.12  05.12  06.12  07.12  08.12  09.12  10.12  11.12  12.12  01.13  02.13  03.13  04.13  05.13  06.13  07.13  08.13  09.13  10.13  11.13  12.13  01.14  02.14  03.14  04.14  05.14  06.14  07.14  09.14  10.14  11.14  12.14  01.15  02.15  03.15  04.15  06.15  09.15  12.15  01.16  03.16  04.16  05.16  06.16  07.16  08.16  09.16  12.16  01.17  02.17  03.17  04.17  06.17  07.17  08.17  09.17  10.17  12.17  01.18  02.18  03.18  04.18  05.18  06.18  07.18  08.18  09.18  11.18  12.18  01.19  02.19  05.19  06.19  08.19  10.19  11.19  05.20  10.20  11.20  12.20  09.21  11.21  12.22 


Sunday, January 27, 2019
 
Python Object Persistence Module, pickle — Quick Notes

Pickling or serialization is the process of converting a Python object to a byte stream; and Unpickling or deserialization is the process of re-creating the original in-memory Python object (not necessarily at the same memory address).

Python's pickle module has the necessary methods to pickle and unpickle Python object hierarchies.

pickle module:

It is possible to pickle a variety of data types including built-in types — numeric types (integer, float, complex numbers), sequence types (lists, tuples), text sequence type (strings), binary sequence types (bytes, bytearray), set types (set), mapping types (dictionary), classes and built-in functions defined at the top level of a module.

Any attempt to pickle an unpicklable object may trigger PicklingError exception.

Couple of gotchas:

eg.,

A trivial example demonstrating the calls to pickle (save data to a binary file) and unpickle (load data from the binary file) a Python data structure.

#!/usr/bin/python

import pickle

EMP = {}
EMP['name'] = 'Gary'
EMP['id'] = 12345

# pickle
with open('employee.db', 'wb') as f:
 pickle.dump(EMP, f, pickle.HIGHEST_PROTOCOL)

print '  Pickled data, EMP     ', EMP

# unpickle
with open('employee.db', 'rb') as f:
 EMP_REC = pickle.load(f)

print 'Unpickled data, EMP_REC ', EMP_REC, '\n'

print '(EMP_REC is EMP)? : ', (EMP_REC is EMP)
print '(EMP_REC == EMP)? : ', (EMP_REC == EMP)

Running the above code shows the following on stdout.

  Pickled data, EMP      {'name': 'Gary', 'id': 12345}
Unpickled data, EMP_REC  {'name': 'Gary', 'id': 12345} 

(EMP_REC is EMP)? :  False
(EMP_REC == EMP)? :  True

dump() method takes a serializable Python object as the first argument; and writes pickled representation of the object (serialized object) to a file. Second argument is the file handle that points to an open file. Rest of the arguments are optional.

load() method reads a pickled object representation (serialized data) from a file and returns the reconstructed object. The protocol version is detected automatically so it is not necessary to specify the protocol version during unpickling process.

In-Memory Pickling/Unpickling Operations

If persistence is not a requirement, dumps() and loads() methods in pickle module can be used to serialize (pickle) and deserialize (unpickle) a Python object in memory. This is useful when sending Python objects over network between compatible applications.

eg.,
#!/usr/bin/python

import pickle

EMP = {}
EMP['name'] = 'Gary'
EMP['id'] = 12345

# in-memory pickling
x = pickle.dumps(EMP, pickle.HIGHEST_PROTOCOL)

print '  Pickled data, EMP     ', EMP

# in-memory unpickling
EMP_REC = pickle.loads(x)

print 'Unpickled data, EMP_REC ', EMP_REC, '\n'

print '(EMP_REC is EMP)? : ', (EMP_REC is EMP)
print '(EMP_REC == EMP)? : ', (EMP_REC == EMP)

Running the above code shows output identical to the output produced by the previous code listing - just that there is no file involved this time.

  Pickled data, EMP      {'name': 'Gary', 'id': 12345}
Unpickled data, EMP_REC  {'name': 'Gary', 'id': 12345} 

(EMP_REC is EMP)? :  False
(EMP_REC == EMP)? :  True

Exceptions

As mentioned earlier, any attempt to pickle or unpickle objects that are not appropriate for serialization fail with an exception. Therefore, it is appropriate to safe guard the code with try-except blocks to handle unexpected failures.

Here is another trivial example demonstrating a pickling exception.

#!/usr/bin/python

import sys
import pickle

try:

 f = open('dummy.txt', 'a')
 x = pickle.dumps(f)
 print 'Pickled file handle'

except Exception, e:
 print 'Caught ', e.__class__.__name__, '-',  str(e)

Running the above code throws a TypeError as shown below.

Caught  TypeError - can't pickle file objects

(Credit: Various Sources including Python Documentation)

Labels:





2004-2019 

This page is powered by Blogger. Isn't yours?