How to use Python pickle to serialize and deserialize objects

With Python pickle it is possible to serialize and later deserialize objects. Numerous data types can be considered for conversion. However, as malicious code can also be stored in a memory file, you should only convert files from trustworthy sources into the original format.

What is Python pickle?

Python pickle may seem to have an unusual name at first, but if you take a closer look at how the module works and what it is used for, you will quickly understand why it was given this name. The module allows you to save objects (i.e. “preserve them”, hence “pickle”) in order to use them at a later time for another project. To do this, the objects are converted into a saveable format. This practice is called serializing. Python pickle can also be used to deserialize objects, i.e. to convert them back to their original format. Python pickle is particularly useful if you want to use objects frequently.

The object is converted into a byte stream, whereby all information is transferred unchanged. In addition, Python pickle provides instructions for successful deserialization, through which the original structure can be reconstructed down to the smallest detail. Using Python pickle saves a lot of time, as once an object is created, it does not have to be recreated for each use. The format for saving is .pkl.

Which data types can be converted?

Python pickle can serialize the following data types:

• Boolean values: “true” and “false”, also “none”
• Integers and complex numbers
• Strings (normal and Unicode)
• Lists
• Sets
• Python tuples
• Directories consisting exclusively of corresponding objects
• Functions
• Python classes

What are the different methods?

There are four methods for working with Python pickle, which are provided within the module:

• pickle.dump(obj, file, protocol=None, *, fix_imports=True, buffer_callback=None): Is used for serialization and creates a file with the desired result
• pickle.dumps(obj, protocol=None, *, fix_imports=True, buffer_callback=None): Is also used for serialization, but returns a byte string
• pickle.load(file, *, fix_imports=True, encoding='ASCII', errors="strict", buffers=None): Is used for deserialization and reads the saved file for this purpose
• pickle.loads(bytes_object, *, fix_imports=True, encoding="ASCII", errors="strict", buffers=None): Is also used for deserialization, but operates with a byte string

To differentiate between the methods, you can remember that the “s” in pickle.dumps and pickle.loads stands for “String.”

How to use Python pickle with an example

To better illustrate how Python pickle works, we will work with a simple example. We will create a simple list containing four colors. This is our code:

import pickle
colors = ['Blue', 'Red', 'Yellow', 'Orange']

Then we open a text file in the .pkl format and use pickle.dump() to save our list there. This is the code we use for this:

with open('colors_file.pkl', 'wb') as f:
	pickle.dump(colors, f)

The abbreviation wb instructs the system to open the file in binary form. This also outputs the contained data as a bytes object. The “colors” list is then saved in this file with dump(). Finally, the file is closed automatically.

How to convert a saved file to its original format

If you now want to deserialize a binary file again, use the Python method pickle.load(). With the following code, it is possible to convert the object back to its original format and initiate an output. We add the abbreviation rb, which stands for “read binary”.

with open('colors_file.pkl', 'rb') as f:
	colors_deserialized = pickle.load(f)
	print(colors_deserialized)

This gives us the following output:

['Blue', 'Red', 'Yellow', 'Orange']

How to serialize dictionary with Python pickle

You can also easily serialize more complex data types such as directories with Python pickle and then convert them back to their original form. To do this, we first create a directory with the name “persons”. Here we store different data for different people:

import pickle
persons = {
	'Person 1': {
		'Name': "Maria", 'Age': 56, 'City': "Los Angeles"
	},
	'Person 2': {
		'Name': "Paul", 'Age': 66, 'City': "Los Angeles"
	},
	'Person 3': {
		'Name': "Lisa", 'Age': 22, 'City': "Chicago"
	},
	'Person 4': {
		'Name': "Lara", 'Age': 34, 'City': "San Francisco"
	}
}

In the following code, we create a new file, convert the data, and then test by converting it back to serialize this directory:

with open("persons_dict.pkl", "wb") as f:
	pickle.dump(persons, f)
with open("persons_dict.pkl", "rb") as f:
	deserialized_dict = pickle.load(f)
	print(deserialized_dict)

The resulting output looks like this:

persons = {
    'Person 1': { 'Name': "Maria", 'Age': 56, 'City': "Los Angeles"},
    'Person 2': { 'Name': "Paul", 'Age': 66, 'City': "Los Angeles"},
    'Person 3': { 'Name': "Lisa", 'Age': 22, 'City': "Chicago"},
    'Person 4': { 'Name': "Lara", 'Age': 34, 'City': "San Francisco"}
}

You can now access the information as usual. We request the following output as an example:

# Define dictionary
deserialized_dict = {
    'Person 1': {'Name': "Maria", 'Age': 56, 'City': "Los Angeles"},
    'Person 2': {'Name': "Paul", 'Age': 66, 'City': "Los Angeles"},
    'Person 3': {'Name': "Lisa", 'Age': 22, 'City': "Chicago"},
    'Person 4': {'Name': "Lara", 'Age': 34, 'City': "San Francisco"}
}
# Print output
print(
    "The name of the third person is"
    + deserialized_dict["Person 3"]["Name"]
    + " and they are "
    + str(deserialized_dict["Person 3"]["Age"])
    + " years old."
)

This is what our output looks like:

The name of the third person is Lisa and she is 22 years old.

How to convert a class to a string

In the next example, we use Python pickle to save a class in a string. This class contains completely different data types, but all of them can be taken into account. We create a class called “ExampleClass” and then serialize it. The corresponding code is:

import pickle
class ExampleClass:
	def __init__(self):
		self.a_number = 17
		self.a_list = [5, 10, 15]
		self.a_tuple = (18, 19)
		self.a_string = "hello"
		self.a_dict = {"color": "blue", "number": 3}
example_object = ExampleClass()
serialized_object = pickle.dumps(example_object)
print(f"This is a serialized object:\n{serialized_object}\n")
example_object.a_dict = None
deserialized_object = pickle.loads(serialized_object)
print(f"This is a_dict from the deserialized object:\n{deserialized_object.a_dict}\n")

After serializing the class and then converting it back to its original format, we get this output:

This is a serialized object:
b'\x80\x03c__main__\nExampleClass\nq\x00)\x81q\x01.'
This is a_dict from the deserialized object:
{'color': 'blue', 'number': 3}

How to compress serialized objects

In principle, files saved with Python pickle are comparatively compact. Nevertheless, it is also possible and, in some cases, advisable to compress the memory files even more. This works, for example, with the free compression program bzip2, which is part of the programming language’s standard library. In the following example, we create a string, serialize it and then apply the compression program:

import pickle
import bz2
exampleString = """Almost heaven, West Virginia
Blue Ridge Mountains, Shenandoah River
Life is old there, older than the trees
Younger than the mountains, growin' like a breeze
Country roads, take me home
To the place I belong
West Virginia, mountain mama
Take me home, country roads."""
serialized = pickle.dumps(exampleString)
compressed = bz2.compress(serialized)

Security instructions for working with Python pickle

Even though Python pickle is a practical and effective method for converting objects, there is one major drawback that you should be aware of when working with the module, which is that there is the possibility of transferring malicious code via serialized data. Although this is not a problem with your own data, you have to be careful when it comes to third-party files. Therefore, only ever deserialize memory files when you know and trust the source!