How to Work With JSON Data Using Python

This tutorial shows how easy it is to use the Python programming language to work with JSON data.

Before I begin the topic, let's define briefly what we mean by JSON. Let's see how JSON's main website defines it:

JSON (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate. It is based on a subset of the JavaScript Programming Language, Standard ECMA-262 3rd Edition - December 1999. JSON is a text format that is completely language independent but uses conventions that are familiar to programmers of the C-family of languages, including C, C++, C#, Java, JavaScript, Perl, Python, and many others. These properties make JSON an ideal data-interchange language.

Thus, JSON is a simple way to create and store data structures within JavaScript. The reason you see JavaScript in the acronym is that a JavaScript object is created when storing data with JSON. But don't worry, you don't need to know JavaScript to work with JSON files—rather it is about the JSON syntax (format) itself.

In brief, JSON is a way by which we store and exchange data, which is accomplished through its syntax and is used in many web applications. The nice thing about JSON is that it has a human-readable format, and this may be one of the reasons for using it in data transmission, in addition to its effectiveness when working with APIs.

An example of JSON-formatted data is as follows:

{
  "name": "Frank",
  "age": 39,
  "isEmployed": true
}

In this tutorial, I will show you how to use Python to work with JSON files. So let's get started!

Python and JSON

Python makes it simple to work with JSON files. The module used for this purpose is the json module. This module should be included (built-in) within your Python installation, and you thus don't need to install any external modules as we did when working with PDF and Excel files, for instance. The only thing you need in order to use this module is to import it:

import json

But what does the json library do? This library mainly parses JSON from files or strings. It also parses JSON into a dictionary or list in Python and vice versa, that is converting a Python dictionary or list into JSON strings.

JSON to Python

Reading JSON means converting JSON into a Python value (object). As mentioned above, the json library parses JSON into a dictionary or list in Python. In order to do that, we use the loads() function (load from a string), as follows:

import json
jsonData = '{"name": "Frank", "age": 39}'
jsonToPython = json.loads(jsonData)

If you want to see the output, do a print jsonToPython, in which case you will get the following output:

{u'age': 39, u'name': u'Frank'}

That is, the data is returned as a Python dictionary (JSON object data structure). So will the statement print jsonToPython['name'] return any output? Go ahead and try it out.

As we just saw, objects in JSON are converted to dictionaries in Python. The conversion of JSON data to Python is based on the following conversion table.

Python to JSON

In the previous section, we saw how to convert JSON into a Python value (i.e. Dictionary). In this section, I will show you how we can convert (encode) a Python value to JSON.

Say that we have the following Dictionary in Python:

import json
pythonDictionary = {'name':'Bob', 'age':44, 'isEmployed':True}
dictionaryToJson = json.dumps(pythonDictionary)

If we print dictionaryToJson, we get the following JSON data:

{"age": 44, "isEmployed": true, "name": "Bob"}

So this output is considered the data representation of the object (Dictionary). The method dumps() was the key to such an operation. The conversion of Python objects to JSON data is based on the following conversion table.

The keys in a Python dictionary have to be converted to a string for them to be used as JSON data. However, a simple string conversion is only feasible for basic types like str, int, float, and bool. For other types of keys, this can result in a TypeError. You can avoid that from happening by setting the value of skipkeys argument to True. This will tell Python to skip all keys which cannot be converted to a string.

import json

class Author:
    def __init__(self, name):
        self.name = name

monty = Author('Monty')

myDictionary = {'tobby': 70, 'adam': 80, monty: 20, 'andrew': 75, 'sally': 99}

# TypeError: keys must be str, int, float, bool or None, not Author
# pythonToJSON = json.dumps(myDictionary)

pythonToJSON = json.dumps(myDictionary, skipkeys=True)

# Outputs: {"tobby": 70, "adam": 80, "andrew": 75, "sally": 99}
print(pythonToJSON)

There is another argument called sort_keys which can be set to True in order to output the dictionary after sorting it by its keys.

import json

myDictionary = {'tobby': 70, 'adam': 80, 'monty': 20, 'andrew': 75, 'sally': 99}

pythonToJSON = json.dumps(myDictionary, sort_keys=True)

# Outputs: {"adam": 80, "andrew": 75, "monty": 20, "sally": 99, "tobby": 70}
print(pythonToJSON)

Back and Forth Conversion of Data

You probably already know that keys for dictionaries in Python can be of different data types like strings, int, or tuples. However, the keys in JSON data can only be strings. This means that when you convert a dictionary into JSON, all its keys will be cast to strings. Conversion of this JSON back to a dictionary will not get you back the original data type of the keys.

import json

squares = {1: 1, 2: 4, 3: 9, 4: 16, 5: 25, False: None}

pythonToJSON = json.dumps(squares)
jsonToPython = json.loads(pythonToJSON)

# Outputs: {1: 1, 2: 4, 3: 9, 4: 16, 5: 25, False: None}
print(squares)

# {"1": 1, "2": 4, "3": 9, "4": 16, "5": 25, "false": null}
print(pythonToJSON)

# {'1': 1, '2': 4, '3': 9, '4': 16, '5': 25, 'false': None}
print(jsonToPython)

Storing Different Data Types or Objects as JSON

It is important to note at this point that JSON cannot store all types of Python objects, but only the following types: Lists, Dictionaries, Booleans, Numbers, Character strings, and None. Thus, any other types need to be converted in order to be stored in JSON.

Let's say we have the following class:

class Employee:
    def __init__(self, name):
        self.name = name

Let's say we created a new object abder, as follows:

abder = Employee('Abder')

What if we wanted to convert this object to JSON? That is json.dumps(abder)? In this case, you would get an error similar to the following:

Traceback (most recent call last):
  File "test.py", line 8, in <module>
    abderJson = json.dumps(abder)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 243, in dumps
    return _default_encoder.encode(obj)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 207, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 270, in iterencode
    return _iterencode(o, 0)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: <__main__.Employee object at 0x10e74b750> is not JSON serializable

But is there a workaround? Fortunately there is. I like the workaround described on the Python Tips website. To solve this issue, we can define a method similar to the following:

def jsonDefault(object):
    return object.__dict__

Then encode the object into JSON as follows:

jsonAbder = json.dumps(abder, default=jsonDefault)

If you print jsonAbder, you should get the following output:

{"name": "Abder"}

We have now encoded a Python object (abder) into JSON.

Conclusion

From this tutorial, we can notice that Python again and again is proving not only its ability to work with different applications, but also its flexibility to work with different issues while working with an application, as we saw in the last part of the tutorial.

If you want to know more about the json module, you can visit the documentation page.

This post has been updated with contributions from Monty Shokeen. Monty is a full-stack developer who also loves to write tutorials and to learn about new JavaScript libraries.