Python – TypeError: Object of type ‘int64’ is not JSON serializable

Posted on

Question :

Python – TypeError: Object of type ‘int64’ is not JSON serializable

I have a data frame that stores store name and daily sales count. I am trying to insert this to Salesforce using the Python script below. However, I get the following error:

TypeError: Object of type 'int64' is not JSON serializable

Below, there is the view of the data frame.

Storename,Count
Store A,10
Store B,12
Store C,5

I use the following code to insert it to Salesforce.

update_list = []
for i in range(len(store)):
    update_data = {
        'name': store['entity_name'].iloc[i],
        'count__c': store['count'].iloc[i] 
    }
    update_list.append(update_data)

sf_data_cursor = sf_datapull.salesforce_login()
sf_data_cursor.bulk.Account.update(update_list)

Getting the error, when the last line above gets executed. Could anyone assist in fixing this? Thanks..

Answer #1:

json does not recognize NumPy data types. Convert the number to a Python int before serializing the object:

'count__c': int(store['count'].iloc[i])
Answered By: DYZ

Answer #2:

You can define your own encoder to solve this problem.

import json
import numpy as np

class NpEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, np.integer):
            return int(obj)
        elif isinstance(obj, np.floating):
            return float(obj)
        elif isinstance(obj, np.ndarray):
            return obj.tolist()
        else:
            return super(NpEncoder, self).default(obj)

# Your codes .... 
json.dumps(data, cls=NpEncoder)
Answered By: Jie Yang

Answer #3:

I’ll throw in my answer to the ring as a bit more stable version of @Jie Yang’s excellent solution.

My solution

numpyencoder and its repository.

from numpyencoder import NumpyEncoder

numpy_data = np.array([0, 1, 2, 3])

with open(json_file, 'w') as file:
    json.dump(numpy_data, file, indent=4, sort_keys=True,
              separators=(', ', ': '), ensure_ascii=False,
              cls=NumpyEncoder)

The breakdown

If you dig into hmallen’s code in the numpyencoder/numpyencoder.py file you’ll see that it’s very similar to @Jie Yang’s answer:


class NumpyEncoder(json.JSONEncoder):
    """ Custom encoder for numpy data types """
    def default(self, obj):
        if isinstance(obj, (np.int_, np.intc, np.intp, np.int8,
                            np.int16, np.int32, np.int64, np.uint8,
                            np.uint16, np.uint32, np.uint64)):

            return int(obj)

        elif isinstance(obj, (np.float_, np.float16, np.float32, np.float64)):
            return float(obj)

        elif isinstance(obj, (np.complex_, np.complex64, np.complex128)):
            return {'real': obj.real, 'imag': obj.imag}

        elif isinstance(obj, (np.ndarray,)):
            return obj.tolist()

        elif isinstance(obj, (np.bool_)):
            return bool(obj)

        elif isinstance(obj, (np.void)): 
            return None

        return json.JSONEncoder.default(self, obj)

Answer #4:

This might be the late response, but recently i got the same error. After lot of surfing this solution helped me.

def myconverter(obj):
        if isinstance(obj, np.integer):
            return int(obj)
        elif isinstance(obj, np.floating):
            return float(obj)
        elif isinstance(obj, np.ndarray):
            return obj.tolist()
        elif isinstance(obj, datetime.datetime):
            return obj.__str__()

Call myconverter in json.dumps() like below.
json.dumps('message', default=myconverter)

Answered By: shiva

Answer #5:

A very simple numpy encoder can achieve similar results more generically.

Note this uses the np.generic class (which most np classes inherit from) and use the a.item() method.

import numpy as np

def np_encoder(object):
    if isinstance(object, np.generic):
        return object.item()

json.dumps(obj, default=np_encoder)
Answered By: conmak

Answer #6:

If you have this error

TypeError: Object of type ‘int64’ is not JSON serializable

You can change that specific columns with int dtype to float64, as example:

df = df.astype({'col1_int':'float64', 'col2_int':'float64', etc..})

Float64 is written fine in Google Spreadsheets

Answered By: Kostiantyn Chichko

Answer #7:

Another option is that when you create the dataframe use dtype=str

For example, if you loaded store from a csv file:

import pandas as pd
store = pd.read_csv('store.csv', dtype=str)

Then everything has a type of str which can be serialized to json.

Answered By: MattCochrane

Leave a Reply

Your email address will not be published. Required fields are marked *