Question :
If I want to create a new DataFrame with several columns, I can add all the columns at once — for example, as follows:
data = {'col_1': [0, 1, 2, 3],
'col_2': [4, 5, 6, 7]}
df = pd.DataFrame(data)
But now suppose farther down the road I want to add a set of additional columns to this DataFrame. Is there a way to add them all simultaneously, as in
additional_data = {'col_3': [8, 9, 10, 11],
'col_4': [12, 13, 14, 15]}
#Below is a made-up function of the kind I desire.
df.add_data(additional_data)
I’m aware I could do this:
for key, value in additional_data.iteritems():
df[key] = value
Or this:
df2 = pd.DataFrame(additional_data, index=df.index)
df = pd.merge(df, df2, on=df.index)
I was just hoping for something cleaner. If I’m stuck with these two options, which is preferred?
Answer #1:
Pandas has assign method since 0.16.0. You could use it on dataframes like
In [1506]: df1.assign(**df2)
Out[1506]:
col_1 col_2 col_3 col_4
0 0 4 8 12
1 1 5 9 13
2 2 6 10 14
3 3 7 11 15
or, you could directly use the dictionary like
In [1507]: df1.assign(**additional_data)
Out[1507]:
col_1 col_2 col_3 col_4
0 0 4 8 12
1 1 5 9 13
2 2 6 10 14
3 3 7 11 15
Answer #2:
What you need is the join function:
df1.join(df2, how='outer')
#or
df1.join(df2) # this works also
Example:
data = {'col_1': [0, 1, 2, 3],
'col_2': [4, 5, 6, 7]}
df1 = pd.DataFrame(data)
additional_data = {'col_3': [8, 9, 10, 11],
'col_4': [12, 13, 14, 15]}
df2 = pd.DataFrame(additional_data)
df1.join(df2, how='outer')
output:
col_1 col_2 col_3 col_4
0 0 4 8 12
1 1 5 9 13
2 2 6 10 14
3 3 7 11 15
Answer #3:
If you don’t want to create new DataFrame from additional_data, you can use something like this:
>>> additional_data = [[8, 9, 10, 11], [12, 13, 14, 15]]
>>> df['col3'], df['col4'] = additional_data
>>> df
col_1 col_2 col3 col4
0 0 4 8 12
1 1 5 9 13
2 2 6 10 14
3 3 7 11 15
It’s also possible to do something like this, but it would be new DataFrame, not inplace modification of existing DataFrame:
>>> additional_header = ['col_3', 'col_4']
>>> additional_data = [[8, 9, 10, 11], [12, 13, 14, 15]]
>>> df = pd.DataFrame(data=np.concatenate((df.values.T, additional_data)).T, columns=np.concatenate((df.columns, additional_header)))
>>> df
col_1 col_2 col_3 col_4
0 0 4 8 12
1 1 5 9 13
2 2 6 10 14
3 3 7 11 15
Answer #4:
All you need to do is create the new columns with data from the additional dataframe.
data = {'col_1': [0, 1, 2, 3],
'col_2': [4, 5, 6, 7]}
additional_data = {'col_3': [8, 9, 10, 11],
'col_4': [12, 13, 14, 15]}
df = pd.DataFrame(data)
df2 = pd.DataFrame(additional_data)
df[df2.columns] = df2
df now looks like:
col_1 col_2 col_3 col_4
0 0 4 8 12
1 1 5 9 13
2 2 6 10 14
3 3 7 11 15
Indices from the original dataframe will be used as if you had performed an in-place left join. Data from the original dataframe in columns with a matching name in the additional dataframe will be overwritten.
For example:
data = {'col_1': [0, 1, 2, 3],
'col_2': [4, 5, 6, 7]}
additional_data = {'col_2': [8, 9, 10, 11],
'col_3': [12, 13, 14, 15]}
df = pd.DataFrame(data)
df2 = pd.DataFrame(additional_data, index=[0,1,2,4])
df[df2.columns] = df2
df now looks like:
col_1 col_2 col_3
0 0 8 12
1 1 9 13
2 2 10 14
3 3 NaN NaN
