Python & Pandas: How to query if a list-type column contains something?

Posted on

Question :

Python & Pandas: How to query if a list-type column contains something?

I have a dataframe, which contains info about movies. It has a column called genre, which contains a list of genres it belongs to. For example:


## returns 

0       ['comedy', 'sci-fi']
1       ['action', 'romance', 'comedy']
2       ['documentary']
3       ['crime','horror']

I want to know how can I query the dataframe, so it returns the movie belongs to a cerain genre?

For example, something may like df['genre'].contains('comedy') returns 0 or 1.

I know for a list, I can do things like:

'comedy' in  ['comedy', 'sci-fi']

However, in pandas, I didn’t find something similar, the only thing I know is df['genre'].str.contains(), but it didn’t work for the list type.

Asked By: cqcn1991


Answer #1:

You can use apply for create mask and then boolean indexing:

mask = df.genre.apply(lambda x: 'comedy' in x)
df1 = df[mask]
print (df1)
0           [comedy, sci-fi]
1  [action, romance, comedy]
Answered By: jezrael

Answer #2:

using sets['comedy']).issubset)

0     True
1     True
2    False
3    False
dtype: bool


0             [comedy, sci-fi]
1    [action, romance, comedy]
dtype: object

presented in a way I like better

comedy = set(['comedy'])
iscomedy = comedy.issubset

more efficient

comedy = set(['comedy'])
iscomedy = comedy.issubset
df[[iscomedy(l) for l in df.genre.values.tolist()]]

using str in two passes
slow! and not perfectly accurate!

df[df.genre.str.join(' ').str.contains('comedy')]
Answered By: piRSquared

Answer #3:

A complete example:

import pandas as pd

data = pd.DataFrame([[['foo', 'bar']],
                    [['bar', 'baz']]], columns=['list_column'])
0  [foo, bar]
1  [bar, baz]

filtered_data = data.loc[
    lambda df: df.list_column.apply(
        lambda l: 'foo' in l
0  [foo, bar]
Answered By: Adrien Renaud

Answer #4:

According to the source code, you can use .str.contains(..., regex=False).

Answered By: HYRY

Answer #5:

One liner using boolean indexing and list comprehension:

searchTerm = 'something'
df[[searchTerm in x for x in df['arrayColumn']]]
Answered By: bloodrootfc

Leave a Reply

Your email address will not be published. Required fields are marked *