Identify duplicate values in a list in Python

Answer #1:

These answers are O(n), so a little more code than using mylist.count() but much more efficient as mylist gets longer

If you just want to know the duplicates, use collections.Counter

from collections import Counter
mylist = [20, 30, 25, 20]
[k for k,v in Counter(mylist).items() if v>1]

If you need to know the indices,

from collections import defaultdict
D = defaultdict(list)
for i,item in enumerate(mylist):
    D[item].append(i)
D = {k:v for k,v in D.items() if len(v)>1}

Answer #2:

Here’s a list comprehension that does what you want. As @Codemonkey says, the list starts at index 0, so the indices of the duplicates are 0 and 3.

>>> [i for i, x in enumerate(mylist) if mylist.count(x) > 1]
[0, 3]

Answer #3:

You can use list compression and set to reduce the complexity.

my_list = [3, 5, 2, 1, 4, 4, 1]
opt = [item for item in set(my_list) if my_list.count(item) > 1]

Answer #4:

simplest way without any intermediate list using list.index():

z = ['a', 'b', 'a', 'c', 'b', 'a', ]
[z[i] for i in range(len(z)) if i == z.index(z[i])]
>>>['a', 'b', 'c']

and you can also list the duplicates itself (may contain duplicates again as in the example):

[z[i] for i in range(len(z)) if not i == z.index(z[i])]
>>>['a', 'b', 'a']

or their index:

[i for i in range(len(z)) if not i == z.index(z[i])]
>>>[2, 4, 5]

or the duplicates as a list of 2-tuples of their index (referenced to their first occurrence only), what is the answer to the original question!!!:

[(i,z.index(z[i])) for i in range(len(z)) if not i == z.index(z[i])]
>>>[(2, 0), (4, 1), (5, 0)]

or this together with the item itself:

[(i,z.index(z[i]),z[i]) for i in range(len(z)) if not i == z.index(z[i])]
>>>[(2, 0, 'a'), (4, 1, 'b'), (5, 0, 'a')]

or any other combination of elements and indices….

Answer #5:

The following list comprehension will yield the duplicate values:

[x for x in mylist if mylist.count(x) >= 2]

Answer #6:

That’s the simplest way I can think for finding duplicates in a list:

my_list = [3, 5, 2, 1, 4, 4, 1]

my_list.sort()
for i in range(0,len(my_list)-1):
               if my_list[i] == my_list[i+1]:
                   print str(my_list[i]) + ' is a duplicate'

Answer #7:

I tried below code to find duplicate values from list

1) create a set of duplicate list

2) Iterated through set by looking in duplicate list.

glist=[1, 2, 3, "one", 5, 6, 1, "one"]
x=set(glist)
dup=[]
for c in x:
    if(glist.count(c)>1):
        dup.append(c)
print(dup)

OUTPUT

[1, ‘one’]

Now get the all index for duplicate element

glist=[1, 2, 3, "one", 5, 6, 1, "one"]
x=set(glist)
dup=[]
for c in x:
    if(glist.count(c)>1):
        indices = [i for i, x in enumerate(glist) if x == c]
        dup.append((c,indices))
print(dup)

OUTPUT

[(1, [0, 6]), (‘one’, [3, 7])]

Hope this helps someone

Answer #8:

The following code will fetch you desired results with duplicate items and their index values.

  for i in set(mylist):
    if mylist.count(i) > 1:
         print(i, mylist.index(i))