Pandas Series: duplicated() function

Indicate duplicate Series values

The duplicated() function is used to indicate duplicate Series values.

Duplicated values are indicated as True values in the resulting Series. Either all duplicates, all except the first or all except the last occurrence of duplicates can be indicated.

Syntax:

Series.duplicated(self, keep='first')

Parameters:

Returns: Series indicating whether each value has occurred in the preceding values.

Example - By default, for each set of duplicated values, the first occurrence is set on False and all others on True:

Python-Pandas Code:

import numpy as np
import pandas as pd
animals = pd.Series(['dog', 'cow', 'dog', 'cat', 'dog'])
animals.duplicated()

Output:

0    False
1    False
2     True
3    False
4     True
dtype: bool

Example - which is equivalent to:

Python-Pandas Code:

import numpy as np
import pandas as pd
animals = pd.Series(['dog', 'cow', 'dog', 'cat', 'dog'])
animals.duplicated(keep='first')

Output:

0    False
1    False
2     True
3    False
4     True
dtype: bool

Example - By using ‘last’, the last occurrence of each set of duplicated values is set on False and all others on True:

Python-Pandas Code:

import numpy as np
import pandas as pd
animals = pd.Series(['dog', 'cow', 'dog', 'cat', 'dog'])
animals.duplicated(keep='last')

Output:

0     True
1    False
2     True
3    False
4    False
dtype: bool

Example - By setting keep on False, all duplicates are True:

Python-Pandas Code:

import numpy as np
import pandas as pd
animals = pd.Series(['dog', 'cow', 'dog', 'cat', 'dog'])
animals.duplicated(keep=False)

Output:

0     True
1    False
2     True
3    False
4     True
dtype: bool

Previous: Remove Pandas series with duplicate values
Next: Test Pandas objects contain the same elements