Prevent adding duplicates to a list in Python

avatar

Borislav Hadzhiev

Last updated: Aug 17, 2022

banner

Photo from Unsplash

Prevent adding duplicates to a list in Python #

To prevent adding duplicates to a list:

  1. Use the not in operator to check if the value is not in the list.
  2. If the value is not in the list, use the list.append() method to add it.
  3. The list won't contain any duplicates.
main.py
my_list = ['a', 'b'] value = 'c' if value not in my_list: my_list.append(value) print(my_list) # 👉️ ['a', 'b', 'c'] # ---------------------------------------- my_list = [] values = ['a', 'a', 'b', 'b', 'c', 'c'] for value in values: if value not in my_list: my_list.append(value) print(my_list) # 👉️ ['a', 'b', 'c']

We used the not in operator to prevent adding duplicates to a list.

The in operator tests for membership. For example, x in l evaluates to True if x is a member of l, otherwise it evaluates to False.

x not in l returns the negation of x in l.

If the value is not in the list, we use the list.append() method to add it.

The list.append() method adds an item to the end of the list.

The method returns None as it mutates the original list.

If you need to prevent adding duplicates to a list while iterating over a collection of values, use the not in operator on each iteration.

main.py
my_list = [] values = ['a', 'a', 'b', 'b', 'c', 'c'] for value in values: if value not in my_list: my_list.append(value) print(my_list) # 👉️ ['a', 'b', 'c']

We used a for loop to iterate over a list of values.

On each iteration, we check if the current value is not present in the other list.

If the condition is met, we use the list.append() method to add the value to the list.

The new list won't contain any duplicates.

If the order of the elements in the list is not important, consider using a set object.

Set objects are an unordered collection of unique elements.
main.py
values = ['a', 'a', 'b', 'b', 'c', 'c'] my_set = list(set(values)) print(my_set) # 👉️ ['b', 'a', 'c']

We used the set() class to convert a list to a set object. Since set objects only store unique values, any duplicates get automatically removed.

However, you should only use this approach if the order of the elements is not important because set objects are unordered.

If you decide to use a set object, you don't have to check if the value is already present in the set.

You can directly use the set.add() method because duplicate values cannot be added to the set.

main.py
jmy_set = set() my_set.add(1) my_set.add(1) print(my_set) # 👉️ {1}

Calling the set.add() method multiple times with the same value has no effect.

I wrote a book in which I share everything I know about how to become a better, more efficient programmer.
book cover
You can use the search field on my Home Page to filter through all of my articles.