Master JavaScript: Top Methods to Remove Duplicates from Arrays

JavaScript developers frequently encounter duplicate elements in arrays or collections. Duplicate removal is critical for improving performance and maintaining data integrity. In this post, we will look at many approaches for removing duplicates from an array in JavaScript, ranging from basic to advance.

Understanding the Problem

When working with arrays, duplicates might occur due to user inputs, API responses, or processing issues.

Example:

const numbers = [1, 2, 3, 2, 4, 1, 5];

The purpose is to eliminate duplicate elements, which results in:

const uniqueNumbers = [1, 2, 3, 4, 5];

Let’s take a look at some of the different approaches to achieve this.

1. Using Set

The Set object in JavaScript stores unique data, making it a simple way to remove duplicates.

const numbers = [1, 2, 3, 2, 4, 1, 5];
const uniqueNumbers = [...new Set(numbers)];
console.log(uniqueNumbers); // [1, 2, 3, 4, 5]

How It Works:

• The Set object filters out duplicates automatically.
• The ... spread operator converts the Set back to an array.

Pros:

• Simple and concise.
• High performance for most use cases.

Cons:

• Only works for primitive values (numbers, strings, etc.). Does not work for arrays of objects (unless the objects are references to the same object).

1. Using Reduce()

The reduce() method uses functional programming to remove duplicates.

const numbers = [1, 2, 3, 2, 4, 1, 5];
const uniqueNumbers = numbers.reduce((acc, value) => {
  if (!acc.includes(value)) {
    acc.push(value);
  }
  return acc;
}, []);
console.log(uniqueNumbers); // [1, 2, 3, 4, 5]

How It Works:

• Reduce() iterates through the array, keeping an accumulator (acc) for unique values.
• Prior to adding an element, the includes() method checks for duplicates.

Pros:

• Flexible and customizable.

Cons:

• Slower than Set for large arrays. It is Inefficient for large arrays (time complexity of O(n²)).

1. Using filter() and indexOf()

How It Works:

• The indexOf() function returns the value's first occurrence in the array.
• The filter() function maintains elements whose current index matches the first occurrence.

Pros:

• Works well with primitive values.

Cons:

• Less efficient for large arrays due to nested iteration.

1. Using forEach() and an Object

You may use an object to keep track of unique values when you go over the array with forEach().

const array = [1, 2, 2, 3, 4, 4, 5];
const uniqueArray = [];
const seen = {};

array.forEach((value) => {
  if (!seen[value]) {
    seen[value] = true;
    uniqueArray.push(value);
  }
});

console.log(uniqueArray); // Output: [1, 2, 3, 4, 5]

How It Works:

• Seen is an object that keeps track of values encountered.
• If a value is not already present, it is added to the uniqueArray and marked as seen.

Pros:

• Efficient for large arrays (time complexity of O(n)).
• Works for primitive values.

Cons:

• Requires additional memory for the seen object.

1. Using Map for Advanced Use Cases

Map may be used to eliminate duplicates from object arrays or to filter them based on a given property.

const users = [
  { id: 1, name: "Alice" },
  { id: 2, name: "Bob" },
  { id: 1, name: "Alice" },
];

const uniqueUsers = Array.from(
  new Map(users.map(user => [user.id, user])).values()
);

console.log(uniqueUsers);
// [ { id: 1, name: "Alice" }, { id: 2, name: "Bob" } ]

How It Works:

• Map stores unique key-value pairs, with the id as the key.
• Array.from() converts the Map back into an array.

Pros:

• Ideal for arrays of objects.
• Efficient for large datasets.

Cons:

• Requires understanding of Map.

Understanding and adopting the appropriate approach ensures that the code is efficient, clean, and maintainable. Happy coding!