Words: 462 487
Time to read: ~ 3 minutes
TL;DR: [System.Collections.Generic.HashSet<T>]
It has been a while ago since I’ve blogged so it seems fitting that this post will be about a question that was asked of me a while ago. The question was along the lines of “Can I join objects in PowerShell but remove duplicates?”.
So this is allowed:
1
2
3
4
But this isn’t allowed:
1
2
3
1
For me, it summed down to “Is there something like UNION
or UNION ALL
in PowerShell?
Luckily, this is something that I had asked before and been told the answer. So here I am, repeating the answer for you all since this is how I learn; repetition and practice.
The PowerShell type:
[System.Collections.Generic.HashSet<T>]
UPDATE: I have been reliably informed that this is a major understatement and I will update when I know more/learn more/am taught more about this.
An Example, not a Speech.
Let’s say we have similar but different objects.
$Boom = 1
$Blast = 2
$And = 3
$Ruin = 4
Now the question that we are asked is, if there is anyway to do a UNION
on these objects?
Absolutely, we’ll create a hashset
object and start putting these objects into the HashSet. I know these objects are integers so I’m going to put [Int]
in for the <T>
type.
#Create the HashSet object.
$HashSet = [System.Collections.Generic.HashSet[Int]]::new()
#Add the objects.
foreach ($Item in $Boom,$Blast,$And,$Ruin) {
$HashSet.Add($Item)
}

Now when you add something to a HashSet using the .Add
method it returns either a True or a False.
Returns
https://docs.microsoft.com/en-us/dotnet/api/system.collections.generic.hashset-1.add?view=netframework-4.7.2#System_Collections_Generic_HashSet_1_Add__0_
Booleantrue
if the element is added to the HashSet<T> object;false
if the element is already present.
If we then check the HashSet object, it returns both of our objects.
#Show us what you GOT!!!
$HashSet

If you don’t want to see the output of the .Add()
method then you can push the output to one of the nulls e.g. $null =
, [void]
, > $null
, | Out-Null
, with the first two being placed at the start of the line and the last two at the end.
# Clear the HashSet.
$HashSet.Clear()
# Add items and hide output.
$null = foreach ($Item in $Boom,$Blast,$And,$Ruin) {
$HashSet.Add($Item)
}
# See if it still worked.
$HashSet

Check for Duplicates.
Now let’s see what happens if we try to add a duplicate object to the HashSet.
#Clear the HashSet
$HashSet.Clear()
#Populate it again.
foreach ($Item in $Boom,$Blast,$And,$Boom){
$HashSet.Add($Item)
}
#Check it again
$HashSet

HashSet sees the duplicate value, gracefully says no (False), and does not add it.
But wait, there’s more!
HashSet is something that I think DBAs will like as it is based on Sets. If you have the time, check out some of the other methods that it has.
#What else you got?
Get-Member -InputObject $HashSet

Take a look!
How about just adding two arrays then using Select-Object?
@(1,2,3) + @(2,3,4) | Select-Object -Unique
Output:
1
2
3
4
Yup, that seems to work! 👍 And being honest, if I was only joining two non-large arrays, then I’m 100% behind you and I’d use Select-Object and -Unique as well.
There’s a couple of extra methods that HashSet has that I want to dive into more – plus the age old question on how to they perform against each over; Select -Unique vs. HashSet
But yeah that method should work and it’s nice and clear! 🙂