Finding Duplicate Objects in Active Directory

Finding Duplicate Objects in Active Directory

Be the first to vote!

For those of you that have embarked upon the trek to Office 365, you’ve undoubtedly run (or at least heard of) IDFix.  It detects and fixes a number of conditions that will cause the directory sync to report errors.

Today, I want to focus on a tool I wrote for a customer almost 2 years ago that addresses conditions not yet identified or remedied by IDFix.  I just updated the tool today (and as of 2018-04-17, even more recently), which reminded me of why I wrote it in the first place:

  1. When two objects of the same class have overlapping values in different attributes (ie, User 1 has a UPN of and User 2 has a UPN of but a proxy address of
  2. When two or more objects of different classes have overlapping values (ie, User 1 has a UPN of and Contact 1 has a mail or proxy address of
  3. When an object has a value in msRTCSIP-PrimaryUserAddress that conflicts with the value of another attribute on another user object.


  • -Auto switch (which is, shockingly, engaged automatically) to try to determine if you have the Active Directory schema extended for either Exchange (to check proxyAddresses) or LCS/OCS/Skype (to check msRTCSIP-PrimaryUserAddress).
  • Color highlighting for matches so you can see exactly where the pesky values are hiding.
  • Apparently, the original version of the script did substring matching (using the -LDAPFilter parameter of Get-ADObject), which can add extra objects to the output ( will match things like user1@domain.com1 and  For those of you who want to use it to find partial matches, I’ve put that functionality in the -LDAPStyle parameter and use a better syntax for finding exact matches.

Case 1: Overlapping attribute values in different attributes

In the first instance, we’re going to look for users where they have an overlapping attribute value in different attributes.  IDFix is good at locating duplicate values when they appear in the same object class and the same attribute–like finding two users with set as the UPN or mail address.  What IDFix doesn’t do is to identify users where these attributes may have been jumbled–User 1 may have an email address of and a UPN of, while User 2 may have an email address of and a UPN of  From AAD’s perspective, the object that gets written to the directory first wins, and all other objects with conflicting attributes result in an error condition.  From the user perspective, I typically err on the side of letting the user who has the value as an email address win and then fixing everything else.  AADConnect will let you know you have an error value, but it only shows you the value for the user in error–not the user that got there first.

Case 2: Overlapping attribute values in dissimilar object classes

This one also happens frequently: someone on the sales team thinks they need a shared mailbox (like  Someone on the sales team thinks they need a security group for the sales team home directory, and the service desk creates a security group.  The service desk has a procedure to put the email address of the person or group responsible for the security group in the mail attribute of the group.  Sounds crazy? I’ve had several enterprise customers that do exactly that. IDFix won’t pick up on this, but Azure AD will let you know in no uncertain terms that you’ve gone out of bounds.

Case 3: Overlap in the msRTCSIP-PrimaryUserAddress attribute

Back in the olden days of Live Communications Server and Office Communications Server, this attribute was used to store the SIP user address.  Chances are, you no longer have these servers around.  Chances are that you also just shut them off like a lot of people did when you started using a new IM/conferencing service.  And, chances are you still have data in this attribute.  I had a customer last year that had legacy data in that attribute that was conflicting with new users being created (for example, a user named John Smith had a UPN and email address value of, but in the msRTCSIP-PrimaryUserAddress attribute when OCS 2007 had been deployed.  Years later, a new user named Jason Smith had somehow been provisioned with as his primary SMTP address, and Azure AD alerted them that there was a conflict.

Why these edge cases are important

So, now that I’ve identified some things that don’t work, let’s talk about why they are important.

In all of these cases, it means that the first object to get synchronized to AAD wins, and all other objects that have conflicting attributes will fail to synchronize.  Undoubtedly, some of these objects are important (in the case of users with mailboxes that have attributes that were updated directly instead of via the Exchange interface or cmdlets).  Additionally, if users have incorrect values, it could mean non-delivery of email, inability to access services, or email gets delivered to the wrong user.  All of those seem not so good.

With that in mind, here’s the abridged version of how to use the tool.

In my lab, I created a similar scenario:

  1. I created a test user ( and synchronized to AAD.
  2. I created a second test user (, but manually added the mail attribute, which will cause a collision, since the data in that indexed attribute can’t exist on any other object (the indexed attributes are UserPrincipalName, Mail, ProxyAddresses, and msRTCSIP-PrimaryUserAddress).

After synchronizing, I received the expected error:


To locate the offending user:

  1. Click on the data in the Error column (InvalidSoftMatch).
  2. Click on the Detail button.
  3. Review the error. In this case, it tells you that the attribute Mail with a value of is conflicting with something else.  Unfortunately, it doesn’t tell you what object that attribute is on (in this window) nor does it tell you the other object holding that value.
  4. Copy the value immediately following the identified attribute (in this case,
  5. Open a PowerShell prompt and navigate to the directory containing the script you downloaded.
  6. Run .\Find-DuplicateValues.ps1 -IncludeExchange -Address
  7. Review the output.  In this case, you can see two user objects match:

The first object is in OU=Test,DC=forest,DC=com and the conflicting object is in OU=Users,OU=Test,DC=forest,DC=com.  The attribute is both the UPN and primary SMTP address for the object in OU=Test,DC=forestc,DC=com, and is only in the mail attribute for the user in OU=Users,OU=Test,DC=forest,DC=com.  It will take a human to determine which one is “correct.”

You can find the full tool here: Find Objects with Duplicate UPN or SMTP Values in Active Directory

Published by Aaron Guilmette

Helping companies conquer inferior technology since 1997. I spend my time developing and implementing technology solutions so people can spend less time with technology. Specialties: Active Directory and Exchange consulting and deployment, Virtualization, Disaster Recovery, Office 365, datacenter migration/consolidation, cheese.

Reader Comments

  1. Tried in new Exchange and windows powershell

    Get-ADObject : This operation returned because the timeout period expired
    At line:1 char:13
    + Get-ADObject <<<< -Server -Filter { Userprincipalname -eq "" -or mail -eq "kare" -or msRTCSIP-PrimaryUserAddress -like "*" -or proxyAddresses -like "*" } -Properties UserPrincipalName,DisplayName,DistinguishedName,objectClass,mail,proxyAddresses,msExchReci
    pientDisplayType,msExchRecipientTypeDetails,mailnickname,targetAddress,msRTCSIP-PrimaryUserAddress | Out-String | Forma
    tColor -StringMatch "" -HighlightColor $()
    + CategoryInfo : NotSpecified: (:) [Get-ADObject], ADException
    + FullyQualifiedErrorId : This operation returned because the timeout period expired,Microsoft.ActiveDirectory.Man

    1. Two questions:
      – How many objects in your environment?
      – Are you running this from a DC?

      I’ve found that for larger environments, running it directly from a DC will provide better results, due to how the AD Web Service works. I’m working on some methods to get make this better.

  2. But this tool looks to be useful in a single domain (or maybe multiple domain forest) – I will have to check it out!

    I had started writing something similar…only to look across multiple forests (4 different exchange orgs, from 6 forests (7 domains being sync’d), 2 of the domains were exchange resource forests (so the ID forest HAD to sync first)) – of source, FIM was used to sync GALs across all 4 exchange org’s…it was getting very convoluted! (IDFIX would detect millions of duplicates…the IDFIX developer made me a custom IDFIX that would dump out all the duplicates (50K at a time) into sequential CSV files…cause doing that manually (50K at a time) was painful to do manually.

    But…i digress…I ended up just writing a query tool that given an address ( would go look across all the objects in all domains where that entry could exist (from an ADSYNC perspective) – UPN, mail, proxyaddressses, mailnickname, msrtcsip…,email forwarding attribute (name eludes me at the moment), etc. It would then remove any “allowed” duplicates (i.e. from the OU’s that galsync was writing to – those were excluded from sync) and then the user would need to determine who was going to need to be corrected. In my time there (about 16 months doing primarily O365 migrations) that was super useful…too bad I couldn’t take that code with me or I’d share it!

    1. You can use it across multiple domains as long as you specify which ones they are. I wrote it specifically to troubleshoot in a multi-domain or multi-forest scenario, but it’s really designed to just troubleshoot a single object.

      I’ve been working on an update for this tool for quite a while (sidetracked by other more interesting projects) that would allow for the type of identification you’re talking about (bulk identification, as well as ignoring OUs and objects that aren’t going to sync or aren’t important). Someday …

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.