How-To: User Mappings

[MrHinsh]

This discussion is for questions, feedback, and community support related to the How-To: User Mappings.

Quick Reference

• Documentation: How-To: User Mappings

How to Use This Discussion

• Ask Questions: If you need help configuring or using this component
• Share Examples: Post your configuration examples or use cases
• Report Issues: Describe any problems you encounter (though bugs should be reported as GitHub Issues)
• Request Features: Suggest improvements or new features

Related Documentation

Please refer to the official documentation for detailed configuration options and examples.

[ValerianHs]

Hi,
do you have an updated how-to for v16.x?
Thankfully

[MrHinsh]

This should be updated in preview with #2858, can you check and see whats what? Give it 5m to get from dev to preview.

[MrHinsh]

https://blue-river-093197403-preview.westeurope.5.azurestaticapps.net/docs/how-to/user-mappings/

[veinito]

Hey Hi,

I'd appreciate any help from your side Guys, as I've already spent half a day on it and still no luck.
Shall I report it in a different way or elsewhere - let me know.

So, trying to run TfsWorkItemMigrationProcessor and TfsUserMappingTool with manually prepared user map .json file (as export failed with same error message given below).

This is my config file:

{
  "$schema": "https://devopsmigration.io/schema/configuration.schema.json",
  "Serilog": {
    "MinimumLevel": "Information"
  },
  "MigrationTools": {
    "Version": "16.2",
    "Endpoints": {
      "Source": {
        "EndpointType": "TfsTeamProjectEndpoint",
        "Collection": "https://dev.azure.com/XXX/",
        "Project": "AAA",
        "Authentication": {
          "AuthenticationMode": "AccessToken",
          "AccessToken": "token"
        },
        "ReflectedWorkItemIdField": "Custom.ReflectedWorkItemId",
        "LanguageMaps": {
          "AreaPath": "Area",
          "IterationPath": "Iteration"
        }
      },
      "Target": {
        "EndpointType": "TfsTeamProjectEndpoint",
        "Collection": "https://dev.azure.com/YYY/",
        "Project": "BBB",
        "Authentication": {
          "AccessToken": "token"
        },
        "ReflectedWorkItemIdField": "Custom.ReflectedWorkItemId",
        "LanguageMaps": {
          "AreaPath": "Area",
          "IterationPath": "Iteration"
        }
      }
    },   
    "CommonTools": {
      "TfsUserMappingTool": {
        "Enabled": true,
        "UserMappingFile": "C:\\projects\\ADO\\user_mappings.json",
        "MatchUsersByEmail": false,
        "IdentityFieldsToCheck": [
          "System.AssignedTo"
          "System.ChangedBy",
          "System.CreatedBy",
          "Microsoft.VSTS.Common.ActivatedBy",
          "Microsoft.VSTS.Common.ResolvedBy",
          "Microsoft.VSTS.Common.ClosedBy"
        ]
      }
    },
    "Processors": [
      {
        "ProcessorType": "TfsWorkItemMigrationProcessor",
        "Enabled": true,
        "UpdateCreatedDate": true,
        "UpdateCreatedBy": true,
        "WIQLQuery": "SELECT [System.Id] FROM WorkItems WHERE [System.TeamProject] = @TeamProject AND [System.ChangedDate] > @today - 8 AND [System.WorkItemType] IN ('Epic') ORDER BY [System.ChangedDate] desc",
        "FixHtmlAttachmentLinks": true,
        "WorkItemCreateRetryLimit": 5,
        "FilterWorkItemsThatAlreadyExistInTarget": true,
        "GenerateMigrationComment": true,
        "SourceName": "Source",
        "TargetName": "Target"
      }
    ]
  }
}

Execution log:

[10:54:00 INF] [16.2.9] Validating::Check that all work item types needed in the Target exist or are mapped 
[10:54:00 INF] [16.2.9] Validating::Check all Target Work Items have the RefectedWorkItemId field 
[10:54:00 INF] [16.2.9] Validating::Check that all Area & Iteration paths from Source have a valid mapping on Target 
[10:54:01 INF] [16.2.9] Validating::Check that all users in the source exist in the target or are mapped! 
[10:54:01 INF] [16.2.9] TfsUserMappingTool::GetUsersInSourceMappedToTarget Loading identities from source server 
[10:54:01 INF] [16.2.9] TfsUserMappingTool::GetUsersListFromServer Found 881 identities (users and groups) in server. 
Error logged to Elmah.io! 
!! Check for latest version - We fix issues constantly - If not, please create a discussion on https://github.com/nkdAgility/azure-devops-migration-tools/discussions so we can get this fixed !!
[10:54:28 WRN] [16.2.9] TfsUserMappingTool::GetUsersListFromServer::[user:S-1-9-1551374245-1204400969-2402986413-2179408616-3-1799548171-706465091-2640529154-3530057890] Failed With Server was unable to process request. ---> There was an error generating the XML document. ---> Instance validation error: '5' is not a valid value for Microsoft.TeamFoundation.Framework.Server.ApplicationGroupSpecialType. 

(...)
50 times above error
(...)

[11:16:40 INF] [16.2.9] TfsUserMappingTool::GetUsersListFromServer 5086 user identities are applicable for mapping 
[11:16:40 FTL] [16.2.9] Error while running TfsWorkItemMigrationProcessor 
System.InvalidOperationException: Sequence contains more than one matching element
   at System.Linq.Enumerable.SingleOrDefault[TSource](IEnumerable`1 source, Func`2 predicate)
   at MigrationTools.Tools.TfsUserMappingTool.GetUsersInSourceMappedToTarget(TfsProcessor processor) in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools.Clients.TfsObjectModel\Tools\TfsUserMappingTool.cs:line 193
   at MigrationTools.Tools.TfsUserMappingTool.GetUsersInSourceMappedToTargetForWorkItems(TfsProcessor processor, List`1 sourceWorkItems) in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools.Clients.TfsObjectModel\Tools\TfsUserMappingTool.cs:line 217
   at MigrationTools.Processors.TfsWorkItemMigrationProcessor.ValidateAllUsersExistOrAreMapped(List`1 sourceWorkItems) in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools.Clients.TfsObjectModel\Processors\TfsWorkItemMigrationProcessor.cs:line 256
   at MigrationTools.Processors.TfsWorkItemMigrationProcessor.InternalExecute() in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools.Clients.TfsObjectModel\Processors\TfsWorkItemMigrationProcessor.cs:line 147
   at MigrationTools.Processors.Infrastructure.Processor.Execute() in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools\Processors\Infrastructure\Processor.cs:line 99
Error logged to Elmah.io! 
!! Check for latest version - We fix issues constantly - If not, please create a discussion on https://github.com/nkdAgility/azure-devops-migration-tools/discussions so we can get this fixed !!
[11:16:40 INF] [16.2.9] TfsWorkItemMigrationProcessor completed in 00:00:00  
[11:16:40 ERR] [16.2.9] TfsWorkItemMigrationProcessor The Processor MigrationEngine entered the failed state...stopping run 
[11:16:40 INF] [16.2.9] Command ExecuteMigrationCommand completed in 00:24:48.7267934

System.Linq.Enumerable.SingleOrDefault throws that Exception.

Cheers
Tomek

[MrHinsh]

Its finding more than one entry that matches... so it throws on that. This prevents invalid data.

What versions are you using? TFS/Azure DevOps Server? Azure DevOps Service?

5086 is a lot of identieis...

[veinito]

it's Azure DevOps Service

Its finding more than one entry that matches..

that matches email or name? Can you point me in a good direction?

Another thing is that I've prepared the .json file on my own (format taken from: https://devopsmigration.io/docs/how-to/user-mappings/) which I failed generating automatically via your tool (for the very same reason: SingleOrDefault exception), to bypass this issue. And I'm wondering why the migration tool is checking on all 5k+ identities, while it shall only read the map from the .json file. My understanding after reading https://devopsmigration.io/docs/how-to/user-mappings was that the typical procedure goes like this:

1. export users to .json file via TfsUserMappingTool and TfsExportUsersForMappingProcessor
2. manipulate that .json file manually to address all users names mismatches etc.
3. run TfsWorkItemMigrationProcessor with TfsUserMappingTool but this time no export shall happen, only mapping users from .json map + adding new work items
   that was my assumption and I wonder if that's correct - is it?

[MareksIf]

We have a lot of users ( is several thousands) and are getting the same error, unfortunately it is not clear - how to fix it.

[09:56:32 INF] [16.3.3] TfsUserMappingTool::GetUsersListFromServer <NUMBER> user identities are applicable for mapping 
[09:56:32 FTL] [16.3.3] Error while running TfsExportUsersForMappingProcessor 
System.InvalidOperationException: Sequence contains more than one matching element
   at System.Linq.Enumerable.SingleOrDefault[TSource](IEnumerable`1 source, Func`2 predicate)
   at MigrationTools.Tools.TfsUserMappingTool.GetUsersInSourceMappedToTarget(TfsProcessor processor) in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools.Clients.TfsObjectModel\Tools\TfsUserMappingTool.cs:line 193
   at MigrationTools.Tools.TfsUserMappingTool.GetUsersInSourceMappedToTargetForWorkItems(TfsProcessor processor, List`1 sourceWorkItems) in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools.Clients.TfsObjectModel\Tools\TfsUserMappingTool.cs:line 217
   at MigrationTools.Processors.TfsExportUsersForMappingProcessor.InternalExecute() in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools.Clients.TfsObjectModel\Processors\TfsExportUsersForMappingProcessor.cs:line 55
   at MigrationTools.Processors.Infrastructure.Processor.Execute() in D:\a\azure-devops-migration-tools\azure-devops-migration-tools\src\MigrationTools\Processors\Infrastructure\Processor.cs:line 96

[MrHinsh]

My guess: You have two users with identical display names in your directory.

targetUser ??= targetUsers.SingleOrDefault(x => x.DisplayName == sourceUser.DisplayName);

There could certainly be a better error message here... would likely be good to actually show the duplicate.

[MrHinsh]

Im submitting a fix #3090 shoudl be in https://github.com/nkdAgility/azure-devops-migration-tools/releases/tag/v16.3.4-Preview.4 once #3091 completes

[MrHinsh]

Note that this will not fix your issue, but it will give you a better message. It should list the offending users by display name and email.

[mkennerley]

Hi, any news on when this will be fixed. It's stopping my Azure DevOps migration at the moment as I get the warning message:

[14:22:46 WRN] [16.3.4-Preview.5] TfsUserMappingTool::GetUsersListFromServer::[user:S-1-9-1551374245-1204400969-2402986413-2179408616-3-1394914849-339404363-2725937125-2549201335] Failed With Server was unable to process request. ---> There was an error generating the XML document. ---> Instance validation error: '5' is not a valid value for Microsoft.TeamFoundation.Framework.Server.ApplicationGroupSpecialType.
Error logged to Elmah.io!
!! Check for latest version - We fix issues constantly - If not, please create a discussion on https://github.com/nkdAgility/azure-devops-migration-tools/discussions so we can get this fixed !!

Is there anything I can do manually to get past this?

[MrHinsh]

Not that I know of. There is a group type that does not match what is expected. Would need investigation of the code,

[winklerja85]

As of version 16.3, the user map json in the live documentation still isn't working. The top comment on this page asked for an updated version, and response links to:

#2858

Which also has a comment saying that the schema does not work. As of this writing, the error looks like:

[18:04:19 ERR] [16.3.3] TfsUserMappingTool::DeserializeUserMap User mapping could not be deserialized from file 'UserMap.json'.

...and happens only during that actual migration process. I've manually verified the JSON. It is valid.

Additionally, the file output by the "TfsUserMappingTool" during write mode has the wrong schema.

[CorrenSoft]

I filed an issue with all I could find related to this topic, working on proposing a solution.