So James Morris & I were engaged in a consulting gig for a UW client awhile ago. This involved helping them sort out how to migrate a set of servers between Windows domains, but not doing the actual migration. For free, we had given them some pointers on what to do, but for many reasons this help wasn’t enough to produce a working solution that they could then use. And after engaging us on a consulting gig, for quite awhile, the problems were enough to stump both James & I.

So there are a couple general purpose tools for this kind of thing. ADMT (active directory migration tool) is an excellent tool, but takes longer to setup, and often has more features then you might want. ADMT is a free download from Microsoft. Subinacl is another excellent tool, takes little to setup, but may not do everything you want. Subinacl is also free, as part of the support tools in the operating system. The two tools have slightly different paradigms which is worthwhile background info to cover first.

ADMT uses it’s own database to determine what should be reACLed. ADMT takes the list of all users and groups that you have previously migrated with ADMT and uses this list as a reference when it encounters entries in an ACL it is being asked to fix. ADMT uses the SID of those users/groups to keep track of those mappings. So if you have not migrated any users or groups with ADMT, then when you ask ADMT to translate security on a computer, it will effectively do nothing. This is one of the reasons why ADMT takes longer, and is more time-consuming. For user/group migration activities, ADMT requires admin level permissions in both domains, which can be a non-starter in some scenarios. For computer activities, ADMT only requires access to the ADMT database which did previously user/group migrations, and admin access on the computer(s) being translated and/or migrated.

Subinacl doesn’t use a database to determine what should be reACLed. You have many choices. You can replace any given account (by name such as nebula2\barkills) with another account (e.g. pottery\barkills). Which isn’t terribly efficient when there are lots of accounts. Or you can replace all accounts with a given domain with another domain. This latter option is much more efficient and there is next to no preparation needed. To do either, you must have access to the domains involved, because ACLs do have SIDs in them, not usernames or group names, and therefore usernames/group names have to be translated to SIDs before work can be done.

So ADMT was thrown out as a solution in this case because there was a small set of servers to migrate, and the users/groups involved had already largely migrated themselves already so there was no reason to do all the work required with ADMT.

Subinacl was the target solution. Specifically, the commandline:

subinacl /subdirectories c:\ /changedomain=pottery.washington.edu=nebula2.washington.edu

in other words, replace all instances of users/groups from the domain pottery.washington.edu with their exact username/groupname complement in nebula2.washington.edu for all files on the c:\ drive.

So for this client, they would run this command on a server in the pottery domain from the context of a nebula2 user account that had admin privs on that server.

So for this client, this command (and a similar one with the /migratetodomain switch), would always result in some kind of error. And this, of course, led to grey and fewer hair for James & I.

Errors we encountered included the following:

1722 Could not find domain name : nebula2
Error finding domain name : 1722 the RPC server is unavailable

1722 Could not find domain name : pottery
Error finding domain name : 1722 the RPC server is unavailable

1355 Could not find domain name : pottery

1355 Could not find domain name : nebula2

5 Access is denied

Early efforts focused on getting a domain trust setup, DNS entries correct, and testing connectivity to ensure that firewalls were not interfering. All of these were part of the problem, but once fixed, the errors continued.

The next hurdle was ensuring that the user account running the command had the ability to “enumerate” users on both domains. The combination of security policy settings like “network access: allow anonymous SID/Name translation=disabled” plus lack of any ACL permitting read access could add up to problems. But this wasn’t the cause.

Next, James & I went on site together in the hopes that a physical visit would shake something loose. I guess something about the physical proximity worked for us both.

James had an idea which identified a new problem. Specifically, the policy on the pottery server was left at the default of windows 2003 server, namely that the LMCompatibilityLevel was set such that it wouldn’t ever negotiate NTLMv2. This worked fine for the pottery domain, but Nebula2 only permitted NTLMv2. So subinacl couldn’t use the same token for the process, and this resulted in the error above. Once we fixed the LMCompatibilityLevel, we ended up with a new error.

This new error brought no end of grief. And for quite awhile we were stumped. Then I stumbled into a solution. I modified the command slightly to:

subinacl /subdirectories c:\ /changedomain=pottery=nebula2.washington.edu

notice that instead of the fully qualified DNS name for the pottery domain, I used the netbios name for the pottery domain.

And this worked. We were so shocked that I think it took us a minute to realize that it had worked and was busily working away.

Then my internal wheels (what Poirot calls the “leetle grey cells”) began to churn because this didn’t, and still doesn’t make sense to me. My best hypothesis on why this worked is tied to the fact that sometimes the form of the domain name is equated with the authentication method, in other words, netbios name=NTLM and fully qualified name=Kerberos, and in this scenario the trust between the domains was an external domain trust so Kerberos could not be negotiated. But the loose tie between the form of the domain name is not always true, and this is a very unsatisfactory explanation for this behavior.

Perhaps you’ve got a good idea or even the explanation and you’d care to share it?

Anyhow, I’ll probably post something more about ADMT in the future. In the meantime, there’s a semi-decent write-up at:

http://www.washington.edu/computing/support/windows/UWdomains/migrateOut.html

that I wrote. Additionally, these pages might have a few other tips not in that doc:

http://staff.washington.edu/barkills/migrationTools.htm
http://staff.washington.edu/barkills/migrationToDo.htm

All of those documents are based on ADMTv2. The current version of ADMT is version 3.0, and there’s 3.1 in beta now. I’ve used 3.0, and there are a few minor differences I’ve noticed which are worth writing up. There’s also no real help with the admt command lines you might want to use in any of those documents.