Briefly: DNS is broken, and an inability to talk to an apparently healthy global catalog seems to be the core reason.
One of our 2 domain controllers started throwing disk errors earlier today, so I built a new DC, but could not add it to the domain (Logon Failure). On checking I found that the global Catalog was held by the remaining server, so turned off the dud box and used ntdsdiag to sieze control of the other FSMO roles and do a metadata cleanup. However, I still couldn't join the domain.
Investigation showed a handful of remaining references to the dead server (cleaned up AD Sites & Services and deleted the appropriate name servers from the interface DNS configuration). However, when I tried to delete the old server from DNS, I was refused. I believe this was because the old server had previously been domain primary.
In researching this I found a suggestion to uninstall and reinstall DNS, which I've done; however I find that now I cannot re-attach the DNS to Active Directory. Any attempt to do so gives an on-screen error saying "The zone cannot be created. The data is invalid" and refuses to create the zone.
At the same time Event Viewer tags the following errors (all NTDS Global Catalog errors):
1869 - Active Directory has located a global catalog in the following site....
1655 - Active Directory attempted to communicate with the following global catalog and the attempts were unsuccessful... Additional Data Error value: 5 Access is denied.
1126 - Active Directory was unable to establish a connection with the global catalog.... Additional Data Error value:
8430 The directory service encountered an internal failure. / Internal ID: 3200c89
The global catalog described here is the server itself, i.e.the local system. For a while I thought this was a replication issue (trying to replicate with the dead server) but I can't see any evidence of that.
dcdiag /v looks like this:
Domain Controller Diagnosis
Performing initial setup:
* Verifying that the local machine ebitnswdc04, is a DC.
* Connecting to directory service on server ebitnswdc04.
* Collecting site info.
* Identifying all servers.
* Identifying all NC cross-refs.
* Found 1 DC(s). Testing 1 of them.
Done gathering initial info.
Doing initial required tests
Testing server: SYD-51DruittSt\EBITNSWDC04
Starting test: Connectivity
* Active Directory LDAP Services Check
The host ee8bb430-9cee-4d41-ad3c-1d902fb063a2._msdcs.ebit.com.au could
not be resolved to an
IP address. Check the DNS server, DHCP, server name, etc
Although the Guid DNS name
(ee8bb430-9cee-4d41-ad3c-1d902fb063a2._msdcs.ebit.com.au) couldn't be
resolved, the server name (ebitnswdc04.ebit.com.au) resolved to the IP
address (10.3.3.82) and was pingable. Check that the IP address is
registered correctly with the DNS server.
......................... EBITNSWDC04 failed test Connectivity
Doing primary tests
Testing server: SYD-51DruittSt\EBITNSWDC04
Skipping all tests, because server EBITNSWDC04 is
not responding to directory service requests
Test omitted by user request: Topology
Test omitted by user request: CutoffServers
Test omitted by user request: OutboundSecureChannels
Test omitted by user request: VerifyReplicas
Test omitted by user request: VerifyEnterpriseReferences
Test omitted by user request: CheckSecurityError
Running partition tests on : ForestDnsZones
Starting test: CrossRefValidation
......................... ForestDnsZones passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... ForestDnsZones passed test CheckSDRefDom
Running partition tests on : DomainDnsZones
Starting test: CrossRefValidation
......................... DomainDnsZones passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... DomainDnsZones passed test CheckSDRefDom
Running partition tests on : Schema
Starting test: CrossRefValidation
......................... Schema passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... Schema passed test CheckSDRefDom
Running partition tests on : Configuration
Starting test: CrossRefValidation
......................... Configuration passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... Configuration passed test CheckSDRefDom
Running partition tests on : ebit
Starting test: CrossRefValidation
......................... ebit passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... ebit passed test CheckSDRefDom
Running enterprise tests on : ebit.com.au
Starting test: Intersite
Skipping site SYD-51DruittSt, this site is outside the scope provided
by the command line arguments provided.
......................... ebit.com.au passed test Intersite
Starting test: FsmoCheck
GC Name: \\ebitnswdc04.ebit.com.au
Locator Flags: 0xe00003fd
PDC Name: \\ebitnswdc04.ebit.com.au
Locator Flags: 0xe00003fd
Time Server Name: \\ebitnswdc04.ebit.com.au
Locator Flags: 0xe00003fd
Preferred Time Server Name: \\ebitnswdc04.ebit.com.au
Locator Flags: 0xe00003fd
KDC Name: \\ebitnswdc04.ebit.com.au
Locator Flags: 0xe00003fd
......................... ebit.com.au passed test FsmoCheck
Test omitted by user request: DNS
Test omitted by user request: DNS
Results of dcdiag /test:dns follow...
Domain Controller Diagnosis
Performing initial setup:
Done gathering initial info.
Doing initial required tests
Testing server: SYD-51DruittSt\EBITNSWDC04
Starting test: Connectivity
The host ee8bb430-9cee-4d41-ad3c-1d902fb063a2._msdcs.ebit.com.au could
not be resolved to an
IP address. Check the DNS server, DHCP, server name, etc
Although the Guid DNS name
(ee8bb430-9cee-4d41-ad3c-1d902fb063a2._msdcs.ebit.com.au) couldn't be
resolved, the server name (ebitnswdc04.ebit.com.au) resolved to the IP
address (10.3.3.82) and was pingable. Check that the IP address is
registered correctly with the DNS server.
......................... EBITNSWDC04 failed test Connectivity
Doing primary tests
Testing server: SYD-51DruittSt\EBITNSWDC04
DNS Tests are running and not hung. Please wait a few minutes...
Running partition tests on : ForestDnsZones
Running partition tests on : DomainDnsZones
Running partition tests on : Schema
Running partition tests on : Configuration
Running partition tests on : ebit
Running enterprise tests on : ebit.com.au
Starting test: DNS
Test results for domain controllers:
DC: ebitnswdc04.ebit.com.au
Domain: ebit.com.au
TEST: Basic (Basc)
Error: No LDAP connectivity
Warning: adapter [00000007] Broadcom NetXtreme 5751 Gigabit Co
ntroller has invalid DNS server: 10.3.3.82 (<name unavailable>)
Error: all DNS servers are invalid
Error: The A record for this DC was not found
Warning: The Active Directory zone on this DC/DNS server was n
ot found (probably a misconfiguration)
TEST: Forwarders/Root hints (Forw)
Error: Root hints list has invalid root hint server: b.root-se
rvers.net. (128.9.0.107)
Error: Root hints list has invalid root hint server: h.root-se
rvers.net. (128.63.2.53)
Error: Root hints list has invalid root hint server: l.root-se
rvers.net. (198.32.64.12)
TEST: Records registration (RReg)
Error: Record registrations cannot be found for all the network a
dapters
Summary of test results for DNS servers used by the above domain contro
llers:
DNS server: 10.3.3.82 (<name unavailable>)
1 test failure on this DNS server
Name resolution is not functional. _ldap._tcp.ebit.com.au. failed
on the DNS server 10.3.3.82
DNS server: 128.63.2.53 (h.root-servers.net.)
1 test failure on this DNS server
This is not a valid DNS server. PTR record query for the 1.0.0.12
7.in-addr.arpa. failed on the DNS server 128.63.2.53
DNS server: 128.9.0.107 (b.root-servers.net.)
1 test failure on this DNS server
This is not a valid DNS server. PTR record query for the 1.0.0.12
7.in-addr.arpa. failed on the DNS server 128.9.0.107
DNS server: 198.32.64.12 (l.root-servers.net.)
1 test failure on this DNS server
This is not a valid DNS server. PTR record query for the 1.0.0.12
7.in-addr.arpa. failed on the DNS server 198.32.64.12
Summary of DNS test results:
Auth Basc Forw Del Dyn RReg Ext
________________________________________________________________
Domain: ebit.com.au
ebitnswdc04 PASS FAIL PASS n/a PASS FAIL n/a
......................... ebit.com.au failed test DNS
ipconfig /all looks like this:
Windows IP Configuration
Host Name . . . . . . . . . . . . : ebitnswdc04
Primary Dns Suffix . . . . . . . : ebit.com.au
Node Type . . . . . . . . . . . . : Hybrid
IP Routing Enabled. . . . . . . . : No
WINS Proxy Enabled. . . . . . . . : No
DNS Suffix Search List. . . . . . : ebit.com.au
com.au
Ethernet adapter Local Area Connection:
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Broadcom NetXtreme 5751 Gigabit Controller
Physical Address. . . . . . . . . : 00-13-20-07-AF-A5
DHCP Enabled. . . . . . . . . . . : No
IP Address. . . . . . . . . . . . : 10.3.3.82
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : 10.3.3.252
DNS Servers . . . . . . . . . . . : 10.3.3.82
Primary WINS Server . . . . . . . : 10.3.3.82
Any ideas of exactly why this (in particular, DNS registration) is failing would be most welcome. I suspect I may otherwise wind up hand-coding the DNS for our zone.
If it matters, we have a second domain in a trust relationship ("they" trust "us" but not vice-versa). However that domain seems to be unaffected except in that ebitnswdc04 is now knocking back their zone transfers.
Also if it matters, the authentication functions of the domain seem to be fine. I can log into that server (and other servers) without any problems. I should add that this was *not* the case before I yanked the server with the failed disk. However, the inability to resolve host names is going to be a problem... to the extent that netbios fails to fill the gap I suppose.
Also if you need additional diagnostics I'm happy to supply them. I tried to cover the high points here but I've undoubtedly missed something important. A couple of pointers to start would definitely be appreciated...
...Ronny Cook