NTDS ISAM / NTDS Replication major issues

[Windows 2003 Server / Exchange 2003]

Problem 1: Server stops accepting logins. Even the admin accounts would not
work (password rejected). No one could login.

I rebooted the machine, and everything seems to be running fine.

Problem 2: The logs start off with an error "Timeout (30000 milliseconds)
waiting for a transaction response from the NtFrs service.

Then we start seeing The Security Account Manager failed a KDC request in an
unexpected way. The error is in the data field. The account name was
SERVERNAME$ and lookup type 0x0.

Found many article online (eventid.net etc..) but no fix.

Cheers
Brett

IPCONFIG /ALL

Microsoft Windows [Version 5.2.3790]
(C) Copyright 1985-2003 Microsoft Corp.

C:\Documents and Settings\admin1>ipconfig /all

Windows IP Configuration

Host Name . . . . . . . . . . . . : hkgmail
Primary Dns Suffix . . . . . . . : mydomain.com
Node Type . . . . . . . . . . . . : Hybrid
IP Routing Enabled. . . . . . . . : Yes
WINS Proxy Enabled. . . . . . . . : No
DNS Suffix Search List. . . . . . : mydomain.com

Ethernet adapter Local1:

Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Intel(R) PRO/1000 MT Dual Port
Network C
nnection
Physical Address. . . . . . . . . : 00-09-6B-89-2B-AC
DHCP Enabled. . . . . . . . . . . : No
IP Address. . . . . . . . . . . . : 192.168.8.17
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : 192.168.8.254
DNS Servers . . . . . . . . . . . : 192.168.0.19
192.168.8.17
192.168.0.20






"Brett Lindsey" wrote:

> [Windows 2003 Server / Exchange 2003]
>
> Problem 1: Server stops accepting logins. Even the admin accounts would not
> work (password rejected). No one could login.
>
> I rebooted the machine, and everything seems to be running fine.
>
> Problem 2: The logs start off with an error "Timeout (30000 milliseconds)
> waiting for a transaction response from the NtFrs service.
>
> Then we start seeing The Security Account Manager failed a KDC request in an
> unexpected way. The error is in the data field. The account name was
> SERVERNAME$ and lookup type 0x0.
>
> Found many article online (eventid.net etc..) but no fix.
>
> Cheers
> Brett
>

EVENTS:
[DIRECTORY SERVICE LOG]

**The first issue within all event logs seems to be the first post below**

Event Type: Error
Event Source: NTDS ISAM
Event Category: Logging/Recovery
Event ID: 471
Date: 7/14/2005
Time: 5:50:30 AM
User: N/A
Computer: HKGMAIL
Description:
NTDS (560) NTDSA: Unable to rollback operation #11615 on database
C:\WINDOWS\NTDS\ntds.dit. Error: -1014. All future database updates will be
rejected.

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.

------------------------------------------------------------------
Event Type: Error
Event Source: NTDS Replication
Event Category: Replication
Event ID: 1977
Date: 7/14/2005
Time: 5:53:24 AM
User: MYDOMAIN\GMDOCS$
Computer: HKGMAIL
Description:
The following domain controller made a replication request for a writable
directory partition that has been denied by the local domain controller. The
requesting domain controller does not have access to a writable copy of this
directory partition.

Requesting domain controller:
faff04ed-d41b-49f2-90eb-ac562414ceec
Directory partition:
DC=mydomain,DC=com

User Action
If the requesting domain controller must have a writable copy of this
partition, verify that the security descriptor on this directory partition
has the correct configuration for the Replication Get Changes All access
right. You may also get this message during the transition period after a
child partition has been removed. This message will cease when knowledge of
the child partition removal has replicated throughout the forest.

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.

------------------------------------------------------------------

Event Type: Error
Event Source: NTDS Replication
Event Category: Replication
Event ID: 1699
Date: 7/14/2005
Time: 5:53:24 AM
User: MYDOMAIN\GMDOCS$
Computer: HKGMAIL
Description:
The local domain controller failed to retrieve the changes requested for the
following directory partition. As a result, it was unable to send the change
requests to the domain controller at the following network address.

Directory partition:
DC=mydomain,DC=com
Network address:
faff04ed-d41b-49f2-90eb-ac562414ceec._msdcs.mydomain.com
Extended request code:
0

Additional Data
Error value:
8451 The replication operation encountered a database error.

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.

------------------------------------------------------------------
Event Type: Warning
Event Source: NTDS KCC
Event Category: Knowledge Consistency Checker
Event ID: 1435
Date: 7/14/2005
Time: 5:54:40 AM
User: NT AUTHORITY\ANONYMOUS LOGON
Computer: HKGMAIL
Description:
The Knowledge Consistency Checker (KCC) encountered an unexpected error
while performing an Active Directory operation.

Operation type:
KccSearch
Object distinguished name:
CN=Inter-Site Transports,CN=Sites,CN=Configuration,DC=mydomain,DC=com

The operation will be retried at the next KCC interval.

Additional Data
Error value:
5 000020EF: SvcErr: DSID-02080183, problem 5012 (DIR_ERROR), data -1090

Internal ID:
f0e0162

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.

------------------------------------------------------------------
Event Type: Warning
Event Source: NTDS KCC
Event Category: Knowledge Consistency Checker
Event ID: 1663
Date: 7/14/2005
Time: 5:54:40 AM
User: NT AUTHORITY\ANONYMOUS LOGON
Computer: HKGMAIL
Description:
The Knowledge Consistency Checker (KCC) did not initialize its configuration
cache.

This operation will be tried again later.

User Action
If this condition continues, restart this domain controller.

Additional Data
Internal ID:
f100098

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.

------------------------------------------------------------------
Event Type: Warning
Event Source: NTDS KCC
Event Category: Knowledge Consistency Checker
Event ID: 1014
Date: 7/14/2005
Time: 5:54:40 AM
User: NT AUTHORITY\ANONYMOUS LOGON
Computer: HKGMAIL
Description:
The Knowledge Consistency Checker (KCC) failed to update the replication
topology for the local domain controller. The KCC will attempt to update the
replication topology at the following scheduled interval.

KCC update interval:
900

By default, updates occur every 15 minutes.

User Action
If this continues to occur, restart the local domain controller.

Additional Data
Error value:
8409 A database error has occurred.
Internal ID:
f0700cb

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.

------------------------------------------------------------------








"Brett Lindsey" wrote:

> [Windows 2003 Server / Exchange 2003]
>
> Problem 1: Server stops accepting logins. Even the admin accounts would not
> work (password rejected). No one could login.
>
> I rebooted the machine, and everything seems to be running fine.
>
> Problem 2: The logs start off with an error "Timeout (30000 milliseconds)
> waiting for a transaction response from the NtFrs service.
>
> Then we start seeing The Security Account Manager failed a KDC request in an
> unexpected way. The error is in the data field. The account name was
> SERVERNAME$ and lookup type 0x0.
>
> Found many article online (eventid.net etc..) but no fix.
>
> Cheers
> Brett
>

EVENTS:
[SYSTEM LOG]

Event Type: Error
Event Source: Service Control Manager
Event Category: None
Event ID: 7011
Date: 7/14/2005
Time: 4:13:02 AM
User: N/A
Computer: HKGMAIL
Description:
Timeout (30000 milliseconds) waiting for a transaction response from the
NtFrs service.

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.

------------------------------------------------------------------

Event Type: Error
Event Source: KDC
Event Category: None
Event ID: 7
Date: 7/14/2005
Time: 5:51:30 AM
User: N/A
Computer: HKGMAIL
Description:
The Security Account Manager failed a KDC request in an unexpected way. The
error is in the data field. The account name was hkgmail$@MYDOMAIN.COM and
lookup type 0x28.

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 01 00 00 c0 ...À

------------------------------------------------------------------
Event Type: Error
Event Source: KDC
Event Category: None
Event ID: 7
Date: 7/14/2005
Time: 5:51:30 AM
User: N/A
Computer: HKGMAIL
Description:
The Security Account Manager failed a KDC request in an unexpected way. The
error is in the data field. The account name was hkgmail$ and lookup type 0x8.

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 01 00 00 c0 ...À

------------------------------------------------------------------
Event Type: Error
Event Source: KDC
Event Category: None
Event ID: 7
Date: 7/14/2005
Time: 5:55:11 AM
User: N/A
Computer: HKGMAIL
Description:
The Security Account Manager failed a KDC request in an unexpected way. The
error is in the data field. The account name was sin004$@MYDOMAIN.COM and
lookup type 0x28.

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 01 00 00 c0 ...À

------------------------------------------------------------------
Event Type: Error
Event Source: KDC
Event Category: None
Event ID: 7
Date: 7/14/2005
Time: 5:55:11 AM
User: N/A
Computer: HKGMAIL
Description:
The Security Account Manager failed a KDC request in an unexpected way. The
error is in the data field. The account name was sin004$ and lookup type 0x8.

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 01 00 00 c0 ...À

------------------------------------------------------------------




"Brett Lindsey" wrote:

> [Windows 2003 Server / Exchange 2003]
>
> Problem 1: Server stops accepting logins. Even the admin accounts would not
> work (password rejected). No one could login.
>
> I rebooted the machine, and everything seems to be running fine.
>
> Problem 2: The logs start off with an error "Timeout (30000 milliseconds)
> waiting for a transaction response from the NtFrs service.
>
> Then we start seeing The Security Account Manager failed a KDC request in an
> unexpected way. The error is in the data field. The account name was
> SERVERNAME$ and lookup type 0x0.
>
> Found many article online (eventid.net etc..) but no fix.
>
> Cheers
> Brett
>

EVENTS:
[DNS LOG]

Event Type: Error
Event Source: DNS
Event Category: None
Event ID: 4015
Date: 7/14/2005
Time: 5:52:04 AM
User: N/A
Computer: HKGMAIL
Description:
The DNS server has encountered a critical error from the Active Directory.
Check that the Active Directory is functioning properly. The extended error
debug information (which may be empty) is "00000070: LdapErr: DSID-0C0413DF,
comment: A jet error was encountered, data fffffbbe, vece". The event data
contains the error.

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 01 00 00 00 ....

------------------------------------------------------------------


Event Type: Error
Event Source: DNS
Event Category: None
Event ID: 4015
Date: 7/14/2005
Time: 5:52:04 AM
User: N/A
Computer: HKGMAIL
Description:
The DNS server has encountered a critical error from the Active Directory.
Check that the Active Directory is functioning properly. The extended error
debug information (which may be empty) is "000020EF: SvcErr: DSID-02080183,
problem 5012 (DIR_ERROR), data -1090". The event data contains the error.

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 01 00 00 00 ....




"Brett Lindsey" wrote:

> [Windows 2003 Server / Exchange 2003]
>
> Problem 1: Server stops accepting logins. Even the admin accounts would not
> work (password rejected). No one could login.
>
> I rebooted the machine, and everything seems to be running fine.
>
> Problem 2: The logs start off with an error "Timeout (30000 milliseconds)
> waiting for a transaction response from the NtFrs service.
>
> Then we start seeing The Security Account Manager failed a KDC request in an
> unexpected way. The error is in the data field. The account name was
> SERVERNAME$ and lookup type 0x0.
>
> Found many article online (eventid.net etc..) but no fix.
>
> Cheers
> Brett
>

EVENTS:
[APPLICATION LOG]

Event Type: Error
Event Source: MSExchangeAL
Event Category: Service Control
Event ID: 8251
Date: 7/14/2005
Time: 5:50:30 AM
User: N/A
Computer: HKGMAIL
Description:
Could not read entry '' on directory hkgmail.lincolnescott.com. Cannot
access Address List information. Make sure that service was installed
properly.

For more information, click http://www.microsoft.com/contentredirect.asp.


------------------------------------------------------------------

Event Type: Error
Event Source: MSExchangeSA
Event Category: General
Event ID: 9188
Date: 7/14/2005
Time: 5:50:49 AM
User: N/A
Computer: HKGMAIL
Description:
Microsoft Exchange System Attendant failed to read the membership of group
'cn=Exchange Domain Servers,cn=Users,dc=mydomain,dc=com'. Error code
'80072020'.

Please check whether the local computer is a member of the group. If it is
not, stop all the Microsoft Exchange services, add the local computer into
the group manually and restart all the services.

For more information, click http://www.microsoft.com/contentredirect.asp.

------------------------------------------------------------------

Event Type: Error
Event Source: Userenv
Event Category: None
Event ID: 1080
Date: 7/14/2005
Time: 5:52:49 AM
User: NT AUTHORITY\SYSTEM
Computer: HKGMAIL
Description:
Windows cannot search for Organizational Unit hierarchy. (1). Group Policy
processing aborted.

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.

------------------------------------------------------------------

Event Type: Error
Event Source: Userenv
Event Category: None
Event ID: 1030
Date: 7/14/2005
Time: 5:52:49 AM
User: NT AUTHORITY\SYSTEM
Computer: HKGMAIL
Description:
Windows cannot query for the list of Group Policy objects. Check the event
log for possible messages previously logged by the policy engine that
describes the reason for this.

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.

------------------------------------------------------------------

Event Type: Error
Event Source: Userenv
Event Category: None
Event ID: 1053
Date: 7/14/2005
Time: 5:57:50 AM
User: NT AUTHORITY\SYSTEM
Computer: HKGMAIL
Description:
Windows cannot determine the user or computer name. (The directory service
is busy. ). Group Policy processing aborted.

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.

------------------------------------------------------------------

Event Type: Error
Event Source: MSExchangeIS Mailbox Store
Event Category: General
Event ID: 7200
Date: 7/14/2005
Time: 6:00:00 AM
User: N/A
Computer: HKGMAIL
Description:
Background thread FDoUpdateCatalog halted on database "First Storage
Group\Mailbox Store (HKGMAIL)" due to error code 0x80004005.

For more information, click http://www.microsoft.com/contentredirect.asp.

------------------------------------------------------------------

Event Type: Error
Event Source: MSExchangeIS Public Store
Event Category: General
Event ID: 7200
Date: 7/14/2005
Time: 6:00:00 AM
User: N/A
Computer: HKGMAIL
Description:
Background thread FDoUpdateCatalog halted on database "First Storage
Group\Public Folder Store (HKGMAIL)" due to error code 0x80004005.

For more information, click http://www.microsoft.com/contentredirect.asp.

------------------------------------------------------------------





"Brett Lindsey" wrote:

> [Windows 2003 Server / Exchange 2003]
>
> Problem 1: Server stops accepting logins. Even the admin accounts would not
> work (password rejected). No one could login.
>
> I rebooted the machine, and everything seems to be running fine.
>
> Problem 2: The logs start off with an error "Timeout (30000 milliseconds)
> waiting for a transaction response from the NtFrs service.
>
> Then we start seeing The Security Account Manager failed a KDC request in an
> unexpected way. The error is in the data field. The account name was
> SERVERNAME$ and lookup type 0x0.
>
> Found many article online (eventid.net etc..) but no fix.
>
> Cheers
> Brett
>

Upon running netdiag /fix:

LDAP test. . . . . . . . . . . . . : Passed
[WARNING] Failed to query SPN registration on DC 'permail.mydomain.com'
..
[WARNING] Failed to query SPN registration on DC 'perdocs.mydomain.com'
..
[WARNING] Failed to query SPN registration on DC 'adldocs.mydomain.com'
..
[WARNING] Failed to query SPN registration on DC 'bkkdocs.mydomain.com'
..
[WARNING] Failed to query SPN registration on DC 'cnsdocs.mydomain.com'
..
[WARNING] Failed to query SPN registration on DC 'hnldocs.mydomain.com'
..
[WARNING] Failed to query SPN registration on DC 'SYDDOCS.mydomain.com'
..



C:\WINDOWS\system32\ntdsutil.exe: domain management
domain management: connections
server connections: connect to server permail.mydomain.com
Disconnecting from lsdc.mydomain.com...
Binding to permail.mydomain.com ...
DsBindW error 0x6ba(The RPC server is unavailable.)
server connections:

[LSDC is my domain naming master]
server connections: connect to server lsdc.mydomain.com
Binding to lsdc.mydomain.com ...
Connected to lsdc.mydomain.com using credentials of locally logged on user.








"Brett Lindsey" wrote:

> [Windows 2003 Server / Exchange 2003]
>
> Problem 1: Server stops accepting logins. Even the admin accounts would not
> work (password rejected). No one could login.
>
> I rebooted the machine, and everything seems to be running fine.
>
> Problem 2: The logs start off with an error "Timeout (30000 milliseconds)
> waiting for a transaction response from the NtFrs service.
>
> Then we start seeing The Security Account Manager failed a KDC request in an
> unexpected way. The error is in the data field. The account name was
> SERVERNAME$ and lookup type 0x0.
>
> Found many article online (eventid.net etc..) but no fix.
>
> Cheers
> Brett
>

If I remember correctly, the SPN issue is a bug. Install the latest Support
tools that came with SP1.

--
Paul Williams
Microsoft MVP - Windows Server - Directory Services
http://www.msresource.net | http://forums.msresource.net

How long's this been happening? What's changed since it was working? Was
it ever working?

--
Paul Williams
Microsoft MVP - Windows Server - Directory Services
http://www.msresource.net | http://forums.msresource.net

I know this isn't the same event source, but the underlying database engine
is the same so it might be worth a try. If you don't have Exchange, there's
a Windows version called esentutil (I think that's the correct name).:
--
http://www.eventid.net/display.asp?e...ce=ESE&phase=1

--
Paul Williams
Microsoft MVP - Windows Server - Directory Services
http://www.msresource.net | http://forums.msresource.net

Looks like you've got major problems with the database on this DC. If you
have another working DC, it will probably be easier to destroy this one, do
the metadata cleanup, remove the computer account, DNS entries and server
object (from sites and services) and then rebuild the dodgy DC and promote
anew.

Otherwise you're going to need to go into DSRM and run some low-level
database commands on the database. I would start with a defrag (which will
cause a reindex).

--
Paul Williams
Microsoft MVP - Windows Server - Directory Services
http://www.msresource.net | http://forums.msresource.net