Virtualized a child DC - need to recover due to USN rollback

I now realize that converting a DC from a physical machine to a virtual
machine has its own set of caveats. That said: I need to recover from a USN
rollback situation.

Scenario:
- Parent PDC Zeus is Windows 2000 Server (corporate.com)
--- Child PDC Alpha is Windows 2000 Server (dev.corporate.com)
------ Member server Vulcan is Windows 2003 Server and a member of
dev.corporate.com.

I converted Alpha (the child domain PDC) from a physical system to a virtual
machine, but didn't realize that Active Directory needs more attention when
this sort of cloning happens. I just shut down the physical box, and booted
the virtual machine. The NetLogon service on Alpha pauses, throwing Event ID
2103: "The Active Directory database has been restored using an unsupported
restoration procedure." I just resumed NetLogon, and it looks happy. I
assumed this would sort itself out after a while, but it hasn't.

After some research, I discovered http://support.microsoft.com/kb/885875/
which indicates the proper solution is a complete reinstall of AD; however, I
don't have a fresh system state backup (though I still have the physical PDC
idling in the server room).

Is there any way to uninstall AD and reinstall AD on this child domain PDC,
without having to redo all the permissions on the member server? Will the
parent PDC just "magically" see the reinstalled child PDC and sync
everything? Or does uninstalling AD on the child PDC mean all the permissions
set on the member server are orphaned?

Would it be possible to add a child domain BDC, promote it, then demote and
uninstall AD from the child PDC? Or would the added child BDC have
bogus/broken replication data, since the USN on the child PDC was rolled back?

My goal is to resolve the child domain NetLogon problems without having to
redo all of the share and permission settings on the child domain member
server.

Hi Troy,
Did you do a backup before that operation on the original DC?

I don't think I have a valid backup. I made a backup the night before doing
the conversion to virtual, but it's suspect after I've reviewed the
restoration options. Is there a particular folder I need to restore? Would
just the entire OS folder (C:\winnt including registry, etc) be all that's
needed?

I still don't know if you have a valid System State backup? Was NOT that
backup taken before the clone procedure? If yes I still recomend to use that
one.

You need systemstate at minimum.
-Assuming that you have that you can use it for DC restore.
-Boot the DC in DSRM and use that backup "SystemState" to restore the DC.
What backup solution are you using? Check
http://technet.microsoft.com/en-us/l.../cc740010.aspx

The Key is to have a valid system state backup.

Just a thought - Maybe he can bring the old DC online, but in a separate
network and not on the same network, run a backup to disk, copy the BKF file
using a USB to the VM, then restore? It will be a few days out of date, but
replication should get it caught up.

That is what I would suggest here, too. I'd put effort into bringing the
old DC online - with network wiring unplugged and run a system state
backup. Restoring the backup on the virtual DC and restoring
non-authoritatively should do the trick. That would involve the least
pain, I guess.

So let me summarize the recommendation:

1) Power up the old DC physical machine, completely off the network.
2) Backup the system state on the old DC; I can use Windows 2000 backup for
this, selecting only the System State item, writing the results into a .BKF
file.
3) Copy the .BKF to removable media and shut down the old DC.
4) Reboot the current virtualized DC in "Directory Services Recovery Mode".
5) Restore the System State via the Windows 2000 backup utility.
6) Restart the virtualized DC.

I've found instructions for performing the restore:
http://technet.microsoft.com/en-us/l...2.aspx#E0LB0AA

Does this cover it? Any caveats?

After some review, I've found that the normal scheduled backup taken
immediately before the clone procedure was incomplete. (I'm beginning to
trust Retrospect less and less...) Further, due to the way Retrospect works,
it's easy to restore an entire volume (aka volume snapshot) or just a single
file--but there is no elegant way to restore "only system state" out of a
backup.

I'll use NTBackup from now on, then back up the .BKF file (which I've been
doing on my main PDC for quite some time... and now I remember why...)

SO... looks like the old physical box is the safest way to get a System
State backup to use for this recovery effort.

- Ok, since that you don't have a valid Systate backup the next option will
be that one.
- I don't know if that will work or crash in other containers... The problem
is that you did the cloning process with both DCs online, the cloned and the
original, If by any chance the cloned DC replicated anything to the other
DC, you will have USN rollbacks any away... That’s why I was insisting in
the original Backup.

If you're decided... Try it and let's know how it did it.

I agree there may be a caveat with possible hardware differences, and may or
may not cause problems. Difficult to tell, but at this point, he has a
non-working DC, and this would sound like the best option, only because of
that reason.

I would be curious how he makes out too!

Ace

That's why I think that he should connect the old DC to the network First.
Then do the tests to guarantee that everything was OK. After that (and
assuming that everything is ok) do the migration using the correct methods
(disconnect from network, do the system state backup, etc...).

IM There's no point to recover a system state from a disconnected DC that no
one knows if it's working correctly and use it in a VM that has different
hardware characteristics, if things go wrong, that is an extra thing to
consider including the problems that he might have in that operation.

In conclusion, first guarantee that the old DC is in fact working correctly.
If it's, then do the things in the right way.

But that's only my opinion :)

I agree. May as well take the VM machine off line and put the old one back
up and make sure it works. If it does and it's replicating, etc, make a good
backup of the System State and flat file backup of the hard drives. Then
install Windows in the VM and simply restore the backup and he will have his
DC up and running in the VM

Ace

Here's the current situation:

- I logged into the old physical server and made a System State backup.
- I restored the System State in DS recovery mode onto the virtualized system.
- It's "ALMOST" working!

I think the final fiddly bit is to figure out what repadmin magic word will
synchronize the machine passwords. Here's what I get now:

Directory Service event log:
Event ID 1869 Information - Active Directory has located the following
global catalog: \\zeus.mylocation.com

Event ID 1655 Warning - The attempt to communicate with global catalog
\\zeus.mylocation.com failed with the following status:

Logon failure: unknown user name or bad password.

The operation in progress might be unable to continue. The directory
service will use the locator to try find an available global catalog server
for the next operation that requires one.

The record data is the status code.
0000: 2e 05 00 00

System Event log shows an Event ID 5722, "The session setup from the
computer ZEUS failed to authenticate. The name of the account referenced in
the security database is mylocation.com. The following error occurred: Access
is denied."

It appears that a repadmin /syncall should fix this, or that I need to reset
the computer account on the parent DC via NETDOM.

The "remove and rejoin the domain" trick sounds unhealthy to do with a child
PDC!

I tried this: http://support.microsoft.com/kb/260575/EN-US/

netdom resetpwd /server:zeus.mylocation.com /userd:childdomain\administrator
/passwordd:*

I entered the password of the child domain Admin when prompted. After a long
pause, it failed with "The machine account password for the local machine
could not be reset. The trust relationship between the primary domain and the
trusted domain failed."

It seems SO close! Any other things to check?

Ok, the machine account is corrupted. That is difficult to fix without
reinstalling. I have yet to get netdom to work with resetting a DC's account
as it would with a non-DC.

Let's look at this another way that may be beneficial to get this resolved
without devoting any more time for this.

Turn off this current image. How about bringing the original DC back up.
Then create/install a new Windows 2000 installation in a VM, call it
Beta.dev.domain.com, and promote it into the child domain as a replica DC.
Then tranfer the FSMO roles from Alpha to Beta, If you are only keeping one
child DC, then I would not recommend making it a GC, because the IM role
will fail on a DC that is a GC in a multi domain forest.

Install DNS on it, make sure communication works, once satisfied, demote
Alpha.

FYI, fault tolerant best practices dictate that every domain should have a
minimal of two DCs. In a multi domain forest, this is especially important
because of the GC and IM incompatibilities.

Ace

"Ace Fekay [Microsoft Certified Trainer]" wrote:

>
> "Troy Thompson" <Troy Thompson@discussions.microsoft.com> wrote in message
> news:BB02807D-0BEC-4BDA-B19C-9B6B2ABA251D@microsoft.com...
> > Here's the current situation:
> >
> > - I logged into the old physical server and made a System State backup.
> > - I restored the System State in DS recovery mode onto the virtualized
> > system.
> > - It's "ALMOST" working!
> >
> > I think the final fiddly bit is to figure out what repadmin magic word
> > will
> > synchronize the machine passwords. Here's what I get now:
> >
> > Directory Service event log:
> > Event ID 1869 Information - Active Directory has located the following
> > global catalog: \\zeus.mylocation.com
> >
> > Event ID 1655 Warning - The attempt to communicate with global catalog
> > \\zeus.mylocation.com failed with the following status:
> >
> > Logon failure: unknown user name or bad password.
> >
> > The operation in progress might be unable to continue. The directory
> > service will use the locator to try find an available global catalog
> > server
> > for the next operation that requires one.
> >
> > The record data is the status code.
> > 0000: 2e 05 00 00
> >
> > System Event log shows an Event ID 5722, "The session setup from the
> > computer ZEUS failed to authenticate. The name of the account referenced
> > in
> > the security database is mylocation.com. The following error occurred:
> > Access
> > is denied."
> >
> > It appears that a repadmin /syncall should fix this, or that I need to
> > reset
> > the computer account on the parent DC via NETDOM.
> >
> > The "remove and rejoin the domain" trick sounds unhealthy to do with a
> > child
> > PDC!
> >
> > I tried this: http://support.microsoft.com/kb/260575/EN-US/
> >
> > netdom resetpwd /server:zeus.mylocation.com
> > /userd:childdomain\administrator
> > /passwordd:*
> >
> > I entered the password of the child domain Admin when prompted. After a
> > long
> > pause, it failed with "The machine account password for the local machine
> > could not be reset. The trust relationship between the primary domain and
> > the
> > trusted domain failed."
> >
> > It seems SO close! Any other things to check?
> >
>
> Ok, the machine account is corrupted. That is difficult to fix without
> reinstalling. I have yet to get netdom to work with resetting a DC's account
> as it would with a non-DC.
>
> Let's look at this another way that may be beneficial to get this resolved
> without devoting any more time for this.
>
> Turn off this current image. How about bringing the original DC back up.
> Then create/install a new Windows 2000 installation in a VM, call it
> Beta.dev.domain.com, and promote it into the child domain as a replica DC.
> Then tranfer the FSMO roles from Alpha to Beta, If you are only keeping one
> child DC, then I would not recommend making it a GC, because the IM role
> will fail on a DC that is a GC in a multi domain forest.
>
> Install DNS on it, make sure communication works, once satisfied, demote
> Alpha.
>
> FYI, fault tolerant best practices dictate that every domain should have a
> minimal of two DCs. In a multi domain forest, this is especially important
> because of the GC and IM incompatibilities.
>
> Ace
>
>

Well, that does it. I'm screwed. Starting up the old physical box (which has
been offlined for more than 30 days) throws the same errors.

So now the problem changes to "can I restore a 50-day old backup onto the
original hardware and get AD to work"? A restoring I go!...

In news:C76D76D0-B6C0-4C8F-A4B1-591F57AE8FAA@microsoft.com,
Troy Thompson <TroyThompson@discussions.microsoft.com>, posted the
following:

> Well, that does it. I'm screwed. Starting up the old physical box
> (which has been offlined for more than 30 days) throws the same
> errors.
>
> So now the problem changes to "can I restore a 50-day old backup onto
> the original hardware and get AD to work"? A restoring I go!...

Yes, you can, as long as it is under 60 days. But the question goes, I'm not
sure what will happen since the prior attempts to bring up a replica "image"
of it in a VM. GIve it a shot, monitor the event logs. Maybe a simple
burflag entry can force it to replicate, as long as the machine account is
not skewed.

Ace