Cluster Service fails to start 1st time following reboot.

Hi,

I have a 6 node (active) and 2 (passive) cluster, running on W2K3 Enterprise
Edition with SP1. The hardware is IBM and the SAN is a HDS, both on the HCL
list for clusters. The cluster works fine in so much as all resources can
move between all nodes. However, when a node is rebooted intially it comes
back online immediately. A few reboots later, there seems to be a delay,
checking the system event log I see event id 1009 'Cluster service could not
join an existing server cluster and could not form a new server cluster.
Cluster service has terminated.' shortly followed by event id 7031 'The
Cluster Service service terminated unexpectedly. It has done this 1 time(s).
The following corrective action will be taken in 60000 milliseconds: Restart
the service.' The service then starts. Further, test reboots result in the
service not starting at all. Am I missing something?

Thanks for taking the time.

--
Regards,
Chalkie

I see this quite often in our hosting environment. From event viewer
investigations, it appears that the cluster service is trying to start
before the NIC teaming is initialized. Later in the event log I see
the NIC teaming software service start, and the cluster service
successfully starts after that.

Do you have a similar issue?

JL!

On Feb 12, 3:31 pm, Chalkie <Chal...@discussions.microsoft.com> wrote:
> Hi,
>
> I have a 6 node (active) and 2 (passive) cluster, running on W2K3 Enterprise
> Edition with SP1. The hardware is IBM and the SAN is a HDS, both on the HCL
> list for clusters. The cluster works fine in so much as all resources can
> move between all nodes. However, when a node is rebooted intially it comes
> back online immediately. A few reboots later, there seems to be a delay,
> checking the system event log I see event id 1009 'Cluster service could not
> join an existing server cluster and could not form a new server cluster.
> Cluster service has terminated.' shortly followed by event id 7031 'The
> Cluster Service service terminated unexpectedly. It has done this 1 time(s).
> The following corrective action will be taken in 60000 milliseconds: Restart
> the service.' The service then starts. Further, test reboots result in the
> service not starting at all. Am I missing something?
>
> Thanks for taking the time.
>
> --
> Regards,
> Chalkie

Hi Jeremy,

I've had a scan through the system log and found that all the NICS appear to
be initialized, then immediately after the clusdisk and clussvc errors
(clusdisk are event id 1209 - 'Cluster service is requesting a bus reset for
\Device\CluskDisk0.'. Then I get Event ID 1009 ClusSvc, followed by Event ID
1122 ''The node (re)established communication with the cluster node
'whatever' on network 'Heartbeat LAN'.
So, the NIC's are active, but the links to the cluster have not been
established and thus the Cluster service fails to join.

Did you follow that?
--
Regards,
Chalkie


"Jeremy Lyons" wrote:

> I see this quite often in our hosting environment. From event viewer
> investigations, it appears that the cluster service is trying to start
> before the NIC teaming is initialized. Later in the event log I see
> the NIC teaming software service start, and the cluster service
> successfully starts after that.
>
> Do you have a similar issue?
>
> JL!
>
> On Feb 12, 3:31 pm, Chalkie <Chal...@discussions.microsoft.com> wrote:
> > Hi,
> >
> > I have a 6 node (active) and 2 (passive) cluster, running on W2K3 Enterprise
> > Edition with SP1. The hardware is IBM and the SAN is a HDS, both on the HCL
> > list for clusters. The cluster works fine in so much as all resources can
> > move between all nodes. However, when a node is rebooted intially it comes
> > back online immediately. A few reboots later, there seems to be a delay,
> > checking the system event log I see event id 1009 'Cluster service could not
> > join an existing server cluster and could not form a new server cluster.
> > Cluster service has terminated.' shortly followed by event id 7031 'The
> > Cluster Service service terminated unexpectedly. It has done this 1 time(s).
> > The following corrective action will be taken in 60000 milliseconds: Restart
> > the service.' The service then starts. Further, test reboots result in the
> > service not starting at all. Am I missing something?
> >
> > Thanks for taking the time.
> >
> > --
> > Regards,
> > Chalkie
>
>
>

Are you using NIC teaming as Jeremy mentioned? I've also seen Internet
Connection Sharing cause this. Check this service and see if it's set to
'automatic'. If it is, disable and repro.
--
Jeff Hughes, MCSE
Support Escalation Engineer
Microsoft Enterprise Platforms Support (Server Core/Cluster)
"Chalkie" <Chalkie@discussions.microsoft.com> wrote in message
news:ACBA9237-EA41-457C-A277-5CEE6ADCC18F@microsoft.com...
> Hi Jeremy,
>
> I've had a scan through the system log and found that all the NICS appear
> to
> be initialized, then immediately after the clusdisk and clussvc errors
> (clusdisk are event id 1209 - 'Cluster service is requesting a bus reset
> for
> \Device\CluskDisk0.'. Then I get Event ID 1009 ClusSvc, followed by Event
> ID
> 1122 ''The node (re)established communication with the cluster node
> 'whatever' on network 'Heartbeat LAN'.
> So, the NIC's are active, but the links to the cluster have not been
> established and thus the Cluster service fails to join.
>
> Did you follow that?
> --
> Regards,
> Chalkie
>
>
> "Jeremy Lyons" wrote:
>
>> I see this quite often in our hosting environment. From event viewer
>> investigations, it appears that the cluster service is trying to start
>> before the NIC teaming is initialized. Later in the event log I see
>> the NIC teaming software service start, and the cluster service
>> successfully starts after that.
>>
>> Do you have a similar issue?
>>
>> JL!
>>
>> On Feb 12, 3:31 pm, Chalkie <Chal...@discussions.microsoft.com> wrote:
>> > Hi,
>> >
>> > I have a 6 node (active) and 2 (passive) cluster, running on W2K3
>> > Enterprise
>> > Edition with SP1. The hardware is IBM and the SAN is a HDS, both on
>> > the HCL
>> > list for clusters. The cluster works fine in so much as all resources
>> > can
>> > move between all nodes. However, when a node is rebooted intially it
>> > comes
>> > back online immediately. A few reboots later, there seems to be a
>> > delay,
>> > checking the system event log I see event id 1009 'Cluster service
>> > could not
>> > join an existing server cluster and could not form a new server
>> > cluster.
>> > Cluster service has terminated.' shortly followed by event id 7031 'The
>> > Cluster Service service terminated unexpectedly. It has done this 1
>> > time(s).
>> > The following corrective action will be taken in 60000 milliseconds:
>> > Restart
>> > the service.' The service then starts. Further, test reboots result in
>> > the
>> > service not starting at all. Am I missing something?
>> >
>> > Thanks for taking the time.
>> >
>> > --
>> > Regards,
>> > Chalkie
>>
>>
>>

Jeff,

Spot on. Many thanks.
--
Regards,
Chalkie


"Jeff Hughes [Microsoft]" wrote:

> Are you using NIC teaming as Jeremy mentioned? I've also seen Internet
> Connection Sharing cause this. Check this service and see if it's set to
> 'automatic'. If it is, disable and repro.
> --
> Jeff Hughes, MCSE
> Support Escalation Engineer
> Microsoft Enterprise Platforms Support (Server Core/Cluster)
> "Chalkie" <Chalkie@discussions.microsoft.com> wrote in message
> news:ACBA9237-EA41-457C-A277-5CEE6ADCC18F@microsoft.com...
> > Hi Jeremy,
> >
> > I've had a scan through the system log and found that all the NICS appear
> > to
> > be initialized, then immediately after the clusdisk and clussvc errors
> > (clusdisk are event id 1209 - 'Cluster service is requesting a bus reset
> > for
> > \Device\CluskDisk0.'. Then I get Event ID 1009 ClusSvc, followed by Event
> > ID
> > 1122 ''The node (re)established communication with the cluster node
> > 'whatever' on network 'Heartbeat LAN'.
> > So, the NIC's are active, but the links to the cluster have not been
> > established and thus the Cluster service fails to join.
> >
> > Did you follow that?
> > --
> > Regards,
> > Chalkie
> >
> >
> > "Jeremy Lyons" wrote:
> >
> >> I see this quite often in our hosting environment. From event viewer
> >> investigations, it appears that the cluster service is trying to start
> >> before the NIC teaming is initialized. Later in the event log I see
> >> the NIC teaming software service start, and the cluster service
> >> successfully starts after that.
> >>
> >> Do you have a similar issue?
> >>
> >> JL!
> >>
> >> On Feb 12, 3:31 pm, Chalkie <Chal...@discussions.microsoft.com> wrote:
> >> > Hi,
> >> >
> >> > I have a 6 node (active) and 2 (passive) cluster, running on W2K3
> >> > Enterprise
> >> > Edition with SP1. The hardware is IBM and the SAN is a HDS, both on
> >> > the HCL
> >> > list for clusters. The cluster works fine in so much as all resources
> >> > can
> >> > move between all nodes. However, when a node is rebooted intially it
> >> > comes
> >> > back online immediately. A few reboots later, there seems to be a
> >> > delay,
> >> > checking the system event log I see event id 1009 'Cluster service
> >> > could not
> >> > join an existing server cluster and could not form a new server
> >> > cluster.
> >> > Cluster service has terminated.' shortly followed by event id 7031 'The
> >> > Cluster Service service terminated unexpectedly. It has done this 1
> >> > time(s).
> >> > The following corrective action will be taken in 60000 milliseconds:
> >> > Restart
> >> > the service.' The service then starts. Further, test reboots result in
> >> > the
> >> > service not starting at all. Am I missing something?
> >> >
> >> > Thanks for taking the time.
> >> >
> >> > --
> >> > Regards,
> >> > Chalkie
> >>
> >>
> >>
>

[bernardes15]

Hi Jeff and All,
I am having the same problem on a 4 node cluster (os windows Server 2003 EE) and I do not have ICS enabled on network cards, but in fact i am using HP Teaming.
The beahviour is the same, server restart, NICs started, then BUS Reset (event id 1209), then cluster fails to join (event id 1009) and after that cluster try again to start and starts succesfully and is able to join the cluster, and of course the active node is already since the reboot other server/node.

Any more information that can help?

Thx,
Berna

> but in fact i
> am using HP Teaming.

Is the behaviour the same if you break the team, and just use single NIC's ?

Do you have Windows Firewall service enabled on these cluster nodes? If so,
try disabling this service and then reboot again

Regards,
John

Visit my blog: http://msmvps.com/blogs/jtoner


"bernardes15" <bernardes15.338pbc@DoNotSpam.com> wrote in message
news:bernardes15.338pbc@DoNotSpam.com...
>
> Hi Jeff and All,
> I am having the same problem on a 4 node cluster (os windows Server
> 2003 EE) and I do not have ICS enabled on network cards, but in fact i
> am using HP Teaming.
> The beahviour is the same, server restart, NICs started, then BUS Reset
> (event id 1209), then cluster fails to join (event id 1009) and after
> that cluster try again to start and starts succesfully and is able to
> join the cluster, and of course the active node is already since the
> reboot other server/node.
>
> Any more information that can help?
>
> Thx,
> Berna
>
>
> --
> bernardes15
> ------------------------------------------------------------------------
> bernardes15's Profile: http://forums.techarena.in/members/39655.htm
> View this thread: Cluster Service fails to start 1st time following reboot.
>
> http://forums.techarena.in
>

Tell me if you are running TrendMicro Anti-Virus.

Thanks.

--
Chuck Timon, Jr.
Microsoft Corporation
Windows Server 2008 Readiness Team
This posting is provided 'AS IS" with no warranties, and confers no rights.
"bernardes15" <bernardes15.338pbc@DoNotSpam.com> wrote in message
news:bernardes15.338pbc@DoNotSpam.com...
>
> Hi Jeff and All,
> I am having the same problem on a 4 node cluster (os windows Server
> 2003 EE) and I do not have ICS enabled on network cards, but in fact i
> am using HP Teaming.
> The beahviour is the same, server restart, NICs started, then BUS Reset
> (event id 1209), then cluster fails to join (event id 1009) and after
> that cluster try again to start and starts succesfully and is able to
> join the cluster, and of course the active node is already since the
> reboot other server/node.
>
> Any more information that can help?
>
> Thx,
> Berna
>
>
> --
> bernardes15
> ------------------------------------------------------------------------
> bernardes15's Profile: http://forums.techarena.in/members/39655.htm
> View this thread: Cluster Service fails to start 1st time following reboot.
>
> http://forums.techarena.in
>

[cedtech23]

sorry posted in the wrong section