Multiple NICs in same segment server 2008

I have anew Server 2008 server with multiple NICs. The Server is going to be
running HYPER-V. I have a Management NIC and 2 more NICs that are bound to
the same network segment and use Public IPs.

The first NIC gets the gateway. The 2nd and thrid NIC I do not assign a
Gateway. I set the order of the NICs so the one with the Gateway is at the
top.

When viewing in Route Print it says ON-LINK for the 2nd and thrid NIC.

I can talk to these NICs locally from other servers on the same segment but
nothing outside the network. If I add a route it works just fine. As you
know if you attempt to assign a Gateway to a second NIC an error message is
thrown telling you that you should not be assigning multiple Gateways.

In the past I have never had routing issues on second or third NICs on the
same segment. This seems to be specific to Server 2008.

Firewall is disabled.
\
I am betting there is a simple solution but I have been unable to find it
hence this post

All of them use Public IPs.

It is the same network and the same segment hence the reason the primary
gateway should affect the second and third nics.

I have the same config on a server 2003 and the additional NICs work fine.

The IPCONFIG is huge as there are 8 NIcs and IPV6 as well (Not being used
but left enabled)

It is simply three NICs. Imagine the IPs below but they are PUBLIC IPS NOT
natted IPs. All going to one GIG Switch in the same VLAN.

I hope that is clear now. Nothing complicated here.

NIC 1 is.

10.1.1.10
255.255.255.0
10.1.1.1
DNS
10.1.1.5
10.1.1.6

NIC 2 is.

10.1.1.11
255.255.255.0
No Gateway
DNS
10.1.1.5
10.1.1.6

NIC 3 is.

10.1.1.12
255.255.255.0
No Gateway
DNS
10.1.1.5
10.1.1.6

"dcohn" wrote:

> I have anew Server 2008 server with multiple NICs. The Server is going to be
> running HYPER-V. I have a Management NIC and 2 more NICs that are bound to
> the same network segment and use Public IPs.
>
> The first NIC gets the gateway. The 2nd and thrid NIC I do not assign a
> Gateway. I set the order of the NICs so the one with the Gateway is at the
> top.
>
> When viewing in Route Print it says ON-LINK for the 2nd and thrid NIC.
>
> I can talk to these NICs locally from other servers on the same segment but
> nothing outside the network. If I add a route it works just fine. As you
> know if you attempt to assign a Gateway to a second NIC an error message is
> thrown telling you that you should not be assigning multiple Gateways.
>
> In the past I have never had routing issues on second or third NICs on the
> same segment. This seems to be specific to Server 2008.
>
> Firewall is disabled.
> \
> I am betting there is a simple solution but I have been unable to find it
> hence this post

"dcohn" <dcohn@discussions.microsoft.com> wrote in message
news:73713195-9AA1-419E-B04A-8818B1455DA2@microsoft.com...
> All of them use Public IPs.
>
> It is the same network and the same segment hence the reason the primary
> gateway should affect the second and third nics.
>
> I have the same config on a server 2003 and the additional NICs work fine.
>
> The IPCONFIG is huge as there are 8 NIcs and IPV6 as well (Not being used
> but left enabled)
>
> It is simply three NICs. Imagine the IPs below but they are PUBLIC IPS
> NOT
> natted IPs. All going to one GIG Switch in the same VLAN.
>
> I hope that is clear now. Nothing complicated here.
>
> NIC 1 is.
>
> 10.1.1.10
> 255.255.255.0
> 10.1.1.1
> DNS
> 10.1.1.5
> 10.1.1.6
>
> NIC 2 is.
>
> 10.1.1.11
> 255.255.255.0
> No Gateway
> DNS
> 10.1.1.5
> 10.1.1.6
>
> NIC 3 is.
>
> 10.1.1.12
> 255.255.255.0
> No Gateway
> DNS
> 10.1.1.5
> 10.1.1.6
>
> "dcohn" wrote:
>
>> I have anew Server 2008 server with multiple NICs. The Server is going
>> to be
>> running HYPER-V. I have a Management NIC and 2 more NICs that are bound
>> to
>> the same network segment and use Public IPs.
>>
>> The first NIC gets the gateway. The 2nd and thrid NIC I do not assign a
>> Gateway. I set the order of the NICs so the one with the Gateway is at
>> the
>> top.
>>
>> When viewing in Route Print it says ON-LINK for the 2nd and thrid NIC.
>>
>> I can talk to these NICs locally from other servers on the same segment
>> but
>> nothing outside the network. If I add a route it works just fine. As
>> you
>> know if you attempt to assign a Gateway to a second NIC an error message
>> is
>> thrown telling you that you should not be assigning multiple Gateways.
>>
>> In the past I have never had routing issues on second or third NICs on
>> the
>> same segment. This seems to be specific to Server 2008.
>>
>> Firewall is disabled.
>> \
>> I am betting there is a simple solution but I have been unable to find it
>> hence this post


Why do you have three NICs in the same IP subnet assigned to the
machine? What purpose do they serve?

If you are planning to use the second and third NICs for vms running
under Hyper-V, the problem will go away. Once the NICs are assigned to a
virtual switch for the vms to use, they do not require an IP address on the
host.

"dcohn" <dcohn@discussions.microsoft.com> wrote in message
news:73713195-9AA1-419E-B04A-8818B1455DA2@microsoft.com...
> All of them use Public IPs.
>
> It is the same network and the same segment hence the reason the primary
> gateway should affect the second and third nics.
>
> I have the same config on a server 2003 and the additional NICs work fine.
>
> The IPCONFIG is huge as there are 8 NIcs and IPV6 as well (Not being used
> but left enabled)
>
> It is simply three NICs. Imagine the IPs below but they are PUBLIC IPS
> NOT
> natted IPs. All going to one GIG Switch in the same VLAN.
>
> I hope that is clear now. Nothing complicated here.

[snipped]

Hello Dcohn,

In addition to Bill's reply, if an interface doesn't have a gateway, and it
receives a request on that specific NIC, it won't be able to respond if it
doesn't have a gate if the NICs are on the same subnet. Why not just team
them for balancing and fault tolerance? You can assign individual IPs on the
VM guests independent of the host.

--
Ace

This posting is provided "AS-IS" with no warranties or guarantees and
confers no rights.

Ace Fekay, MCSE 2003 & 2000, MCSA 2003 & 2000, MCSA Messaging, MCT
Microsoft Certified Trainer
aceman@mvps.RemoveThisPart.org

For urgent issues, you may want to contact Microsoft PSS directly. Please
check http://support.microsoft.com for regional support phone numbers.

"Efficiency is doing things right; effectiveness is doing the right
things." - Peter F. Drucker
http://twitter.com/acefekay

That is correct HYPER V. But if you create a Virtual Switch the NIC must
first have assigned IPs and gateways NO?????? QUESTION THAT IS ACTUALLY....
Or is it simply based on the network address assigned to the VMs and where
the NIC used for the VS is physically cabled?

I thought I had to make the NICs working before assigning them to a switch.
Wasn't that true of Virtual Server 2005?

I cannot team them to use with HYPER-V to answer another post.

What would be different if I select Private or public when creating the
VSwitch as well.



"Bill Grant" wrote:

Why do you have three NICs in the same IP subnet assigned to the
machine? What purpose do they serve?

If you are planning to use the second and third NICs for vms running
under Hyper-V, the problem will go away. Once the NICs are assigned to a
virtual switch for the vms to use, they do not require an IP address on the
host.

CAN I TEAM NICS I AM USING for a Virtual Switch?? I thought this was a NO NO?


You said

Why not just team
them for balancing and fault tolerance? You can assign individual IPs on the
VM guests independent of the host.

"Ace Fekay [Microsoft Certified Trainer]" wrote:

"dcohn" <dcohn@discussions.microsoft.com> wrote in message
news:E4CB35EA-3C18-4D55-994F-0E54804F9A0C@microsoft.com...
> CAN I TEAM NICS I AM USING for a Virtual Switch?? I thought this was a NO
> NO?
>
>
> You said
>
> Why not just team
> them for balancing and fault tolerance? You can assign individual IPs on
> the
> VM guests independent of the host.
>
> "Ace Fekay [Microsoft Certified Trainer]" wrote:
>
>


I was speaking of the machine NICs itself. I had a server running with
teamed NICs using Virtual Server without a problem. I just assigned the
guests their own IP configs.

Ace

"Ace Fekay [Microsoft Certified Trainer]" <aceman@mvps.RemoveThisPart.org>
wrote in message news:ek9#CUK6JHA.1096@TK2MSFTNGP06.phx.gbl...
> "dcohn" <dcohn@discussions.microsoft.com> wrote in message
> news:E4CB35EA-3C18-4D55-994F-0E54804F9A0C@microsoft.com...
>> CAN I TEAM NICS I AM USING for a Virtual Switch?? I thought this was a
>> NO NO?
>>
>>
>> You said
>>
>> Why not just team
>> them for balancing and fault tolerance? You can assign individual IPs on
>> the
>> VM guests independent of the host.
>>
>> "Ace Fekay [Microsoft Certified Trainer]" wrote:
>>
>>
>
>
> I was speaking of the machine NICs itself. I had a server running with
> teamed NICs using Virtual Server without a problem. I just assigned the
> guests their own IP configs.
>
> Ace
>

There were problems with NIC teaming and Hyper-V early on. The
manufacturers were slow to bring out drivers which were compatible with the
Hyper-V networking software. The latest versions from Broadcom and Intel
should handle NIC teaming, if you really need it.

Networking in Hyper-V is very different from Virtual Server.

"Bill Grant" <not.available@online> wrote in message
news:OhKUieK6JHA.1420@TK2MSFTNGP04.phx.gbl...
>
> There were problems with NIC teaming and Hyper-V early on. The
> manufacturers were slow to bring out drivers which were compatible with
> the Hyper-V networking software. The latest versions from Broadcom and
> Intel should handle NIC teaming, if you really need it.
>
> Networking in Hyper-V is very different from Virtual Server.
>

I wasn't aware of the driver issues early on, but aware the two are
different. Are you saying the later drivers, and possibly 2008 SP2 covers
the issue?

Ace

"Ace Fekay [Microsoft Certified Trainer]" <aceman@mvps.RemoveThisPart.org>
wrote in message news:OMk2EJL6JHA.5932@TK2MSFTNGP03.phx.gbl...
> "Bill Grant" <not.available@online> wrote in message
> news:OhKUieK6JHA.1420@TK2MSFTNGP04.phx.gbl...
>>
>> There were problems with NIC teaming and Hyper-V early on. The
>> manufacturers were slow to bring out drivers which were compatible with
>> the Hyper-V networking software. The latest versions from Broadcom and
>> Intel should handle NIC teaming, if you really need it.
>>
>> Networking in Hyper-V is very different from Virtual Server.
>>
>
> I wasn't aware of the driver issues early on, but aware the two are
> different. Are you saying the later drivers, and possibly 2008 SP2 covers
> the issue?
>
> Ace
>
>
>
>
I don't run NIC teaming myself (although I do have multiple NICs in the
host), but postings in the forums indicate that the latest NIC teaming
software works fine with Hyper-V-enabled NICs. There were heaps of problems
when Hyper-V was first released.

Not sure about any improvements with SP2. There are certainly changes
and improvements to networking in Hyper-V with R2!

"dcohn" <dcohn@discussions.microsoft.com> wrote in message
news:69ACF9A8-17F7-44B0-B038-9773FD0D6314@microsoft.com...
> That is correct HYPER V. But if you create a Virtual Switch the NIC must
> first have assigned IPs and gateways NO?????? QUESTION THAT IS
> ACTUALLY....
> Or is it simply based on the network address assigned to the VMs and where
> the NIC used for the VS is physically cabled?
>


When you create an external virtual switch in Hyper-V (ie one linked to
a physical NIC) it does not work like installing VMNS in Virtual Server. See
this article from Ben's blog.

http://blogs.msdn.com/virtual_pc_guy...h-hyper-v.aspx

If you administer the server remotely, it is recommended to reserve one
NIC in the host for this purpose (ie do not create a virtual network linked
to this NIC) and use the other NIC(s) for vms. This is easier in R2 as they
have made it more obvious in the GUI.

"Bill Grant" <not.available@online> wrote in message
news:uTUTZKM6JHA.4116@TK2MSFTNGP04.phx.gbl...
>
> I don't run NIC teaming myself (although I do have multiple NICs in the
> host), but postings in the forums indicate that the latest NIC teaming
> software works fine with Hyper-V-enabled NICs. There were heaps of
> problems when Hyper-V was first released.
>
> Not sure about any improvements with SP2. There are certainly changes
> and improvements to networking in Hyper-V with R2!

Thanks, Bill!

Thanks

I have a dedicated NIC just for management. I also have taken your advice
that the NIC settings are irrelevant if I am creating a Virtual switch. But
I did not see this before I setup and it is not 100% clear anyway as they
seem to refer to just a single NIC. The issue was multiple NICs and whether
they required any settings BEFORE Creating a VS.

I NOW BELIEVE the best thing would have been to take bare unconfigured NICs
and use them as the VS NICs as long as they are physically plugged into the
correct segment for the access desired.

Using your info I removed the gateways from the Virtual Nics (they took the
settings I had applied to the physical NICs). Everything works fine without
those gateways as you stated.

It seems I Do NOT need to bind an IP address to the Virtual NICs? RIGHT?

Since I already have created the VS can I just change those NIC settings to
obtain automatically (we have NO DHCP Servers as this is a data center with
only servers) and then I would not be wasting those IPs.

Seems like a PLAN.

I don't know why I assumed I needed the IP address but it is really not
stated anywhere and habit caused me to do it. I follow now and it is quite
clear.

"Bill Grant" wrote:

>
>
> "dcohn" <dcohn@discussions.microsoft.com> wrote in message
> news:69ACF9A8-17F7-44B0-B038-9773FD0D6314@microsoft.com...
> > That is correct HYPER V. But if you create a Virtual Switch the NIC must
> > first have assigned IPs and gateways NO?????? QUESTION THAT IS
> > ACTUALLY....
> > Or is it simply based on the network address assigned to the VMs and where
> > the NIC used for the VS is physically cabled?
> >
>
>
> When you create an external virtual switch in Hyper-V (ie one linked to
> a physical NIC) it does not work like installing VMNS in Virtual Server. See
> this article from Ben's blog.
>
> http://blogs.msdn.com/virtual_pc_guy...h-hyper-v.aspx
>
> If you administer the server remotely, it is recommended to reserve one
> NIC in the host for this purpose (ie do not create a virtual network linked
> to this NIC) and use the other NIC(s) for vms. This is easier in R2 as they
> have made it more obvious in the GUI.
>
>
>
>

No, the NIC does not need an IP address on the host. If you set the vms to
get their network config automatically, they will get their config from the
DHCP on the LAN.

As far as the networking software is concerned, each vm connected through
this NIC is just another machine on the LAN (the virtual switch works just
like a physical switch plugged into this port) and the host machine is
invisible through this NIC.

To get this set up in Virtual Server, you had to clear all the boxes
except VMNS in the TCP/IP settings on the host (to get a NIC dedicated to
the guests only) or clear only VMNS (to get a NIC dedicated to the host).

"dcohn" <dcohn@discussions.microsoft.com> wrote in message
news:39F39184-D66D-4A7D-A48E-AC49E22B65CE@microsoft.com...
> Thanks
>
> I have a dedicated NIC just for management. I also have taken your advice
> that the NIC settings are irrelevant if I am creating a Virtual switch.
> But
> I did not see this before I setup and it is not 100% clear anyway as they
> seem to refer to just a single NIC. The issue was multiple NICs and
> whether
> they required any settings BEFORE Creating a VS.
>
> I NOW BELIEVE the best thing would have been to take bare unconfigured
> NICs
> and use them as the VS NICs as long as they are physically plugged into
> the
> correct segment for the access desired.
>
> Using your info I removed the gateways from the Virtual Nics (they took
> the
> settings I had applied to the physical NICs). Everything works fine
> without
> those gateways as you stated.
>
> It seems I Do NOT need to bind an IP address to the Virtual NICs? RIGHT?
>
> Since I already have created the VS can I just change those NIC settings
> to
> obtain automatically (we have NO DHCP Servers as this is a data center
> with
> only servers) and then I would not be wasting those IPs.
>
> Seems like a PLAN.
>
> I don't know why I assumed I needed the IP address but it is really not
> stated anywhere and habit caused me to do it. I follow now and it is
> quite
> clear.
>
> "Bill Grant" wrote:
>
>>
>>
>> "dcohn" <dcohn@discussions.microsoft.com> wrote in message
>> news:69ACF9A8-17F7-44B0-B038-9773FD0D6314@microsoft.com...
>> > That is correct HYPER V. But if you create a Virtual Switch the NIC
>> > must
>> > first have assigned IPs and gateways NO?????? QUESTION THAT IS
>> > ACTUALLY....
>> > Or is it simply based on the network address assigned to the VMs and
>> > where
>> > the NIC used for the VS is physically cabled?
>> >
>>
>>
>> When you create an external virtual switch in Hyper-V (ie one linked
>> to
>> a physical NIC) it does not work like installing VMNS in Virtual Server.
>> See
>> this article from Ben's blog.
>>
>> http://blogs.msdn.com/virtual_pc_guy...h-hyper-v.aspx
>>
>> If you administer the server remotely, it is recommended to reserve
>> one
>> NIC in the host for this purpose (ie do not create a virtual network
>> linked
>> to this NIC) and use the other NIC(s) for vms. This is easier in R2 as
>> they
>> have made it more obvious in the GUI.
>>
>>
>>
>>

Thanks very much.

Just some minor confusion.

I also checked my W2K3 server with muttiple NICs and the additional NICs are
on private networks where they are used simply for access to local DATA ona
Layer 2 switch. NO routing.

The only other place I have multiple NICs I have them Teamed also.

The real lesson learned is to use NICs with no configuration plugged into
the segment desired as Virtual Switches. They just work and I have tons of
bandwiidth on this box as well.

THANKS

Doug

"Bill Grant" wrote:

>
>
> "Ace Fekay [Microsoft Certified Trainer]" <aceman@mvps.RemoveThisPart.org>
> wrote in message news:ek9#CUK6JHA.1096@TK2MSFTNGP06.phx.gbl...
> > "dcohn" <dcohn@discussions.microsoft.com> wrote in message
> > news:E4CB35EA-3C18-4D55-994F-0E54804F9A0C@microsoft.com...
> >> CAN I TEAM NICS I AM USING for a Virtual Switch?? I thought this was a
> >> NO NO?
> >>
> >>
> >> You said
> >>
> >> Why not just team
> >> them for balancing and fault tolerance? You can assign individual IPs on
> >> the
> >> VM guests independent of the host.
> >>
> >> "Ace Fekay [Microsoft Certified Trainer]" wrote:
> >>
> >>
> >
> >
> > I was speaking of the machine NICs itself. I had a server running with
> > teamed NICs using Virtual Server without a problem. I just assigned the
> > guests their own IP configs.
> >
> > Ace
> >
>
> There were problems with NIC teaming and Hyper-V early on. The
> manufacturers were slow to bring out drivers which were compatible with the
> Hyper-V networking software. The latest versions from Broadcom and Intel
> should handle NIC teaming, if you really need it.
>
> Networking in Hyper-V is very different from Virtual Server.
>
>
>