SBS 2008 becomes slow a few days after restart

After restart my SBS 2008 is perfect. It is fast and memory usage is 2.5Gb
out of 10Gb. 2-3 days later, memory usage grous to 7Gb, opening active
directory users and computers takes a few minutes and so does opening and
saving word and excel files in network locations on my workstations.
Restarting the server repairs it all for 2 days.

Please help me solve this situation which makes my SBS 2008 almos useless.

Unfortunately you've given us very little to go on and it doesn't sound like
you have much of an idea where the problem even lies.

For instance, you tell us how much memory your system is using, but not what
processes are using them. Exchange and SQL will both take "free" memory and
use it, so seeing your memory usage climb is NOT uncommon and certainly not
a problem in most installations.

Could be anything from a bad disk controller to a bad network card to any
number of other things in between (3rd party apps on the server?!?)

I'd start by doing a search for "performance monitoring" and "windows 2008"
and reading. There are plenty of blogs, posts, and books that cover setting
up some data gathering information and you'll be MUCH better armed in at
least diagnosing the problem.

please post us the rest of the hardware specification (plz be as exact as
possible), especially those around the storage subsystem (RAID-Controller,
HDDs, Cache etc.).

BIOS, Firmware and drivers are all up-to-date?

Remark: As a Databaseserver (Exchange, AD and so on are one too) consuming
as much RAM (for caching) as possible (just beneath the max. amount of
physical RAM) is GOOD (for the all-over performance).

I have been investigating this problem for a while. The only unconvincing
result says it is a matter of Opportunisting Locking. My feeling is the the
active directory is very slow but I could not find the reason.

My server consist of a SuperMicro mainboard using 2x 2Ghz Dual Opteron,
NVIDIA N Force Ethernet and LSI MegaRaid SAS 8208 using 4x 512GB SATA disks
in RAID1.
My server is running Exchange 2007 and Symantec BE 12.5 which has it's own
SQL 2005. It does serve 20 users. 2 additional Win2003 server are connected:
One running SQL2005 and the other is a Terminal Server.

Please feel free to ask for any any additional information.

I did forget to say that all 2008 software are up to date and that
originally the server was SBS2000 with no such problems.

Am I to understand that this is a whitebox/homebuilt server then? Has the
hardware been thoroughly tested for end-to-end compatibility and were
burn-in tests done to make sure the components (RAM/NIC) don't fail under
extended load *before* the OS was installed? (a little tough to do those
tests when an OS is running...)

how about BIOS, Firmware, drivers? Are they also up-to-date and
supported/certified under Windows Server 2008 64-bit?

Where did you read that about "Opportunisting Locking" and "active
directory"?

Can you send us a "ipconfig /all" from your SBS and some Clients/Server?

Did you check the free space on all your partition (>=40% free space for
best performance on NTFS files systems) and defraged it?

Sound ok for your scenario (a RAID Controller has better performance if it
can use BBU Cache >=128MB) , but i prefer tested Server from vendors like
HP, IBM, Fujitsu, Dell and so on..

What is your Security-Software solution and how is it configured?

Did you keep in mind the following KB-Articles:

Recommended [AV-Software] file and folder exclusions for Microsoft products
http://support.microsoft.com/kb/943556/en-us

Overview of Exchange Server [..] and Antivirus Software
http://support.microsoft.com/?id=823166

Virus scanning recommendations on a Windows 2000 or on a Windows Server [..]
domain controller
http://support.microsoft.com/?id=822158

Consideration for a Virus Scanner on a Computer That Is Running SQL Server
http://support.microsoft.com/?id=309422

Are there any warnings, errors or other information in the eventlogs you can
share?

I don't see a mention of the actual physical RAM installed?

See my answers below your text:

The BIOS is part of the SuperMicro H8DME-2 board which is specified as
Windows 2008 64 bit complient. Most drivers are Microsoft original. I did
change the ethernet driver to Nvidia 2008 complient driver because the
original Microsoft did fail.

My RAID controller has no Cache.
A commercial Server of the same capacity will cost 4 times as my server.

Here is the mos significant one:
The User accounts cannot be added into GroupPolicy 'AllSBSUsers'. Please
Verify the GroupPolicy 'AllSBSUsers' exists.

This server was built by myself from best cost/performance parts. I have a
long experiance in that matter. The mainboard is specified by SuperMicro as
Windows 2008 64bit complient. The memory comes from Kingston and the disks
from Hitachi.

So again: BIOS, FIRMWARE and DRIVERS of all your (self-tested)
hardware-components are all UP-TO-DATE..?

Did you implement (-> disable the Chimney and RSS feature) the following:

Information about the TCP Chimney Offload, Receive Side Scaling, and Network
Direct Memory Access features in Windows Server 2008
http://support.microsoft.com/kb/951037/en-us
[..]
netsh int tcp show global

netsh int tcp set global chimney=disabled
netsh int tcp set global rss=disabled
[..]

so, did you read this too?

"As of now there is no file lock / opslock which i would suspect, opslock is
a feature which is only specific when accesisng file over a network, then
file operations are converted into network requests. so please confirm and
tweak the opslock only if your application is dealing with file related
activity." Source:
http://social.technet.microsoft.com/...f-95a249df3d37

That's why i asked about "Opportunisting Locking" AND "active directory",
because one is file/share (which use SMB1) related, the other is a database
connection (Active Directory is a LDAP-Database where data is pulled from
LDAP-clients via LDAP-Connections on TCP-Port 389 - s. e.g.
http://technet.microsoft.com/en-us/l.../cc961766.aspx)

Got it:

It isn't my intention to second-guess your experience in building boxes. It
is, however, my experience, that custom built boxes experience *FAR* more
incompatibility issues than major vendor servers. The problem does not lie
with the system builder, as they can buy parts that are obviously on the HCL
and *should* work together. So this is by no means a criticism. It is
simply the nature of *so* many moving parts that when one thing goes wrong,
the whole system is affected. I've seen motherboards that, because of a
BIOS/Firmware issue, are way more sensitive to RAM timing issues. So RAM
that *should* work still causes problems. Or a NIC that leaves it's PXE
boot code in memory, which on 99% of motherboards works fine, but on 2%
conflicts with something else the motherboard is trying to do. And on and
on, I could keep listing examples.

So yes, I asked if it was a white-box server because, based on both the
description of the problem (to which I alluded to before) and now with this
added information, I suspect that the problem is hardware related. It is
because of these incompatibilities and the difficulties in tracking them
down that you'll find most long-time server admins will recommend a major
brand for a server purchase. The difference in price, the fact that
desktops are usually NOT mission critical, but servers are, the overnight
warranties you can get with most servers, and the cost of troubleshooting
just doesn’t just justify home-built servers whereas a homebuilt desktop is
perfectly acceptable.

Anyways, that is just my opinion. I'd still start with setting up some
performance monitors. That should at least let you narrow down which piece
of hardware is giving you problems. But ultimately I suspect you'll find
that this is a compatibility issue that you'll need to fix by changing out
"stuff."

I did start to proceed according to your instructions. Here is a diary of
actions and error messages. Your comments will be appreciated.

12/09/2009 server restart sharepoint stopped

14/09/2009

netsh int tcp set global chimney=disabled
netsh int tcp set global rss=disabled

16/09/2009

Server become slow.
Typically the following occures:
Open network view of server
Check properies of Share AMOS, take a few minutes to open
Check security, a string of numbers apper instead of user Amos for a few
minutes.

From the Event Viewer:
System

The processing of Group Policy failed. Windows attempted to read the file
\\equip8.local\sysvol\equip8.local\Policies\{6AC1786C-016F-11D2-945F-00C04fB984F9}\gpt.ini
from a domain controller and was not successful. Group Policy settings may
not be applied until this event is resolved. This issue may be transient and
could be caused by one or more of the following:
a) Name Resolution/Network Connectivity to the current domain controller.
b) File Replication Service Latency (a file created on another domain
controller has not replicated to the current domain controller).
c) The Distributed File System (DFS) client has been disabled.

The Group Policy settings for the computer were processed successfully.
There were no changes detected since the last successful processing of Group
Policy.

Microsoft windows- Small buisness.

The User accounts cannot be added into GroupPolicy 'AllSBSUsers'. Please
Verify the GroupPolicy 'AllSBSUsers' exists.

17/09/2009 After restart and Sharepoint turn on

From the Event Viewer:

Application

The update cannot be started because the content sources cannot be accessed.
Fix the errors and try the update again.

Context: Application 'Search', Catalog 'index file on the search server
Search'

System

A RADIUS message was received from the invalid RADIUS client IP address
10.0.0.100.

The problem was solved as I did expect: Nothing was wrong with the hardware,
it was all SBS 2008. I did install Windows Search service, it did create a
250,000 item index and my server flies like a bird. A month ago, I did
install the indexing service which was useless so I did uninstall it. I still
do not understand the difference berween the 3 indexing types available on
SBS 2008.

It does still remains to be seen if the system will deteriorate with time.

I guess, your scenario is too complex for discussing it in this newsgroup.
To analyse your underlying problems deeper, one has to "see" (e.g. via RDP
or teamviewer) your live infrastructure and make there some tests.

[Khadgar]

#Amos Sobel

I am expirience the exact same problem as you ... so what exactly helped you out here?