Printer Spooler Keeps Crashing - Windows Server 2003 SP2 R2

[AndyTC@community.nospam]

My print server keeps crashing constantly, a few times a week. It
crashes to the point of where I have to restart the spooler server to
get it to work.

We have 43 printers installed.
This is what we have for models of printers:

Brother HL-4040CN
Brother HL-5070N
Brother HL-5170DN
Brother HL-5250DN BR-Script3
Brother HL-5280DW BR-Script3
Brother HL-7050 Series
Canon iR C5180-H1 PCL5c
Canon iR3235/iR3245 PCL6
Canon iR3530 PCL6
Canon iR 3570/iR4570 PS3
Canon iR 5570/iR6570 PCL6
HP Color Laserjet 2500 PS
HP Color Laserjet 3000 PCL 6
HP Color Laserjet 5550 PCL 6
HP Designjet 500PS 42
HP Laserjet 4200 PS
HP Laserjet 2420
Lexmark C520
Lexmark C532
Lexmark T642 PS3
Ricoh Aficio AP400N
Ricoh Aficio CL2000
Xerox WCM20

This is the latest entry in Application Events:

Faulting application spoolsv.exe, version 5.2.3790.3959, faulting module
unknown, version 0.0.0.0, fault address 0x00000000.

This is another error shortly before the spooler crashed (I removed the
username):

Event Type: Error
Event Source: Print
Event Category: None
Event ID: 6161
Date: 2/27/2009
Time: 9:06:28 AM
User: HQ\xxx.xxxx
Computer: MERIDIA
Description:
The document Microsoft Word - 13C1F528.doc owned by xxx.xxxx failed to
print on printer Proposal Room. Data type: NT EMF 1.008. Size of the
spool file in bytes: 131072. Number of bytes printed: 0. Total number of
pages in the document: 1. Number of pages printed: 0. Client machine:
\\TCI428586. Win32 error code returned by the print processor: 0. The
operation completed successfully.

I have changed all HP printers to use WinPrint print processor, but its
still crashing.

Any idea's?

[Thee Chicago Wolf (MVP)]

See if the suggestions in KB947477 help you some.

Also, not sure if they'll install on R2 but have a look at and try
installing these hotfixes. I've had really good success with them.
http://support.microsoft.com/kb/955455
http://support.microsoft.com/kb/953546
http://support.microsoft.com/kb/952206
http://support.microsoft.com/kb/953155

[AndyTC@community.nospam]

I will check those out. In the meantime, the spool has crashed again.

This time, these two errors are in the event viewer
Event Type: Warning
Event Source: Server
Event Category: None
Event ID: 2510
Date: 2/27/2009
Time: 10:28:56 AM
User: N/A
Computer: MERIDIA
Description:
The server service was unable to map error code 1727.

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.

Event Type: Warning
Event Source: Server
Event Category: None
Event ID: 2510
Date: 2/27/2009
Time: 10:28:56 AM
User: N/A
Computer: MERIDIA
Description:
The server service was unable to map error code 1722.

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.

The second one has about 10 entries

[Thee Chicago Wolf (MVP)]

Check this KB article for this Event ID:
http://support.microsoft.com/kb/319335 (kind of useless but hey)

Also, look at what's mentioned about Xerox printers in this KB article
(pertains to Server 2000). Might work on 2003!
http://support.microsoft.com/kb/888206

also look at
http://support.microsoft.com/kb/194333
http://support.microsoft.com/kb/822219
http://support.microsoft.com/kb/826897
http://support.microsoft.com/kb/923360
http://support.microsoft.com/kb/158751

[AndyTC@community.nospam]

Unfortunately none of those KB's seem to apply to my environment. The
only thing I've done so far is removed a couple non-default Print
Monitors from the registry. Is there a way to enable logging to see
which printer/print driver is crashing the spooler?

I thought maybe dr. watson could do this, but I can't find any dr watson
log.

I am going to try the hyper threading hotfix tonight.
However, it has crashed again with the following error:

Application exception occurred:
App: C:\WINDOWS\system32\spoolsv.exe (pid=3936)
When: 2/27/2009 @ 16:11:52.103
Exception number: c0000005 (access violation)

Event Type: Error
Event Source: Application Error
Event Category: (100)
Event ID: 1000
Date: 2/27/2009
Time: 4:11:51 PM
User: N/A
Computer: MERIDIA
Description:
Faulting application spoolsv.exe, version 5.2.3790.3959, faulting module
rpcrt4.dll, version 5.2.3790.4115, fault address 0x00019e6d.

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 41 70 70 6c 69 63 61 74 Applicat
0008: 69 6f 6e 20 46 61 69 6c ion Fail
0010: 75 72 65 20 20 73 70 6f ure spo
0018: 6f 6c 73 76 2e 65 78 65 olsv.exe
0020: 20 35 2e 32 2e 33 37 39 5.2.379
0028: 30 2e 33 39 35 39 20 69 0.3959 i
0030: 6e 20 72 70 63 72 74 34 n rpcrt4
0038: 2e 64 6c 6c 20 35 2e 32 .dll 5.2
0040: 2e 33 37 39 30 2e 34 31 .3790.41
0048: 31 35 20 61 74 20 6f 66 15 at of
0050: 66 73 65 74 20 30 30 30 fset 000
0058: 31 39 65 36 64 19e6d

It has just occured to me that we have a separate Terminal Services
Server that serves about 15 users. What is the correct way to have
printers setup in terminal sessions? Will it use their local setup
connections? Do I need drivers on the TS?

here is the results of user.dmp

0:023> k
ChildEBP RetAddr
WARNING: Stack unwind information not available. Following frames may be
wrong.
02c5fe90 77c68ad6 rpcrt4!I_RpcTransConnectionAllocatePacket+0x1e4
02c5fea4 77c6e8db rpcrt4!I_RpcTransGetThreadEvent+0x1b4
02c5ff04 77c6e7b4 rpcrt4!I_RpcLogEvent+0x59c
02c5ff18 77c7b799 rpcrt4!I_RpcLogEvent+0x475
02c5ff84 77c7b9b5 rpcrt4!NdrFullPointerInsertRefId+0xa43
02c5ff8c 77c8872d rpcrt4!NdrFullPointerInsertRefId+0xc5f
02c5ffac 77c7b110 rpcrt4!I_RpcFree+0xb6b
*** ERROR: Symbol file could not be found. Defaulted to export symbols
for kern
el32.dll -
02c5ffb8 77e64829 rpcrt4!NdrFullPointerInsertRefId+0x3ba
02c5ffec 00000000 kernel32!GetModuleHandleA+0xdf
0:023>

[Thee Chicago Wolf (MVP)]

You might be able to tweak Dr. Watson to give you more info.

1. Start > Run > drwtsn32
2. Check on an boxes you may feel appropriate.
3. Select the bullet for "FULL" crashdump type.

By the way, all of your printer drivers fully current as can be?
Nothing old in there?

Also forgot to mention that when you do the Dr. Watson tweaks, also do
a Start > Run > drwtsn32.exe -i to make it the default debugger.

If you have a user dump of this failure you can find the faulting stack
trace using ntsd.exe

ntsd -z user.dmp

this will launch a new window and you will see bunch of module load
information

type k at the prompt

this will display the stack. Don't worry about symbols not found spew.

Look for hp driver names there is one that is fairly nasty hpzui4wm.dll.
HP has updated a bunch of drivers that use this module.

if you don't see anything that indicates a print driver send the stack trace

[Mervyn Zhang [MSFT]]

According to your description, I understand that:

The print server keeps crashing. There are several errors in log.

If I have misunderstood the problem, please don't hesitate to let me know.

Please try Alan’s suggestions and Wolf’s suggestions to check if you can
find the cause of this issue.

Also, the Event 6161 may be caused by insufficient memory. I suggest
monitoring system resource to check if memory or Network is in peak hour
when the error occurs.

You can try the steps below to narrow down the cause of this issue.

1. If possible, please check all printer drivers and make sure them are
WHQL signed printer drivers.

2. Test every printer on the server locally.

3. Please refer to the following KB to remove third party print monitors
and use the default print processors to test this issue again.

260142 How to troubleshoot printing problems in Windows 2000 and in Windows
Server 2003
http://support.microsoft.com/default...b;EN-US;260142

4. To isolate the exact printer driver that causes the spooler failure, we
will need to collect and analyze the dump file. Unfortunately, we cannot
analyze dump file in newsgroup threads. To do so, you will need to contact
Microsoft Customer Support Services (CSS) so that a dedicated Support
Professional can assist with this request. Thank you for your understanding.

To obtain the phone numbers for specific technology request please take a
look at the web site listed below.
http://support.microsoft.com/default...S;PHONENUMBERS

If you are outside the US please see http://support.microsoft.com for
regional support phone numbers.

[Alan Morris [MSFT]]

this is the Remote Procedure Call binary.

Okay no we need to dig a bit deeper. If you already called customer
support, they can handle this

==============

If you have a good internet connection on the client, set the symbol path
and reveal the stack trace with the corrected function calls.

The initial debugger output reveals useful information. After displaying the
command line used to start the application, it shows the search path for
finding symbol files (PDBs). The symbol search path is important and, in
this case, I had specified it beforehand by setting the _NT_SYMBOL_PATH
system variable before running ntsd.exe. The symbol path shown here is
SRV*c:\Files\websymbols*http://msdl.microsoft.com/download/symbols, which
identifies a symbol server URL and the location where downloaded symbol
files may be cached. That is, when ntsd needs to load symbols for a binary,
it connects to the symbol server at the specified URL, and requests PDBs
based on unique characteristics of the binary such as the name of the binary
and its timestamp. The URL here is for the Microsoft public symbol server,
which provides public Windows symbols for many versions of Windows. Of
course, I also need to make sure that ntsd can find the symbols for my
application. If I wish to add, say, the location "C:\Files" to the symbol
path, then I can use the debugger command .sympath+ C:\Files. The command
..symfix+ adds the URL for the Microsoft public symbol server if is not on
the path. After changing the symbol search path, I always use the .reload
command to force the debugger to reload all symbols.

[Adventure777]

Have you found any fixes for the 6161 errors and the spooler crashes?

I have a Clustered print spooler with the same issues and I am struggling to find a fix from the vendors.

I had the same problem and I think it started after a Microsoft Update. I found a solution if the KB260142 and have cut part of it and added my comments.

The problem is more than likely a printer that is configured wrong.

The below sequence will get you to a default setup so that you can then trouble shoot the issue.

Start Registry Editor.
Locate the Local Port value under the following key in the registry:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Print\Monitors\Local Port
Double-click the Driver subkey, and then edit the value. Change the string value to Localspl.dll, and then click OK.

Check the following registry key for third-party monitors. Remove any non-default monitors:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Print\Monitors
The default port monitors are:

AppleTalk Printing Devices (When Services for Macintosh is installed)
BJ Language Monitor
Local Port
PJL Language Monitor
Standard TCP/IP Port
USB Monitor
Windows NT Fax Monitor

** LPR Port
NOTE: Do not remove LPR Port Monitor unless advised by a Microsoft Support Professional. Create a folder called Monitors on the desktop. Export each non standard monitor key and save as the export as a reg file in the Monitors, to be used later. Once each key is exported, delete the key.

Check the following registry key for third-party print providers. Remove any non-default print providers:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Print\Providers

The default print providers are:
Internet Print Provider
LanMan Print Services

Create a folder called Providers on the desktop. Export each non standard provider key and save as the export as a reg file in the Providers folder, to be used later. Once each key is exported, delete the key.

Check the following registry key for third-party print processors. Remove any non-default print processors:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Print\Environments\Windows NT x86\Print Processors

The default print processor is:
WinPrint

Create a folder called Processors on the desktop. Export each non standard provider key and save as the export as a reg file in the Processors folder, to be used later. Once each key is exported, delete the key.

To find out what printer is using the print processor, use the Microsoft Product Support Reporting Tool (MPS_REPORTS) tool to open MachineName_PRINTDRIVERS.TXT, and then search for the third-party print processor and for the queues that are using the print processor.
Change the third-party print processor to WinPrint.
Click Start, point to Settings, and then click Control Panel.
Double-click Printers, right-click the printer, and then click Properties.
On the Advanced tab, click Print Processor.
In the Print Processor box, click WinPrint.
Click OK two times.
Quit Registry Editor.
After you edit the registry, restart the print spooler. To do so, start Microsoft Management Console (MMC) and add the Computer Management or the Services snap-in. Right-click Print Spooler Service, and then click Start.

Once you have removed all of the non standard keys as stated above. You should be able to start the Print Spooler with out it crashing. Test the spooler by opening the Printers from the Start Menu. If the Print Spooler is working, you should be able to see at least some of the printers. The next step is to click on the reg files that you created one at a time (to add them back to the registry) and restart the Print Spooler after each one.

When you find the key that stops the spooler from restarting then you know where to trouble shoot.

MY issue was in the HP Standard TCP/IP Ports key. I had 8 subkeys under Ports which each represented a printer port on the print server. They export that I did in the above process before deleting it had all 8 ports. So I cut it up into 8 separate reg files and added them one at a time until I found the one that was the problem. It turned out that a Xerox printer was one of the HP ports. I left it out and reconfigured the printer as a Standard TCP/IP port.

[scaughto]

Removing the monitors did the trick for me as well. Be warned...DO NOT install "HP Color LaserJet CP2020 Series PCL6 Network Express Install", as it will destroy your print server. If you happen to make this fatal mistake, ensure to remove all un-needed monitors, as suggested above!

Thank you so much for your help! You saved me many, many hours of server and print queue rebuilding!