Cyberarmor viexpf2k.sys conflicts with MS's Driver Verifier

I had a BSOD (Blue Screen of Death) on my Windows XP laptop today. It freaked me out as I don't remember changing any system configurations previously, so I thought that it might be a virus. I've extracted the juicy part from the BSOD messages below:

--------------------------------------------------------------------
Technical information:

*** STOP: 0x000000c9 (0x00000007, 0xA8655492, 0x8ADACF68, 0x00000000)

*** viexpf2k.sys - Address A8655492 base at A8655000, DateStamp 4372d07b

Beginning dump of physical memory
--------------------------------------------------------------------

A little googling shows that "viexpf2k.sys" is either a malware, or a driver from CyberArmor. I have the Cyberarmor firewall (Corporate version) installed, so I investigated from this angle. But just in case, I also ran CureIt AntiVirus from DrWeb.

At first, I couldn't get into Safe Mode as it gets stuck during the driver loading phase. But CHKDSK pops up after a few hard reboots, and Safe Mode boots fine after the disk check.

For some reason, my laptop didn't generate any dump file (company policy, no direct control), so I could not confirm my suspicion with WinDbg. I had to do it the "hacker" way: I renamed "viexpf2k.sys" to "~viexpf2k.sys" in both the "C:\Program Files\CyberArmor" and "C:\WINDOWS\system32\drivers", and then my XP managed to boot normally, but without CyberArmor.

So it's confirmed that CyberArmor is the problem, but why? More googling shows stop code C9 to be DRIVER_VERIFIER_IOMANAGER_VIOLATION, and it hit me! I must have turned on Microsoft's Driver Verifier by mistake yesterday when I was researching on kernel dump debugging.

So I restored "viexpf2k.sys" and ran:

C:> verifier /reset

Voila! Every thing's back to normal after a reboot.

Performance tuning for Samba on Fedora 9

When I first set up Samba on my Fedora 9 server, the access speed was not very impressive to say the least.

So I googled a little, and I found out that I can get some speedups if my disk do not record the file access time as my application reads and writes to files very often. I added the noatime option to the entry for my share partition in "/etc/fstab" like:
UUID=????? /data ext3 defaults,noatime 1 1

I did get some speed increase after this, but it is still slower than a native Windows Share.

So I googled some more, and I found:
http://oreilly.com/catalog/samba/chapter/book/appb_02.html

I added the following options to "/etc/samba/smb.conf":

socket options = TCP_NODELAY IPTOS_LOWDELAY
dead time = 15

The speed increase is quite dramatic, but it is still slower than a native Windows Share. Still not sure if the "dead time" option helps, but I just don't have to time to test it out.

Symbol Server for WinDbg

Today, I used Microsoft's WinDbg debugger to check out a kernel memory dump to see if I can determine which is the last program in memory before my Windows 2003 server (Mystic Hare) crashed. I guessed that it may give me a clue about which is the offending program that is crashing the server.

I haven't used WinDbg in years, and I didn't have the symbol package for Windows 2003 with me. Downloading it is out of the question as my internet connection speed is slow and the package is a few hundred megabytes.

Lucky for me, WinDbg can now download symbols as needed if internet access is available, from what I read in http://www.microsoft.com/whdc/devtools/debugging/debugstart.mspx

I created the "c:\websymbols" directory to download the symbols to.

1. Start a debugging session.

2. Set the debugger symbol path as:
SRV*c:\websymbols*http://msdl.microsoft.com/download/symbols

Now, I can start the debugging process.

Unfortunately, the debugging didn't go far as the symbols for an external program's .sys file is not available, but it's another story.

Editing fstab from "Repair Filesystem" prompt in Fedora 9

My Fedora 9 server is a laptop with a non-standard harddisk setup. In a mis-guided attempt to reduce the heat produced by the laptop, I removed the internal SATA II harddisk and put the Fedora's / root partition on a external SATA II harddisk connected by a SATA ExpressCard. As my laptop do not detect the main harddisk connected by the SATA ExpressCard, I created the /boot partition on a 1GB USB disk.

Here comes the problem, the USB disk is detected as either "/dev/sda1" or "/dev/sdb1" at different boot times. Thus I have the problem of deciding what to put in the fstab. You may ask when don't I use the UUID since these devices are controlled by udev anyway. It's because sometimes the UUID used to identify the USB disk do not work during boot up and Fedora drops me into the "Repair Filesystem" prompt.

Usually, I'll insert the Fedora DVD, boot into "Rescue" mode, chroot /mnt/sysimage/ and edit the fstab file. But when I rebooted and encountered the same problem last night, I got fed up of getting the DVD again.

I found the following forum while googling and the answer provided is exactly what I need:
http://www.linuxquestions.org/questions/linux-software-2/how-modify-fstab-when-its-ro-in-repair-filesystem-mode-562021/

In the "Repair Filesystem" prompt, login and remount the filesystem with:
> mount -o remount,rw /

I can modify the fstab file now.

Also, I found out that the system can boot up perfectly fine without mounting the /boot partition when I removed the entry from fstab. But I put the /boot entry back in case I forget to remount /boot when I upgrade the kernel in the future.

VBScript to backup Windows System, Application and Security event logs

To diagnose my crashing Windows 2003 server (Mystic Hare) mentioned previously, I needed to backup the server's System, Application and Security event logs every half hour. At first, I thought of just copying the *.evt files to another location, using the Windows Scheduler. But this might not be the best idea as the latest entries might not be in the event logs. So I clobbered together a VBScript from the following links:
http://msdn.microsoft.com/en-us/library/aa394593.aspx
http://www.microsoft.com/technet/scriptcenter/guide/sas_log_pcna.mspx?mfr=true
http://www.informit.com/content/images/9780672329470/downloads/SPSiteBackup.wsf.txt

The script will backup the event logs using standard WMI methods to "C:\scripts\" and append the current date and time to the saved event log files.

'*******************************************************************
'********************** Begin Script *******************************
'*******************************************************************

dtmThisSecond = PadDigits(Second(Now), 2)
dtmThisMinute = PadDigits(Minute(Now), 2)
dtmThisHour = PadDigits(Hour(Now), 2)
dtmThisDay = PadDigits(Day(Now), 2)
dtmThisMonth = PadDigits(Month(Now), 2)
dtmThisYear = Year(Now)


strBackupName = dtmThisYear & "-" & dtmThisMonth _
& "-" & dtmThisDay & "_" & dtmThisHour & "-" & dtmThisMinute & "-" & dtmThisSecond
strComputer = "."

Set objWMIService = GetObject("winmgmts:" _
& "{impersonationLevel=impersonate,(Backup, Security)}!\\" & _
strComputer & "\root\cimv2")


Set colLogFiles = objWMIService.ExecQuery _
("Select * from Win32_NTEventLogFile " _
& "Where LogFileName='Application'")
For Each objLogfile in colLogFiles
objLogFile.BackupEventLog("c:\scripts\" _
& strBackupName & _
"_application.evt")
' WScript.Echo "File saved: " & strBackupName & _
' "_application.evt"
Next

Set colLogFiles = objWMIService.ExecQuery _
("Select * from Win32_NTEventLogFile " _
& "Where LogFileName='System'")
For Each objLogfile in colLogFiles
objLogFile.BackupEventLog("c:\scripts\" _
& strBackupName & _
"_system.evt")
Next



Set colLogFiles = objWMIService.ExecQuery _
("Select * from Win32_NTEventLogFile " _
& "Where LogFileName='Security'")
For Each objLogfile in colLogFiles
objLogFile.BackupEventLog("c:\scripts\" _
& strBackupName & _
"_security.evt")
Next


'===================================================================
' Functions
'===================================================================
' This function is used to pad date variables that contain only on digit.
Function PadDigits(n, totalDigits)
If totalDigits > len(n) then
PadDigits = String(totalDigits-len(n),"0") & n
Else
PadDigits = n
End If
End Function

'*******************************************************************
'************************ End Script *******************************
'*******************************************************************

PS: The script was only tested on a Windows XP system.

Dummy write to Windows' System event log

I have a Windows 2003 server, let's call it Mystic Hare, which keeps crashing intermittently. The problem I have in diagnosing the problem, is that the last 3 hours of the Windows System logs are always empty.

So either:
1. The last 3 hours of the System log are wiped out during the crash; or
2. There is no event written to the System log during those times.

So I decided to write to the System event log with dummy entries as an experiment.

Googling brings up a lot of VBScripts, JavaScripts, C#, WSH scripts, etc. But I needed a simple app that I can just set in the Windows scheduler. Thank goodness I found what I need at:
http://www.winserverkb.com/Uwe/Forum.aspx/windows-server-scripting/8696/Write-To-System-Log-On-A-Specified-Server-When

The command I'm using is:
> eventcreate /L System /T Information /id 1 /d "For testing only"

This is what I see for the dummy System log entry in the Event Viewer"

New "e1000" network adapter for Fedora 9 guest in VMWare Server 2

I read somewhat that in VMWare Server 2, I can get better network performance if I use the new Intel "e1000" network adapter instead of the default AMD one. So here's the process of replacing the adapter.

For each guest, add the following line to each VMX file:
ethernet0.virtualDev = "e1000"

For Windows 2003 guests, the new hardware will be automatically recognized and a new "Local Area Network 2" connection will be created.

For Fedora 9 guests, eth0 will fail to work during boot time as the new hardware is recognized as eth1. Modifying the network config files under "/etc/sysconfig/networking" do not help because in Fedora 9, udev controls the hardware. The following file needs to be modified:

/etc/udev/rules.d/70-persistent-net.rules

Remove or comment out the original line for the AMD network adapter.
# Advanced Micro Devices [AMD] 79c970 [PCnet32 LANCE] (rule written by anaconda)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:0c:29:9b:f4:58", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"


The new e1000 adapter should be in the file as:
# PCI device 0x8086:0x100f (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:0c:29:d2:28:34", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"


change NAME="eth1" to "eth0" so we get:

# PCI device 0x8086:0x100f (e1000)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:0c:29:d2:28:34", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

Reboot and the network service should start fine at boot time.

My first attempt at blogging!

Wow! I still can't believe that I've finally became a blogger. I used to tell myself never to become one, but guess what? Thanks to a friend, I found out that the blog is actually a great place to put all the technical information that I learnt; instead of just creating lots of little text files in my systems (that I can't seem to find the time to organize).