Tuesday, May 12, 2009

NFS trouble

on Linux, do a mount command, if you see anything mapping to a nfs storage

bcdtnf01.server.ux.acme.local:/data/nfs/tst/acme_tst_wls on /data/share/acme_tst_wls type nfs (rw,nosuid,hard,intr,addr=11.81.21.162)


then you might be exposed to the Linux NFS Mount Problem

We are talking about (uname -a):
Linux bctst06.acme.local 2.6.18-92.1.6.el5 #1 SMP Fri Jun 20 02:36:06 EDT 2008 x86_64 x86_64 x86_64 GNU/Linux

Do a nslookup bcdtnf01.server.ux.acme.local

nslookup bcdtnf01.server.ux.v.local
Server: 10.80.23.1
Address: 10.80.23.1#53

Name: bcdtnf01.server.ux.acme.local
Address: 11.81.21.161

so you know that the IP of your NFS storage is 11.81.21.161.

With route, you can find the routing data for that destination:

route -ee
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface MSS Window irtt
11.81.16.0 * 255.255.252.0 U 0 0 0 eth0 0 0 0
11.81.21.0 * 255.255.252.0 U 0 0 0 eth1 0 0 0
11.81.24.0 * 255.255.252.0 U 0 0 0 eth2 0 0 0
169.254.0.0 * 255.255.0.0 U 0 0 0 eth2 0 0 0
default 11.81.16.254 0.0.0.0 UG 0 0 0 eth0 0 0 0

then use http://en.wikipedia.org/wiki/Traceroute traceroute


http://en.wikipedia.org/wiki/Dmesg

if you do dmesg you get:

dmesg
Linux version 2.6.18-92.1.6.el5 (brewbuilder@ls20-bc2-13.build.redhat.com) (gcc version 4.1.2 20071124 (Red Hat 4.1.2-42)) #1 SMP Fri Jun 20 02:36:06 EDT 2008
Command line: ro root=/dev/vg00/root notsc divider=10 elevator=noop
BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000009f800 (usable)
BIOS-e820: 000000000009f800 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000ca000 - 00000000000cc000 (reserved)
BIOS-e820: 00000000000dc000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 00000000efef0000 (usable)
BIOS-e820: 00000000efef0000 - 00000000efeff000 (ACPI data)
BIOS-e820: 00000000efeff000 - 00000000eff00000 (ACPI NVS)
BIOS-e820: 00000000eff00000 - 00000000f0000000 (usable)
BIOS-e820: 00000000fec00000 - 00000000fec10000 (reserved)
BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
BIOS-e820: 00000000fffe0000 - 0000000100000000 (reserved)
BIOS-e820: 0000000100000000 - 0000000271000000 (usable)
DMI present.
ACPI: RSDP (v000 PTLTD ) @ 0x00000000000f6c60
ACPI: RSDT (v001 PTLTD RSDT 0x06040000 LTP 0x00000000) @ 0x00000000efefab5a
ACPI: FADT (v001 INTEL 440BX 0x06040000 PTL 0x000f4240) @ 0x00000000efefef06
ACPI: MADT (v001 PTLTD APIC 0x06040000 LTP 0x00000000) @ 0x00000000efefef7a
ACPI: BOOT (v001 PTLTD $SBFTBL$ 0x06040000 LTP 0x00000001) @ 0x00000000efefefd8
ACPI: DSDT (v001 PTLTD Custom 0x06040000 MSFT 0x0100000d) @ 0x0000000000000000
No NUMA configuration found
Faking a node at 0000000000000000-0000000271000000
Bootmem setup node 0 0000000000000000-0000000271000000
Memory for crash kernel (0x0 to 0x0) notwithin permissible range
disabling kdump
On node 0 totalpages: 2458094
DMA zone: 2686 pages, LIFO batch:0
DMA32 zone: 964648 pages, LIFO batch:31
Normal zone: 1490760 pages, LIFO batch:31
ACPI: PM-Timer IO Port: 0x1008
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
Processor #0 7:7 APIC version 17
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
Processor #1 7:7 APIC version 17
ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Setting APIC routing to physical flat
Using ACPI (MADT) for SMP configuration information
Nosave address range: 000000000009f000 - 00000000000a0000
Nosave address range: 00000000000a0000 - 00000000000ca000
Nosave address range: 00000000000ca000 - 00000000000cc000
Nosave address range: 00000000000cc000 - 00000000000dc000
Nosave address range: 00000000000dc000 - 0000000000100000
Nosave address range: 00000000efef0000 - 00000000efeff000
Nosave address range: 00000000efeff000 - 00000000eff00000
Nosave address range: 00000000f0000000 - 00000000fec00000
Nosave address range: 00000000fec00000 - 00000000fec10000
Nosave address range: 00000000fec10000 - 00000000fee00000
Nosave address range: 00000000fee00000 - 00000000fee01000
Nosave address range: 00000000fee01000 - 00000000fffe0000
Nosave address range: 00000000fffe0000 - 0000000100000000
Allocating PCI resources starting at f1000000 (gap: f0000000:ec00000)
SMP: Allowing 2 CPUs, 0 hotplug CPUs
Built 1 zonelists. Total pages: 2458094
Kernel command line: ro root=/dev/vg00/root notsc divider=10 elevator=noop
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 32768 bytes)
Console: colour VGA+ 80x25
Dentry cache hash table entries: 2097152 (order: 12, 16777216 bytes)
Inode-cache hash table entries: 1048576 (order: 11, 8388608 bytes)
Checking aperture...
PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
Placing software IO TLB between 0x282b000 - 0x682b000
Memory: 9736580k/10240000k available (2459k kernel code, 240824k reserved, 1247k data, 196k init)
Calibrating delay using timer specific routine.. 6060.56 BogoMIPS (lpj=3030280)
Security Framework v1.0.0 initialized
SELinux: Initializing.
SELinux: Starting in permissive mode
selinux_register_security: Registering secondary module capability
Capability LSM initialized as secondary
Mount-cache hash table entries: 256
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 6144K
SMP alternatives: switching to UP code
ACPI: Core revision 20060707
Using local APIC timer interrupts.
result 4105065
Detected 4.105 MHz APIC timer.
SMP alternatives: switching to SMP code
Booting processor 1/2 APIC 0x1
Initializing CPU#1
Calibrating delay using timer specific routine.. 6002.11 BogoMIPS (lpj=3001057)
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 6144K
Intel(R) Xeon(R) CPU E5450 @ 3.00GHz stepping 08
Brought up 2 CPUs
testing NMI watchdog ... <4>WARNING: CPU#0: NMI appears to be stuck (0->0)!
Disabling vsyscall due to use of PM timer
time.c: Using 3.579545 MHz WALL PM GTOD PM timer.
time.c: Detected 2999.627 MHz processor.
sizeof(vma)=176 bytes
sizeof(page)=56 bytes
sizeof(inode)=560 bytes
sizeof(dentry)=216 bytes
sizeof(ext3inode)=760 bytes
sizeof(buffer_head)=96 bytes
sizeof(skbuff)=240 bytes
migration_cost=3
checking if image is initramfs... it is
Freeing initrd memory: 3203k freed
NET: Registered protocol family 16
No dock devices found.
ACPI: bus type pci registered
PCI: Using configuration type 1
ACPI: Interpreter enabled
ACPI: Using IOAPIC for interrupt routing
ACPI: PCI Root Bridge [PCI0] (0000:00)
PCI: Probing PCI hardware (bus 00)
PCI quirk: region 1000-103f claimed by PIIX4 ACPI
PCI quirk: region 1040-104f claimed by PIIX4 SMB
Boot video device is 0000:00:0f.0
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 9 10 11 14 *15)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 *9 10 11 14 15)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 7 9 10 *11 14 15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 7 9 *10 11 14 15)
Linux Plug and Play Support v0.97 (c) Adam Belay
pnp: PnP ACPI init
pnp: PnP ACPI: found 12 devices
usbcore: registered new driver usbfs
usbcore: registered new driver hub
PCI: Using ACPI for IRQ routing
PCI: If a device doesn't work, try "pci=routeirq". If it helps, post a report
NetLabel: Initializing
NetLabel: domain hash size = 128
NetLabel: protocols = UNLABELED CIPSOv4
NetLabel: unlabeled traffic allowed by default
PCI-GART: No AMD northbridge found.
PCI: Bridge: 0000:00:01.0
IO window: disabled.
MEM window: disabled.
PREFETCH window: disabled.
PCI: Setting latency timer of device 0000:00:01.0 to 64
NET: Registered protocol family 2
IP route cache hash table entries: 524288 (order: 10, 4194304 bytes)
TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 262144 bind 65536)
TCP reno registered
Simple Boot Flag at 0x36 set to 0x80
audit: initializing netlink socket (disabled)
audit(1240702275.849:1): initialized
Total HugeTLB memory allocated, 0
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
SELinux: Registering netfilter hooks
Initializing Cryptographic API
ksign: Installing public key data
Loading keyring
- Added public key 5C9DD0FD4CC9C295
- User ID: Red Hat, Inc. (Kernel Module GPG key)
io scheduler noop registered (default)
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered
Limiting direct PCI/PCI transfers.
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
ACPI: Processor [CPU0] (supports 8 throttling states)
ACPI: Processor [CPU1] (supports 8 throttling states)
ACPI Exception (acpi_processor-0681): AE_NOT_FOUND, Processor Device is not present [20060707]
ACPI: Getting cpuindex for acpiid 0x2
ACPI Exception (acpi_processor-0681): AE_NOT_FOUND, Processor Device is not present [20060707]
ACPI: Getting cpuindex for acpiid 0x3
Real Time Clock Driver v1.12ac
Non-volatile memory driver v1.2
Linux agpgart interface v0.101 (c) Dave Jones
agpgart: Detected an Intel 440BX Chipset.
agpgart: AGP aperture is 256M @ 0x0
Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing enabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 0) is a 16550A
serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
00:09: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
00:0a: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
RAMDISK driver initialized: 16 RAM disks of 16384K size 4096 blocksize
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
PIIX4: IDE controller at PCI slot 0000:00:07.1
PIIX4: chipset revision 1
PIIX4: not 100% native mode: will probe irqs later
ide0: BM-DMA at 0x1050-0x1057, BIOS settings: hda:DMA, hdb:pio
Probing IDE interface ide0...
hda: VMware Virtual IDE CDROM Drive, ATAPI CD/DVD-ROM drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
Probing IDE interface ide1...
ide-floppy driver 0.99.newide
usbcore: registered new driver hiddev
usbcore: registered new driver usbhid
drivers/usb/input/hid-core.c: v2.6:USB HID core driver
PNP: PS/2 Controller [PNP0303:KBC,PNP0f13:MOUS] at 0x60,0x64 irq 1,12
serio: i8042 KBD port at 0x60,0x64 irq 1
serio: i8042 AUX port at 0x60,0x64 irq 12
mice: PS/2 mouse device common for all mice
md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
TCP bic registered
Initializing IPsec netlink socket
NET: Registered protocol family 1
NET: Registered protocol family 17
ACPI: (supports S0 S1 S4 S5)
Freeing unused kernel memory: 196k freed
Write protecting the kernel read-only data: 476k
input: AT Translated Set 2 keyboard as /class/input/input0
input: ImPS/2 Generic Wheel Mouse as /class/input/input1
ohci_hcd: 2005 April 22 USB 1.1 'Open' Host Controller (OHCI) Driver (PCI)
USB Universal Host Controller Interface driver v3.0
SCSI subsystem initialized
Fusion MPT base driver 3.04.05
Copyright (c) 1999-2007 LSI Corporation
Fusion MPT SPI Host driver 3.04.05
GSI 16 sharing vector 0xA9 and IRQ 16
ACPI: PCI Interrupt 0000:00:10.0[A] -> GSI 17 (level, low) -> IRQ 169
mptbase: ioc0: Initiating bringup
ioc0: LSI53C1030 B0: Capabilities={Initiator}
scsi0 : ioc0: LSI53C1030 B0, FwRev=01032920h, Ports=1, MaxQ=128, IRQ=169
Vendor: VMware Model: Virtual disk Rev: 1.0
Type: Direct-Access ANSI SCSI revision: 02
target0:0:0: Beginning Domain Validation
target0:0:0: Domain Validation skipping write tests
target0:0:0: Ending Domain Validation
target0:0:0: FAST-40 WIDE SCSI 80.0 MB/s ST (25 ns, offset 127)
SCSI device sda: 20971520 512-byte hdwr sectors (10737 MB)
sda: test WP failed, assume Write Enabled
sda: cache data unavailable
sda: assuming drive cache: write through
SCSI device sda: 20971520 512-byte hdwr sectors (10737 MB)
sda: test WP failed, assume Write Enabled
sda: cache data unavailable
sda: assuming drive cache: write through
sda: sda1 sda2
sd 0:0:0:0: Attached scsi disk sda
Vendor: VMware Model: Virtual disk Rev: 1.0
Type: Direct-Access ANSI SCSI revision: 02
target0:0:1: Beginning Domain Validation
target0:0:1: Domain Validation skipping write tests
target0:0:1: Ending Domain Validation
target0:0:1: FAST-40 WIDE SCSI 80.0 MB/s ST (25 ns, offset 127)
SCSI device sdb: 31457280 512-byte hdwr sectors (16106 MB)
sdb: test WP failed, assume Write Enabled
sdb: cache data unavailable
sdb: assuming drive cache: write through
SCSI device sdb: 31457280 512-byte hdwr sectors (16106 MB)
sdb: test WP failed, assume Write Enabled
sdb: cache data unavailable
sdb: assuming drive cache: write through
sdb: sdb1
sd 0:0:1:0: Attached scsi disk sdb
libata version 3.00 loaded.
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.11.5-ioctl (2007-12-12) initialised: dm-devel@redhat.com
EXT3-fs: INFO: recovery required on readonly filesystem.
EXT3-fs: write access will be enabled during recovery.
kjournald starting. Commit interval 5 seconds
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with ordered data mode.
SELinux: Disabled at runtime.
SELinux: Unregistering netfilter hooks
audit(1240702282.382:2): selinux=0 auid=4294967295 ses=4294967295
input: PC Speaker as /class/input/input2
hda: ATAPI 1X CD-ROM drive, 32kB Cache, UDMA(33)
Uniform CD-ROM driver Revision: 3.20
parport: PnPBIOS parport detected.
parport0: PC-style at 0x378, irq 7 [PCSPP,TRISTATE]
Intel(R) PRO/1000 Network Driver - version 7.3.20-k2-NAPI
Copyright (c) 1999-2006 Intel Corporation.
GSI 17 sharing vector 0xB1 and IRQ 17
ACPI: PCI Interrupt 0000:00:11.0[A] -> GSI 18 (level, low) -> IRQ 177
PCI: Setting latency timer of device 0000:00:11.0 to 64
sd 0:0:0:0: Attached scsi generic sg0 type 0
sd 0:0:1:0: Attached scsi generic sg1 type 0
e1000: 0000:00:11.0: e1000_probe: (PCI:66MHz:32-bit) 00:50:56:a7:14:bb
e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
GSI 18 sharing vector 0xB9 and IRQ 18
ACPI: PCI Interrupt 0000:00:12.0[A] -> GSI 19 (level, low) -> IRQ 185
PCI: Setting latency timer of device 0000:00:12.0 to 64
Floppy drive(s): fd0 is 1.44M
FDC 0 is a post-1991 82077
e1000: 0000:00:12.0: e1000_probe: (PCI:66MHz:32-bit) 00:50:56:a7:5f:c6
e1000: eth1: e1000_probe: Intel(R) PRO/1000 Network Connection
GSI 19 sharing vector 0xC1 and IRQ 19
ACPI: PCI Interrupt 0000:00:13.0[A] -> GSI 16 (level, low) -> IRQ 193
PCI: Setting latency timer of device 0000:00:13.0 to 64
e1000: 0000:00:13.0: e1000_probe: (PCI:66MHz:32-bit) 00:50:56:a7:75:2b
e1000: eth2: e1000_probe: Intel(R) PRO/1000 Network Connection
shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
piix4_smbus 0000:00:07.3: Found 0000:00:07.3 device
piix4_smbus 0000:00:07.3: Host SMBus controller not enabled!
lp0: using parport0 (interrupt-driven).
lp0: console ready
ACPI: AC Adapter [ACAD] (on-line)
ACPI: Power Button (FF) [PWRF]
ibm_acpi: ec object not found
md: Autodetecting RAID arrays.
md: autorun ...
md: ... autorun DONE.
device-mapper: multipath: version 1.0.5 loaded
EXT3 FS on dm-0, internal journal
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-2, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-3, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-4, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-5, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on sda1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
VMware hgfs: HGFS is disabled in the host
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-8, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-7, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
Adding 1048568k swap on /dev/vg00/swap. Priority:-1 extents:1 across:1048568k
IA-32 Microcode Update Driver: v1.14a
e1000: eth0: e1000_watchdog_task: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
e1000: eth1: e1000_watchdog_task: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
e1000: eth2: e1000_watchdog_task: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
VMware memory control driver initialized
vmmemctl: started kernel thread pid=2696
e1000: eth0: e1000_set_tso: TSO is Enabled
e1000: eth1: e1000_set_tso: TSO is Enabled
e1000: eth2: e1000_set_tso: TSO is Enabled
VMware hgfs: HGFS is disabled in the host
FS-Cache: Loaded
FS-Cache: netfs 'nfs' registered for caching
BUG: soft lockup - CPU#0 stuck for 10s! [opcmon:29290]
CPU 0:
Modules linked in: nfs lockd fscache nfs_acl autofs4 vmmemctl(U) sunrpc vmhgfs(U) dm_multipath video sbs backlight i2c_ec button battery asus_acpi acpi_memhotplug ac lp i2c_piix4 i2c_core floppy shpchp sg e1000 parport_pc parport serio_raw ide_cd pcspkr cdrom dm_snapshot dm_zero dm_mirror dm_mod ata_piix libata mptspi mptscsih mptbase scsi_transport_spi sd_mod scsi_mod ext3 jbd uhci_hcd ohci_hcd ehci_hcd
Pid: 29290, comm: opcmon Tainted: G 2.6.18-92.1.6.el5 #1
RIP: 0010:[] [] __handle_mm_fault+0x7e/0xe23
RSP: 0018:ffff8100c8a0bb38 EFLAGS: 00000202
RAX: ffff81026dace100 RBX: ffff81026dace100 RCX: 0000000000000001
RDX: ffff810080bf5000 RSI: ffff8100bfcfad98 RDI: ffff81026dace100
RBP: ffff810100000000 R08: 0000000087aa4a48 R09: 0000000000000005
R10: 000000009f8c1ca1 R11: 000000007b0af9e2 R12: 0000000000000000
R13: ffff8100cdaf80c0 R14: 0000000087aa4a48 R15: 0000000000000005
FS: 0000000000000000(0000) GS:ffffffff8039f000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00000000ffadaf0b CR3: 00000000c947f000 CR4: 00000000000006e0

Call Trace:
[] __handle_mm_fault+0xdee/0xe23
[] do_page_fault+0x4cb/0x830
[] prio_tree_insert+0x182/0x231
[] error_exit+0x0/0x84
[] copy_user_generic+0xae/0x16c
[] arch_align_stack+0x12/0x23
[] load_elf32_binary+0x1233/0x197b
[] zone_statistics+0x3e/0x6d
[] search_binary_handler+0xbb/0x26d
[] do_execve+0x18c/0x243
[] sys_execve+0x36/0x4c
[] stub_execve+0x67/0xb0

BUG: soft lockup - CPU#0 stuck for 10s! [opcmon:29290]
CPU 0:
Modules linked in: nfs lockd fscache nfs_acl autofs4 vmmemctl(U) sunrpc vmhgfs(U) dm_multipath video sbs backlight i2c_ec button battery asus_acpi acpi_memhotplug ac lp i2c_piix4 i2c_core floppy shpchp sg e1000 parport_pc parport serio_raw ide_cd pcspkr cdrom dm_snapshot dm_zero dm_mirror dm_mod ata_piix libata mptspi mptscsih mptbase scsi_transport_spi sd_mod scsi_mod ext3 jbd uhci_hcd ohci_hcd ehci_hcd
Pid: 29290, comm: opcmon Tainted: G 2.6.18-92.1.6.el5 #1
RIP: 0010:[] [] flush_tlb_page+0x57/0xda
RSP: 0018:ffff8100c8a0baf8 EFLAGS: 00000286
RAX: ffff8100cdaf80c0 RBX: ffff81026dace100 RCX: 0000000000000000
RDX: ffff810100000000 RSI: ffff81026dace3d8 RDI: ffff8100c8a0bb18
RBP: 000000007b0af9e2 R08: 0000000087aa4a48 R09: 0000000000000005
R10: 000000009f8c1ca1 R11: 000000007b0af9e2 R12: ffff81026dace100
R13: 0000000000000282 R14: ffff810108aa7d40 R15: ffffffff800c5b6e
FS: 0000000000000000(0000) GS:ffffffff8039f000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00000000ffadaf0b CR3: 00000000c947f000 CR4: 00000000000006e0

Call Trace:
[] flush_tlb_page+0xac/0xda
[] __handle_mm_fault+0xdee/0xe23
[] do_page_fault+0x4cb/0x830
[] prio_tree_insert+0x182/0x231
[] error_exit+0x0/0x84
[] copy_user_generic+0xae/0x16c
[] arch_align_stack+0x12/0x23
[] load_elf32_binary+0x1233/0x197b
[] zone_statistics+0x3e/0x6d
[] search_binary_handler+0xbb/0x26d
[] do_execve+0x18c/0x243
[] sys_execve+0x36/0x4c
[] stub_execve+0x67/0xb0

No comments: