/var/log/messages not sorted correctly

From: David White 
------------------------------------------------------
I inherited the task to support a large website running on a cPanel server
(I hate cpanel and refuse to touch it as much as possible) that is hosted
and managed by a different company (not me).

After today, it might be time for me to convince the site owners to switch
to me. =)

Anyone ever heard of a CentOS /var/log/messages file showing entries that
are completely (and randomly) out of order?

I've engaged the web hosting provider, as the website is currently giving
off 500s. I manually went through the access
===============================================================
From: Stephen Kraus 
------------------------------------------------------
Sounds compromised, either way seems like its going to need to be re-done,
so might as well push the client onto a more supportable back end


===============================================================
From: Lynn Dixon 
------------------------------------------------------
Check the system clock for drift and then try restarting rsyslog

Is it setup for ntp? I I can tell you if it's running on VMware and getting
it's time from what the os thinks is the "hardware", clock its going to
drift. Vmware has a hard time keeping time especially on vmotion hosts.


===============================================================
From: Mike Harrison 
------------------------------------------------------

I'll vote for "both".

While it could be a clock problem, and I would start with that.
Then install from a properly running similar system, a known good ps, top, ps, 
du, etc.. etc.. in an odd directory to poke around with. For example, 
compare the output of /clean/ps with /usr/bin/ps

Side note: On a fresh system I have often made a '/root/clean.zip'
wih what I consider the system debug essentials: ps, ls, ..