From kximdo at gmail.com  Mon Mar  1 15:14:13 2010
From: kximdo at gmail.com (zzz kxmdo)
Date: Mon, 1 Mar 2010 22:14:13 +0800
Subject: [mpm-itk] mpm-itk@err.no
Message-ID: <e257ca8d1003010614o6c0d769ftbe33485bc753dfd1@mail.gmail.com>

Hi,


We are testing MPM-itk with php5_module on FreeBSD.

with httpd-vhosts.conf:

    <IfModule mpm_itk_module>
>           AssignUserId  testUser wheel
>           MaxClientsVHost 12
>     </IfModule>
>


But It seems very strange that phpinfo() shows the" Environment" with:

    USER:     *root *



Is something wrong?  Or should that be *testUser* ?



Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.err.no/pipermail/mpm-itk/attachments/20100301/5e13a3e0/attachment.htm>

From knut at auvor.no  Mon Mar  1 15:45:35 2010
From: knut at auvor.no (Knut Auvor Grythe)
Date: Mon, 1 Mar 2010 15:45:35 +0100
Subject: [mpm-itk] mpm-itk@err.no
In-Reply-To: <e257ca8d1003010614o6c0d769ftbe33485bc753dfd1@mail.gmail.com>
References: <e257ca8d1003010614o6c0d769ftbe33485bc753dfd1@mail.gmail.com>
Message-ID: <20100301144535.GJ17219@stud.ntnu.no>

On Mon, Mar 01, 2010 at 10:14:13PM +0800, zzz kxmdo wrote:
>     <IfModule mpm_itk_module>
>>           AssignUserId  testUser wheel
>>           MaxClientsVHost 12
>>     </IfModule>
> 
> But It seems very strange that phpinfo() shows the" Environment" with:
> 
>     USER:     *root *
> 
> Is something wrong?  Or should that be *testUser* ?

mpm-itk just does a setuid() with friends, it does not change the
environment. The vhost is probably running as testUser.

Try creating a php page with <? echo `id -a` ?> instead (or something
similar if your PHP settings do not allow running external programs)

-- 
Knut Auvor



From lawrencewilliams at lcwsoft.com  Mon Mar  1 15:58:49 2010
From: lawrencewilliams at lcwsoft.com (Lawrence Williams)
Date: Mon, 01 Mar 2010 11:28:49 -0330
Subject: [mpm-itk] mpm-itk@err.no
In-Reply-To: <20100301144535.GJ17219@stud.ntnu.no>
References: <e257ca8d1003010614o6c0d769ftbe33485bc753dfd1@mail.gmail.com>
	<20100301144535.GJ17219@stud.ntnu.no>
Message-ID: <4B8BD629.5000103@lcwsoft.com>

That is normal. I think it's probably just a glitch with the environment 
variable. To see if you have ITK configured properly, create a file 
called getuser.pl on your site and paste the following into it

#!/usr/bin/perl
print "Content-Type: text/html\n\n";

my $userid = $<;

# get user info
my ($name, $pass, $uid, $gid, $quota, $comment, $gcos, $dir, $shell, 
$expire) = getpwuid($userid);

# get logged-in user name
print "Current user is " . $name;
exit;

View getuser.pl from your web browser. If it displays the correct user 
name, everything is fine.

Regards,
Lawrence

Knut Auvor Grythe wrote:
> On Mon, Mar 01, 2010 at 10:14:13PM +0800, zzz kxmdo wrote:
>   
>>     <IfModule mpm_itk_module>
>>     
>>>           AssignUserId  testUser wheel
>>>           MaxClientsVHost 12
>>>     </IfModule>
>>>       
>> But It seems very strange that phpinfo() shows the" Environment" with:
>>
>>     USER:     *root *
>>
>> Is something wrong?  Or should that be *testUser* ?
>>     
>
> mpm-itk just does a setuid() with friends, it does not change the
> environment. The vhost is probably running as testUser.
>
> Try creating a php page with <? echo `id -a` ?> instead (or something
> similar if your PHP settings do not allow running external programs)
>
>   
> ------------------------------------------------------------------------
>
>
> No virus found in this incoming message.
> Checked by AVG - www.avg.com 
> Version: 8.5.435 / Virus Database: 271.1.1/2716 - Release Date: 03/01/10 07:34:00
>
>   




From kximdo at gmail.com  Mon Mar  1 16:20:49 2010
From: kximdo at gmail.com (zzz kxmdo)
Date: Mon, 1 Mar 2010 23:20:49 +0800
Subject: [mpm-itk] mpm-itk@err.no
In-Reply-To: <4B8BD629.5000103@lcwsoft.com>
References: <e257ca8d1003010614o6c0d769ftbe33485bc753dfd1@mail.gmail.com>
	<20100301144535.GJ17219@stud.ntnu.no> <4B8BD629.5000103@lcwsoft.com>
Message-ID: <e257ca8d1003010720n27836ae0kb6c11ba512521c14@mail.gmail.com>

Very thanks!

We wanna to put mpm-itk in some production Env,
but still not sure if it is safe enough, as it has to setuid(), it runs as
root until request parsed ...

Any suggestions?



2010/3/1 Lawrence Williams <lawrencewilliams at lcwsoft.com>

> That is normal. I think it's probably just a glitch with the environment
> variable. To see if you have ITK configured properly, create a file called
> getuser.pl on your site and paste the following into it
>
> #!/usr/bin/perl
> print "Content-Type: text/html\n\n";
>
> my $userid = $<;
>
> # get user info
> my ($name, $pass, $uid, $gid, $quota, $comment, $gcos, $dir, $shell,
> $expire) = getpwuid($userid);
>
> # get logged-in user name
> print "Current user is " . $name;
> exit;
>
> View getuser.pl from your web browser. If it displays the correct user
> name, everything is fine.
>
> Regards,
> Lawrence
>
> Knut Auvor Grythe wrote:
>
>> On Mon, Mar 01, 2010 at 10:14:13PM +0800, zzz kxmdo wrote:
>>
>>
>>>    <IfModule mpm_itk_module>
>>>
>>>
>>>>          AssignUserId  testUser wheel
>>>>          MaxClientsVHost 12
>>>>    </IfModule>
>>>>
>>>>
>>> But It seems very strange that phpinfo() shows the" Environment" with:
>>>
>>>    USER:     *root *
>>>
>>> Is something wrong?  Or should that be *testUser* ?
>>>
>>>
>>
>> mpm-itk just does a setuid() with friends, it does not change the
>> environment. The vhost is probably running as testUser.
>>
>> Try creating a php page with <? echo `id -a` ?> instead (or something
>> similar if your PHP settings do not allow running external programs)
>>
>>  ------------------------------------------------------------------------
>>
>>
>> No virus found in this incoming message.
>> Checked by AVG - www.avg.com Version: 8.5.435 / Virus Database:
>> 271.1.1/2716 - Release Date: 03/01/10 07:34:00
>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.err.no/pipermail/mpm-itk/attachments/20100301/ae9af208/attachment.htm>

From jani+mpm-itk at ifi.uio.no  Mon Mar  1 19:12:36 2010
From: jani+mpm-itk at ifi.uio.no (Jan Ingvoldstad)
Date: Mon, 01 Mar 2010 19:12:36 +0100
Subject: [mpm-itk] mpm-itk@err.no
In-Reply-To: <e257ca8d1003010720n27836ae0kb6c11ba512521c14@mail.gmail.com>
	(zzz kxmdo's message of "Mon\, 1 Mar 2010 23\:20\:49 +0800")
References: <e257ca8d1003010614o6c0d769ftbe33485bc753dfd1@mail.gmail.com>
	<20100301144535.GJ17219@stud.ntnu.no> <4B8BD629.5000103@lcwsoft.com>
	<e257ca8d1003010720n27836ae0kb6c11ba512521c14@mail.gmail.com>
Message-ID: <oasocj7vrl7.fsf@luke.ifi.uio.no>

On Mon, 1 Mar 2010 23:20:49 +0800, zzz kxmdo <kximdo at gmail.com> said:

> Very thanks!
> We wanna to put mpm-itk in some production Env,
> but still not sure if it is safe enough, as it has to setuid(), it runs as
> root until request parsed ...

Any service that listens on a privileged port needs some system
privileges.  You can, of course, limit these privileges to a certain
degree by configuring your system appropriately.

> Any suggestions?

Run it in a VM, so that the potential damage from any security
vulnerability is limited.

If you plan to allow scripting (Java, Perl, PHP, Python, Ruby, etc.),
you probably also want to limit the fallout from privilege
escalation.

And it makes cloning + migration reasonably easy-ish.
-- 
In the beginning was the Bit, and the Bit was Zero.  Then Someone
said, Let there be One, and there was One.  And Someone blessed them,
and Someone said unto them, Be fruitful, and multiply, and replenish
the Word and subdue it: and have dominion over every thing that is.



From kximdo at gmail.com  Mon Mar  1 19:46:33 2010
From: kximdo at gmail.com (zzz kxmdo)
Date: Tue, 2 Mar 2010 02:46:33 +0800
Subject: [mpm-itk] mpm-itk@err.no
In-Reply-To: <oasocj7vrl7.fsf@luke.ifi.uio.no>
References: <e257ca8d1003010614o6c0d769ftbe33485bc753dfd1@mail.gmail.com>
	<20100301144535.GJ17219@stud.ntnu.no> <4B8BD629.5000103@lcwsoft.com>
	<e257ca8d1003010720n27836ae0kb6c11ba512521c14@mail.gmail.com>
	<oasocj7vrl7.fsf@luke.ifi.uio.no>
Message-ID: <e257ca8d1003011046wc157f8ex51dc8b58404ce839@mail.gmail.com>

>
> Run it in a VM, so that the potential damage from any security
> vulnerability is limited.
>

If you plan to allow scripting (Java, Perl, PHP, Python, Ruby, etc.),
> you probably also want to limit the fallout from privilege
> escalation.
>

We'll drop itk in FreeBSD Jails (may 100 users per Jail),  with Perl,
PHP and Python enabled, SSL disabled.  :-)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.err.no/pipermail/mpm-itk/attachments/20100302/abbaa01f/attachment.htm>

From jani+mpm-itk at ifi.uio.no  Mon Mar  1 21:59:40 2010
From: jani+mpm-itk at ifi.uio.no (Jan Ingvoldstad)
Date: Mon, 01 Mar 2010 21:59:40 +0100
Subject: [mpm-itk] mpm-itk@err.no
In-Reply-To: <e257ca8d1003011046wc157f8ex51dc8b58404ce839@mail.gmail.com> (zzz
	kxmdo's message of "Tue\, 2 Mar 2010 02\:46\:33 +0800")
References: <e257ca8d1003010614o6c0d769ftbe33485bc753dfd1@mail.gmail.com>
	<20100301144535.GJ17219@stud.ntnu.no> <4B8BD629.5000103@lcwsoft.com>
	<e257ca8d1003010720n27836ae0kb6c11ba512521c14@mail.gmail.com>
	<oasocj7vrl7.fsf@luke.ifi.uio.no>
	<e257ca8d1003011046wc157f8ex51dc8b58404ce839@mail.gmail.com>
Message-ID: <oasbpf7vjur.fsf@luke.ifi.uio.no>

On Tue, 2 Mar 2010 02:46:33 +0800, zzz kxmdo <kximdo at gmail.com> said:

> We'll drop itk in FreeBSD Jails (may 100 users per Jail),  with Perl,
> PHP and Python enabled, SSL disabled.  :-)

If your users are allowed to install and maintain their own software,
I'd say that that's trouble enough.  ;)

The usual disclaimer for MPM-ITK AFAICR is that _if_ there is an
exploitable bug in Apache, you're screwed (in that jail).

-- 
In the beginning was the Bit, and the Bit was Zero.  Then Someone
said, Let there be One, and there was One.  And Someone blessed them,
and Someone said unto them, Be fruitful, and multiply, and replenish
the Word and subdue it: and have dominion over every thing that is.



From kximdo at gmail.com  Tue Mar  2 15:12:13 2010
From: kximdo at gmail.com (zzz kxmdo)
Date: Tue, 2 Mar 2010 22:12:13 +0800
Subject: [mpm-itk] mpm-itk@err.no
In-Reply-To: <oaspr3m52yb.fsf@luke.ifi.uio.no>
References: <e257ca8d1003010614o6c0d769ftbe33485bc753dfd1@mail.gmail.com>
	<4B8BD629.5000103@lcwsoft.com>
	<e257ca8d1003010720n27836ae0kb6c11ba512521c14@mail.gmail.com>
	<oasocj7vrl7.fsf@luke.ifi.uio.no>
	<e257ca8d1003011046wc157f8ex51dc8b58404ce839@mail.gmail.com>
	<oasbpf7vjur.fsf@luke.ifi.uio.no>
	<e257ca8d1003011803x7c3b51f9jcf2a867cddff7a22@mail.gmail.com>
	<oaswrxv3z9o.fsf@luke.ifi.uio.no>
	<e257ca8d1003020156p3d09d10fr25251038235d5326@mail.gmail.com>
	<oaspr3m52yb.fsf@luke.ifi.uio.no>
Message-ID: <e257ca8d1003020612r281bc23at6a8b2d0c7ea0301@mail.gmail.com>

>
> Well, it does add a bit of overhead compared to running plain Apache
> with MPM-worker and suexec.
>

Guess, you are running ONE APACHE with suexec for 2000+ vhosts?
that seems a big challenge, is each vhost limitted strictly on sys
resources?



> If I recall correctly, our estimates for MPM-ITK landed us at around
> 100-500 vhosts per physical server, all depending on the sunspots and
> molecular harmonic distortion modulators or something.  ;)


Are you sure?

But there's a test which said mpm-itk was much better than suexec,
but a bit slower than peruser.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.err.no/pipermail/mpm-itk/attachments/20100302/28718cbe/attachment.htm>

From marzona at knoway.info  Fri Mar  5 20:01:54 2010
From: marzona at knoway.info (ml@knoway.info)
Date: Fri, 05 Mar 2010 20:01:54 +0100
Subject: [mpm-itk] I can't figure out..
Message-ID: <1267815714.29216.304.camel@diefenbaker>

Hi all

I got some weird problem with apache2-mpm-itkit's weird..

As for now, it's impossibile to startup a specified start-server
processes..

Even if I put obviously wrong values, the behaviour doesn't change (I
tried negative values, or non numerical values..) . 7 childrens.. It
ignores the configuration directives in apache.conf, but it cares about
doing suid as stated in vhost config files. What's wrong?

Running on ubuntu hardy lts (and ubuntu lucid):

[Fri Mar 05 05:26:46 2010] [notice] SIGHUP received.  Attempting to
restart
apache2: Could not reliably determine the server's fully qualified
domain name, using 127.0.1.1 for ServerName
[Fri Mar 05 05:26:46 2010] [warn] NameVirtualHost *:80 has no
VirtualHosts
[Fri Mar 05 05:26:46 2010] [notice] Apache/2.2.14 (Ubuntu) configured --
resuming normal operations
[Fri Mar 05 05:26:46 2010] [info] Server built: Jan 15 2010 04:48:52
[Fri Mar 05 05:26:46 2010] [debug] itk.c(1100): AcceptMutex: sysvsem
(default: sysvsem)

root at ubuntu:/etc/apache2# dpkg -l | grep apache
ii  apache2                               2.2.14-5ubuntu1
Apache HTTP Server metapackage
ii  apache2-mpm-itk                       2.2.14-5ubuntu1
multiuser MPM for Apache 2.2
ii  apache2-utils                         2.2.14-5ubuntu1
utility programs for webservers
ii  apache2.2-bin                         2.2.14-5ubuntu1
Apache HTTP Server common binary files
ii  apache2.2-common                      2.2.14-5ubuntu1
Apache HTTP Server common files
rc  libapache2-mod-php5                   5.2.4-2ubuntu5.6
server-side, HTML-embedded scripting languag






From joerg at alea.gnuu.de  Sun Mar  7 13:39:20 2010
From: joerg at alea.gnuu.de (=?iso-8859-1?Q?J=F6rg?= Sommer)
Date: Sun, 7 Mar 2010 13:39:20 +0100
Subject: [mpm-itk] RLimitCPU and RLimitMem support for itk?
Message-ID: <20100307123920.GA9635@alea.gnuu.de>

Hi,

if my understanding of the description of RLimitCPU and RLimitMem is
correct, these limits aren't set for Apache's child processes and the itk
patch doesn't change this. But I think it would be helpful to limit the
cpu and memory consumtion of the processes serving the web sites,
especially those using modules like PHP.

http://httpd.apache.org/docs/2.2/mod/core.html#rlimitcpu
http://httpd.apache.org/docs/2.2/mod/core.html#rlimitmem

Is it planed to add support for RLimitCPU and RLimitMem as part of the
AssignUserID step? Or is such limitation already available?

Regards, J?rg.
-- 
Der Wille und nicht die Gabe macht den Geber.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature http://en.wikipedia.org/wiki/OpenPGP
URL: <http://lists.err.no/pipermail/mpm-itk/attachments/20100307/659bfab7/attachment.sig>

From izghitu at gmail.com  Wed Mar 17 01:34:49 2010
From: izghitu at gmail.com (George)
Date: Wed, 17 Mar 2010 02:34:49 +0200
Subject: [mpm-itk] mpm-itk poor performance when using network file systems
Message-ID: <948a6d891003161734gca61dd7v4c1725ea195b803e@mail.gmail.com>

Hi,

I have CentOS 5 64bit with latest mpm-itk and httpd-2.2.3-31. I get
extremely poor performance during benchmarks using network file
systems.

Below are the anvironments and tests I performed:
1. One nginx server as load balancer. Two apache with mpm-itk servers
connected to memcached, each one connected to its own mysql server in
master/master replication and using one lustrefs server as storage.

When running 200-1000 clients test against a single big file(5MB) I
was getting very good performance.
When running 200-1000 clients test against a Joomla website with lots
of small files I was getting extremely poor performance. I could not
even pass the 200 clients test.

2. One nginx server as load balancer. Two apache with mpm-itk servers
connected to memcached, each one connected to its own mysql server in
master/master replication and using one NFS server as storage.

When running 200-1000 clients test against a single big file(5MB) I
was getting very good performance.
When running 200-1000 clients test against a Joomla website with lots
of small files I was getting extremely poor performance. I could not
even pass the 200 clients test

3. One nginx server as load balancer. Two apache with mpm-itk servers
connected to memcached, each one connected to its own mysql server in
master/master replication and using LOCAL(to each apache) file system.

I got very good performance in both single big file and many small files tests.

4. One nginx server as load balancer. Two apache servers with the
standard HTTPD running as the apache user connected to memcached, each
one connected to its own mysql server in master/master replication and
using one NFS server as storage.

I got very good performance in both single big file and many small files tests.
The same goes for local filesystem

All servers have a lot of resources(6-12 CPU cores and 4-12GB of RAM).
The external bandwidth is 100Mbps. There's no open file limit(it's
insanely high). While running the tests there's no slowness from
memcached or both mysql servers or nginx.

My question is: Why there's extremely poor performance with itk when
using a network filesystem(I tried lustrefs, nfs and glusterfs) but it
works fine with local file system? Why both local and network
filesystems work fine when NOT using the ITK(this proves that the
problem is not with the network filesystems)?

Please help



From knut at auvor.no  Wed Mar 17 08:28:17 2010
From: knut at auvor.no (Knut Auvor Grythe)
Date: Wed, 17 Mar 2010 08:28:17 +0100
Subject: [mpm-itk] mpm-itk poor performance when using network file
	systems
In-Reply-To: <948a6d891003161734gca61dd7v4c1725ea195b803e@mail.gmail.com>
References: <948a6d891003161734gca61dd7v4c1725ea195b803e@mail.gmail.com>
Message-ID: <20100317072817.GR12349@stud.ntnu.no>

On Wed, Mar 17, 2010 at 02:34:49AM +0200, George wrote:
> 4. One nginx server as load balancer. Two apache servers with the
> standard HTTPD running as the apache user connected to memcached, each
> one connected to its own mysql server in master/master replication and
> using one NFS server as storage.

What do you mean with "standard HTTPD"? MPM-ITK is based on the Prefork
mpm, so I would recommend benchmarking against that.

> My question is: Why there's extremely poor performance with itk when
> using a network filesystem(I tried lustrefs, nfs and glusterfs) but it
> works fine with local file system? Why both local and network
> filesystems work fine when NOT using the ITK(this proves that the
> problem is not with the network filesystems)?

Other large sites (ntnu.no comes to mind) are running mpm-itk with NFS
with no trouble at all. Maybe you have a suboptimal configuration. Do
you see any messages in the apache error log?

-- 
Knut Auvor



From izghitu at gmail.com  Wed Mar 17 10:18:50 2010
From: izghitu at gmail.com (George)
Date: Wed, 17 Mar 2010 11:18:50 +0200
Subject: [mpm-itk] mpm-itk poor performance when using network file
	systems
In-Reply-To: <20100317072817.GR12349@stud.ntnu.no>
References: <948a6d891003161734gca61dd7v4c1725ea195b803e@mail.gmail.com>
	<20100317072817.GR12349@stud.ntnu.no>
Message-ID: <948a6d891003170218p745538u4f84a5a3f772d583@mail.gmail.com>

Hi,

>
> What do you mean with "standard HTTPD"? MPM-ITK is based on the Prefork
> mpm, so I would recommend benchmarking against that.
>

With standard httpd I mean the default httpd without the mpm-itk. I
guess it's prefork mpm.

> Other large sites (ntnu.no comes to mind) are running mpm-itk with NFS
> with no trouble at all. Maybe you have a suboptimal configuration. Do
> you see any messages in the apache error log?

The fact that the NFS works very fast when I am not using mpm-itk
proves it's not a NFS issue. I do not see any errors in the apache
error log.



From sgunderson at bigfoot.com  Wed Mar 17 10:36:13 2010
From: sgunderson at bigfoot.com (Steinar H. Gunderson)
Date: Wed, 17 Mar 2010 10:36:13 +0100
Subject: [mpm-itk] mpm-itk poor performance when using network file
 systems
In-Reply-To: <948a6d891003161734gca61dd7v4c1725ea195b803e@mail.gmail.com>
References: <948a6d891003161734gca61dd7v4c1725ea195b803e@mail.gmail.com>
Message-ID: <20100317093613.GF22081@uio.no>

On Wed, Mar 17, 2010 at 02:34:49AM +0200, George wrote:
> My question is: Why there's extremely poor performance with itk when
> using a network filesystem(I tried lustrefs, nfs and glusterfs) but it
> works fine with local file system? Why both local and network
> filesystems work fine when NOT using the ITK(this proves that the
> problem is not with the network filesystems)?

FWIW, have you tried tcpdumping on your servers to see if there's an
increased amount of network filesystem traffic when running under mpm-itk?

/* Steinar */
-- 
Homepage: http://www.sesse.net/



From knut at auvor.no  Wed Mar 17 10:41:18 2010
From: knut at auvor.no (Knut Auvor Grythe)
Date: Wed, 17 Mar 2010 10:41:18 +0100
Subject: [mpm-itk] mpm-itk poor performance when using network file
	systems
In-Reply-To: <948a6d891003170218p745538u4f84a5a3f772d583@mail.gmail.com>
References: <948a6d891003161734gca61dd7v4c1725ea195b803e@mail.gmail.com>
	<20100317072817.GR12349@stud.ntnu.no>
	<948a6d891003170218p745538u4f84a5a3f772d583@mail.gmail.com>
Message-ID: <20100317094118.GT12349@stud.ntnu.no>

On Wed, Mar 17, 2010 at 11:18:50AM +0200, George wrote:
>> What do you mean with "standard HTTPD"? MPM-ITK is based on the Prefork
>> mpm, so I would recommend benchmarking against that.
> 
> With standard httpd I mean the default httpd without the mpm-itk. I
> guess it's prefork mpm.

I suggest you check instead of guess.

>> Other large sites (ntnu.no comes to mind) are running mpm-itk with NFS
>> with no trouble at all. Maybe you have a suboptimal configuration. Do
>> you see any messages in the apache error log?
> 
> The fact that the NFS works very fast when I am not using mpm-itk
> proves it's not a NFS issue. I do not see any errors in the apache
> error log.

I am talking about your apache configuration, not NFS.

-- 
Knut Auvor



From izghitu at gmail.com  Wed Mar 17 10:52:45 2010
From: izghitu at gmail.com (George)
Date: Wed, 17 Mar 2010 11:52:45 +0200
Subject: [mpm-itk] mpm-itk poor performance when using network file
	systems
In-Reply-To: <20100317093613.GF22081@uio.no>
References: <948a6d891003161734gca61dd7v4c1725ea195b803e@mail.gmail.com>
	<20100317093613.GF22081@uio.no>
Message-ID: <948a6d891003170252u32a79465sa716d145762d3976@mail.gmail.com>

Hi,

>
> FWIW, have you tried tcpdumping on your servers to see if there's an
> increased amount of network filesystem traffic when running under mpm-itk?

Did not try that. Thanks for the idea. I'll let you know how it goes.



From izghitu at gmail.com  Wed Mar 17 10:55:34 2010
From: izghitu at gmail.com (George)
Date: Wed, 17 Mar 2010 11:55:34 +0200
Subject: [mpm-itk] mpm-itk poor performance when using network file
	systems
In-Reply-To: <20100317094118.GT12349@stud.ntnu.no>
References: <948a6d891003161734gca61dd7v4c1725ea195b803e@mail.gmail.com>
	<20100317072817.GR12349@stud.ntnu.no>
	<948a6d891003170218p745538u4f84a5a3f772d583@mail.gmail.com>
	<20100317094118.GT12349@stud.ntnu.no>
Message-ID: <948a6d891003170255w3897df0ek5905318979ebbc31@mail.gmail.com>

Hi,


>> With standard httpd I mean the default httpd without the mpm-itk. I
>> guess it's prefork mpm.
>
> I suggest you check instead of guess.

I checked and it's prefork.

>> The fact that the NFS works very fast when I am not using mpm-itk
>> proves it's not a NFS issue. I do not see any errors in the apache
>> error log.
>
> I am talking about your apache configuration, not NFS.

The MaxClients and MaxServers settings are high. MaxClientsVHost is
high too so I am not hitting any limit that would create time outs.

Please advise



From sgunderson at bigfoot.com  Wed Mar 17 13:30:36 2010
From: sgunderson at bigfoot.com (Steinar H. Gunderson)
Date: Wed, 17 Mar 2010 13:30:36 +0100
Subject: [mpm-itk] mpm-itk poor performance when using network file
 systems
In-Reply-To: <948a6d891003170218p745538u4f84a5a3f772d583@mail.gmail.com>
References: <948a6d891003161734gca61dd7v4c1725ea195b803e@mail.gmail.com>
	<20100317072817.GR12349@stud.ntnu.no>
	<948a6d891003170218p745538u4f84a5a3f772d583@mail.gmail.com>
Message-ID: <20100317123036.GH22081@uio.no>

On Wed, Mar 17, 2010 at 11:18:50AM +0200, George wrote:
> The fact that the NFS works very fast when I am not using mpm-itk
> proves it's not a NFS issue. 

Not really; for instance, the uid switching could cause some kind of problems
for the client-side caching.

I'd guess you'll have to go in and take a look at what actually happens on
the machine. Is it largely idle? Is there lots of NFS traffic? Where do the
Apache children spend their time? Does an oprofile show anything interesting?
Etc., etc. etc... There's no easy way to nail down such problems except to
gather as much information as you can and try to make sense of the whole
picture.

/* Steinar */
-- 
Homepage: http://www.sesse.net/



From sgunderson at bigfoot.com  Thu Mar 18 12:59:00 2010
From: sgunderson at bigfoot.com (Steinar H. Gunderson)
Date: Thu, 18 Mar 2010 12:59:00 +0100
Subject: [mpm-itk] mpm-itk poor performance when using network file
 systems
In-Reply-To: <20100317123036.GH22081@uio.no>
References: <948a6d891003161734gca61dd7v4c1725ea195b803e@mail.gmail.com>
	<20100317072817.GR12349@stud.ntnu.no>
	<948a6d891003170218p745538u4f84a5a3f772d583@mail.gmail.com>
	<20100317123036.GH22081@uio.no>
Message-ID: <20100318115900.GA25372@uio.no>

On Wed, Mar 17, 2010 at 01:30:36PM +0100, Steinar H. Gunderson wrote:
> I'd guess you'll have to go in and take a look at what actually happens on
> the machine. Is it largely idle? Is there lots of NFS traffic? Where do the
> Apache children spend their time? Does an oprofile show anything interesting?
> Etc., etc. etc... There's no easy way to nail down such problems except to
> gather as much information as you can and try to make sense of the whole
> picture.

For interested parties: I was given access to the system in question, and we
more or less nailed it down to extensive use of require_once() in the PHP
scripts.

Basically, due to PHP's pretty, erm, unique of the module system,
require_once() needs to canonicalize all paths given to it, so you couldn't
include the same file twice via different paths. It does this by calling
realpath(), which by design needs to lstat() all components of the path (so
for /foo/bar/baz there's lstat() of /foo, /foo/bar and /foo/bar/baz). On
local disk this is bad enough, but on NFS, especially with the old kernel in
use (a custom kernel derived from 2.6.18, on top of Xen) this caused huge
amounts of NFS calls, and a lot of time being spent in the kernel. (I don't
know why they couldn't just stat() the file instead and check the inode
number, essentially reducing the problem to one stat per file instead of 7-8,
but I guess there's some subtleties in here I haven't thought of.)

PHP's ?realpath cache? (new from 5.1.0), which is supposed to mitigate such
issues (although in a pretty primitive and limited way) was ineffective here,
because it's not shared between Apache children, and mpm-itk effectively
starts a new Apache child for each request, blowing the cache away for each
request. 

There are many possible solutions to this, but none are simple flip-the-flag
fixes; the _right_ fix would probably be simply not using require_once (or
not using PHP...), but I guess that's not always an option. Someone who likes
to fiddle into PHP's internals could probably move the realpath cache into
shared memory, which would help many other cases but this specific one; it
shouldn't be more than a week or two for a semi-competent C programmer.

/* Steinar */
-- 
Homepage: http://www.sesse.net/



From andrew at cloudaccess.net  Thu Mar 18 16:10:22 2010
From: andrew at cloudaccess.net (Andrew Godziuk)
Date: Thu, 18 Mar 2010 16:10:22 +0100
Subject: [mpm-itk] mpm-itk poor performance when using network file
	systems
Message-ID: <c216af351003180810y1f2b9b4ax36dffab9232abeb0@mail.gmail.com>

Not using PHP is not really an option in a company devoted to hosting
Joomla. We also can't fix the code, the Joomla developers are very
uneager to accept any patches to their mess.

Thank you very much for your expertise. If getting a hardware RAID
won't help with the issue, I'll probably write the patch you
mentioned. I already looked at PHP interpreter's implementation of
realpath cache and making it shared across instances looks doable (fun
fact: it uses linear search).

-- 
Andrzej Godziuk
http://CloudAccess.net/



From gogi at gogi.tv  Thu Mar 18 16:24:40 2010
From: gogi at gogi.tv (Giuliano Gagliardi)
Date: Thu, 18 Mar 2010 16:24:40 +0100
Subject: [mpm-itk] per-vhost ErrorLog owned by root?
In-Reply-To: <20100318115900.GA25372@uio.no>
References: <948a6d891003161734gca61dd7v4c1725ea195b803e@mail.gmail.com>
	<20100317123036.GH22081@uio.no> <20100318115900.GA25372@uio.no>
Message-ID: <201003181624.40884.gogi@gogi.tv>

Hello,

is it the intended behaviour, that per-vhost ErrorLogs are owned by root?
Can I change that somehow? Is it a bad idea for some reason?



From azurit at pobox.sk  Fri Mar 19 10:17:04 2010
From: azurit at pobox.sk (azurIt)
Date: Fri, 19 Mar 2010 10:17:04 +0100
Subject: [mpm-itk] per-vhost ErrorLog owned by root?
Message-ID: <0da6471d1918449693bf64633521418f@pobox.sk>

Just chown logs if you want to change it (you can do it also via logrotate), but i don't see a reason for this.

azur


>-----P?vodn? spr?va-----
>Od: Giuliano Gagliardi [mailto:gogi at gogi.tv]
>Komu: mpm-itk at err.no
>Predmet: [mpm-itk] per-vhost ErrorLog owned by root?
>
>
>Hello,
>
>is it the intended behaviour, that per-vhost ErrorLogs are owned by root?
>Can I change that somehow? Is it a bad idea for some reason?
>
>_______________________________________________
>mpm-itk mailing list
>mpm-itk at err.no
>http://lists.err.no/mailman/listinfo/mpm-itk



From christoph at iway.ch  Fri Mar 19 12:25:09 2010
From: christoph at iway.ch (Christoph Roethlisberger)
Date: Fri, 19 Mar 2010 12:25:09 +0100
Subject: [mpm-itk] mpm-itk poor performance when using network file
	systems
Message-ID: <A6758C5DFD82457A80A08646FCFD42BC@brigitte>

Maybe an opcode cacher for PHP would minimize these problems, as they tend to look for changes in the files not by timestamp of last 
modify, but rather after a specific amount of time. So, this should reduce these NFS calls to once every 30 seconds or so (depends 
on the confuguration of the opcacher) which helps greatly in a heavy use environment.

We use xCache (http://xcache.lighttpd.net/) for this purpose and it served us very well in production environments.
The only downside of this opcachers is the development of php apps on such systems - it's just a pain in the ass to always wait for 
the cacher to reread the changes in the script, you just made...

Christoph Roethlisberger

-------------------------------------------------
 http://www.iway.ch
-------------------------------------------------