[gentoo-user] OT: iptables w/ 2 web servers

[gentoo-user] OT: iptables w/ 2 web servers

[James]

Hello,


I have one static  IP with DNS (primary and secondary)
performed by my isp. I'm setting up a second web server
with a different domain name. It is setup already by the ISP
for DNS. Could someone post
some simple iptable examples of how to route 2 different
web server traffic streams to 2 different machines?

Both are inside the same DMZ....2 different machines
with different (NAT) IP addresses.


Right now, all port 80 traffic is auto forwarded to
a single NAT address on the firewall. Simple. Now I have 
to figure out how to  forward different web traffic streams 
to 2 different NAT ip addresses, each on a different ip 
address and a different machine.


I do not want to put the sites on the same machine, for a variety
of reasons, beside one machine moves in a few months to a 
different physical location (and network numbers).


Suggestions or a good book for example would be keen.
I use raw IPtables/netfilter on the firewall. All servers
are gentoo.


James

Re: [gentoo-user] OT: iptables w/ 2 web servers

[kashani]

James wrote:
> Hello,
> 
> I have one static  IP with DNS (primary and secondary)
> performed by my isp. I'm setting up a second web server
> with a different domain name. It is setup already by the ISP
> for DNS. Could someone post
> some simple iptable examples of how to route 2 different
> web server traffic streams to 2 different machines?
> 
> Both are inside the same DMZ....2 different machines
> with different (NAT) IP addresses.
> 
> Right now, all port 80 traffic is auto forwarded to
> a single NAT address on the firewall. Simple. Now I have 
> to figure out how to  forward different web traffic streams 
> to 2 different NAT ip addresses, each on a different ip 
> address and a different machine.
> 
> I do not want to put the sites on the same machine, for a variety
> of reasons, beside one machine moves in a few months to a 
> different physical location (and network numbers).
>  
> Suggestions or a good book for example would be keen.
> I use raw IPtables/netfilter on the firewall. All servers
> are gentoo.

	I'm not sure it's possible via firewall rules because they are 
operating at the IP level and you'd really need to be doing deep looks 
into the packets to read the http request headers in order to figure out 
which server should be getting the connection.
	The simplest solution is to run a reverse proxy on your firewall that 
actually accepts the http connection, reads the http request, and then 
forwards it on to the correct web server. You can do this in apache via 
proxypass, Squid which is your most powerful and flexible option, ngnix, 
lighttpd, or Varnish.
	There are some security concerns with this type of setup, ie running 
daemons open to the public on your firewall, reverse proxies need to be 
locked down, hard to do IP based restrictions on the webserver, etc.

kashani

Re: [gentoo-user] OT: iptables w/ 2 web servers

[Stroller]

On 23 Sep 2009, at 22:35, James wrote:
> ...
> I have one static IP
> ... Could someone post
> some simple iptable examples of how to route 2 different
> web server traffic streams to 2 different machines?
>
> Both are inside the same DMZ....2 different machines
> with different (NAT) IP addresses.

Can't be done.

Inherently, a client requesting a webpage looks up the IP address for  
bigbreastedmommas.com, finds its 24.73.161.102 (or whatever) and send  
an http request to port 80 of that IP.

There is no way for IPtables to distinguish between an http request to  
bigbreastedmommas.com at 24.73.161.102 and an http request to  
bouncyboobs.com at 24.73.161.102, assuming both are on port 80.

I would LOVE to be proved wrong on this, because it would be immensely  
useful for NATted geeks. However an iptables module to do this  
certainly did not exist in the past, and I'm guessing there are  
protocol-specific reasons why it can't be done.

What you can do is forward port 80 to one machine and have that proxy  
to the other when necessary. Or have it redirect to the other, which  
listens on port 81.

Stroller.

[gentoo-user] Re: OT: iptables w/ 2 web servers

[James]

Stroller <stroller <at> stellar.eclipse.co.uk> writes:


> > I have one static IP
> > ... Could someone post
> > some simple iptable examples of how to route 2 different
> > web server traffic streams to 2 different machines?

> > Both are inside the same DMZ....2 different machines
> > with different (NAT) IP addresses.

> Can't be done.

Ok, that explains why I drew a blank on how to proceed.





> There is no way for IPtables to distinguish between an http request to  
> bigbreastedmommas.com at 24.73.161.102 and an http request to  
> bouncyboobs.com at 24.73.161.102, assuming both are on port 80.


So the best I can do is forward all traffic( 80, 443, etc) for the
group of websites to a proxy behind the firewall, then use software
such as what kashani suggested (proxypass, Squid, ngnix, 
lighttpd, or Varnish) and parse the traffic with some form of 
vhosts implementation on a single server (nated IP)?

I definately do not want to run anything additional on the firewall,
unless it is absolutely secure and then it would have to have an
light loading of firewall resources.

Then if the load of the combined virtual hostings becomes too large,
I use a group (cluster) of servers that and implement some sort of load
balancing across the machines that each contain complete copies of each website?

Then there is the question of how to keep the individual machines
'in sync' and the limitation that once a machine is saturated (performance
suffers too much due to insufficient  resources) there 
is no solution for expansion?

One last thing. I can get a small subnet of say 5 IP address from my
ISP for an additional 20/month. That that help me? I want to put up
dozens of small charitable web sites. None will have a huge user base,
but I was going to stream some limited video from each of them.


Any other architectual solutions here?  (outside hosting is not an option).
My ISP is very cool, and will even let me run my own primary and secondary
name service, if that helps redirect the traffic?


Ideas?



James

Re: [gentoo-user] Re: OT: iptables w/ 2 web servers

[kashani]

James wrote:
> So the best I can do is forward all traffic( 80, 443, etc) for the
> group of websites to a proxy behind the firewall, then use software
> such as what kashani suggested (proxypass, Squid, ngnix, 
> lighttpd, or Varnish) and parse the traffic with some form of 
> vhosts implementation on a single server (nated IP)?

That's not quite correct.

Let's assume you don't install anything on the firewall. Instead you'll 
forward port 80 to a single server internally on port 4080 which you've 
set Squid, Varnish, Ngnix, or Lighttpd to listen on.

internet -> firewall:80 -> server1:4080

Your proxy accepts the connection and then looks at its config or in 
most case the proxy is smart enough to use DNS to go to the server it 
needs. Using DNS might be an issue in your case since the IPs will 
resolve to the single public IP.

site1 -> server1:80
site2 -> server2:80
site3 -> server3:3128
site4 -> server1:80
site5 -> server123.dreamhost.com:80
site6 -> localhost:80
site6/newapp -> server7:80
site6/newapp1 -> server8:80
and so on. You can really do just about anything here.

All connections are going to come through your proxy, but the serving of 
the pages will be done by the web servers. I would not worry about the 
number of connections to your proxy, all the proxy solutions list above 
about are capable of handling a few thousand connections.

Here's the link to the Apache proxy module. It should give you some 
ideas on what you can do. I recommend using some other proxy software 
than Apache just to simplify the setup and make it easier to hold the 
system in your head. Also prefork Apache is the slowest and uses the 
most resources of your options which is another reason to use a seperate 
proxy.
http://httpd.apache.org/docs/2.2/mod/mod_proxy.html

kashani

Re: [gentoo-user] Re: OT: iptables w/ 2 web servers

[Stroller]

On 24 Sep 2009, at 16:30, James wrote:
> ...
> So the best I can do is forward all traffic( 80, 443, etc) for the
> group of websites to a proxy behind the firewall, then use software
> such as what kashani suggested (proxypass, Squid, ngnix,
> lighttpd, or Varnish) and parse the traffic with some form of
> vhosts implementation on a single server (nated IP)?

I think you can simply forward to server A. If the site is on server A  
then it's served, if it's on server B then in the vhosts for that site  
on server A you can proxy for server B. Of course if server A goes  
down then you're stuffed.

> Then if the load of the combined virtual hostings becomes too large,
> I use a group (cluster) of servers that and implement some sort of  
> load
> balancing across the machines that each contain complete copies of  
> each website?
>
> Then there is the question of how to keep the individual machines
> 'in sync' and the limitation that once a machine is saturated  
> (performance
> suffers too much due to insufficient  resources) there
> is no solution for expansion?

This surely exceeds what you'll be hosting on a NATted home connection?

> One last thing. I can get a small subnet of say 5 IP address from my
> ISP for an additional 20/month. That that help me? I want to put up
> dozens of small charitable web sites. None will have a huge user base,
> but I was going to stream some limited video from each of them.

Yes, this certainly overcomes the original problem. You have a  
separate IP for each server and the DNS for each site directs  
appropriately.

Not all routers support this configuration and, 5 years ago, I found  
it a little cumbersome to set it up in Linux (it's called "bridging").  
No doubt the situation has improved a lot since then.

Stroller.

Re: [gentoo-user] Re: OT: iptables w/ 2 web servers

[Etaoin Shrdlu]

On Thursday 24 September 2009 16:30:51 James wrote:

> One last thing. I can get a small subnet of say 5 IP address from my
> ISP for an additional 20/month. That that help me? 

Possibly. If you manage to get two public IPs, each website using one, you can 
then DNAT requests arriving at the first IP to the first website, and requests 
arriving at the second IP to the second website.