Howto use Squid3 in front of Zope with ESI

Get Squid3 and Zope

By writing this Squid3 did not reach final. I used PRE3 for this. Download Squid3 from http://www.squid-cache.org/Versions/v3/3.0/.

Also Zope 2.7 didnt reach final yet. But it runs quite fine. Get the latest release candidate from http://zope.org.

Configure Zope

Install Zope as usual and create at least one instance.

Squid can round robin multiple backend ZEO instances easily. You need at least one. Any instance needs http and icp listener enabled.

For Zope 2.7 this looks like this in $INSTANCE_HOME/etc/zope.conf:

<http-server>
  # valid keys are "address" and "force-connection-close"
  address 8081
  # force-connection-close on
</http-server>

<icp-server>
  # valid key is "address"
  address 3131
</icp-server>

This opens the http port 8081 and icp port 3131, which are both needed in squid.conf again.

Compile Squid3

extract squid tarball, configure and compile it

>>> ./configure --prefix=/usr/local/squid3 --enable-gnuregex \ 
         --with-pthreads --enable-esi --enable-storeio=ufs,aufs \
         --with-aufs-threads=10 --enable-useragent-log \
         --enable-referer-log -enable-ssl --enable-x-accelerator-vary \
         --with-dl
>>> make

become root

>>> make install

This compiles Squid3 with the required functionality.

Configure Squid3

Edit the Squid3 configuration file etc/squid.conf inside the squid installation dir (/usr/local/squid3 in the example above).

The following configuration parts require changing the default configuration:

http_port 80 vhost
#hierarchy_stoplist cgi-bin ?

cache_mem 256 MB
maximum_object_size 4096 KB
cache_replacement_policy lru
memory_replacement_policy lru

cache_dir ufs /usr/local/squid/var/cache 1024 16 256

logformat combined %>a %ui %un [%tl] "%rm %ru HTTP/%rv" %Hs %<st "%{Referer}>h" "%{User-Agent}>h"

access_log /usr/local/squid3/logs/access.log combined

redirect_program /usr/local/squid3/iredir/iRedirector.py
redirect_children 90
redirect_rewrites_host_header off

client_lifetime 1 hour

acl in_backendpool dstdomain backendpool

cache_peer 127.0.0.1 parent 8081 3131 no-digest no-netdb-exchange round-robin
#cache_peer 127.0.0.1 parent 8082 3132 no-digest no-netdb-exchange round-robin
#cache_peer 127.0.0.1 parent 8083 3133 no-digest no-netdb-exchange round-robin
                                                           
cache_peer_access 127.0.0.1 allow in_backendpool
cache_peer_access 127.0.0.1 deny all

http_access allow all

cache_mgr you@yourdomain.com

visible_hostname www.yourdomain.com

# these were for squid2
#httpd_accel_host virtual
#httpd_accel_port 0
#httpd_accel_with_proxy on
#httpd_accel_uses_host_header on

never_direct allow all

ie_refresh on

To easily keep the squid.conf structure search for each of the configuration option names (all of them are already inside the default configuration but not active).

Please note the cache_peer lines which each of them configures one backend ZEO client with http port (eg. 8081) and icp port (eg. 3131). ZEO clients can exist on other machines as well. Just change the ip to match them. Dont forget to allow other ips with the cache_peer_access options as well.

Squid3 redirector (iredir)

Get a Squid redirector to rewrite the urls to the backendpool. I recommend my own iredir. To get iredir look at the iredir homepage. You can use any redirector you like but don't forget to adapt the redirect_program configuration to your redirector if you choose another.

Edge Side Includes (ESI)

Squid3 supports ESI (see http://www.esi.org/overview.html if you dont know ESI). We compiled Squid3 with ESI already and its activated per default with upper configuration.

Put the following templates and scripts inside a Zope folder:

set_esi_header (Script (Python)):

request = container.REQUEST
RESPONSE =  request.RESPONSE
RESPONSE.setHeader('Surrogate-Control','max-age=30+60,content="ESI/1.0"')

test1 (PageTemplate):

<table tal:define="dummy here/set_esi_header" > 
<tr>
<td colspan="2">
<esi:try>
 <esi:attempt>
   <esi:include
      src="top.html"
      tal:attributes="src string:${here/absolute_url}/top.html"
      onerror="continue" />
 </esi:attempt>

 <esi:except>
   <!--esi
     This spot is reserved for something 
   -->
 </esi:except>
</esi:try>
</td></tr>
<tr><td>esi test1</td></tr>
</table>

top.html (PageTemplate):

<h1>Included by ESI</h1>

Thats it. Now point a browser to the test1 PageTemplate (through squid of course) and see ESI including the top.html file. Please note that you need to call the set_esi_header script in any PageTemplate which makes use of ESI markup. Without these response headers ESI is just ignored.