How to setup an ultra-fast CDN using Nginx and Varnish

If your website takes more than 3 seconds to load, there is a big chance to loose your site visitors and that is a risk we are dealing with today due to the big competition in the industry.

There are lots of reasons which effects on the site’s speed

• Large multimedia contents
• DNS look up issue
• Server response time
• Multiple redirects

are few from a lengthy list.

If you are using a good CDNs to serve contents, then that will give a good boost while loading the site. Last month I had to setup up Our own CDN for WSO2 Oxygentank Developer Portal and i’m trying to summaries steps that I followed.

###Installing Nginx and varnish

    apt-get install nginx
    apt-get install varnish

###Nginx Config

• /etc/nginx/sites-enabled/default

    server {
           open_file_cache_valid 20m;
            listen 81;
            server_name mydomain.com;
            access_log   /var/log/cdn/mydomain.com/logs/access.log;
            root   /cdn_document_path/;

            location ~* \.(gif|jpg|jpeg|png|wmv|avi|mpg|mpeg|mp4|htm|html|js|css|mp3|swf|ico|flv)$ {
                            open_file_cache_valid 120m;
                            expires 7d;
                            open_file_cache max=1000 inactive=20s
            }

             location = /50x.html {
                            root   /var/www/nginx-default;
            }

            # No access to .htaccess files.
            location ~ /\.ht {
                deny  all;
            }
    }

###Varnish Config

• /etc/default/varnish

    START=yes
    NFILES=131072
    MEMLOCK=82000
    INSTANCE=$(uname -n)
    DAEMON_OPTS="-a :80 \
                 -T localhost:6082 \
                 -f /etc/varnish/default.vcl \
                 -S /etc/varnish/secret \
                 -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,1G"

• /etc/varnish/default.vcl

    backend default {
        .host = "127.0.0.1";
        .port = "81";
    }

    sub vcl_recv {
     if (req.url ~ “\.(js|css|jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf)$”) {
      return (lookup);
     }
    }

    sub vcl_fetch {
     if (req.url ~ “\.(js|css|jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf)$”) {
      unset obj.http.set-cookie;
     }
    }

###Start Nginx and Varnish

    /etc/init.d/nginx start
    /etc/init.d/varnish restart

Once you start Nginx and Varnish. Varnish will start to server port 80 web traffic. If it is a static content varnish will server and Nginx will look after all dynamic contents.

How to Setup smstools in debian

3G dongle is the only bandwidth service I have when i’m home. (If my neighbor didn’t turn on his WIFI connection ). Due to heavy usage of mobile broadband, my 3g Dongle frequently get caught to service disruptions by exceeding the reserved data bundle, Most of the time I had to call 3G service provider to get the connection temporary to do an online payment.

Normally service provider send a SMS when reaching the reserved quota (Probably when reaching 75%). But as I can’t use Mobile Partner software in linux, I always missed those SMSs, if I didn’t connect my dongle in to a Windows powered machine (As I hardly use Windows OS, most of the time I was in trouble).

As a solution I installed and configured smstools to avoid this mess and getting alone without the Internet.

###Install smstools

Install the package

    sudo apt-get install smstools

Find the device. ex: /dev/ttyUSB0

    dmesg | grep usb

###Configuration (/etc/smsd.conf)

Change the smstools config file according to your service provider

    sudo vim /etc/smsd.conf

    devices = GSM1
    outgoing = /var/spool/sms/outgoing
    checked = /var/spool/sms/checked
    incoming = /var/spool/sms/incoming
    logfile = /var/log/smstools/smsd.log
    infofile = /var/run/smstools/smsd.working
    pidfile = /var/run/smstools/smsd.pid
    outgoing = /var/spool/sms/outgoing
    checked = /var/spool/sms/checked
    failed = /var/spool/sms/failed
    incoming = /var/spool/sms/incoming
    sent = /var/spool/sms/sent
    receive_before_send = no
    autosplit = 3

    [GSM1]
    device = /dev/ttyUSB0
    incoming = yes
    baudrate = 19200
    memory_start = 1

Start smstools after changing configurations

    sudo /etc/init.d/smstools start ()

To check incoming sms

    cd  /var/spool/sms/incoming

If you configured the smstools properly you will get sms to /var/spool/sms/incoming

###Test your settings

Send a test sms to your 3G Dongle from a mobile phone, then

    ls /var/spool/sms/incoming

If the dongle received the sms, you will see a file, name is slimier to GSM1.AuvV6s. To read the sms

    vim  /var/spool/sms/incoming/$filename

###Debug your setings

    sudo tail -f /var/log/smstools/smsd.log

Note: I didn’t test sms sending through the 3G dongle as my service provider has blocked the facility.