Thursday, December 13, 2012

LVS+PGBouncer+RubyRep

We gonna build 3 server with different services. LVS (for load balancing), PGBouncer (connection pooling) and RubyRep (master-master replication).

Here's our simple topology,






And now, here steps



PostgreSQL 9.0

1. Install postgresql 9.0
2. Edit /etc/postgresql/9.0/main/postgresql.conf,

listen_addresses = '*'

3. Edit /etc/postgresql/9.0/main/pg_hba.conf,

host    all             all             127.0.0.1/32              trust
host    all             all             10.211.55.0/24          trust 

4. Now your postgresql accept every connection from 10.211.55.0/24 network.


RubyRep

1. You can install rubyrep on your all DB servers. 
2. Login with postgres user.
3. Please download ruby rep from, http://rubyforge.org/frs/?group_id=7932
4  As a root user please install ruby, rubygems and jruby
5. Still with root user, please install ruby-pg with gem,

     gem install ruby-pg
6. Unzip ruby-1.x.x.zip file to postgres user
7. You can using this configuration template file,

RR::Initializer::run do |config|
  config.left = {
    :adapter  => 'postgresql',
    :database => 'dodol',
    :username => 'postgres',
    :password => '',
    :host     => '10.211.55.12'
  }

  config.right = {
    :adapter  => 'postgresql',
    :database => 'dodol',
    :username => 'postgres',
    :password => '',
    :host     => '10.211.55.13'
  }
  config.options[:sync_conflict_handling] = :right_wins
  config.options[:replication_conflict_handling] = :right_wins
  config.add_table_options 'emp',
  :sync_conflict_handling => :left_wins,
  :replication_conflict_handling => :left_wins
  config.include_tables /./
end

8. Test you configuration with,

./rubyrep/bin/rubyrep scan -c your-rubyrep.conf 

9. If success, now you can run rubyrep with,

./rubyrep/bin/rubyrep replicate -c your-rubyrep.conf 

10. Now you already master-master replication postgresql server.
11. For auto running after you restarting the server, please put following line to /etc/rc.local file,

/usr/bin/ruby /var/lib/postgresql/rubyrep/bin/rubyrep replicate -c /var/lib/postgresql/your-rubyrep.conf &



PGBouncer

1. Install with,

apt-get install pgbouncer

2. Here's my /etc/pgbouncer/pgbouncer.ini file,

[databases]

* = host=10.211.55.12 port=5432   ;;; this is your DB1 server
* = host=10.211.55.13 port=5432   ;;;this is your DB2 server

foodb =
bardb = 
forcedb = 
nondefaultdb = 

[pgbouncer]
logfile = /var/log/postgresql/pgbouncer.log
pidfile = /var/run/postgresql/pgbouncer.pid
listen_addr = * 
listen_port = 3170
unix_socket_dir = /var/run/postgresql
auth_type = trust
auth_file = /etc/pgbouncer/userlist.txt
admin_users = postgres
stats_users = postgres
pool_mode = transaction
server_reset_query = DISCARD ALL;
ignore_startup_parameters = application_name
server_check_query = select 1
server_check_delay = 10
max_client_conn = 1000
default_pool_size = 20
log_connections = 1
log_disconnections = 1
log_pooler_errors = 1
server_round_robin = 1
server_lifetime = 1200


3. Here's my /etc/default/pgbouncer file,

# Set to 1 after you've configured pgbouncer in
# /etc/pgbouncer

START=1

# It is possible to change the options the daemon is started with.
# The default setting is shown below, uncomment and apply your changes
# if you need to modify it.

#OPTS="-d /etc/pgbouncer/pgbouncer.ini"

4. Start your pgbouncer

/etc/init.d/pgbouncer start

5. Done


LVS Server

1. Install ipvsadm. If you using ubuntu/debian please edit /etc/ipvsadm.rules file,

-A -t 10.211.55.100:3170 -s wlc -p 600
-a -t 10.211.55.100:3170 -r 10.211.55.10:3170 -m -w 100
-a -t 10.211.55.100:3170 -r 10.211.55.11:3170 -m -w 100

As we can see from above, 10.211.55.100 its our virtual IP and we use port 3170. For real server we put 10.211.55.10 and 10.211.55.11 and port 3170

Please make sure that we using PGBouncer port (3170).

2. Start the ipvsadm services.
3. Now you can test connect to your DB server,

psql -U postgres -h 10.211.55.100 -p 3170 

Make sure you using virtual ip to connect to your DB server.

No comments:

Post a Comment