about.me/v1nc3ntlaw

Sheng-Zhe Lin 林聖哲

Rails developer and linux system administrator at Techbang.

Powered by Rails

http://www.techbang.com.tw
http://digiphoto.techbang.com.tw
http://game.techbang.com.tw
http://t17.techbang.com.tw

Rails Production Server

https://github.com/jnstq/rails-nginx-passenger-ubuntu
http://www.rubyenterpriseedition.com/

Rails and Server Monitoring

New Relic

Monitor, Troubleshoot and Tune Your Production Application

http://newrelic.com/

Scout

Monitoring Server Cluster

https://scoutapp.com/

Open Source Solution

Nagios + Cacti

Nagios

services current status, unusual situation or crash alert

http://www.nagios.org/

Cacti

devices data graphing solution

http://www.cacti.net/

God

Process Monitoring Framework in Ruby

http://rubygems.org/gems/god

Nginx and Passenger Tuning

Break Limits, Unleashed Power

/opt/nginx/conf/nginx.conf

worker_processes 16;        # cat /proc/cpuinfo
worker_rlimit_nofile 32768;

events {                   # max_clients =
  worker_connections 8192; # worker_processes * worker_connections
  use epoll;
}

/etc/security/limits.conf

rails soft nofile 32768
rails hard nofile 32768

Passenger Configuration

$ passenger-memory-stats
----- Passenger processes ------
PID    VMSize    Private   Name
--------------------------------
3054   365.8 MB  150.2 MB  Rack: /home/rails/project/current
passenger_max_pool_size 20; # Private 150.2 * 20 = 3624 MB
http://blog.scoutapp.com/articles/2009/12/08/production-rails-tuning-with-passenger-passengermaxprocesses

Nginx upload module with Paperclip

moved hard work from Ruby to C

http://matthewhutchinson.net/2010/1/6/nginx-upload-module-with-paperclip-on-rails

Load Balancing on Rails

HAProxy F5 BIG-IP

Rails Asset Cache

Problem with Rails Asset Cache

Preheat Rails Asset Cache: Jammit

An industrial strength asset packaging library for Rails

task :preheat_assets, :roles => [:web] do
  run "cd #{release_path} && bundle exec jammit --base-url http://www.techbang.com.tw"
end

# before symblo link to current and touch tmp/restart.txt
before "deploy:symlink", "my_tasks:preheat_assets"
http://documentcloud.github.com/jammit/

Memcached Distribute and Failover

config/enviroments/production.rb

MEMCACHE_SERVERS = ["192.168.100.3:11211", "192.168.100.4:11211"]
config.cache_store = :mem_cache_store, MEMCACHE_SERVERS, {
  :namespace => "techbang"
}

Rails Backup Solution

Whenever + Backup Gems

Whenever

Cron jobs in Ruby

Linux Crontab Format
# min hour dom mon dow command

Whenever DSL
every :hour every 3.hours every 1.hour, :at => 30
every 1.day, :at => '4:30 am' every '0 0 27-31 * *'

Whenever

job_type: command, rake task

# whenever config/schedule.rb
every 1.day, :at => "23:45" do
  command "echo 'Say Hello!'"
  rake "mail:notification"
end
# generated crontab
45 23 * * * /bin/bash -l -c 'echo '\''Say Hello!'\'''
45 23 * * * /bin/bash -l -c 'cd project && \
RAILS_ENV=production bundle exec rake mail:notification'

Whenever

Capistrano integration, update crontab with deploy

# config/deploy.rb
set :whenever_command, "bundle exec whenever"
require "whenever/capistrano"
http://rubygems.org/gems/whenever

Backup (RubyGem)

Backup (RubyGem)

backup perform -t local_backup

Backup::Model.new(:local_backup, "Backup!") do
  backup_path = "/mnt/backup_copy/project"
  backup_list = []
  backup_list << "/home/rails/project/shared/system"
  backup_list << "/home/rails/project/shared/config"

  if File.directory? backup_path
    system("rsync -ravh --delete #{backup_list.join(" ")} #{backup_path}")
  end
end

Backup has a great wiki, explains each component in detail with many examples.

http://rubygems.org/gems/backup

Backup Target

project/
├── current -> /home/rails/project/releases/20110819032217
├── releases
│   ├── 20110728073852
│   ├── ...
│   └── 20110819032217
└── shared
    ├── bundle
    ├── cached-copy
    ├── config => current/config/database.yml
    ├── log
    ├── pids
    └── system => current/public/system

Configuration Stay with Project

Whenever config/schedule.rb
Backup config/backup.rb

backup run at one main server only

# config/schedule.rb
host = `hostname`.chomp
if host == "main_server"
  every 6.hours do
    command "cd /home/rails/project/current && \
    AWS_CALLING_FORMAT=SUBDOMAIN backup perform -t local_backup \
    --config_file 'config/backup.rb'"
  end
end

Q & A

Thanks!