Several months ago I ordered a root server by the German hosting provider Hetzner called EQ4. It is quite powerful: an Intel Core i7-920 Quad-Core CPU, 8 GB RAM and two 750 GB HDDs for only 45,- €/month. Since they only provide several Linux flavors (openSuSE, Fedora, CentOS) at first glance I decided to use CentOS. I already some very good experiences with it a couple of years ago. The installation process was very easy.
After a couple of months without much time to fiddle with the server it just sat there in its rack and got bored.
After the very inspiring NoSQL meeting in Berlin last Thrusday I decided to spend some time with my server installing Erlang, CouchDB and nginx as a reverse proxy to do authentication and SSL stuff.
Installing the software packages went very well. Some of them I grabbed via yam others I installed from source. Connecting to my system via a ssh session worked very well but there was a very strange iptables setup in the CentOS installation which drove me crazy. I could not reach the proxy from outside and after several hours I decided to try a reinstall. At Hetzner one can reboot the server in a so called rescue mode. This mode can be of course Linux, but also FreeBSD and OpenSolaris. Digging a little further I discovered a site in the Hetzner wiki describing how to install OpenSolaris through this rescue system.
I used JollyFastVNC to establish a VNC session to the rescue system and used the graphical OpenSolaris installer to install it on the first HDD. After installation I used my directions from an earlier post to create a ZFS mirror using both HDDs.
This is my hardware configuration discovered by OpenSolaris:
# prtdiag -v System Configuration: MSI MS-7522 BIOS Configuration: American Megatrends Inc. V8.2 04/20/2009 ==== Processor Sockets ==================================== Version Location Tag -------------------------------- -------------------------- Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz CPU 1 ==== Memory Device Sockets ================================ Type Status Set Device Locator Bank Locator ----------- ------ --- ------------------- ---------------- other in use 0 DIMM0 BANK0 other in use 0 DIMM1 BANK1 other in use 0 DIMM2 BANK2 other empty 0 DIMM3 BANK3 other in use 0 DIMM4 BANK4 other empty 0 DIMM5 BANK5 FLASH in use 0 ==== On-Board Devices ===================================== ==== Upgradeable Slots ==================================== ID Status Type Description --- --------- ---------------- ---------------------------- 1 available PCI PCI1 2 available PCI Express PCIE2 3 available PCI Express PCIE3 4 available PCI Express PCIE4
Next I used the CouchDB directions in the Joyent Wiki to install the entire required software stack from source. After some fiddling with directory write permissions I had my CouchDB system up and running.
To install nginx I used the official site. I wanted to have a password authentication on my site. Since nginx doesn’t come with htpasswd I used it on my Mac:
$ htpasswd -nbd user password user:TYVlO9aeSogv6
I copied the output line into the file /etc/nginx/htpasswd on my server.
To create a self signed certificate in the folder /etc/nginx I used the following commands:
# openssl req -new -nodes -keyout selfsigned.key -out selfsigned.csr Generating a 1024 bit RSA private key ............................................................................................................................++++++ ........................++++++ writing new private key to 'selfsigned.key' ... # openssl x509 -req -days 1095 -in selfsigned.csr -signkey selfsigned.key -out selfsigned.crt Signature ok ... Getting Private key
My nginx setup file contents are:
#/etc/nginx/nginx.conf
#user nobody;
worker_processes 2;
#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;
#pid logs/nginx.pid;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
#log_format main '$remote_addr - $remote_user [$time_local] $request '
# '"$status" $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';
#access_log logs/access.log main;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 65;
#gzip on;
server {
listen 80;
server_name localhost;
#charset koi8-r;
#access_log logs/host.access.log main;
location / {
auth_basic "Restricted";
auth_basic_user_file /etc/nginx/htpasswd;
rewrite /couchdb/(.*) /$1 break;
proxy_pass http://localhost:5984;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
#error_page 404 /404.html;
# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
#
# HTTPS server
#
server {
listen 443;
server_name localhost;
ssl on;
ssl_certificate /etc/nginx/selfsigned.crt;
ssl_certificate_key /etc/nginx/selfsigned.key;
ssl_session_timeout 5m;
ssl_protocols SSLv2 SSLv3 TLSv1;
ssl_ciphers ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP;
ssl_prefer_server_ciphers on;
location / {
auth_basic "Restricted";
auth_basic_user_file /etc/nginx/htpasswd;
rewrite /couchdb/(.*) /$1 break;
proxy_pass http://localhost:5984;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
}
Now when I open http://my.secret.server.ipaddress/ I can log in with the created user credentials stored in htpasswd and get the warm CouchDB welcome message: ‘{“couchdb”:”Welcome”,”version”:”0.10.0″}’. I can also use the secure entry at https://my.secret.server.ipaddress/.
After every successful step I made a ZFS snapshot which is the greatest feature I can use now. By the way: a nice ZFS cheat sheet can be found here.
I don’t know why it worked so well with OpenSolaris and I had so many problems with CentOS. Maybe my system is now wide open and completely insecure, but this way I like it much better because now I can close all the open doors step by step and make it more secure.
Next I will move my domain also to Hetzner and let it point to my server. Then I will setup a mail server, maybe install some Ruby on Rails stuff (http://www.redmine.org/) and will write an Adobe Flex application for a customer which will rely completely on CouchDB #bliss.
. The script should not run at the same time and produce a decent logfile.
utput,:error_output,:cmd,:exec_time
def initialize(cmd,cmd_id)
@output = ""
@error_output = ""
@exec_time = 0
@cmd = cmd
@cmd_id = cmd_id
end
def run
start_time = Time.now
begin
$LOG.info "[#{@cmd_id}] Starting command: #{@cmd}..."
Open3.popen3(@cmd) do |stdin, stdout, stderr|
@output = stdout.read
@error_output = stderr.read
end
rescue Exception => e
@error_output += e.to_s
ensure
@exec_time = Time.now - start_time
$LOG.info "[#{@cmd_id}] Command completed in #{@exec_time} seconds."
end
end
# Log the stdio and stderr outputs
def log_results
$LOG.info "[#{@cmd_id}] #{@cmd}:"
if @error_output.length > 0
@error_output.split("\n").each { |line| $LOG.error "[#{@cmd_id}] #{line}" }
end
if @output.length > 0
@output.split("\n").each { |line| $LOG.info "[#{@cmd_id}] #{line}" }
end
end
# Returns false if the command hasn't been executed yet
def run?
return @exec_time > 0
end
# Returns true if the command was successful.
def success?
return @error_output.length == 0
end
end
# Define for each host which user accounts are being backed up and which files should be excluded
default_excludes = ['.Trash', 'Downloads', 'Desktop', 'Music/iTunes/iTunes Music/Podcasts',
'Library/Caches', 'Library/Logs']
# format: hostname => { username => [excluded_files] }
HOSTS={ 'host1' => { 'user1' => default_excludes },
'host2' => { 'user2' => default_excludes, 'user1' => default_excludes },
'host3' => { 'user2' => default_excludes + ['Music/iTunes']},
'host4' => { 'user3' => default_excludes }
}
$LOG.info "****** Backup started... *******"
# Make a ZFS snapshot
snapshot_name = "#{ZFS_POOL}@backup-#{Time.now.strftime('%y-%m-%d_%H:%M')}"
$LOG.info "Creating ZFS snapshot #{snapshot_name}"
`zfs snapshot #{snapshot_name}`
pending_commands = {}
HOSTS.each do |hostname,user_data|
$LOG.info "Calling #{hostname} ..."
if Ping.pingecho(hostname)
user_data.each do |user,excluded_files|
exclude_file = Tempfile.new("tempfile")
excluded_files.each { |filepath| exclude_file << filepath << "\n" }
exclude_file.close
user_hostname = "#{user}@#{hostname}"
$LOG.info "Backing up #{user_hostname} ..."
local_backup_path = "#{LOCAL_BACKUP_PATH}/#{hostname}/#{user}"
FileUtils.mkdir(local_backup_path) unless File.exists? local_backup_path
command = "rsync -#{DRY_RUN ? 'n' : ''}avz --delete --partial --exclude-from=#{exclude_file.path} #{user_hostname}: #{local_backup_path}/"
rsync = ExecCmd.new(command, user_hostname)
pending_commands[user_hostname] = rsync
Thread.new do
rsync.run
end
end
else
$LOG.warn "#{hostname} does not respond!"
end
end
# Wait for the backup processes to complete
while pending_commands.length > 0
pending_commands.each do |user_hostname, exec_cmd|
if exec_cmd.run?
exec_cmd.log_results
pending_commands.delete(user_hostname)
end
end
if pending_commands.length > 0
$LOG.info "Still #{pending_commands.length} tasks backing up #{pending_commands.keys.join(', ')}"
sleep 60
end
end
$LOG.info "****** Backup complete. *******\n"
to fullfil my long lasting wish to setup my own fileserver running Solaris and ZFS. Since I wanted to use it also as a potential test server for my projects I decided to use a slightly bigger processor. I ordered a BTO system at www.alternate.de with the following configuration:
