How to rule remote shell sessions with tmux and mosh

If you're like most ssh users when your connection breaks it's bad news for you. Not only do you have to reconnect but your session gets destroyed and you have to make all the moves to restore the previous state. This doesn't have to be that way. I'd like to say some words about two tools that solve these problems in the most elegant way possible.

tmux

tmux is a terminal multiplexer: it enables a number of terminals to be created, accessed, and controlled from a single screen. tmux may be detached from a screen and continue running in the background, then later reattached.

In the world of tmux there are windows and panes within windows. You can think of tmux windows as workspaces on the desktop that are aligned in a horizontal manner. It's like having a number of virtual monitors next to each other each running different shell sessions. You can move across these windows as desired. With the use of panes you can split individual windows horizontally and/or vertically as desired, each pane housing a different session. This is pretty useful for tailing various log files in different panes and monitoring them at once.

You simply have to run the tmux command to create a new tmux session. Once a session exists upon reconnecting over ssh you have to invoke tmux attach to reconnect to your already existing session.

If you're like me you may want to use tmux by default upon ssh'ing to servers. To make this happen you have to include export LC_TMUX_SESSION_NAME=yourusername into your ~/.bashrc and wrap scp on the client side and invoke tmux automatically on the server side. On a related note you can also take a look at my tmux.conf which I believe defines more intuitive shortcuts than the default configuration.

There are a number of alternatives to tmux that I'd like to list starting with the most powerful towards the least powerful. GNU Screen is yet another terminal multiplexer but its feature set, usability and configurability is rather limited compared to tmux. dtach is like a minimalistic tmux featuring one pane inside one window and it only provides a minimal set of options. Finally, with the use of the nohup command you can make your (typically long-running) script immune to hangups and hence it can survive ssh disconnects.

mosh

Remote terminal application that allows roaming, supports intermittent connectivity, and provides intelligent local echo and line editing of user keystrokes.

mosh is the other piece of the puzzle leading to the remote shell nirvana. After apt-getting mosh on the client and mosh-server on the server instead of invoking ssh yourserver.com invoke mosh yourserver.com. From this point on you don't have to worry about reconnecting to ssh or having to wait for the server to echo back your characters anymore.

Caching static server-side resources the easy way

Static resources are static because those never change thus always should be cached forever, right? Well, more often than not some of those files eventually change in which case the files in question must be renamed to be updated by the client which is a pain, especially if you use revision control (which you should).

Lately I came up with a new way to cache static server-side resources as efficiently and effortlessly as possible. Consider the following nginx configuration fragment:

From this point on you can reference a resource like http://yoursite.com/images/background.png as http://yoursite.com/static/1/images/background.png to be cached forever which you can change to http://yoursite.com/static/2/images/background.png in case the contents of this file gets updated. Alternatively, instead of incremental numeric values you may want to use the hash of the current Git commit or any other identifier.

Ethernet cable labelling

Ever felt that you have too many cables in your home and don't know which one leads where?  Welcome to my world!  Here's my recipe to how to make the situation easier.

It all starts with a sticky tape.

ethernet-cable-labelling-1

Now comes the pen.

ethernet-cable-labelling-2

Let's cut the tape with scissors.

ethernet-cable-labelling-3

At this point you should remove the protective foil from the back of the tape to expose the glue and shape a loop.

ethernet-cable-labelling-4

As a final move, pull the cable through the loop and push the tape against the cable to make it stick.

ethernet-cable-labelling-5

When doing all the above properly, you'll get a of sight that makes the pants of any nerd wet.

Make your OpenWrt router automatically enable your home server and notify you

Here's the situation: I have a Mini-ITX home server in my LAN that sometimes gets disabled when the power goes down. I could setup its BIOS to get automatically enabled when it gets power but considering the nature of power outages it may happen that the power goes out and on in rapid succession over a short amount of time which isn't really good for the hardware.

I'd rather choose my server to be checked on a 5 minute basis and have it automatically woken up and I also wanna be notified by email on such an occasion so that I can SSH into it and uncrypt the crypto partition.

I'm about to carry out this task using my OpenWrt driven router. Everything is pretty straightforward, except that I can't send emails through GMail SMTP using Openwrt because mini_sendmail lacks SASL support so I'll just fetch a PHP script that'll actually send the mail.

openwrt:/usr/bin/supervise-host

Be aware that the above wol utility is a home-cooked one.

openwrt:/etc/crontab/root

# m h  dom mon dow   command
0,5,10,15,20,25,30,35,40,45,50,55 * * * * supervise-host mybox http://mydomain.com/notifyme.php

At this ponit restart cron on OpenWrt.

mydomain.com:/var/www/notifyme.php

< ?php
mail('admin@mydomain.com', $_GET['host'] . ' has been rebooted', 'uncrypt me!');
?>

Enjoy!

Streamlined OpenVPN configuration for LANs

I have a reoccuring task of setting up OpenVPN for the LANs of small enterprises and adding / removing users.  Usually they have a dumb little TP-Link or D-Link router facing the public Internet, we bring a relatively powerful PC to their office and my job is to configure the PC as an OpenVPN gateway (among other things).  OpenVPN traffic gets forwarded to our PC through the dumb little router using port forwarding.  Well, this is not particularly challenging to me but I was looking for a way to automate this process as much as I can because managing clients can be cumbersome.

Let's clarify a task at hand: An OpenVPN gateway has to be set up for a /24 LAN in order to provide access to all hosts on the LAN.  Privilege management will be implemented using PKI.  On top of that we'll use tls-auth so the HMAC firewall will only answer if the received packet signature is valid, thus effectively making the OpenVPN service undetectable by any scanning techniques.

The LAN should reside on a class A private subnet (10.x.y.0/24) where x and y should be randomly choosen because it'll minimize the probability of address collision with other subnets used with OpenVPN.

First of all, the PKI should not reside on the server on which the OpenVPN daemon runs for security reasons.  I store it on my home partition which is heavily encrypted and regularly backed up.  I create a directory under ~/openvpn for every OpenVPN installations where I store the server and client configuration files and the PKI.  Only the needed files will be transferred to the server or to the clients.

This post will describe the implementation of the above configuration and will provide a set of scripts to make the task very efficient.

1) Set up the ~/openvpn infrastructure

mkdir ~/openvpn
cd ~/openvpn

# User credentials will be temporarily published under the directory below for user download.  This should be a trusted host.
# It's probably needless to say but I mention that $PUBLISH_URL should not under any circumstances be listable by the web server.
cat >config <<END
PUBLISH_PATH=yourhost:/var/www/pki
PUBLISH_URL=http://yourhost.com/pki
END

wget /wordpress/wp-content/uploads/openvpn-scripts.tar.bz2
tar xjf openvpn-scripts.tar.bz2 -C ~/bin
rm openvpn-scripts.tar.bz2

2) Set up the server directory

cd ~/openvpn
mkdir SERVERNAME
cd SERVERNAME

3) Set up the PKI

mkdir easy-rsa
cp -r /usr/share/doc/openvpn/examples/easy-rsa/2.0/* easy-rsa
cd easy-rsa
# Edit the all the KEY_* variables in ./vars so you won't have to type them anymore.
. ./vars
./clean-all
./build-ca
./build-key-server server
./build-dh
cd ..
mkdir ccd

4) Create server configuration

openvpn --genkey --secret ta.key

cat >server.conf << END
mode server
local 10.X.Y.Z
tls-server
dev tun
proto udp
port 1194
client-config-dir ccd
ifconfig 10.8.0.1 10.8.0.2
push "route 10.X.Y.0 255.255.255.0"
push "route 10.8.0.0 255.255.255.0"
route 10.8.0.0 255.255.255.0
keepalive 10 120
ca ca.crt
cert server.crt
key server.key
dh dh1024.pem
tls-auth ta.key 0
log server.log
verb 3
END

# This will be used by the synchronization script to rsync the configuration to the server through SSH.
echo SERVERHOSTNAME > server.hostname

5) Create general client configuration

# This is the client configuration from which the all individual client configurations will be generated.
# Don't touch "username" as it will be automatically replaced with the name of the relevant user during the generation process.

cat >client.conf << END
dev tun
proto udp
nobind
remote OPENVPN-GATEWAY-HOST 1194
client
ca server.crt
tls-auth server-ta.key 1
cert username.crt
key username.key
verb 3
END

6) Add users

openvpn-add-user username1
openvpn-add-user username2
...

# The configuration will be automatically transferred to the server.

7) Publish client credentials

openvpn-publish-user-credentials username1
openvpn-publish-user-credentials username2
...

# Which outputs something like this:
# User credentials are accessible from http://yourhost.com/pki/servername-username1-65378842373270.zip
# User credentials are accessible from http://yourhost.com/pki/servername-username2-10200344763221.zip
# ...

# These URLs are meant to be mailed to the relevant users and removed eventually.

8) Unpublish client credentials

openvpn-unpublish-user-credentials username1
openvpn-unpublish-user-credentials username2
...

# Which removes the relevant files from the server.

9) Revoke client credentials

openvpn-revoke-user-credentials username

# The configuration will be automatically transferred to the server.

Setting up HTTPS with Apache using a CAcert certificate

openssl req -new -nodes -out yourdomain.com.csr -keyout yourdomain.com.pem
# Type your domain name to the Common Name field.

# Log in to CAcert, go to Server Certificates > New, select "Sign by class 3 root certificate", hand the CSR and get the CRT which you should place to your server as /etc/apache2/ssl/yourdomain.com.crt

cat << END > /etc/apache2/sites-available/yourdomain.com
NameVirtualHost *:443
<VirtualHost *:443>
DocumentRoot /var/www
Options FollowSymLinks
SSLEngine on
SSLCertificateFile /etc/apache2/ssl/yourdomain.com.crt
SSLCertificateKeyFile /etc/apache2/ssl/yourdomain.com.pem
</VirtualHost>
END

a2ensite yourdomain.com
a2enmod ssl
apache2ctl restart