Fed up with vim 7.2 on CentOS 6?

The vim RPM that ships with CentOS7, currently at version 7.4.160, can be rebuilt and installed on a CentOS 6 box.

First, install the source RPM.

$ rpm -Uvh http://vault.centos.org/7.0.1406/os/Source/SPackages/vim-7.4.160-1.el7.src.rpm

Make a minor change to the vim.spec file with this patch.

$ cd ~/rpmbuild/SPECS
$ wget -O - \
   https://gist.githubusercontent.com/renecunningham/897bc360210c2901d6d5/raw | \
   patch -p0 vim.spec

Install the required build dependencies.

$ yum install ncurses-devel 'perl(ExtUtils::Embed)' libacl-devel autoconf \
    gpm-devel ruby-devel ruby gtk2-devel gtk2-devel libSM-devel libXt-devel \
    libXpm-devel

Build the vim-7.4.160 RPM.

$ rpmbuild -ba vim.spec

The vim-7.4.160 the RPMs ready to be installed will be stored ~/rpmbuilds/RPMS/$(uname -p).

$ ls ~/rpmbuild/RPMS/$(uname -p)
vim-common-7.4.160-1.el6.x86_64.rpm      vim-minimal-7.4.160-1.el6.x86_64.rpm
vim-enhanced-7.4.160-1.el6.x86_64.rpm    vim-X11-7.4.160-1.el6.x86_64.rpm

SIP commonly runs over UDP but there are times when you may need to run it over TCP.

To allow SIP TCP clients to connect with asterisk update sip.conf with the following

[general]
tcpenable=yes
tcpbindaddr=0.0.0.0

Within your SIP clients definition you have to add transport=tcp for each individual connection.

[client001]
callerid="Client 001" <001>
username=client001
secret=password
type=friend
host=dynamic
context=internal
canreinvite=yes
mailbox=001@default
transport=tcp
disallow=all
allow=alaw
nat=route
dtmfmode=inband

Reload sip within the asterisk console and confirm that asterisk is now listening on 5060/tcp with netstat.

server*CLI> sip reload
server*CLI> quit
Executing last minute cleanups
$ sudo netstat  -tlpn | grep 5060
tcp        0      0 0.0.0.0:5060            0.0.0.0:*               LISTEN      17414/asterisk  

Discover Googles Netblocks

September 19, 2014

Do you need to discover what IP netblocks are owned and operated by google to perhaps add to your firewall ACLs?

Use dig.

$ dig -t txt _netblocks.google.com

Have you ever used SSH within a script or perhaps with rsnapshot and had to accept the servers host key?

$ ssh server.example.org
The authenticity of host 'server.example.org (192.0.2.100)' can't be established.
ECDSA key fingerprint is 63:f1:fe:d1:a5:4e:2e:94:33:13:46:58:9f:51:32:f1.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'server.example.org,192.0.2.0' (ECDSA) to the list of known hosts.

By using StrictHostKeyChecking and UserKnownHostsFile SSH client options you can bypass this check.

$ ssh server.example.org -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null
Warning: Permanently added 'server.example.org (192.0.2.100)' (ECDSA) to the list of known hosts.
root@server:~#

Alternatively you could update ${HOME}/.ssh/config

Host                    server.example.org
    HostName                server.example.org
    User                    root
    Compression             yes
    ForwardAgent            no
    ForwardX11              no
    StrictHostKeyChecking   no
    UserKnownHostsFile      /dev/null

Security Warning

It’s important that you trust the server that you are connecting to before changing the default StrictHostKeyChecking & UserKnownHostsFile options. All an attacker would need to do is poison your DNS and import your SSH public key to fake what you would expect to be the server you are connecting to.

Good Sysadmins always produce great documentation and descriptive network diagrams.

It’s good form to use appropriate network addresses within documentation, example code and diagrams.

Ever seen a Fictitious IP address in a movie?

Well someone should of read RFC5735.

192.0.2.0/24 – This block is assigned as “TEST-NET-1″ for use in documentation and example code. It is often used in conjunction with domain names example.com or example.net in vendor and protocol documentation.

198.51.100.0/24 – This block is assigned as “TEST-NET-2″ for use in documentation and example code. It is often used in conjunction with domain names example.com or example.net in vendor and protocol documentation.

203.0.113.0/24 – This block is assigned as “TEST-NET-3″ for use in documentation and example code. It is often used in conjunction with domain names example.com or example.net in vendor and protocol documentation.

So next time you write some documentation, create a diagram or reference a network in example code consider using one of the networks in RFC5735.

Ever logged into an Ubuntu box and seen this?

Welcome to Ubuntu 14.04.1 LTS (GNU/Linux 3.13.0-29-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

  System information as of Sat Sep 13 23:12:33 UTC 2014

  System load:  0.08              Processes:           118
  Usage of /:   12.7% of 7.74GB   Users logged in:     1
  Memory usage: 16%               IP address for eth0: 10.1.1.1
  Swap usage:   0%

  Graph this data and manage this system at:

https://landscape.canonical.com/

  Get cloud support with Ubuntu Advantage Cloud Guest:

http://www.ubuntu.com/business/services/cloud

You can configure /etc/pam.d/sshd to not include the pam_motd.so module when logging in. Just hash out the following lines in /etc/pam.d/sshd.

session    optional     pam_motd.so  motd=/run/motd.dynamic noupdate
session    optional     pam_motd.so # [1]

I was notified that one of my slave MySQL DB’s stop replicating. An immediate investigation showed that because I dont slave the mysql.user table (is there a reason to do this?), an SQL statement that required a certain user and privilege failed on the slave and broke replication.

mysql-slave> SHOW SLAVE STATUS\G;
...
Last_Errno: 1449
Last_Error: Error 'There is no 'user_foo'@'ip_bar' registered' on query. Default database: 'database_fye'. Query: 'insert into ...'
...

Ouch.

To fix this I stopped the replication and took note of the master log position, Exec_Master_Log_Pos and slave’s master log file, Relay_Log_File. Both are found in the output of SHOW SLAVE STATUS.

I then added the mysql user account with appropriate privileges, reset the slave, manually pointed mysql to the master log position and then started to slave again.

mysql-slave> STOP SLAVE;
mysql-slave> SHOW SLAVE STATUS\G;
mysql-slave> GRANT ALL PRIVILEGES ON database_fye.* TO user_foo@'ip_bar' IDENTIFIED BY 'password_foe';
mysql-slave> FLUSH PRIVILEGES;
mysql-slave> RESET SLAVE;
mysql-slave> CHANGE MASTER TO MASTER_HOST='MASTER_IP', MASTER_USER='SLAVE_USER', MASTER_PASSWORD='SLAVE_PASS', MASTER_LOG_FILE='mysql-bin.000801', MASTER_LOG_POS=17122711;
mysql-slave> START SLAVE;

A couple of seconds later the slave caught up with the master.

virsh is the main command line tool to manage virtual guest domains on Linux. Several Linux distributions use polkit, a toolkit for handling unprivileged access to processes, to manage access to the libvirt virtualisation layer.

libvirt ships with a set of polkit actions defining operations that clients (example: virsh) can request from privileged processes (example: libvirtd). These action files are stored in /usr/share/polkit-1/actions and can be viewed with the pkactions command.

The polkit action we’re interested in is org.libvirt.unix.manage.

$ pkaction --verbose --action-id org.libvirt.unix.manage
org.libvirt.unix.manage:
  description:       Manage local virtualized systems
  message:           System policy prevents management of local virtualized systems
  vendor:
  vendor_url:
  icon:
  implicit any:      auth_admin_keep
  implicit inactive: auth_admin_keep
  implicit active:   auth_admin_keep

polkit can be configured to allow clients that belong to a particular unix group to run an action of org.libvirt.unix.manage.

[libvirt Admin Access]
Identity=unix-group:virt
Action=org.libvirt.unix.manage
ResultAny=yes
ResultInactive=yes
ResultActive=yes

Any invocations of this action are logged to /var/log/secure.

Now we need to add users to the virt group.

# usermod -a -G virt rene

virsh can connect to remote hypervisors running libvirtd. To configure virsh to connect to libvirtd running on a local server by default, we need to define the URI of qemu:///system as a environment variable.

if test -x `which virsh`; then
  export LIBVIRT_DEFAULT_URI=qemu:///system
fi

Users within the virt group should now be able to run virsh commands without having to be root.

Nagios Remote Plugin Executor (NRPE), is used by nagios to run commands remotely on linux servers.

For example, to discover the load, a measure of computional work, on a linux server you would run uptime which provides you with the load average.

$ uptime
 22:41:02 up 14 min,  1 user,  load average: 0.00, 1.02, 2.40

Without NRPE, nagios is unable to run this command remotely.

NRPE works by running as a daemon on the remote linux server listening on a port. Nagios connects to the NRPE daemon and runs a NRPE command which typically calls a nagios plugin such as check_load, returning some info aswell as a status of OK, WARNING, CRITICAL or UNKNOWN.

First, install and start the NRPE server on the remote linux server.

$ sudo yum install nrpe
$ sudo service nrpe start

NRPE by default listens on port 5666/tcp which you will need to allow through if your remote linux server is running a firewall. Be sure to only expose the NRPE service to your nagios server by specifying the IP address of your nagios server with -s in your iptables rules.

-A INPUT -m state --state NEW -m tcp -p tcp --dport 5666 -s 10.0.0.1 -j ACCEPT
$ service iptables restart

On the nagios server, install the check_nrpe plugin.

$ sudo yum install nagios-plugins-nrpe

check_nrpe is the executable that nagios will use to query the remote linux server.

$ /usr/lib64/nagios/plugins/check_nrpe -H 10.0.1.254
CHECK_NRPE: Error - Could not complete SSL handshake.

NRPE will only allow certain hosts to connect which is configured by the allowed_hosts option in /etc/nagios/nrpe.cfg. Let’s add our nagios server and restart the NRPE service.

allowed_hosts=127.0.0.1,10.0.0.1
$ sudo service nrpe restart

Again, let’s run check_nrpe from our nagios server.

$ /usr/lib64/nagios/plugins/check_nrpe -H 10.0.1.254
NRPE v2.14

By default, NRPE should be configured with a check_load command.

command[check_load]=/usr/lib64/nagios/plugins/check_load -w 15,10,5 -c 30,25,20

Call this command from the nagios server via NRPE using the check_nrpe plugin.

$ /usr/lib64/nagios/plugins/check_nrpe -H 10.0.1.254 -c check_load
OK - load average: 0.00, 0.00, 0.00|load1=0.000;15.000;30.000;0; load5=0.000;10.000;25.000;0; load15=0.000;5.000;20.000;0;

Once we’ve confirmed that the nagios server is able to communicate with the remote linux server over NRPE we can configure nagios and reload.

define service {
  use                            generic-service
  host_name                      webserver.example.org
  service_description            NRPE - Load
  check_command                  check_nrpe!check_load
}
$ sudo service nagios reload

virsh provides Linux System Administrators with the ability to dynamically scale allocated memory to virtual guests during runtime.

$ virsh dommemstat web-server
actual 2097152
rss 903040

dommemstat shows that the domain web-server has a memory allocation limit of 2097152 kB and is currently using 903040 kB. You can confirm this with ps.

$ ps -C qemu-kvm -o rss,cmd
  RSS CMD
903040 /usr/bin/qemu-kvm -name web-server -S -M pc-1.2 -enable-kvm -m 2048 -smp

The memory allocation limit is set with setmaxmem and can only be performed whilst the domain is inactive.

$ virsh setmaxmem web-server 1048576 --config

By passing setmaxmem the config option the memory allocation limit is written to the virsh configuration file.

The current memory allocation for a guest domain is set with setmem.

$ virsh setmem web-server 786432 --config --live

When passing setmem the live option, a memory balloon is performed on the running guest and the change happens instantly. This can be verified with dominfo.

$ virsh dominfo web-server
Id:             3
Name:           web-server
UUID:           a1a0ee38-20ff-144b-a0ef-2bb14e4ce167
OS Type:        hvm
State:          running
CPU(s):         2
CPU time:       33.9s
Max memory:     1048576 KiB
Used memory:    786432 KiB
Persistent:     yes
Autostart:      disable
Managed save:   no
Security model: selinux
Security DOI:   0
Security label: system_u:system_r:svirt_t:s0:c782,c903 (enforcing)