The OpenBSD c2k6 Hackathon is currently in progress in Calgary, Canada. The first outputs are
Timedelta sensors for OpenNTPD
Further KernelTrap.org has a nice introduction of some OpenBSD developers at the Hackathon. Many of the developers also talk about their goals for the Hackathon.
Some photos can be found here.
UPDATE:
HP donated gear for the Hackathon. One of the machines has 24GB RAM and is used for the SCSI development.
FreeBSD uses make options while building ports in order to build a certain extension or (de)activate certain features.
MySQL 4.1 for example can be build with the following options:
WITH_CHARSET=charset
WITH_XCHARSET=list
WITH_COLLATION=collate
WITH_OPENSSL=yes
WITH_LINUXTHREADS=yes
WITH_PROC_SCOPE_PTH=yes
BUILD_OPTIMIZED=yes
BUILD_STATIC=yes
WITHOUT_INNODB=yes
WITH_NDB=yes
There are two ways of utilizing these options. If the port uses OPTIONS in the Makefile, you can configure the options with
# cd /usr/ports/databases/mysql41-server
# make config
You will get an ncurses screen from which you can choose your options. Your configuration will be saved in /var/db/ports/PORTNAME/options and each time you build this port you get the same options. Reconfiguration is done through calling make config in the port directory again.
But many ports do not use the OPTIONS framework in their Makefiles. MySQL for example does not.
If you want to use some of the build options you have to do it like this:
# cd /usr/ports/databases/mysql41-server
# make -DBUILD_OPTIMIZED install clean
The problem is that you build options are not saved. So when you have to update MySQL, you can’t just use portupgrade -a because your options will be lost. You have to upgrade MySQL yourself (make && make deinstall && make install clean) and remember to use all your options again.
The author of portupgrade though of this problem and introduced /usr/local/etc/pkgtools.conf. With pkgtools.conf you can specify arguments that portupgrade uses while upgrading a port.
MAKE_ARGS = {
# a) Separate them with the space
‘databases/mysql41-’ => ‘WITH_LINUXTHREADS=1 BUILD_STATIC=1’,
# b) Specify them using an array
‘databases/mysql41-’ => [
‘WITH_LINUXTHREADS=1’,
‘BUILD_STATIC =1’,
],
}
So you just edit all your ports with the correct build options here and you’re done right? No.
Only portupgrade will use this file so if you happen to build any of the ports yourself, your options are not used. And worse, if a port gets updated THROUGH portupgrade as a dependency of another port, the options will NOT BE USED by portupgrade.
So the only solution left is to use /etc/make.conf. In /etc/make.conf you can specify global make options.
CPUTYPE?= pentium3
CFLAGS= -O2 -mmmx -msse -pipe
COPTFLAGS= -O2 -pipe
WITHOUT_X11=yes
WITHOUT_GUI=yes
The problem with this approach is that these options will be set in every make invocation. So all my ports are build without a GUI and without X11 if this options exists in their Makefiles.
So the correct solution is to conditionally set the make variables in /etc/make.conf because I want BUILD_STATIC only set if the mysql port is compiled. You can do this with checking the CURDIR make variable that points to the current directory:
.if ${.CURDIR:M/databases/mysql}
BUILD_STATIC=yes
BUILD_OPTIMIZED=yes
WITH_LINUXTHREADS=yes
.endif
Now every time make is called in databases/mysql41-server (or better in databases/mysql* so that if I compile the client or decide to move to MySQL 5, the options are still set) make will have my options set. Even if you use portupgrade everything works as portupgrade just calls make.
Not pretty but the only real solution if you want to preserve make options through upgrades automatically.
Andrey V. Elsukov ported the variant symlinks from DragonFlyBSD to FreeBSD. Variant symlinks are symlinks with variables in them and when you access the symlinks the variables are replaced with their contents. The variables can be set per-process, per-user, or per-system.
An example can be found here and the patchset for CURRENT can be found here. This is not commited yet.
libc’s malloc(3) implementation has been replaced with Jason Evans’ newer version that should be faster for SMP systems. For now, all debugging, sanity, and statistics gathering options are enabled in order to find any problems. So your programs will run slower and willl have a larger memory footprint until debugging is disabled.
The update to Ruby 1.8.4 is pending and testers are wanted. The patch (get it here) requires some testing as the last update (to Ruby 1.8.3) was rolled back due to plist issues. After the 1.8.4 update, Pav Lucistnik wants to phase out Ruby 1.6 support.
Four security advisories were released (1,2,3,4). The most serious one affects ipfw as ICMP IP fragments can crash the firewall.
(There are also two security fixes and two reliability fixes for OpenBSD, check the errata for details)
Max Laier called for the FreeBSD status reports. The reports are due on January 20th and the report will be published shortly afterwards.
UPDATE:
Ruby 1.8.4 is now in the tree.
UPDATE 2:
There is another security advisory for FreeBSD about a IEEE 802.11 buffer overflow. Details here.
I just got my OpenBSD 3.8 pre-order.
The official release date is November 1, 2005. Order a CD and support the project!
UPDATE:
OpenBSD 3.8 is now officially released. Further this article got a Chinese Slashdotting.
Arthur Barrett from the CVSNT team (a fork of CVS) asked on the openbsd-tech mailing list if the CVSNT/OpenCVS project effort can perhaps be consolidated.
His point was since the goals are similar (stay compatible to GNU CVS, be secure as possible, and provide a better access control) and CVSNT already had five years of stable releases, joining efforts would benefit both parties.
Apart from the argument of re-licensing (OpenCVS will be BSD licensed), Theo had some interesting information:
We were fully aware of there being other CVS projects, and we do not feel that their stuff can help us towards our goals at all. A lot of our goals are- as yet -not disclosed.
and continued to “explain” why they did not choose to use one of the various CVS implementations:
That said, we have no interest in furthering GPL codebases. Not just because of the licenses, but also because of the obvious bloat that always happens with these codebases designed to “work on every stupid variation of system even written in the past”.
Finally he ended the discussion in his way:
OK, let me be more clear. When people who know nothing about anything write software, in the GNU-style, they write bloated bloated bloated crap when it is not neccessary.
When it is done, OpenCVS will run fine on other systems.
Like OpenSSH.
Without the boatloats of bloat that is common in GNU-style projects.
Anyways, I think our conversation is over.
I personally agree with Theo and that’s why I really like the OpenBSD approach. That’s why I don’t like J2EE and love Ruby & Ruby on Rails.
You can find the whole thread on marc.
OpenSSH 4.2 has just been released. It will be available from the mirrors listed at http://www.openssh.com/ shortly.
OpenSSH 4.2 also includes some security fixes:
- SECURITY: Fix a bug introduced in OpenSSH 4.0 that caused GatewayPorts to be incorrectly activated for dynamic (“-D”) port forwardings when no listen address was explicitly specified.
- SECURITY: sshd in OpenSSH versions prior to 4.2 allow GSSAPI credentials to be delegated to users who log in with methods other than GSSAPI authentication (e.g. public key) when the client requests it. This behaviour has been changed in OpenSSH 4.2 to only delegate credentials to users who authenticate using the GSSAPI method. This eliminates the risk of credentials being inadvertently exposed to an untrusted user/host (though users should not activate GSSAPIDelegateCredentials to begin with when the remote user or host is untrusted).
For details see the announce.
I received some mails about how to exactly configure rsnapshot/rsync to use another user than root to do remote backups as described in my article.
Im my article I wrote:
But using root for this work is not secure because the ssh-key can not be protected by a passphrase as it is used automatically in the background. Therefore create a backup-user on the remote host for this task and allow him to use sudo rsync without a password in /etc/sudoers. Really paranoid can then also allow only this command in .ssh/autorized_keys. See the rsnapshot mailling list for more information on how to do this.
The danger of using root for the backups is that when somebody hacks your backup-server he will gain access to all SSH private keys. With these he will be able to login to your servers as the private keys are configured to use no password. And if the user that for that key on the remode side is root, you’re lost. As the information on the rsnapshot mailing list is maybe not enough, I will describe in detail how to configure rsnapshot without using root. A hacker with access to the SSH private keys will only be able to backup your files.
First create a backup user on the production server (that means the server you want to backup).
remote# adduser
Enter the desired information. Make sure that this user in not in the group wheel as this should be a non-priviledged user, that is the hole reason behind this article.
As described in this article on securityfocus.com, create a ssh-key for the user on the local machine and copy the public key to the remote server.
local# ssh-keygen -t rsa
local# scp id_rsa.pub ssh-remote-server:id_rsa_backup.pub
local# ssh ssh-remote-server
remote# cat id_rsa_backup.pub >> authorized_keys
The last command will tell sshd to allow connections without a password to this account if they have the private key to this public key. So far, nothing new. Now instead of using root we ca now login with the user backup without a password to the remote host. So lets tell rsnapshot to use our new backup user:
Change
backup root@www.example.com:/etc/ www.example.com/etc/
to
backup backup@www.example.com:/etc/ www.example.com/etc/
Are we done now? Unfortunately no as the user backup has not the rights to do the backup. He is not root and therefore does not have access to many files like special files in /etc or the files of other users.
We need to allow the backup user to have access to all files when using rsync .
Therefore we need to install sudo on the remote server and allow the backup user to call sudo rsync without using a password.
On FreeBSD
remote# cd /usr/ports/security/sudo
remote# make install clean
remote# echo “backup remote = NOPASSWD: /usr/local/bin/rsync_real” >> /usr/local/etc/sudoers
Of course you should use visudo to edit sudoers (the config file of sudo).
This line will allow the local user backup to use the command /usr/local/bin/rsync_real as root without a password.
But wait, why rsync_real and not rsync? rsnapshot will call rsync but we have to call sudo rsync in order to gain root privileges for this command. So we have to create a wrapper script that will call sudo rsync for rsnapshot as rsnapshot is not aware of sudo.
remote# cd /usr/local/bin/
remote# mv rsync rsync_real
Now create the wrapper script:
remote# cat /usr/local/bin/rsync
#!/bin/sh
/usr/local/bin/sudo /usr/local/bin/rsync_real ”$@”
Now rsnapshot should be able to use the remote user backup to backup using rsync over ssh. But we can do more. As I wrote:
Really paranoid can then also allow only this command in .ssh/autorizedkeys
Edit /home/backup/.ssh/authorized_keys to allow the backup user that is logged in through the ssh private key only a certain command and to connect form only certain hosts.
remote# cat .ssh/authorized_keys
from=”backup.example.com”, command=”/home/backup/validate-rsync.sh” ssh-rsa AADDB39z….....
Why not allowing /usr/local/bin/rsync? Because sshd will only allow the command as written in the authorized_keys file, the problem is that we want to call rsync with different arguments and this is not possible with the command=”” directive in authorized_keys. Therefore we have to create another wrapper and call the real (better the other wrapper) rsync. Luckily sshd will set the original command issued over SSH as an variable () that we can use in the vaildate-rsync.sh script.
I got the following validate-rsync.sh from here (notice my change in the rsync-case).
remote# chown backup validate-rsync.sh
remote# chmod u+x validate-rsync.sh
remote# cat validate-rsync.sh
#!/bin/sh
case ”$SSH_ORIGINAL_COMMAND” in
&)
echo “Rejected”
;;
;*)
echo “Rejected”
;;
rsync)
$SSH_ORIGINAL_COMMAND
;;
*)
echo “Rejected”
;;
esac
Puh! Hard work for the paranoid. Let’s walk through the hole process again.
The backup machine will connect to the remote server through rsync over ssh using the user backup (backup@remote in rsnapshot.conf). The connection will use the private key of the user on the backup machine to identify itself to the remote sshd (.ssh/authorized_keys). The SSH server will only allow this connection form certain hosts (from-directive) and limit the user to the validate-rsync.sh command (command-directive).
Now the validate-rsync.sh script will call rsync with the arguments that the backup user originally used (rsnapshots commands). But we have to use sudo in order to gain access to all files so rsync is another wrapper around sudo rsync_real.
In the end rsync_real (the real rsync :-) is called with super-user rights and will actually do the backup.
More complicated then just using root? Yes
More secure? Definitely!
Truly paranoid can further edit the validate-rsync.sh script to validate the exact rsync calls (check rsnapshot.conf). Now a hacker will only be able to backup your files to your server!
UPDATE:
There is an interesting thread on the rsnapshot mailing list with some improvements on my HOWTO.
Thanks to Jason for pointing it out.
Just a quick note for people running OpenBSD with memory-based filesystems like described in my article.
If also get double for filesystems (e.g. /var ) after booting:
# mount
/dev/wd0a on / type ffs (...)
mfs:26303 on /var type mfs (...)
mfs:30728 on /dev type mfs (...)
mfs:6263 on /var type mfs (...)
Make sure to include the noauto option for the “double” filesystem:
# cat /etc/fstab
/dev/wd0a / ffs ro 1 1
swap /var mfs rw,-P=/proto/var,-s=32768,noexec,nosuid,nodev,noauto 0 0
swap /dev mfs rw,-P=/proto/dev,-s=1200,-i=128,noexec,nosuid 0 0
The problem is that /var was also mounted from /etc/rc. Got this from misc@openbsd.org.
Further a new version of Flashdist for OpenBSD 3.7 and Current is out.
If you ever wanted to have scheduled remote backups but Amanda or Bacula were too complicated for your needs and just cron jobs with rsync were not enough, you should have a look at rsnapshot.
I started with using rsync over ssh to backup my servers and workstations to a central backup server.
rsync -vaz—del user@remote:/etc /backups/remote/etc
The problem with his approach is that with rsync it is easy to have a synchronized version of your live system but when you want to be able to access several version to the past it is getting complicated. You have to write shell scripts and cron jobs in order to have several versions of the live systems like backing up every system once a week and keeping up to 4 weekly backups. Then maybe you want to also keep 12 monthly backups and so on. Your scripts are getting bigger and bigger and disk space is starting to shrink.
rsnapshots is trying to solve these problems. It is a remote filesystem snapshot utility based on rsync. It can do all the things I described while keeping the needed disk space at a minimum through using hard links. Therefore if in your weekly backups of /etc only 3 file changed, only these files will be copied but the rest will also be accessible as hard links to the backup store where they last changed.
That means that I can have daily/weekly/monthly/yearly snapshots of whole filesystems but the used disk space will be minimal.
Configuration is very easy. All configuration is done on the backup server. This server will use rsync over ssh to connect to the servers that should be backed up and copy the given filesystem(-changes) to the backup store.
After installing rsnapshot (and rsync), edit rsnapshot.conf. First set where you want to store the backups and enable cross-plattform backup of special files (e.g. FIFOs):
snapshot_root /backup/snapshots/
link_dest 1
We are not done with the general configuration. Now we have to specify the intervals for backing up. The default of hourly/daily/weekly/monthly is reasonable and useful and will not hurt your disk space because of the hard links. For my demands, hourly is not needed, so I comment it out:
#interval hourly 6
interval daily 7
interval weekly 4
interval monthly 3
When you use your own intervals be sure to use the smallest first. Larger intervals use the smaller ones as a start. That means that after 4 weeks the oldest weekly snapshot will become the newest monthly backup. The numbers mean that the hourly snapshot will be taken 6 times a day (or every four hours), the daily snapshot will be taken 7 times a week and so on.
Now just tell rsnapshot what to backup and where to put it:
# localhost
backup /root/ localhost/root/
backup /etc/ localhost/etc/
backup /usr/local/etc/ localhost/usr_local_etc/
This will backup the directories /root, /etc, and /usr/local/etc to a directory named localhost in snapshot_root on the backup machine. For remote backups just give ssh the user and host:
# www.example.com
backup root@www.example.com:/root/ www.example.com/root/
backup root@www.example.com:/etc/ www.example.com/etc/
backup root@www.example.com:/usr/local/etc/ www.example.com/usr_local_etc/
backup root@www.example.com:/usr/local/www/ www.example.com/usr_local_www/
backup root@www.example.com:/usr/local/home/ www.example.com/home/
The use the hostname of the remote host as the directory name on the local machine is a often used convention.
For this to work, the user root must be able to log in to www.example.com with a RSA or DSA authentication key. Use ssh-keygen to create these or read here for more information on how to do this. Further root has to be able to read all the directories on the remote server. This is why we start with root.
But using root for this work is not secure because the ssh-key can not be protected by a passphrase as it is used automatically in the background. Therefore create a backup-user on the remote host for this task and allow him to use sudo rsync without a password in /etc/sudoers. Really paranoid can then also allow only this command in .ssh/autorizedkeys. See the rsnapshot mailling list for more information on how to do this.
rsnapshot can also run scripts and backup their output. This is useful for backup of databases, e.g. the script runs pg_dump and rsnapshot backups the dump. Sadly this can only be done for local systems, on remote systems you need a cron job that dumps the database and then rsnapshot can backup the directory with the dump.
We are now nearly finished, the only thing left is to add the cron jobs that call rsnapshot. The daily cron job has to run once a day and just call
rsnapshot daily
The weekly cron job has to run one a week and call
rsnapshot weekly
You get the picture, just call rsnapshot and tell it which interval to back up. On FreeBSD there are already the directories /etc/periodic/{daily, weekly, monthly}. Just insert your script there and it will just work.
After running rsnapshot for some weeks, your backup directory should look like this:
$ cd /backup/snapshots/
$ ls -l
... daily.0
... daily.1
... daily.2
... daily.3
... daily.4
... daily.5
... daily.6
... weekly.0
... weekly.1
... weekly.2
... weekly.3
... monthly.0
$ ls -l daily.0
... localhost
... www.example.com
$ ls-l daily.0/localhost
... etc
... root
... usr_local_etc
I can so access snapshots of whole filesystems of all my servers and workstations several weeks and months to the past without wasting disk space. And the configuration took me only 15 minutes!
See the official HOWTO for a more detailed guide and information on giving normal users access to the snapshots through Samba of NFS.
UPDATE:
See my follow-up article for more information on how to use another user than root for the backups.
After discussing some issues with Thomas, we have now an improved solution for OpenBSD 3.7 on WRAP.
First, we combined the two memory-based filesystems on /var and /tmp into one and symlinked /tmp to /var/tmp.
Then we increased the mfs-filesystem to 16 MB (or even 32 MB) because the default 8 MB are nearly full after the default install.
So the /etc/fstab now looks like this (16MB):
/dev/wd0a / ffs ro 1 1
swap /var mfs rw,-P=/proto/var,-s=32768,noexec,nosuid,nodev 0 0
swap /dev mfs rw,-P=/proto/dev,-s=1200,-i=128,noexec,nosuid 0 0
Then there was the problem of keeping the logs, mail and other important stuff of /var over reboots. As /var is mounted on a memory-filesystem, its content is vanished if the machine reboots. On a new boot it is populated from /proto/var, a cpoy of the “real” /var after installing. My solution was calling rsync from /etc/rc.shutdown:
/usr/local/bin/rsync -vaz /var/ /proto/var/
This syncs the actual /var with /proto/var before shutdown or a reboot so that when /var is populated from /proto/var on a boot, everything is sync and current. No logs, mails or the content of /var/db is lost. rsync can be installed easily from a package.
pkg_add ftp://my/path/to/packages/i386/rsync-2.6.3.tgz
I had to complete first OpenBSD 3.7 CD on a local HTTP mirror. Worked fine. Do not wonder if the rsync call is reporting such errors:
rsync: mknod ”/proto/var/cron/tabs/.sock” failed: Invalid argument (22)
...
rsync: mknod ”/proto/var/empty/dev/log” failed: Invalid argument (22)
...
rsync error: some files could not be transferred (code 23) at /usr/obj/i386/rsync-2.6.3/rsync-2.6.3/main.c(702)
That means that rsync could not create the sockets for cron and bind. Nothing to worry about because as far as I can tell they are created on boot on /var and this is the only place where they are needed.
Another problem was that OpenBSD only detected 64 MB RAM instead of 128 MB. The solution is to update the Bios to at least 1.07. We installed 1.08 through xmodem upload. This was a hard fight for me because I could not get minicom to enter the Bios or tip starting the xmodem upload. With tip I was able to enter the Bios but I could not start an upload with lrzsz as tip was no longer reacting to the escape characters. I tried minicom but with minicom I could not enter the Bios. After OpenBSD booted, everything worked through minicom, but pushing “s” in order to enter the WRAP Bios resulted in no reaction. After some hours of fun with minicom/c-kermit/tip/cu/lrzsz Thomas finally got it.
The trick was to boot the WRAP first and then connect through minicom. Only so we were able to enter the Bios and upload the new 1.08 Bios with lrzsz from within minicom. We had always connected with minicom first and than booted the WRAP because you had to be fast in order to push “s” during memory counting. For more information on the Bios update see this page about Bios update on the very similar Soekris NET4801.
After this hurdle my WRAP uses 128 MB RAM under OpenBSD and can netboot with PXE.
Further Thomas has a new webpage dedicated to OpenBSD 3.7 and WRAP. There you can also find some information on how to control the LEDs.
UPDATE:
You can get the Bios from here: <a href=”http:// www.pcengines.ch/wbios108.zip”> www.pcengines.ch/wbios108.zip
UPDATE 2:
See my follow-up article for a small issue with double mounted filesystems.
BSDCan 2005 should be over now. If you were so unfortunately as myself and could not attend be sure to read the presentations and papers that are now slowly published.
Colin Percival published his paper about a vulnerability in Hyperthreading that can lead to information disclosure on various Operating Systems:
permits local information disclosure, including allowing an unprivileged user to steal an RSA private key being used on the same machine. Administrators of multi-user systems are strongly advised to take action to disable Hyper-Threading immediately; single-user systems (i.e., desktop computers) are not affected.
This vulnerability was the “security issue” that caused some misunderstanding between Theo de Raadt/OpenBSD and Colin Percival/FreeBSD back in March.
Kris Kennaway, the FreeBSD ports “master”, held a presentation about the FreeBSD package cluster that is used to build and test all FreeBSD ports on all supported platforms. You can find his kris/”>website.
Richard Bejtlich, author of the fabulous The Tao Of Network Security Monitoring, gave a talk about keeping FreeBSD up-to-date. The talk is based on his paper on the same topic.
Henning Brauer presented OpenBGPD, an older presentation can be found here. I’m sure that the presentation he held was very similar.
These were the presentations or papers that I could already find. More should be published soon.
Yesterday Thomas and I received our WRAP boxes. They are cheaper than the Soekris net4801 but lack the IDE connector, the PCI slot, the USB interface, and have only one serial port. But for our purposes they suffice.
We are planning to do some testing with CARP+PF like described here. We will hopefully move on to create a redundant OpenBSD Hardware Firewall consisting of two or more WRAP/Soekris/VIA C3/?? boxes configured by a web-interface (maybe Ruby on Rails :-). The whole thing would be placed in a 1U Rack.
