Creating FreeBSD ports for ruby gems

Posted by Jonathan

Creating FreeBSD ports for ruby gems involves creating a pkg-plist that lists all files and directories for a given port. This kind of sucks for gems that have a lot of rdoc documentation (like ActiveSupport). There are a couple of automatic solutions like pkg_trackinst but none is perfect.

While updating the ports for Rails 1.2.1 I had enough and created a small Ruby script to help with the generation of pkg-plists. Updating a Rubygem port now looks like this:

# cd /usr/ports/devel/rubygem-activesupport
# vim Makefile
(update PORTVERSION)
# make makesum

At first update the port version and create the new checksum. Now install the new version (without updating the pkg-plist, so will will get a lot of errors during deinstall).

# make install

Now run the script to generate the new pkg-plist out of the installed gem:

# /tmp/gen_plist.rb activesupport-1.4.0 > pkg-plist

The script assembles a pkg-plist of of all files and directories under /gems and /doc. Any other files like the rails executable for the Rails gem must be added by hand as they will not be installed under /usr/local/lib/ruby/gems/1.8/.

What’s left to do now is to cleanup manually the installation and test the pkg-plist through creating a package and deinstalling the port.

The small script looks like this:

#! /usr/bin/env ruby  

package_name = ARGV[0]

raise ArgumentError if (package_name.nil? || package_name == '')

prefix = "/usr/local" 
gem_lib_dir = "#{prefix}/lib/ruby/gems/1.8/gems/#{package_name}" 
gem_doc_dir = "#{prefix}/lib/ruby/gems/1.8/doc/#{package_name}" 

pkg_plist = ["%%GEM_CACHE%%", "%%GEM_SPEC%%"]

# add lib
lib_files = `find #{gem_lib_dir} -type f`.split("\n")
lib_dirs = `find #{gem_lib_dir} -type d`.split("\n").reverse
lib_dirs = lib_dirs.collect{|d| "@dirrm #{d}"}

# add doc
doc_files = `find #{gem_doc_dir} -type f`.split("\n")
doc_dirs = `find #{gem_doc_dir} -type d`.split("\n").reverse
doc_dirs = doc_dirs.collect{|d| "@dirrm #{d}"}

# assemble
pkg_plist += lib_files + doc_files + lib_dirs + doc_dirs

# change PLIST_SUBs
pkg_plist.each do |line|
  line.gsub!(gem_doc_dir, "%%GEM_DOC_DIR%%")
  line.gsub!(gem_lib_dir, "%%GEM_LIB_DIR%%")
end

# print out pkg-plist
pkg_plist.each do |line|
  puts line
end

Rails 1.1.1 and script.aculo.us 1.6.1 released

Posted by Jonathan

Rails 1.1.1 was just released!

I was waiting with upgrading the FreeBSD port as Rails 1.1 had some issues and 1.1.1 was in the planning. I hopefully have the port updated be the end of the weekend. The upgrade to Rake 0.7 is already submitted.

Further script.aculo.us 1.6.1 was also released some minutes ago. It includes Protoype 1.5.0_rc0 and fixed some ugly IE bugs.

Have fun with all the upgrades.

Rails on FreeBSD

Posted by Jonathan

I’ve updated the Rails port to include optional dependencies on ruby-fcgi and memcache-client:

In order to get a Rails installation with FastCGI, MemCache-Client and native MySQL/PostgreSQL/SQLite support just do:

# cd /usr/ports/www/rubygem-rails
# make install clean

That’s it!

Better Support for ActiveRecord in the Ports Tree

Posted by Jonathan

Roderick van Domburg sent me a patch for the ActiveRecord FreeBSD port that lets you choose to depend on the native MySQL bindings.

I’ve updated the patch to support PostgreSQL and SQLite3 too. When you now install ActiveRecord or Rails through the FreeBSD ports tree you will see this screen:

You can choose support for the native bindings. The ports tree will install them for you and will remember the dependencies. If you want to change these settings just re-configure the port:

# cd /usr/ports/databases/rubygem-activerecord
# make config

I’m thinking about adding such options for www/rubygem-rails so that it can depend on www/ruby-fcgi. My goal is to get installing Rails on FreeBSD very easy.

Upgrading ports and preserve make options

Posted by Jonathan

FreeBSD uses make options while building ports in order to build a certain extension or (de)activate certain features.

MySQL 4.1 for example can be build with the following options:

WITH_CHARSET=charset
WITH_XCHARSET=list
WITH_COLLATION=collate
WITH_OPENSSL=yes
WITH_LINUXTHREADS=yes
WITH_PROC_SCOPE_PTH=yes
BUILD_OPTIMIZED=yes
BUILD_STATIC=yes
WITHOUT_INNODB=yes
WITH_NDB=yes

There are two ways of utilizing these options. If the port uses OPTIONS in the Makefile, you can configure the options with

# cd /usr/ports/databases/mysql41-server
# make config

You will get an ncurses screen from which you can choose your options. Your configuration will be saved in /var/db/ports/PORTNAME/options and each time you build this port you get the same options. Reconfiguration is done through calling make config in the port directory again.

But many ports do not use the OPTIONS framework in their Makefiles. MySQL for example does not.

If you want to use some of the build options you have to do it like this:

# cd /usr/ports/databases/mysql41-server
# make -DBUILD_OPTIMIZED install clean

The problem is that you build options are not saved. So when you have to update MySQL, you can’t just use portupgrade -a because your options will be lost. You have to upgrade MySQL yourself (make && make deinstall && make install clean) and remember to use all your options again.

The author of portupgrade though of this problem and introduced /usr/local/etc/pkgtools.conf. With pkgtools.conf you can specify arguments that portupgrade uses while upgrading a port.

MAKE_ARGS = {
# a) Separate them with the space
‘databases/mysql41-’ => ‘WITH_LINUXTHREADS=1 BUILD_STATIC=1’,

# b) Specify them using an array
‘databases/mysql41-
’ => [
‘WITH_LINUXTHREADS=1’,
‘BUILD_STATIC =1’,
],
}

So you just edit all your ports with the correct build options here and you’re done right? No.

Only portupgrade will use this file so if you happen to build any of the ports yourself, your options are not used. And worse, if a port gets updated THROUGH portupgrade as a dependency of another port, the options will NOT BE USED by portupgrade.

So the only solution left is to use /etc/make.conf. In /etc/make.conf you can specify global make options.

CPUTYPE?= pentium3
CFLAGS= -O2 -mmmx -msse -pipe
COPTFLAGS= -O2 -pipe
WITHOUT_X11=yes
WITHOUT_GUI=yes

The problem with this approach is that these options will be set in every make invocation. So all my ports are build without a GUI and without X11 if this options exists in their Makefiles.

So the correct solution is to conditionally set the make variables in /etc/make.conf because I want BUILD_STATIC only set if the mysql port is compiled. You can do this with checking the CURDIR make variable that points to the current directory:

.if ${.CURDIR:M/databases/mysql}
BUILD_STATIC=yes
BUILD_OPTIMIZED=yes
WITH_LINUXTHREADS=yes
.endif

Now every time make is called in databases/mysql41-server (or better in databases/mysql* so that if I compile the client or decide to move to MySQL 5, the options are still set) make will have my options set. Even if you use portupgrade everything works as portupgrade just calls make.

Not pretty but the only real solution if you want to preserve make options through upgrades automatically.

FreeBSD news

Posted by Jonathan

Variant Symlinks

Andrey V. Elsukov ported the variant symlinks from DragonFlyBSD to FreeBSD. Variant symlinks are symlinks with variables in them and when you access the symlinks the variables are replaced with their contents. The variables can be set per-process, per-user, or per-system.

An example can be found here and the patchset for CURRENT can be found here. This is not commited yet.

New malloc commited to CURRENT

libc’s malloc(3) implementation has been replaced with Jason Evans’ newer version that should be faster for SMP systems. For now, all debugging, sanity, and statistics gathering options are enabled in order to find any problems. So your programs will run slower and willl have a larger memory footprint until debugging is disabled.

Ruby 1.8.4 testers wanted

The update to Ruby 1.8.4 is pending and testers are wanted. The patch (get it here) requires some testing as the last update (to Ruby 1.8.3) was rolled back due to plist issues. After the 1.8.4 update, Pav Lucistnik wants to phase out Ruby 1.6 support.

Security Advisories

Four security advisories were released (1,2,3,4). The most serious one affects ipfw as ICMP IP fragments can crash the firewall.

(There are also two security fixes and two reliability fixes for OpenBSD, check the errata for details)

Call for FreeBSD Status Reports

Max Laier called for the FreeBSD status reports. The reports are due on January 20th and the report will be published shortly afterwards.


UPDATE:
Ruby 1.8.4 is now in the tree.


UPDATE 2:
There is another security advisory for FreeBSD about a IEEE 802.11 buffer overflow. Details here.

Typo 2.6.0 on FreeBSD

Posted by Jonathan

A quick howto install typo 2.6.0 on a blank FreeBSD installation. It is quite easy with Rails 1.0 and typo 2.6.0 in the ports.

Install

# cd /usr/ports/www/typo
# make install clean

Choose your database

# cd /usr/ports/databases/ruby-{mysql | sqlite | postgres }
# make install clean

Configure

# cd /usr/local/www/typo-2.6.0
# vim config/database.yml

Now start your database and create the database with the included schema in db/schema.{mysql | sqlite | psql }.sql

Run

# ruby script/server

Rails 1.0 now in the FreeBSD ports tree 0

Posted by Jonathan

I updated all Rails related ports and they are now commited:

So have fun with Rails 1.0 on FreeBSD:

# cvsup /etc/ports-supfile
# cd /usr/ports && make fetchindex
# cd www/rubygem-rails

# make install clean

or

# portupgrade rubygem-\*

If you have Rails already installed through the ports tree

Some FreeBSD news

Posted by Jonathan

Release schedule 2006

Scot Long posted the release schedule for 2006:

Jan 30: Freeze RELENG_5 and RELENG_6
Mar 20: Release FreeBSD 6.1
Apr 3: Release FreeBSD 5.5
Jun 12: Freeze RELENG_6
Jul 31: Release FreeBSD 6.2
Oct 23: Freeze RELENG_6
Dec 11: Release FreeBSD 6.3

5.5 will be the final release from the RELENG_5 tree. After this final release, the security team will provide security update support through 2007 but all FreeBSD users are strongly encouraged to evaluate FreeBSD 6. The primary focus for the 6.x releases is on bugfixes, performance enhancements, and incremental functionality and driver additions. FreeBSD 7.0 that is scheduled for June 2007 will bring new features.

XFS (read-only) support committed to CURRENT

Craig Rodrigues commited read-only XFS support that is based on the GPL sources by SGI. XFS partitions can be mounted with mount -t xfs device and additional utilities such as mkfs.xfs are available in the sysutils/xfsprogs port.

BSDInstaller Beta 2 release

The second release of FreeBSD install CD’s based on the BSD Installer was announced by Andrew Turner.

The new Lua backend is now being used in this release rather than the older, deprecated C version.

The CD image is available from ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/bsdinstaller/7.0-BSDINSTALLER-BETA-2-i386-disc1.iso.gz or your local mirror.

Changes Since BETA-1
* Can now install Source
* Can now install the Ports tree
* Can now install Packages
* Change to the new Lua backend

Known Problems
* Can only install to one drive, can’t have / on ad0 and /usr on ad1
* fdisk doesn’t alter the geometry

FreeBSD Status Report Third Quarter 2005

Posted by Jonathan

Max Laier posted the July-October 2005 FreeBSD Status Report.

The status report contains some updates about the various Google Summer of Code projects such as Fuse for FreeBSD or gjournal.

FreeRADIUS SSL CA Annoyances

Posted by Jonathan

The last couple of days I’ve struggling with FreeRADIUS. Apart from not having a good documentation, a configuration mess, and not compiling out-of-the-box on OpenBSD, its OpenSSL CA scripts are broken.

FreeRADIUS ships a script that should help you set up your own CA for a self-signed certificate for EAP-TTLS. This script (CA.all in the scripts directory of the distribution) uses OpenSSL’s CA.pl script to do the SSL magic for you. The only problem is that the path to CA.pl is hardcoded:

SSL=/usr/local/ssl
export PATH=${SSL}/bin/:${SSL}/ssl/misc:${PATH}

...

echo “newreq.pem” | /usr/local/ssl/misc/CA.pl -newca

The hardcoded path will of course not work on OS X, FreeBSD or OpenBSD. As the good guys from FreeRADIUS do not use any type of error handling, this problem will manifest later with this error message:

Error opening CA private key ./demoCA/private/cakey.pem
1254:error:02001002:system library:fopen:No such file or directory:bss_file.c:278:fopen(’./demoCA/private/cakey.pem’,’r’)
1254:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:280:
unable to load CA private key

Took me a while to understand why the demoCA stuff was not created. The funny thing is that the path stuff would not work anyway as CA.pl is located in /usr/src/crypto/openssl/apps/ in FreeBSD and in /System/Library/OpenSSL/misc/ under OS X.

I hate developer that assume that UNIX = Linux.

New FreeBSD Logo announced 0

Posted by Jonathan

The winner of the FreeBSD logo competition is Anton K. Gural.

Jun Kuriyama announced the winner on announce@freebsd.org.

The details can be viewed here. The first reactions on the mailing lists are divided.

Personally I have to adjust to the new logo, but the longer I look at it the better it looks.

FreeBSD 6.0-RELEASE coming!

Posted by Jonathan

Scott Long from FreeBSD’s release engineering team just announced that 6.0-RELEASE will be shortly available:

All,

Wanted to let everyone know that the testing on RC1 has gone well enough that we’ve decided to skip RC2 and go straight to 6.0-RELEASE. Everyone that we have talked to has applauded the stability and functionality of the system, so we are really pleased and really eager to wrap it up and get it out to everyone. Over the next 24 hours we will be synchronizing the RELENG_6_0 tree to get in all of the appropriate bug fixes, then we will do some test builds as a final verification. Everyone is still welcome to update their sources on the RELENG_6_0 branch and provide feedback for the next 48 hours or so. The release will likely be announced by the end of the weekend or early next week, at the latest.

Scott

I personally use 6.0-RC1 to host this blog had no issues.

www.freebsd.org updated 0

Posted by Jonathan

Thanks to the DragonFlyBSD Digest I noticed that FreeBSD got a new homepage. Emily Boyd created it in her Google Summer of Code project. I did not saw any announcements on the FreeBSD lists and the old page is still in Google’s cache of yesterday so the new page must be online since today.

It is definitely better than the old design.

UPDATE:
Some folks on stable@freebsd.org noticed the new page, so it is online since today.

Rubygems in the ports tree

Posted by Jonathan

It seems like my efforts to integrate Rubygems into the FreeBSD ports tree were not only for my own needs. Ports for the RGL and Stream rubygems were submitted by Ulrich Spoerlein and are now part of the ports tree.

RGL is a framework for graph datastructures and algorithms implemented in Ruby while Stream is an interface for external iterators.

I hope that this will be continue to be useful for FreeBSD users. My work on the OpenBSD integration is at a halt on the moment because I wait for the Ruby 1.8.2 integration. Due to the upcoming 3.8 release commits on the ports are bug-fixing only so this will be delayed until 3.8 will be released in November.