Creating FreeBSD ports for ruby gems

Posted by Jonathan

Creating FreeBSD ports for ruby gems involves creating a pkg-plist that lists all files and directories for a given port. This kind of sucks for gems that have a lot of rdoc documentation (like ActiveSupport). There are a couple of automatic solutions like pkg_trackinst but none is perfect.

While updating the ports for Rails 1.2.1 I had enough and created a small Ruby script to help with the generation of pkg-plists. Updating a Rubygem port now looks like this:

# cd /usr/ports/devel/rubygem-activesupport
# vim Makefile
(update PORTVERSION)
# make makesum

At first update the port version and create the new checksum. Now install the new version (without updating the pkg-plist, so will will get a lot of errors during deinstall).

# make install

Now run the script to generate the new pkg-plist out of the installed gem:

# /tmp/gen_plist.rb activesupport-1.4.0 > pkg-plist

The script assembles a pkg-plist of of all files and directories under /gems and /doc. Any other files like the rails executable for the Rails gem must be added by hand as they will not be installed under /usr/local/lib/ruby/gems/1.8/.

What’s left to do now is to cleanup manually the installation and test the pkg-plist through creating a package and deinstalling the port.

The small script looks like this:

#! /usr/bin/env ruby  

package_name = ARGV[0]

raise ArgumentError if (package_name.nil? || package_name == '')

prefix = "/usr/local" 
gem_lib_dir = "#{prefix}/lib/ruby/gems/1.8/gems/#{package_name}" 
gem_doc_dir = "#{prefix}/lib/ruby/gems/1.8/doc/#{package_name}" 

pkg_plist = ["%%GEM_CACHE%%", "%%GEM_SPEC%%"]

# add lib
lib_files = `find #{gem_lib_dir} -type f`.split("\n")
lib_dirs = `find #{gem_lib_dir} -type d`.split("\n").reverse
lib_dirs = lib_dirs.collect{|d| "@dirrm #{d}"}

# add doc
doc_files = `find #{gem_doc_dir} -type f`.split("\n")
doc_dirs = `find #{gem_doc_dir} -type d`.split("\n").reverse
doc_dirs = doc_dirs.collect{|d| "@dirrm #{d}"}

# assemble
pkg_plist += lib_files + doc_files + lib_dirs + doc_dirs

# change PLIST_SUBs
pkg_plist.each do |line|
  line.gsub!(gem_doc_dir, "%%GEM_DOC_DIR%%")
  line.gsub!(gem_lib_dir, "%%GEM_LIB_DIR%%")
end

# print out pkg-plist
pkg_plist.each do |line|
  puts line
end

Comments

Leave a response

  1. MaledictusJanuary 25, 2007 @ 03:05 PM
    Thanks for all your great work, Jonathan! But I ask myself if the FreeBSD Ports should repeat the mess with the p5- Ports with rubygem-? gem is the first time I install stuff into my system that is not coming out of the ports. And I really love it, so cool to have multiple versions installed at the same time. I thought a bit about this and still don't know if there would be a clean way to integrate foreign package systems (like rubygems) cleanly into the ports. BUT if it would work one could also cleanup perl, php and python stuff. Maybe more. What do you think about it?
  2. JonathanJanuary 25, 2007 @ 03:31 PM
    Integrating rubygems with a package manager is not trivial as you have problems with native C-extensions. Further if you package gems as 'normal' Ruby libs and put them in site_ruby a lot of confusion can happen if a user later installs the same gem through rubygems. A standard Ruby script would always use the lib in site_ruby and not the gem. I do not know of a better solution then to just wrap around 'gem install/uninstall' with the package tools. If you are interested, the same discussion is right now on the Mongrel and the Rubygem mailing lists. No silver bullet yet :-(
  3. MaledictusJanuary 25, 2007 @ 04:21 PM
    I didn't thought of packaging them as normal ruby libs. That wouldn't really help as we would always have to run after the gems. I thought more in the direction of teaching portupgrade that there are gems und how to upgrade them or list them on portversion -vl "<". As a first step. I think by problems with C-extensions you mean that it doesn't compile under FreeBSD? Do you have an example of such a gem? I'll look at the other lists if I can find the time, thanks for the hint.
  4. JonathanJanuary 25, 2007 @ 07:51 PM
    With C-extensions I ment that you do not know before compiling them, what files are included so there is no automatic way to get the pkg-plist. I now understand what you mean by integration with portupgrade. If you would let portupgrade automatically install/upgrade gems (without creating/updating a specific port) you would lose some stability. The ports system does not only give you a package manager but also stability because a FreeBSD person used this new version of a port on FreeBSD and made sure that it works as it should. Autoinstallations of new gems could result in a non-working version of a gem. This mean that you need somebody who says "this version of rails is now clear for FreeBSD". This is exaclty what the rubygem port as it is now does. The only extra is that it lists the package contents. This could be better autogenerated.
  5. GeraudJanuary 26, 2007 @ 01:28 AM
    There's also this nice hack (courtesy of Rui Lopes) that was spread in some of his ports:
    x-generate-plist:
    	(${PORTSDIR}/Tools/scripts/plist -d -m ${MTREE_FILE} ${PREFIX} \
    	| ${SED} -E \
    		's,.*share/nls/.+$$,,g \
    		;s,.*etc/rc.d/.+$$,,g \
    		;s,^${GEM_CACHE}$$,%%GEM_CACHE%%,g \
    		;s,${GEM_DOC_DIR}(/.+)?$$,%%GEM_DOC_DIR%%\1,g \
    		;s,${GEM_LIB_DIR}(/.+)?$$,%%GEM_LIB_DIR%%\1,g \
    		;s,^${GEM_SPEC}$$,%%GEM_SPEC%%,g \
    		;s,^${GEMS_BASE_DIR}/(.+)$$,\1,g \
    		;s,^@dirrm (${SPEC_DIR}|${GEMS_DIR}|lib/ruby).*$$,,g \
    		' | ${TR} -s '\n') > temp-pkg-plist
    
    I used this a couple of times when I submitted some PRs.
    make depends
    mkdir -p ROOT/lib/ruby/gems/1.8/doc
    setenv ROOT $PWD/ROOT
    make PREFIX=$ROOT install x-generate-plist
    make PREFIX=$ROOT deinstall
    make clean
    [Check the temp-pkg-plist to be sure]
    make clean
    mv temp-pkg-plist pkg-plist
    
    Provided that the x-generate-plist target makes it into ruby-gems/Makefile.common, I guess the second part could be easily scripted. What do you think?
  6. GeraudJanuary 26, 2007 @ 01:30 AM
    Oops, seems like an extra "make clean" made it into my comment. Please discard it. :)
  7. MaledictusJanuary 29, 2007 @ 02:13 PM
    I have not that many gems installed, but so far have not found one not working. Anyway, stability is a valid point. I think this could be achieved by some kind of list which gems are stable for FreeBSD. This would prevent us from duplicating every gem into the ports while preserving the feature of having multiple versions installed. Thoughts?
  8. JonathanJanuary 29, 2007 @ 02:33 PM
    AFAIK I've seen MogileS not working at one point on FreeBSD. A list of stable versions sounds better. This would remove the duplication. I have to think about an implementation but the idea sounds good.
  9. GeraudJanuary 30, 2007 @ 02:19 AM
    A problem in Ruby land is that there is no canonical way to install applications yet. A few months ago I tried to gem-ify Typo (There are some PRs pending but I asked to put a hold on them). If you ask the community whether gem is The Way to install apps, you'll get three answers namely "Gems suck", "Gems rock" and "Depends..." Worse, if you have a module installed through the install.rb way, you seriously need to bend gem to make it recognize it as a dependency for a new gem you want to install. After hitting some PREFIX compliance problems, I lost interest in this issue though. I still have some files idling somewhere if someone is interested. On FreeBSD, you can cpan(p?) a Perl module and it will show in the ports as a `bsdpan-Foo-Bar-x.xx` making it accessible to pkg_remove and even portupgrade if the module makes it into the ports. Having a similar feature for gems would be great. This would allow to let only the unstable apps make it into the ports (when CONFIGURE or BUILD tweaks are required) and the stable ones out. But that's just my 0.02€. G.
  10. JonathanJanuary 31, 2007 @ 11:26 AM
    A similar implementation like done for CPAN would be great and sounds similar to what Maledictus was saying.
  11. Roman <romanbsd@yahoo.com>February 07, 2007 @ 10:26 AM
    There's a problem here if the gem installs something into $PREFIX/bin (e.g. /usr/local/bin).