Holistic Engineering

A random assortment of shit with sprinkles.

Gem Activation and You, Part 3: System Packaging

| Comments

The short of it — packaging ruby gems is hard.

System packaging is a touchy subject to begin with; it’s arguably one of the more prominent cultural divides I’ve encountered coming to ops from a developer background. The notion of packaging something custom before it gets installed, versus installing it with an automating tool such as chef or puppet is a constant source of debate.

This is a rationale on when you should and should not package a ruby gem as a system package, and provide alternative solutions for those situations where it is not a good idea. You should probably read my first two articles before continuing down this one for some of the topics we’ll cover.

The Fundamental Problem with System Packaging

The problem is very simple: gem activation does not care about your package manager, and your package manager doesn’t care about ruby gems.

Gem activation works very similar to the dynamic linker on your system. The app itself will pick out the library it needs, based on what’s available on the system. You can see this in your packaging system: you’ll have libmysqlclient10 and libmysqlclient15, and mysql and some_other_mysql_program using one or the other. Your system picks these libraries based on what the programs are linked to. Rubygems works similarly, but your system package manager knows nothing about this. Technically it knows nothing about your dynamic linker either, but it’s been architected to handle it.

As of this writing, there is likely not a system packager that exists that can handle rubygems properly with regard to multiple concurrent gems. I’m speaking from plenty of experience here.

This is because rubygems can handle multiple versions of a gem, and your system packager only cares about one of them. You have libdbi-ruby in debian and it has one version — 0.4.0. If 0.5.0 is released, either debian releases a new package and deprecates the old one, or they don’t move. Either way, rubygems is given one option.

Ruby programs don’t work like this. The other articles go into detail as to why.

And this is all before the problem of packaging the right ruby installation for your application.

Either way, here are some scenarios you may run into while considering system packaging.

Scenario 1: Many apps with a common set of gems

This is the time to system package. However, this is rare in practice — most apps have uncommon dependencies with each other and this all needs to be accounted for, especially with regard to deep dependencies of common libraries such as the json gem. A 1.8.x versus 1.5.x is going to give one or both of those apps hives if you’ve only system packaged one of them. Strong advice: use a tool like fpm to manage this problem and test regularly.

Scenario 2: Many apps with a diverse set of gems

This is the best opportunity to use bundler and bundle exec if you can. This doesn’t always make sense however — if you need a “bare” ruby, consider omnibus-ruby if you can invest the effort. The point is, you will save yourself endless hell by completely isolating your dependencies. Rails apps definitely fit into this ballpark as well.

Scenario 3: One app with a very specific set of gems

omnibus-ruby. omnibus-ruby. omnibus-ruby.

Keep off your system ruby entirely if you can manage it. It will be cleaner, safer, and less hassle in the long run.

Packaging is cool, yo

You may have been burned by system packages in the past, or system rubies, etc. This series tries to cover the issues in detail so you can understand the fundamental problems and wield bundler, fpm and omnibus-ruby to glory while mastering the problem in a constructive, less frustrating way.

I hope it’s become a valuable resource.

Comments