Ahhh this took forever to figure out…

First find out which fonts you can use:

jpoz$ identify -list font

Find one you like, then set the font path:

g = Gruff::Line.new
g.font  = "/Library/Fonts/Arial Bold.ttf"

Stupid freakin’ ImageMagick