After starting to using rails-assets to manage dependencies, we noticed that asset compilation was taking 500% longer than normal -- up to 5 minutes, sometimes. After two days of digging, I found that the assets_hash method was calling Sprockets' each_logical_path method, which iterates over every directory, even the ones in our new vendored bower assets. So for stuff like jquery-ui, with tons of subdirectories and images, ruby-stylus was building up an assets_hash that included all of them.
I'm not sure there's a fix for this in this library's current implementation, but I would definitely hope it gets considered a bug. I'd imagine that when the assets_hash method was implemented, this edge case was simply not thought of.