Sharing gem source paths

[dtognazzini]

This shows how I'd like to reuse Bundler's path across project-local gems.

The structure

The application depends on blorgh and lorde, from Gemfile.lock:

    blorgh (0.0.1)
      lorde
      rails (~> 4.0.10)
      throrg
    lorde (0.0.1)
      rails (~> 4.0.10)
      yayaya
    throrg (0.0.1)
      rails (~> 4.0.10)

A dependency tree (all arrows denote "depends on"):

require_gemfile_example
    |          |
    v          v  
  blorgh----->lorde
    |          |
    v          v
  throrg     yayaya

The top-level subsystems are in engines/

lorde uses a gem internal to just lorde: yayaya. yayaya is in engines/lorde/gems/

The mechanics

1. Each level that is composed of internal gems (e.g. require_gemfile_example and lorde) has a sources.gemfile that is required (ala require_gemfile) by the internal gems comprising the "level". Internal gems require this file to pull in gem source specifications for resolving external gems dependencies. What?

   So....

   • require_gemfile_example/sources.gemfile is required by all internal engines/gems blorgh, throrg and lorde (lorde is special, read on). For each of the internal gems (i.e. blorgh, throrg and lorde) this line says "I don't know how to resolve external gems, leave it to the thing (require_gemfile_example) that requires me."
   • lorde is also a gem composed of internal gems, so it too defines a sources.gemfile that is used by its internal gem, yayaya, which says the same thing: "I don't know how to resolve external gems, leave it to the thing (lorde) that requires me." lorde/sources.gemfile only knows how to resolve dependencies local to its own gem sources. It doesn't know how to resolve dependencies external to itself. But... its parent (require_gemfile_example) does know how to resolve these dependencies. lorde/sources.gemfile delegates external dependency resolution to its parent.
   • Each level that is composed of internal gems (e.g. require_gemfile_example and lorde) uses its own sources.gemfile in its Gemfile to avoid copying-and-pasting: require_gemfile_example's Gemfile, lorde's Gemfile

2. Each level that is composed of internal gems and is required by something else (i.e. lorde) defines a paths.gemfile which specifies path sources of where to find gem dependencies internal to the "level". What?

   So....

   • lorde is composed of yayaya and lorde is required by both require_gemfile_example and blorgh. lorde defines a paths.gemfile that specifies where lorde's dependencies can be found.
   • lorde is the only "level" like this in this project. However, if, for some reason, something wanted to aggregate the outer project with another project, require_gemfile_example itself could define a paths.gemfile and this pattern could support this aggregation.

The benefits

1. It scales. The system is both infinitely decomposable and infinitely composable.
2. Contained blast radius. Decomposing lorde into a 1000 internal gems doesn't affect any other code (Gemfile, gemspec, sources.gemfile, paths.gemfile) outside of lorde.
3. Deferred external dependency resolution. The resolution of dependencies external to the project are deferred to the top level (i.e. require_gemfile_example), which is in the position of authority for deciding how the dependencies of the project as a whole are resolved.