Speed up "from X import A; from X import B; from X import C; ..."

[gvanrossum]

Over in Pyston-land someone remarked that this:

from X import A, B, C, ..., Z

is much faster than

from X import A
from X import B
from X import C
.
.
.
from X import Z

We can't easily rewrite the latter as the former, because in edge cases the semantics are different (though maybe Black could take the liberty to do it anyway...?). But we could try to make the latter execute faster -- in theory everything needed is just in sys.modules['X'], though in practice there appear to be some caches that are reset between from X import statements but that are kept around between A, B, ...., Z. Maybe there's some wiggle room or a quicker way to check for cache invalidation (which is probably rare).

Note that X could well be a package path like foo.bar.baz. Also, if X is a package, then A, B, ..., Z could be submodules; but if X is a module, A...Z must be variables (or functions, classes, etc.) defined in X.

[stuaxo]

Forgetting edge cases for a moment (I assume there are many).

My initial thought was a cache that holds only the previously loaded module, this is the dumbest implementation (avoiding the lookup in sys.modules even).

[gvanrossum]

That would seem like a good idea. I haven't actually looked at the code to try to understand what's slowing this down though, there may be a reason there explaining why this is difficult. (Another reason may not be obvious from the code: it's possible that there's other (non-import) code between the first and the second import that might invalidate the cache.)

[stuaxo]

I'm just going to note a 3rd style I've seen in some larger projects (it may well a variant on from X import A, B, C, ..., Z but just in case:

from X import (
  A,
  B,
  C,
  ..., 
   Z
)

[sthagen]

@stuaxo this variant is in my experience often used to make the linters happy ... and black called on a file with a from module import comma, separated, list line exceeding the set line length automatically rewrites that way (using the parentheses).

[gvanrossum]

There is no difference in execution speed between

from X import A, B, ..., Z

and

from X import (
    A,
    B,
    ...,
    Z
)

(at least assuming we're executing unmarshalled bytecode -- it is the same, as you can tell by experimenting with dis.py.)

[markshannon]

"someone remarked that X is much faster than Y" is a bit vague. Any actual numbers?