Skip to content
This repository was archived by the owner on Nov 28, 2018. It is now read-only.
This repository was archived by the owner on Nov 28, 2018. It is now read-only.

URL for remote are too long for pip when caching is enabled #16

@alexd-siaras

Description

@alexd-siaras

When acting as a caching proxy, the URLs referring to a remote proxy are very long. For example, python-saharaclient will yield the following URL: http://sheeva:8176/packages/source/p/python-saharaclient/python-saharaclient-0.7.0.tar.gz?remote=http%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fp%2Fpython-saharaclient%2Fpython-saharaclient-0.7.0.tar.gz

When pip caching is enabled, the tarball will be attemped to be saved as: 'http%3A%2F%2Fsheeva%3A8176%2Fpackages%2Fsource%2Fp%2Fpython-saharaclient%2Fpython-saharaclient-0.7.0.tar.gz%3Fremote%3Dhttp%253A%252F%252Fpypi.python.org%252Fpackages%252Fsource%252Fp%252Fpython-saharaclient%252Fpython-saharaclient-0.7.0.tar.gz.content-type'

This file name is longer than the allowed 255 filename characters in most modern linux filesystems, and pip will result in an error:

2014-05-12 11:55:04.653 | Downloading/unpacking python-saharaclient>=0.6.0 (from tempest==2014.1.dev3940.g744617c)
2014-05-12 11:55:04.655 | Storing download in cache at /var/cache/pip/http%3A%2F%2Fsheeva%3A8176%2Fpackages%2Fsource%2Fp%2Fpython-saharaclient%2Fpython-saharaclient-0.7.0.tar.gz%3Fremote%3Dhttp%253A%252F%252Fpypi.python.org%252Fpackages%252Fsource%252Fp%252Fpython-saharaclient%252Fpython-saharaclient-0.7.0.tar.gz
2014-05-12 11:55:04.658 | Cleaning up...
2014-05-12 11:55:04.660 | Exception:
2014-05-12 11:55:04.663 | Traceback (most recent call last):
2014-05-12 11:55:04.666 | File "/usr/local/lib/python2.7/dist-packages/pip-1.4.1-py2.7.egg/pip/basecommand.py", line 134, in main
2014-05-12 11:55:04.668 | status = self.run(options, args)
2014-05-12 11:55:04.671 | File "/usr/local/lib/python2.7/dist-packages/pip-1.4.1-py2.7.egg/pip/commands/install.py", line 236, in run
2014-05-12 11:55:04.675 | requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
2014-05-12 11:55:04.677 | File "/usr/local/lib/python2.7/dist-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 1092, in prepare_files
2014-05-12 11:55:04.680 | self.unpack_url(url, location, self.is_download)
2014-05-12 11:55:04.683 | File "/usr/local/lib/python2.7/dist-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 1238, in unpack_url
2014-05-12 11:55:04.686 | retval = unpack_http_url(link, location, self.download_cache, self.download_dir)
2014-05-12 11:55:04.688 | File "/usr/local/lib/python2.7/dist-packages/pip-1.4.1-py2.7.egg/pip/download.py", line 630, in unpack_http_url
2014-05-12 11:55:04.691 | cache_download(cache_file, temp_location, content_type)
2014-05-12 11:55:04.693 | File "/usr/local/lib/python2.7/dist-packages/pip-1.4.1-py2.7.egg/pip/util.py", line 582, in cache_download
2014-05-12 11:55:04.695 | fp = open(target_file+'.content-type', 'w')
2014-05-12 11:55:04.698 | IOError: [Errno 36] File name too long: '/var/cache/pip/http%3A%2F%2Fsheeva%3A8176%2Fpackages%2Fsource%2Fp%2Fpython-saharaclient%2Fpython-saharaclient-0.7.0.tar.gz%3Fremote%3Dhttp%253A%252F%252Fpypi.python.org%252Fpackages%252Fsource%252Fp%252Fpython-saharaclient%252Fpython-saharaclient-0.7.0.tar.gz.content-type'

Now, granted that this is a limitation of pip, however if could be avoided if the "?remote" parameter did not contain the entire URL, but perhaps a hash of the remote URL.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions