Sunday, September 2, 2012

virtualenv and pip

Update: virtualenv 1.8.1 released on September 3 would fix this problem with --never-download.

pip 1.2 was released about 10 hours ago. It is so new and shiny that it breaks my current deployment process that has worked fine with pip 1.1.

Before today, I have a requirements.txt file that specifies all dependencies for my Python web app. My deployment script basically creates a virtual environment with virtualenv and then does a 2-pass pip'ing like this:

pip install --download=pipcache -r requirements.txt
pip install --find-links file://absolute/path/to/pipcache -r requirements2.txt

requirements2.txt is derived from requirements.txt to eliminate all URLs. For example, requirements.txt has this line,

http://code.google.com/p/pymysql/source/browse/?r=66#egg=PyMySQL-0.3

while requirements2.txt has this:

PyMySQL-0.3

The reason for two passes is to cache downloaded packages and ignore PyPI. The option --download-cache does help with the first goal but always need PyPI.

That deployment process was working fine, until 10 hours ago. Suddenly pip errors out in the second pass, saying PyMySQL cannot be found. Looking at the log file, I suspect that pip does a case sensitive string comparison between pymysql and PyMySQL. Where pip gets the string pymysql from, I do not know.

Then I immediately looked for a way to force virtualenv to pin pip 1.1. Of course, it's not possible. Which is kind of ironic because virtualenv encourages pip over setuptools, yet latest pip fails. A case of oversight, I guess.

So, the solution is to let virtualenv pick the latest pip as usual, and then use pip to degrade itself to 1.1.

No comments:

Post a Comment