Is this Python issue worth solving?

I am facing some challenges with python. They are far from urgent, and can probably be safely ignored, so my question should read: Is this a problem worth solving?

About a month ago, I posted about issues upgrading Mint to version 20. The issues were related to python, I could not fix them, so I returned to 19.3, which works just fine.

Since then, I have found a few postings with tiny fixes, biggest among these are upgrading pip to 20.2.2 and installing pip3. PIP runs in Python 3.6.9
The default python is 2.7.17 and python3 is 3.6.9.
Then I found this article:
https://dev.to/serhatteker/how-to-upgrade-to-python-3-7-on-ubuntu-18-04-18-10-5hab

I did it in the following order:
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.6 1
sudo apt-get install python3.7
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.7 2

When I ran sudo apt update && sudo apt upgrade I found I was able to recreate the issues with the Mint upgrade, that is, a module could not be found called apt_pkg. I ran pip3 check and it said it could not find a source for missing dependencies (apologies that this all happened without copying the information).
On rebooting, the entire Cinnamon experience was inoperable. From CLI I restored to yesterday’s snapshot, which is something I can live with.
Now, when I run sudo update-alternatives --config python3 it gives me only one choice, python 3.6.9

Is this problem worth spending time on? I never use python directly, and really like Mint 19.3, so doing nothing has its own appeal.

Python is crap. Don’t waste time on it.

I would submit that it’s not Python that’s the problem, but rather the Debian (and derivatives’) packaging of Python. The Debian Python maintainers, to be blunt, are on crack, and their crazy decisions to shunt Python modules into a python3/dist-packages/ directory only under the /usr/ and /usr/local/ prefixes breaks a lot more than it solves.

On Fedora 32 the current default Python is python3-3.8.5-5.fc32, which installs as /usr/bin/python3.8 and /usr/bin/python3. Its sys.path loads from /usr/lib64/python3.8, /usr/lib/python3.8/site-packages, and /usr/lib64/python3.8/site-packages.

But we also have packages available for all of the following:

  • python39-3.9.0 (installs /usr/bin/python39, packages in /usr/{lib,lib64}/python3.9)
  • python37-3.7.8 (installs /usr/bin/python37, packages in /usr/{lib,lib64}/python3.7)
  • python36-3.6.11 (installs /usr/bin/python36, packages in /usr/{lib,lib64}/python3.6)
  • python35-3.5.9 (installs /usr/bin/python35, packages in /usr/{lib,lib64}/python3.5)

You can install any or all of those packages on a system with the default python3 installed, and everything will work fine. That’s because each version doesn’t try to take over for or share installed packages with any of the others — there’s no common /usr/lib/python3 directory full of compiled extensions that are going to break any Python other than the system default one when they try to use it, and /usr/bin/python3 never means anything other than /usr/bin/python3.8.

If we want to use any Python packages with any of the non-default Python releases, they have to either be installed with pip and managed that way, or built and installed locally targeting the appropriate Python version.

IMHO you should definitely not even try to use /usr/bin/alternatives to point /usr/bin/python3 to any Python other than the system-supported default, @cliffsloane, because the system Python modules (especially the compiled ones in /usr/lib/python3/dist-packages) will, in practice and despite the Debian maintainers’ apparent hopes to the contrary, generally only work with the Python they were packaged for. Changing the meaning of /usr/bin/python3 on the system will just confuse and break lots of system packages, as you saw.

If you want to alias python3 to /usr/bin/python3.7.2 in your shell, there’s nothing wrong with that… but you should definitely always make sure that anything on the system that uses /usr/bin/python3 in a shebang line or etc. will get the Python it’s expecting.

You can install a pip for an alternate Python under your user account (for instance, by downloading the get-pip.py bootstrapping script and running it with an explicit version, e.g:

# Get pip
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py

# Install pip for python3.7, which will install files/directories including...
# * $HOME/.local/lib/python3.7/site-packages/pip
# * $HOME/.local/bin/pip3.7
# * $HOME/.local/bin/pip3
/usr/bin/python3.7.2 get-pip.py --user

# Probably a good idea
rm $HOME/.local/bin/pip3

# Will install $HOME/.local/lib/python3.7/site-packages/wheel
# (Assuming $HOME/.local/bin/ is in your $PATH)
pip3.7 install --user wheel

# Will use the just-installed package to unpack a wheel
/usr/bin/python3.7.2 -m wheel unpack ${name}.whl

The $HOME/.local/bin/pip3 thing is unfortunate; I’ve never figured out if there’s a way to suppress that.

2 Likes

Extremely helpful, qualified and thorough explanation of one of the many aspects that makes Python unsuitable for what it is generally considered in the developer part of the world. Thank you for that.

1 Like

Thank you Frank. But you could have saved a lot of time and effort just saying “No it isn’t”

2 Likes