Installing Python from Source (Linux)

Package repositories are great, but when it comes to Python they usually only distribute one particular version. If you're a serious Python developer, then at some point you'll need a version that's not available through the package manager and you'll have little choice but to install one from source. But have no fear! It's not as daunting as it may seem, even if youre doing it for the first time.

1. Download Source Code

Head over to https://www.python.org/downloads/ and grab yourself the tarball and signature for whichever version of Python you'll be installing. I haven't installed 3.10 yet so that's the version I'll install in this guide, but the steps are identical for all supported versions.

Verify that the checksum is correct

$ md5sum Python-3.10.0.tar.xz 
3e7035d272680f80e3ce4e8eb492d580  Python-3.10.0.tar.xz

and that the signature matches.

$ gpg --verify Python-3.10.0.tar.xz.asc
gpg: assuming signed data in 'Python-3.10.0.tar.xz'
gpg: Signature made Mon 04 Oct 2021 09:56:39 AM AKDT
gpg:                using RSA key CFDCA245B1043CF2A5F97865FFE87404168BD847
gpg: Good signature from "Pablo Galindo Salgado <pablogsal@gmail.com>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: A035 C8C1 9219 BA82 1ECE  A86B 64E6 28F8 D684 696D
     Subkey fingerprint: CFDC A245 B104 3CF2 A5F9  7865 FFE8 7404 168B D847

See the GPG getting started guide for details on how to import public keys into your chain of trust. Links to the signers keys are found at the bottom of the Python downloads page.

Now extract the tarball and change directory to the extracted folder.

$ tar -xf Python-3.10.0.tar.xz
$ cd Python-3.10.0

2. Install Development Packages

Python is able to build without installing any additional dependencies, however, interpreter history and some builtin modules won't work unless you install some extra packages. Don't worry though, you can uninstall them again after the build has completed.

Ubuntu:

$ sudo apt install build-essential pkg-config \
      libbz2-dev libffi-dev libgdbm-dev liblzma-dev \
      libncurses5-dev libreadline6-dev libsqlite3-dev \
      libssl-dev tk-dev uuid-dev zlib1g-dev

CentOS / RHEL:

$ sudo yum install yum-utils
$ sudo yum-builddep python3

Fedora:

$ sudo dnf install dnf-plugins-core
$ sudo dnf builddep python3

Arch:

$ sudo pacman -Sy gcc make pkgconf bzip2 libffi \
      gdbm lib32-xz ncurses readline sqlite \
      openssl tk lib32-util-linux zlib

3. Compile and Install

Python is compiled in typical C fashion by running a configure script followed by make. When running configure, you have the opportunity to customize some compile options. Here is what I would recommend:

$ ./configure --enable-optimizations \
              --enable-shared \
              --enable-loadable-sqlite-extensions \
              --prefix /usr/local \
              LDFLAGS=-Wl,-rpath=/usr/local/lib

• --enable-optimizations will enable Profile-Guided Optimizations (PGO) which will may cause the build to take significantly longer, but will result in a better optimized binary.
• --enable-shared will build a shared Python library. Python builds distributed through a package manager typically have this enabled as you will need it for any software that expects to be able to link to Python.
• --enable-loadable-sqlite-extensions will compile the _sqlite module to support loadable extensions. If you don't enable this, you may encounter import errors when trying to import dependencies that use the builtin sqlite module.
• --prefix determines where Python will be installed during the final step.
• LDFLAGS is used to add a non-standard search path for the shared library. This is only necessary if you set --prefix to a non-standard directory (such as /opt).

Now compile the code with Make. You can make use of multiple cores by passing the -j flag to get Make to run tasks in parallel.

$ make -j8

Depending on the version of Python you're installing, this could take anywhere between 5 and 45 minutes (Python 3.7 runs an especially large test suite which takes a long time).

Test that the build worked by running the python executable. Since we built it using a shared library and we have not properly installed it to the system yet, we need to tell the dynamic linker to search the current directory for the libpython*.so file by setting the LD_LIBRARY_PATH environment variable.

$ LD_LIBRARY_PATH=. ./python
Python 3.10.0 (default, Nov 15 2021, 20:19:38) [GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> print("Hello, World!")
Hello, World!

Finally, install Python as an alternate installation

$ sudo make altinstall

and verify that it worked

$ python3.10
Python 3.10.0 (default, Nov 15 2021, 20:19:38) [GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> print("Hello, World!")
Hello, World!

That's it! If you have any questions or comments, send them to contact@askaholic.io!

Troubleshooting

If you get to the final step and you see this error when trying to run python

$ python3.10
python3.10: error while loading shared libraries: libpython3.10.so.1.0: cannot open shared object file: No such file or directory

then you probably need to add the LDFLAGS argument to your configure command. Luckily you don't need to recompile everything, just the final produced binary.

$ ./configure ... LDFLAGS=-Wl,-rpath=/usr/local/lib
$ rm ./python
$ make
$ sudo make altinstall