Python

Table of Contents

Packaging

setup.py

setuptools is the distribution package currently being used. 

#!/usr/bin/env python
try:
    from setuptools import setup
except ImportError:
    from distutils.core import setup

setup(
    name='foo',
    version='1.0',
    py_modules=['foo']
)

Then to create a "source distribution", we simply run the shell command 

python setup.py sdist

running sdist
running check
warning: check: missing required meta-data: url

warning: check: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) must be supplied

warning: sdist: manifest template 'MANIFEST.in' does not exist (using default file list)

warning: sdist: standard file not found: should have one of README, README.txt

file foo.py (for module foo) not found
writing manifest file 'MANIFEST'
creating foo-1.0
making hard links in foo-1.0...
foo-1.0
Creating tar archive
removing 'foo-1.0' (and everything under it)

sdist will create an archive file (e.g., tarball on Unix, ZIP file on Windows) containing your setup script setup.py, and your module foo.py. The archive file will be named foo-1.0.tar.gz (or .zip), and will unpack into a directory foo-1.0. 

ls dist

foo-1.0.tar.gz

And then to install this in a different environment/host: 

tar zxf dist/foo-1.0.tar.gz && python setup.py install

Built distribution

OKAY, I think I've got it:

  • Specify the files you want to include in MANIFEST.in
  • In your setup.py you set include_package_data to True
    • This will make it so that we also include data files specified in MANIFEST.in when building our bdist, not just as we get when we build our source distribution (sdist)
  • All the files specified in MANIFEST.in will now be included in your built distribution, yay!

Including files

This here is required when building a wheel distribution. 

setup(
    # ...
    package_data={"your_package": ["files_n_stuff"]},
    include_package_data=True,
)

I think that when building an egg, you can also use the data_files keyword.

Accessing files

When building a distribution as an egg, which is really just a zip-file, you can't just read any resource-file as you would a normal binary file since it's in zip-format.

Thus you have a couple of options:

  • Denpendent on whether or not your main directory ends in .egg or .zip, read as zipped file, otherwise read as usual
  • Best: use pkg_resources package from setuptools to handle and resources required
pkg_resources

Usually you want to do the following to read a file as you would normally do: 

import pkg_resources

filename = pkg_resources.resource_filename('your_package_name', 'path/to/file')

<path_to_file> needs to be specified relative to the package directory, but without using a relative path!! Yah feel me? So if you have your_package_name/data/stuff.txt then this becomes data/stuff.txt. Aight? Cool.

Even if you build an egg, in which case your files will actually be zipped, using resource_filename will first extract the resources and put them in a local cache folder, from which you will be served these files.

You also have methods like:

  • resource_stream

MANIFEST.in

  • Defines non-python files to be included
  • Generates the MANIFEST file

Most typically you'll use these patterns:

  • recursive-include <dir> <file_pat_1> <file_pat_2> ...
  • include path/to/file (here you can use relative paths and such)

Notes

  • Not sure what the difference between declaring non-python files in MANIFEST.in and data_packages in setup.py, but when I only used MANIFEST.in the file would not be included when I built an egg. It did work though, when I also included the declaration in data_packages. It even said in the SOURCES.txt inside the EGG-INFO that the file should be there, but naaaah. To check this stuff you simply extract the egg file (as it is in fact just a zip) and see if the file is actually there or not.
    • Seems like MANIFEST.in is merely for a source-distribution (sdist)
    • Actually, you need a MANIFEST.in AND specifying the package_data option in setup.py to include files.

Extending via C / C++

Packages