This repository is for the python package skyhookdm-py. This package contains the beginnings of a python interface for SkyhookDM, which stores and manages tabular data in the Ceph object storage system.
The python library is simply skyhookdm, and the source code is available in the aptly named
directory.
At this moment, this repository is not the official python API for SkyhookDM. But, this repository is hopefully where my implementation will be coalesced with the existing python API for SkyhookDM.
Dependency management for this repository is handled by poetry. The number of dependencies is purposefully kept to a minimum at the moment, and there are a few dependencies that must be kept carefully in sync with Skyhook:
The reason for this is that these dependencies involve data formats, and some versions can deprecate features or break backwards compatibility. While this isn't going to be the case for every update, minor version updates should still be tested before transitioning to them.
Python version management is handled by pyenv. The main purpose is simply to make it
easy to isolate potentiall differences between python versions, but there is a secondary, very
important reason for needing to explicitly manage the python version: compatibility with the
rados library. The storage subsystem used by Ceph is called RADOS, and the
python rados library is a binding to the librados code. As of this writing, the
newest version of python that librados supports is python 3.6.9 (to my knowledge). Also, newer
versions of numpy and arrow have dependencies on python >= 3.8, so the environment can be tricky to
change in some cases.
Finally, Flake8 is the python linter that my environment is setup to use, and so I have included a .flake8 configuration file in this repository that holds my particular settings.
I personally use VIM for my editor, and here is my vim configuration. I use flake8 with the syntastic VIM plugin, in case you're wondering how I use flake8.