About API discoverability.

[KelSolaar]

Quoting @brandondube here:

I think the issue is that too much is at the top level. Colour (@0.3.10) exposes 510 things at the top level. Numpy and Scipy both expose ~600 things, and have way more features than Colour.

For example, for computations that have different techniques, e.g. munsel_value, every variant is exposed. It would be preferred to just have colour.munsel_value with a method= or technique= in the function signature and a sensible default. Probably 20% of the top-level API can be removed by doing that, and part of the zen of python is that there should be only one way of doing something.

Colour is big and it will grow bigger, with that in mind, there is a concern about discoverability of API features and it is a good time to discuss about it.

One issue is that we ship a quite big dataset and collections of functions, e.g. colourspaces that are exposed at the root level and inflates the root namespace objects count.

Our documentation is currently 99% generated from the code and it makes it hard to discover API features.

I would be keen on knowing what is people preferred way to get a good grasp of large APIs. My way is usually to first go through documentation and/or examples/tutorials and codebase second and potentially direct introspection in the interpreter third. However I have never introspected Scipy or Numpy code directly in the Python interpreter but I have done that for smaller project.

My gut feeling is that we should improve the documentation and maybe spend some time manually write more stuff instead of relying on the 99% automatically generated one. Adding examples like we did recently in the README.rst file is a good step in the right direction but is not enough.

We can also cull some objects from the root namespace.

@MichaelMauderer any thoughts on that?

[KelSolaar]

Something I forgot to add is that the metadata system will be very helpful for that and is a reason why it is being implemented in the first place.

[MichaelMauderer]

I agree and think that moving functionality from the top-level to submodules could be a good move.
We already have the submodule structure, so we wouldn't actually have to do much in that regard, but deprecate the top-level access. We could have a look at whether the current sub-modules are the best and most sensible way to group and access functionality though (but I am not aware of any issues there atm).

For me, API discovery usually happens through online documentation, but also docstrings available in IDES are very important. In general, I am all for additional example pages for the online-docs which can be read to get an idea for most of the usages.

[MichaelMauderer]

It would be neat to have "usage stats" to decide what should remain top-level though. I wonder whether there are tools to crawl GitHub to see how an API is used.

[KelSolaar]

deprecate the top-level access

I'm not sure how we will be able to go that, there is maybe a way to intercept the root namespace getattr, we will have to investigate that.

It would be neat to have "usage stats" to decide what should remain top-level though.

Yeah I'm a bit clueless here really.

[KelSolaar]

In regard to top level module deprecation, this could be useful:

• https://stackoverflow.com/questions/2447353/getattr-on-a-module
• https://github.com/pallets/werkzeug/blob/master/werkzeug/__init__.py

[KelSolaar]

I have implemented a deprecation scheme in a soon-to-be-published branch and I'm working through the list of stuff to cull here: https://docs.google.com/spreadsheets/d/1X136CFx7fpG6ZMy4kS0PYNAA2XeSGTNEv6DDGapPDxA/edit?usp=sharing

[KelSolaar]

@MichaelMauderer: I have implemented the deprecation scheme and the new top level API looks like what is in the aforementioned spreadsheet. We can/should finetune what should be exposed but basically spreadsheet legend is a follows:

• Access: Access is moved to relevant sub-package.
• Ignore: Ignored, i.e. stays as is.
• Keep: The actual new top level API.
• Remove: Will be entirely removed from public API, even in sub-package.

I have not looked at the sub-packages themselves and not planned to do it now but we could do the same later.

The branch is here: https://github.com/colour-science/colour/tree/feature/deprecation and is accounting for the various PR branches to be reviewed/merged, the idea is basically to intercept the attribute access and emit relevant warnings as needed. colour/__init__.py file is a good example but I have also implemented some deprecation jazz in colour/plotting/__init__.py and colour/colorimetry/spectrum.py

[KelSolaar]

And an example run:

import colour
import colour.plotting
from collections import namedtuple

print colour.chromatic_adaptation_forward_CMCCAT2000
print colour.oetf_sRGB
print colour.tristimulus_weighting_factors_ASTME202211
try:
    print colour.SpectralMapping
except AttributeError as error:
    print(error)

print colour.SMPTE_C_RGB_COLOURSPACE.name
print colour.oetf_BT1886

print colour.plotting.planckian_locus_CIE_1960_UCS_chromaticity_diagram_plot

spd = colour.SpectralPowerDistribution()
try:
    print spd.data
except AttributeError as error:
    print(error)

try:
    for s in spd:
        print(s)
except AttributeError as error:
    print(error)

print spd.title

print spd.clone()

/Users/kelsolaar/Documents/Development/colour-science/colour/colour/utilities/verbose.py:140: ColourWarning: "colour.chromatic_adaptation_forward_CMCCAT2000" object access will be removed in a future release.
  warn(*args, **kwargs)
/Users/kelsolaar/Documents/Development/colour-science/colour/colour/utilities/verbose.py:140: ColourWarning: "colour.oetf_sRGB" object access is deprecated and will change to "colour.models.oetf_sRGB" in a future release.
  warn(*args, **kwargs)
/Users/kelsolaar/Documents/Development/colour-science/colour/colour/utilities/verbose.py:140: ColourWarning: "colour.tristimulus_weighting_factors_ASTME202211" object access will be removed in a future release.
  warn(*args, **kwargs)
/Users/kelsolaar/Documents/Development/colour-science/colour/colour/utilities/verbose.py:140: ColourWarning: "colour.SMPTE_C_RGB_COLOURSPACE" object has been renamed to "colour.models.SMPTE_240M_COLOURSPACE".
  warn(*args, **kwargs)
/Users/kelsolaar/Documents/Development/colour-science/colour/colour/utilities/verbose.py:140: ColourWarning: "colour.oetf_BT1886" object has been renamed to "colour.models.eotf_reverse_BT1886".
  warn(*args, **kwargs)
/Users/kelsolaar/Documents/Development/colour-science/colour/colour/utilities/verbose.py:140: ColourWarning: "colour.plotting.planckian_locus_CIE_1960_UCS_chromaticity_diagram_plot" object has been renamed to "colour.plotting.planckian_locus_chromaticity_diagram_plot_CIE1960UCS".
  warn(*args, **kwargs)
<function chromatic_adaptation_forward_CMCCAT2000 at 0x106ec70c8>
<function oetf_sRGB at 0x18109c3230>
<function tristimulus_weighting_factors_ASTME202211 at 0x1810c5ab90>
"colour.SpectralMapping" object has been removed from the API.
SMPTE 240M
<function eotf_reverse_BT1886 at 0x18109b4488>
<function planckian_locus_chromaticity_diagram_plot_CIE1960UCS at 0x1815b26e60>
"SpectralPowerDistribution.data" object has been removed from the API.
"SpectralPowerDistribution.__iter__" object has been removed from the API.
SpectralPowerDistribution (103442504592)
[None None]