Removal of baseline images from matplotlib_baseline_images package #17793

SidharthBansal · 2020-06-29T19:16:57Z

PR Summary

Generated a matplotlib.tests wheel that can be uploaded to PyPI as a separate PyPI package ("distribution", in distutils parlance), to make it possible to install test data from PyPI.
This is useful e.g. for mplcairo, whose test suite relies on matplotlib's one.
Also, useful for Baseline image problem optimisation.
Created the test_data folder needed for additional test images apart from the baseline images
Created the logic for the generation of the baseline images on fresh install of matplotlib
In case, the baseline images exists exist and have changed then the logic for modification of the baseline images is created.
Creation of a plugin to generate images for both matplotlib and mpl_toolkits folders
Added related documentation.

Fixes #16447

attn: @anntzer

PR Checklist

Has Pytest style unit tests
Code is Flake 8 compliant
New features are documented, with examples if plot related
Documentation is sphinx and numpydoc compliant
Added an entry to doc/users/next_whats_new/ if major new feature (follow instructions in README.rst there)
Documented in doc/api/api_changes.rst if API changed in a backward-incompatible way

lib/matplotlib/testing/decorators.py

tacaswell · 2020-07-01T14:42:29Z

lib/matplotlib/testing/decorators.py

+                        stdout=subprocess.DEVNULL),
+                        universal_newlines=True)
+        # Clone mpl repo to tmpvenv and run pytests from new mpl repo created
+        subprocess.run(["git", "clone", str(mplRepoPath), "tmp/testenv"],


What version of git do we get worktree? I suspect that would be both faster and small footprint.

lib/matplotlib/testing/decorators.py

tacaswell · 2020-07-01T14:45:21Z

lib/matplotlib/testing/decorators.py

+                        stderr=subprocess.STDOUT,
+                        stdout=subprocess.DEVNULL,
+                        universal_newlines=True)
+        subprocess.run(["python", "-mpip", "install", "-e", mplRepoPath],


should this be the new venv python?

tacaswell · 2020-07-01T14:46:40Z

lib/matplotlib/testing/decorators.py

+                        stderr=subprocess.STDOUT,
+                        stdout=subprocess.DEVNULL,
+                        universal_newlines=True)
+        subprocess.run(["python", "-mpytest"],


Should also install pytest and all of the extra dependencies (pandas, ...) to make sure we generate all of the images or should we only match what the user has installed in their development environment?

I think installing all extra dependencies is a better option as testing aims to check integrity amongst the components too, which can't be achieved if only some of the installs are done.
First time all the images will be generated, later on we can move on specific missing installs and modifications of the baseline images.

What do you think @anntzer ?

SidharthBansal · 2020-07-08T15:21:31Z

Series of commands executed in the terminal:

python3 -mpytest lib/matplotlib/tests -m "baseline_image_generation_test"
mkdir lib/matplotlib/tests/baseline_images
cp -r result_images/test_agg lib/matplotlib/tests/baseline_images/
python3 -mpytest lib/matplotlib/tests -m "baseline_image_generation_test"

The first pytest command generates the baseline images with the baseline images not found error. Then, in the second pytest command, tests passes.
Similar approach for mpl_toolkits_tests need to be worked upon.

tacaswell · 2020-09-07T16:09:09Z

doc/devel/contributing.rst

+
+Baseline image generation
+-------------------------
+The idea is to eventually remove all of the images but we have options about where devs get the baselines from.


This needs a paragraph explaining the current state of things to the reader. To us (who have been thinking about this all summer), this makes perfect sense, but to a new contributor who does not even know we have baseline images checked into the repo this is going to be consufing.

…ectory and matplotlib directory

jklymak · 2021-05-08T20:45:17Z

@anntzer this seems like a good start, but is there any reason to keep it open?

anntzer · 2021-05-08T21:00:49Z

Let's close this for now; we can always revisit...

anntzer added the status: waiting for other PR label Jun 29, 2020

SidharthBansal force-pushed the removal_of_baseline_images_from_baseline_package branch 2 times, most recently from 25eb8c5 to 678b778 Compare June 30, 2020 15:03

anntzer reviewed Jun 30, 2020

View reviewed changes