The python2.6 failure seems unrelated...

I think you need to be a little bit more descriptive about what you mean by letting tight_layout do the right thing.

I kicked the test to restart.

A less intrusive patch would be to change the default value from None -> gridspec.GridSpec(1, 1)?

Does this still do the right thing the user creates multiple figures using a method which does not otherwise add _subplotspec will result in multiple axes all thinking they are the only axes?

I am not super familiar with this section of the code base, but my knee-jerk reaction is that this patch will have nasty side effects (that we apparently don't test?).

Here is the resulting image without the patch:

and this is after the patch (which is basically looking the same as not using tight_layout):

So it seems this patch corrects the picture to match the result of not using tight_layout (which I meant above with "doing the right thing").

@tacaswell: I'm unsure, if there are other cases, where get_subplotspec_list could be called on, so I added it to the _AxesBase, so also Axes would have it inherited.
I don't understand the second question...

I also expected side effects and wanted to post this to see some failing tests...

A picture is worth a million words. I now see what you are talking about.

What I think is happening is that the location of the colorbar axes is being defined by fractions of the figure space outside of the subplot spec system. Tight layout adjusts the spacing of items defined by the subplot spec system. So, it is completely unaware of other axes elements that were added to the figure in other ways.

So, I think the important question here is whether or not this is really a bug in the first place? tight_layout is doing exactly what it is supposed to be doing -- finding an optimal layout for axes in the subplot grid spec. It just so happens that the user has other axes in the figure that are custom placed.

I would argue that this actually introduces a bug, because now the colorbar is no longer placed where I told it to be. Indeed, I do know of people who like to put their colorbars in custom locations, oftentimes inside a plot area, and would be pissed to no end if tight_layout all the sudden changed this.

Perhaps we need to improve the colorbar api to allow for different intentions?

If I understand your "bug" correctly, the colorbar inside the plot area should look like this:

This picture was created with the patch and tight_layout. All I did is I uncommented the line fig.subplots_adjust(right=0.8) from the script above.
So if you don't adjust the subplots you should obtain the colorbar at your location without moving the underlying plot away, isn't it?

For comparison, the picture without colorbar looks like this:

So it is indeed a bit moved/smaller than with the colorbar, but I'd say it's possible to put the colorbar inside the main picture or I didn't understand your bug...

indeed... I guess I am not fully understanding what is going on here, then.

This doesn't involve the colorbar API; the Figure.add_axes() method is making the colorbar Axes.
Colorbar already has a 'use_gridspec' option which is active when it is making its own axes by stealing space from another axes.
I suspect the real solution here might involve using a gridspec-based method of making the axes for the colorbar, instead of Figure.add_axes().

The example above was based on this example and I think that one should be(come) supported also with tight_layout.

The updated patch looks now much nicer and tight_layout now just uses another default value when a figure does not implement the gridspec yet. Furthermore, it cleans the code a bit as all is contained in tight_layout.py and the same warning as before is retained.

The last commit adds a test for the case of one picture and a colorbar. That one is identical to the output before, but makes sure to not break it in the future. My testcase for two figures unfortunately cuts off the labels a bit, so that one is not yet fully finished for regtesting and I like to leave that for a future improvement...

Just a ping to remind you we are alive and have not forgotten you.

I am quite sure that the checks were completing. Will try to look into the failure soonish.

It is probably the text. Sense you did this we a) now have a way to install a specific version of freetype for testing and b) have zero-tolerance testing on the image comparisons.

Should be fine now. It took me a while to notice that python setup.py test is not the same as running tests.py, maybe the former could call the latter, so that it is more obvious that not the right freetype fonts were used?

setup.py test is hoefully going away see #5434

Great, thanks

The appveyor HTTPError: 404 Client Error failure seems unrelated.

re appveyor: I agree :-)

Error: HTTPError: 404 Client Error: Not Found for url: https://conda.anaconda.org/conda-forge/win-64/msinttypes-r26-0.tar.bz2: https://conda.anaconda.org/conda-forge/win-64/msinttypes-r26-0.tar.bz2

-> conda can't find its packages...

Note: instead of removing the warnings, they could be made more informative as to how make the Axes compatible with tight_layout.

After rebasing, the new tests added above don't pass anymore.
I'll look into this.

I didn't remove the warnings, I just refactored them to another place. But I'll also look into making them more informative, while at it.

I'm not quite sure why this PR works, or if it does for anything other than the test. If the gridspecs don't know about each other, I'm not sure how they negotiate which should be placed where.

If we changed the gridspec hierarchy so that the gridspecs knew about the figure that contains them, then we could make this all work but thats not how GridSpec currently works by default.

For fun, you could try constrained_layout, which was meant for exactly this problem.

fig, axes = plt.subplots(nrows=2, sharex=True, sharey=True, constrained_layout=True)

CS = plt.contour(xi, yi, zi, 15, linewidths=0.5, colors='k')
CS = axes[1].contourf(xi, yi, zi, 15, cmap=plt.cm.rainbow,
                  vmax=abs(zi).max(), vmin=-abs(zi).max())
CS = axes[0].contourf(xi, yi, zi, 15, cmap=plt.cm.rainbow,
                  vmax=abs(zi).max(), vmin=-abs(zi).max())
cbar = fig.colorbar(CS, ax=axes, shrink=0.8)
plt.show()

With constrained_layout it looks better as the original code from above at least. Left is the constrained_layout version and on the right the original code from above with this branch:

You know for sure more about the gridspec machinery than I do. If I read the gridspec docs correctly, the gridspec of (1, 1) does add a single subgrid to every new item, which it did not have any grid specs for, which seems to at least work for the case from above.

I'm going to close this - this just allows tight_layout to be called, but tight_layout doesn't do anything, so its better to error rather than mislead the user that something is happening.