I'm 50/50 on actually merging this.

I think we probably need to either merge this or potentially do a pin on setuptools_scm for this branch, otherwise all backport PRs will fail Azure CI, which may make realizing real failures on backport harder.

We could go an alternative route of not pinning but explicitly installing newer setuptools which would unbreak CI but could leave a thorny path for people installing from source on the backport branch (That's one of the main reasons I just pinned on main)

But doing that would not (officially) be a development/version support change on a point release, though in practice there is some added constraint in compatibility (though unfortunately only flagged/enforced at runtime)

I am less worried about the numpy build time issues than I am by overly pinning deps.

This also only matters for source installations?

It is breaking CI on other backports: #26939

Yes, it only matters for source installations (but that includes CI)

Seems very low risk; let's try it, to get things moving again.