Strangely, it has become harder and harder to find this information via search engines. The results that come up are pointing to WinUI 3 solutions (which is different from UWP) or Windows Phone.
The location is roughly around here:
C:\Program Files (x86)\Windows Kits\10\DesignTime\CommonConfiguration\Neutral\UAP\10.0.22621.0\Generic
The path will vary depending on what version of the Windows SDK you have installed.
Hopefully, this article preserves this information on the web for future UWP developers, and for a future me as well.