Customizing Private Repos

I was able to generate an initial diagram for a private repo via a PAT just fine!

However, when I tried to Customize or Regenerate I just see an error:
Could not fetch repository file tree. Repository might not exist, be empty or private.

Am I doing something wrong?