Well, it turns out I'm a programmer.
I've been working for a little while on PRisk, a Chrome extension that highlights items in a PR that might need extra attention if you're the reviewer. At the top of a pull request, you'll see overall risk factors. If the author of the PR doesn't know the code base, that's a risk. If there's a lot of complexity amongst the diffs, that's a risk. If there are a lot of files, that's a risk. If there are a lot of changes, that's a risk.
For each diff within a pull request, PRisk gets more specific. Is the code perhaps too complex? Is it a file that gets a lot of activity? Are there a lot of contributors to the file? Is the file relatively young? And finally, the extension figures out some likely owners of the file, so you could ping them as a reviewer. I have more plans for this section, but this is a good start.
The Clunky UI
There is a UI, but it needs some work. Depending on the settings for a given repo, you might need to generate an access token for PRisk. Once you've generated it, click on the P that appeared when you installed the extension and fill in your username and the access token. That token will get used for API calls against private repos.