On any larger (50+) album, if you choose to play a slideshow with it, at some point the slides will skip from being in order to a random spot earlier. In some cases, the slideshow will continue, in others it will play a few earlier slides and quit. This has been reported to Apple since TV OS 9.0, as you can see in 9.2 it is still there. (It occurs in three different Photos libraries on three different Macintosh machines, all running the current (at that time) OS X, iTunes and Photos versions and has persisted through all versions since the Apple TV 4th generation was released.)
This video demonstrates it. It has the numbers 1-10, capital letters A-Z, and lower case letters a-z. It is in a new photo library with only one album, but the same occurs in larger photo libraries with 1000s of photos and 50+ albums. It doesn’t seem to be impacted by the time between slides or the transitions (e.g. Classic, Ken Burns etc).

Here is a 2 minute video showing the problem:

And one of several Apple Support forum discussions:
https://discussions.apple.com/message/29961343

Occurs in Apple tvOS / iOS 9.0 – 9.2 (March 21, 2016 current)

 

Update: The May 13, 2016 Apple TV/tvOS update seems to have fixed the bug based on a quick test.  Let’s hope so!