The logic puzzle is solved because the prisoners gain information by observing that others have not yet left.
From “Can you solve the famously difficult green-eyed logic puzzle? - Alex Gendler” — TED-Ed
see it said at 2:32
Did you know?
Surprising facts — each proven at the exact second of its video