Good news: I have solved the puzzle of missing steps with fast rotary encoder rotations. It was a piece of inefficient code, which has now been rearranged using AVR's Pin Change INTerrupts ("PCINT").
The 96 states / 24 cycles per rotation encoder is now read properly: it doesn't miss a step.
Now the FR/IFR firmware has to be adapted to a new form of interaction...