With that information, I think my problem was a mixture between an
already running roscore, rviz and another node. Because those remained
running when I restarted the playback of the bag file, new data arrived
which was slightly in the past, at times that were already observed.

rviz is one of the few things that will handle backward jumps in time, but you need to have rosbag broadcasting on the /clock topic:

rosbag play --clock

rviz will then detect backwards jumps in time and reset itself accordingly.

You can also manually reset rviz's sense of time using the "Reset" button on the Time panel.

Josh