Actually, we have noticed the situation improved with history set to 5. It was not entirely how we expected it to work (we thought lower history to be more “strict” - but it is not).
It is hard to say without seeing the data, but I think it is because of the featurized slots. As soon as they are set, they are used as features for all next time steps