Hello Dao Ha,
we're not on 7.4, but that sounds a lot like "Stateful navigation" to me (which is a feature in 7.3 and 7.31, too).
In those versions you can disable it on "Content Administration > Portal Display > Framework Page Configuration (Detailed Navigation)". Choose your Framework Page from the dropdownlist at the top of the site and then look through the properties for the Framework Page. In our version it's in the middle of the page.
See my answer here for a pretty picture: Disable the "last application visit " in top level navigation
Regards,
Steffi.