Depends on what you mean much slower? Only network ? If you just been in the folder background process for getting icons or extended columns data (depending on what columns you show) may still be running
Enabling filesytem log in Core settings and you see what it does and can compare difference..
There is more work if the path is already cached. But that should not really be that noticeable, processing if a cached items should be keep or not should be really fast. 50.000 items are nothing for a CPU.