I have gone back to a save a couple of times. Each time, it starts out fine, but then artifacts no longer appear. I noticed that it's not only the containers, but trash artifacts no longer appear on the ground in new areas also.
I didn't go back to before Jhereth Deeps, maybe I'll try that next.