It may well be he has issues being separated from you and feels this is a way to get back home. *( wild guess)

But you took toys away once and it worked, so when it started you took toys away...