Successful brands who launch their Facebook Page through an agency will always expect to be guaranteed some 'ownership' of it. It's not uncommon for companies of a certain size to switch agency every few years, simply to embrace a fresh skill-set or on the whim of internal politics. I've been in that position myself and dutifully handed over usernames and passwords for the public entities I controlled to the hungry new whippersnappers.
As someone who acknowledges the value of being "behind the scenes", it would never cross my mind to withhold access to a vehicle I had been paid to build for someone else should they wish to take the wheel from me. Not just for reasons of commercial vanity but more so because you owe it to the users, fans and members of whatever you have created who should never be concerned with conflicts on an administration level.