Upgrade your standard fittings to a pad bolt.

This would include the hinges being bolted to both the shed door and panel that its attached to, making the door as secure as possible