First of all, one short description of outsourcing. It's a relatively new term and from my experience most candidates that I see for an interview have heard of it but don't know what it means or don't know how to explain it clearly.
Outsourcing means giving one of your internal services to an external provider. What can you give? Accounting, HR, IT and many more. What do I mean by "giving"? Signing a contract with an external provider who with his internal means will provide a service that we as a company no longer do internally.
What advantages does outsourcing have? Well, the service is provided by an already-trained team, the provider has all the software needed internally and they are responsible for licenses and legal requirements. Moreover, outsourcing can be a huge cost saver if the provider team is located in a low cost country like India, Romania, Bulgaria and so on.
So, should you outsource your payroll?
No, if you are a small company with 5 employees. Payroll for you can be easily done in an Excel file or even in Open Office Calc (free spreadsheet software) by someone handling HR or accounting. There's no point in paying additional money to an external provider.
Yes, if you are a larger company - 50, 100 employees or even more. There's not a clear line to be drawn between numbers. You don't outsource when you have exactly 50 employees. It's just your decision. Outsourcing the payroll can be beneficial for a larger company because it saves you the headache. The provider has a proper software, a proper team and the proper relationship with the authorities, all of this making payroll simple for you.