MailScanner scans incoming mail for spam, and both incoming and outgoing mail for viruses and dangerous file attachments. However, it is designed primarily to protect users on your server from incoming spam and viruse and is not a good solution for an outgiong spam problem.
There are many better ways to detect and prevent outgoing spam. Some suggestions:
1) Use a malware detection system such as cxs (ConfigServer eXploit Scanner) to detect exploits that may be used for sending spam, and check any reported files and remove them if necessary.
2) Configure relay tracking in csf, monitor the alerts that are sent when the relay limit is exceeded, and check the reported accounts for vulnerable scripts.
3) If you can get the header of one of the outgoing spams, search the exim log to find out what script is sending the mail.
4) Check this webpage on our site which gives some suggestions for finding spammers on your server:
http://www.configserver.com/free/spammers.html#outbound
5) Set WHM > Tweak Settings > Mail > Max hourly emails per domain to a low figure such as 100 to ensure that thousands of messages are not sent out before you detect it.
6) If you have users that forward email on to yahoo or hotmail accounts (for example), make sure that they are educated NOT to mark forwarded mail as spam in their yahoo/hotmail account, as that could lead to your server being blacklisted. MailScanner can help with this if you configure it to delete all spam messages.