This guide explain how to setup Exim to be able to send emails from a LAMP server without needing to setup postfix or sendmail.
Though something in my setup seems to change my FQDN or similar so email sending stops working after a while. Will continue to investigate.