http://www.esumsoft.com/products/pop-peeper/
QuotePOP Peeper is an email notifier that runs in your Windows taskbar and alerts you when you have new email on any of your accounts.
DL : http://www.esumsoft.com/download/?prod=pppro
The zip installer is portable AFAIK.