Please post the complete logs of where this smtp response happened - else it's not possible to find out which parameter needs to be tuned (and if it's actually a PMG setting that's at fault (and not the downstream server)
Else the maximum message size can be set in GUI->Configuration->Mail Proxy->Options (Message size(bytes))
I hope this helps!