I contacted MAGMA and for the record:
Windows apparently did something weird with their definition of a "long" integer in the 64bit operating system, so MAGMA on Windows is only supported in 32bit. The maximum memory MAGMA can draw as a 32bit application is 1.3GB. As this is a system limit and not an internal limit `GetMemoryLimit()` will still return $0$.
The only solution is to run MAGMA in 64bit on either OSX or Linux.