Large-Address awareness on 64 bit systems

[Corinna Vinschen]

Hi gang,

I just read something interesting on MSDN which only affects 64 bit systems:

"If the application has the IMAGE_FILE_LARGE_ADDRESS_AWARE flag set in the
image header, each 32-bit application receives 4 GB of virtual address
space in the WOW64 environment. If the IMAGE_FILE_LARGE_ADDRESS_AWARE
flag is not set, each 32-bit application receives 2 GB of virtual address
space in the WOW64 environment."

So, with this flag set, a WOW64 process has the full 4 Gigs address
space and none of that is taken by the OS? That sounds cool.

That means, if we make all Cygwin binaries large address aware, somebody
using a 64 bit system could rebase all Cygwin DLLs, except for the
Cygwin DLL itself into the virtual memory area beyond 0x80000000. Given
that this memory area is not at all used by Windows itself, it's free
for usage by Cygwin alone. This would affect the memory allocation not
only for Cygwin DLLs, but also for mmaps and for thread stacks, which
both use the MEM_TOP_DOWN flag.

If I get this right, this should ease the fork pain due to memory collisions
on 64 bit systems quite a lot.

So I made an experiment, using the Cygwin DLL from CVS to get the new
extended /proc/$PID/maps output. I converted all .exe files in /bin to
have the large-address awareness flag on, then I rebased all DLLs into
the high memory area, kind of like this:

C:\> cd \cygwin\bin
C:\cygwin\bin> copy dash.exe ..
C:\cygwin\bin> copy peflags.exe ..
C:\cygwin\bin> path C:\cygwin\bin;%path%
C:\cygwin\bin> ..\dash
$ ../peflags --bigaddr=1 *.exe
$ rebaseall -b 0xe0000000

This really works. If you check the /proc/$PID/maps file, you see how
the Cygwin DLLs, the PEB and TEBs, (non-fixed) mmaps and thread stacks
are placed into the large address area.

The downside is that moving the DLLs into the large address area potentially
breaks executables which are not large-address aware. They will run fine
as long as they don't try to fork. I tried that with bash. It's not fun:

[~]$ peflags --bigaddr=0 /bin/bash
[~]$ bash
bash: fork: retry: Resource temporarily unavailable
bash: fork: retry: Resource temporarily unavailable
bash: fork: retry: Resource temporarily unavailable
bash: fork: retry: Resource temporarily unavailable
bash: fork: Resource temporarily unavailable

bash-4.1$ exit
[~]$ peflags --bigaddr=1 /bin/bash
[~]$ bash

***@vmw2k8r2 ~/tests
$

This obviously also affects all non-distro executables, such as self-
compiled stuff. All of them need the --bigaddr flag set, too.

Therefore it might be quite tricky to maintain a sane system, given that
gcc creates executables with the large address awareness flag switched
off, and given the fact that the flag is reset for a distro executable
every time it gets updated via setup.

However, if anybody feels confident to test this on the own machine,
I'd be curious if that has the desired effect of letting fork work
more reliable. I'm not sure I have the required use cases to test
this sufficiently.


Corinna

[Ryan Johnson]

This post might be inappropriate. Click to display it.

[Corinna Vinschen]

This post might be inappropriate. Click to display it.

[Corinna Vinschen]

Post by Corinna Vinschen
That's why we could need a better rebase algorithm. Some kind of
rebaseall executable or perl script which scans all Cygwin DLLs and
notes the used and free address slots as well as the size of the DLLs
and then, afterwards, uses this knowledge to move the DLLs around, as
collision-free as possible. It would also allow to rebase just a single
DLL into a as yet free memory slot instead of having to specify the
address on the command line.

I just noticed what a bad idea it is to convert rebaseall into a perl
script. Scratch that. I guess the right thing to do is to extend the
rebase executable with this functionality.


Corinna

[Ryan Johnson]

Post by Corinna Vinschen

In general, how does ASLR affect things? Do heaps and system dlls risk
wandering into high memory?

Try it. AFAICS, Windows does not use the large address area at all.

Hmm. That could be a double-edged sword: If we rebase everything into
high memory and also mark the dlls ASLR-aware, windows might very well
stuff everything back into the crowded low address space.

Unfortunately I don't have time this week to play around with this, or
I'd be reporting actual results instead of speculation...

Ryan

[Corinna Vinschen]

Post by Ryan Johnson

Post by Corinna Vinschen

In general, how does ASLR affect things? Do heaps and system dlls risk
wandering into high memory?

Try it. AFAICS, Windows does not use the large address area at all.

Hmm. That could be a double-edged sword: If we rebase everything
into high memory and also mark the dlls ASLR-aware, windows might
very well stuff everything back into the crowded low address space.

Sure. I didn't mean to imply to test both things together. Maybe
I should have chose another thread for the ASLR suggestion. The ASLR
flag might help in case of collisions, while the large address for DLLs
might help to avoid collisions.

Right now I was just wondering if you tested (or would like to test)
your collision testcase with the ASLR flag switched on in the DLL
headers.

Post by Ryan Johnson
Unfortunately I don't have time this week to play around with this,
or I'd be reporting actual results instead of speculation...

No hurry.


Corinna

[Corinna Vinschen]

Hi Ryan,

Post by Ryan Johnson
- deliberately dlopen-ing two dlls with the same base address and
trying to fork

http://cygwin.com/ml/cygwin-apps/2011-06/msg00047.html

Did you try the ASLR flag for your test DLLs? Does it help, perhaps?
If the ASLR flag works as I understand it, the DLLs should be moved
into a less dangerous memory area and be then fixed there.


Corinna

[Ryan Johnson]

Post by Corinna Vinschen
Hi Ryan,

Post by Ryan Johnson
- deliberately dlopen-ing two dlls with the same base address and
trying to fork

http://cygwin.com/ml/cygwin-apps/2011-06/msg00047.html
Did you try the ASLR flag for your test DLLs? Does it help, perhaps?
If the ASLR flag works as I understand it, the DLLs should be moved
into a less dangerous memory area and be then fixed there.

ASLR works pretty well for dlls having static use patterns (including
apps which always open the same dlls), from what I can tell. The reason
I hadn't done it for that test app is because dlls created and loaded
dynamically by apps like gcc bootstrap wouldn't have it set either, and
I wanted a consistent way to reproduce the erratic fork problems that
they see as a result of the odd base address collision. Similarly, I
purposefully avoided rebasing or ASLR-ing my cygwin install while
testing the fork patches.

Ryan

[Yaakov (Cygwin/X)]

Post by Corinna Vinschen
That means, if we make all Cygwin binaries large address aware, somebody
using a 64 bit system could rebase all Cygwin DLLs, except for the
Cygwin DLL itself into the virtual memory area beyond 0x80000000. Given
that this memory area is not at all used by Windows itself, it's free
for usage by Cygwin alone. This would affect the memory allocation not
only for Cygwin DLLs, but also for mmaps and for thread stacks, which
both use the MEM_TOP_DOWN flag.
If I get this right, this should ease the fork pain due to memory collisions
on 64 bit systems quite a lot.

Sounds great.

Post by Corinna Vinschen
This obviously also affects all non-distro executables, such as self-
compiled stuff. All of them need the --bigaddr flag set, too.

It would be helpful if there were a short option for --bigaddr.

Post by Corinna Vinschen
Therefore it might be quite tricky to maintain a sane system, given that
gcc creates executables with the large address awareness flag switched
off, and given the fact that the flag is reset for a distro executable
every time it gets updated via setup.

Are there any downsides to large address awareness, particularly for
32bit systems or those with less than 4GB? IOW is there any reason not
to switch this on by default in gcc/ld?

Post by Corinna Vinschen
However, if anybody feels confident to test this on the own machine,
I'd be curious if that has the desired effect of letting fork work
more reliable. I'm not sure I have the required use cases to test
this sufficiently.

But I think I do. :-) I have rebased and reflagged my system
accordingly, and so far (running GNOME 3 desktop) so good. I'll keep
you posted.


Yaakov

[Corinna Vinschen]

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen
That means, if we make all Cygwin binaries large address aware, somebody
using a 64 bit system could rebase all Cygwin DLLs, except for the
Cygwin DLL itself into the virtual memory area beyond 0x80000000. Given
that this memory area is not at all used by Windows itself, it's free
for usage by Cygwin alone. This would affect the memory allocation not
only for Cygwin DLLs, but also for mmaps and for thread stacks, which
both use the MEM_TOP_DOWN flag.
If I get this right, this should ease the fork pain due to memory collisions
on 64 bit systems quite a lot.

Sounds great.

Post by Corinna Vinschen
This obviously also affects all non-distro executables, such as self-
compiled stuff. All of them need the --bigaddr flag set, too.

It would be helpful if there were a short option for --bigaddr.

Post by Corinna Vinschen
Therefore it might be quite tricky to maintain a sane system, given that
gcc creates executables with the large address awareness flag switched
off, and given the fact that the flag is reset for a distro executable
every time it gets updated via setup.

Are there any downsides to large address awareness, particularly for
32bit systems or those with less than 4GB? IOW is there any reason not
to switch this on by default in gcc/ld?

I don't see any problem. Cygwin does not use the high bit of a pointer
as a flag for something and hopefully no other DLL or applications does
that.

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen
However, if anybody feels confident to test this on the own machine,
I'd be curious if that has the desired effect of letting fork work
more reliable. I'm not sure I have the required use cases to test
this sufficiently.

But I think I do. :-) I have rebased and reflagged my system
accordingly, and so far (running GNOME 3 desktop) so good. I'll keep
you posted.

Great, thanks! As I just wrote in my reply to Ryan, I even rebased the
Cygwin DLL(*) and it runs fine for me.


Corinna

(*) First I called rebaseall -b 0xe0000000(**), then I copied over the
Cygwin DLL to cygwin1.X.dll, rebased it to 0xe1000000, exited
Cygwin and moved cygwin1.X.dll over to cygwin1.dll.

(**) Btw., to avoid collisions in apps using mmap a lot, it might be
better to rebase to a lower address. One has to keep in mind how
big the large address area is, it's the same size *again* a process
has on a default installation. Therefore something like rebaseall
-b 0xa0000000 might be more helpful.

[Corinna Vinschen]

Post by Corinna Vinschen

Post by Yaakov (Cygwin/X)
Are there any downsides to large address awareness, particularly for
32bit systems or those with less than 4GB? IOW is there any reason not
to switch this on by default in gcc/ld?

I don't see any problem. Cygwin does not use the high bit of a pointer
as a flag for something and hopefully no other DLL or applications does
that.

Well, there's of course still a chance that an application or the Cygwin
DLL itself uses signed ints or longs to compare addresses or to do
adress arithmetic. I hope there's no such problem lurking in a dark and
musty corner of the sources...


Corinna

[Yaakov (Cygwin/X)]

Post by Corinna Vinschen

Post by Corinna Vinschen

Post by Yaakov (Cygwin/X)
Are there any downsides to large address awareness, particularly for
32bit systems or those with less than 4GB? IOW is there any reason not
to switch this on by default in gcc/ld?

I don't see any problem. Cygwin does not use the high bit of a pointer
as a flag for something and hopefully no other DLL or applications does
that.

Well, there's of course still a chance that an application or the Cygwin
DLL itself uses signed ints or longs to compare addresses or to do
adress arithmetic. I hope there's no such problem lurking in a dark and
musty corner of the sources...

That would only be triggered when rebasing to the upper addresses,
right? My question was simply about adding

%{!shared:--large-address-aware}

to gcc's link specs.


Yaakov

[Corinna Vinschen]

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen

Post by Corinna Vinschen

Post by Yaakov (Cygwin/X)
Are there any downsides to large address awareness, particularly for
32bit systems or those with less than 4GB? IOW is there any reason not
to switch this on by default in gcc/ld?

I don't see any problem. Cygwin does not use the high bit of a pointer
as a flag for something and hopefully no other DLL or applications does
that.

Well, there's of course still a chance that an application or the Cygwin
DLL itself uses signed ints or longs to compare addresses or to do
adress arithmetic. I hope there's no such problem lurking in a dark and
musty corner of the sources...

That would only be triggered when rebasing to the upper addresses,
right? My question was simply about adding
%{!shared:--large-address-aware}
to gcc's link specs.

I think that would be the right thing to do.


Corinna

[Yaakov (Cygwin/X)]

Post by Corinna Vinschen

Post by Yaakov (Cygwin/X)
But I think I do. :-) I have rebased and reflagged my system
accordingly, and so far (running GNOME 3 desktop) so good. I'll keep
you posted.

Great, thanks! As I just wrote in my reply to Ryan, I even rebased the
Cygwin DLL(*) and it runs fine for me.

I still have been getting the occasional "fork: Resource temporarily
unavailable" error with make, but I haven't seen the "unable to remap
dll" error. I have also been getting SIGABRT from throwing exceptions
across C++ DLLs, with GDB pointing to RtlUpdateClonedSRWLock() in ntdll,
but I was seeing that sometimes before this as well. (This is with
1.7.9; recent snapshots haven't been working for me but I haven't had
the time to track down why.)

I'd say that it's not a panacea, but a significant improvement; it
certainly hasn't hurt, and with all the DLLs in Ports, we can use all
the address space we can get.


Yaakov

[Corinna Vinschen]

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen

Post by Yaakov (Cygwin/X)
But I think I do. :-) I have rebased and reflagged my system
accordingly, and so far (running GNOME 3 desktop) so good. I'll keep
you posted.

Great, thanks! As I just wrote in my reply to Ryan, I even rebased the
Cygwin DLL(*) and it runs fine for me.

I still have been getting the occasional "fork: Resource temporarily
unavailable" error with make, but I haven't seen the "unable to remap
dll" error. I have also been getting SIGABRT from throwing exceptions
across C++ DLLs, with GDB pointing to RtlUpdateClonedSRWLock() in ntdll,
but I was seeing that sometimes before this as well. (This is with
1.7.9; recent snapshots haven't been working for me but I haven't had
the time to track down why.)

Oh, please try to track it down. I'm running the latest from CVS on
W7 32 bit and 2008 R2 64 bit daily, and I don't have problems. If the
new stuff to avoid collisions works as designed, the chance to see the
fork problem should be much reduced.

Having said that, even without rebasing to large addresses I had a
strange problem a few days ago. For some reason perl was suddenly
broken. Trying to start perl generated a stackdump from within the
constructor loop in per_module::run_ctors in dll_init.cc. Rebasing
didn't help. Reinstalling helped. What could that be? I'm pretty
sure it has something to do with rebasing.

Post by Yaakov (Cygwin/X)
I'd say that it's not a panacea, but a significant improvement; it
certainly hasn't hurt, and with all the DLLs in Ports, we can use all
the address space we can get.

Good to know. Thanks for testing!


Corinna

[Ryan Johnson]

Post by Corinna Vinschen

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen

Post by Yaakov (Cygwin/X)
But I think I do. :-) I have rebased and reflagged my system
accordingly, and so far (running GNOME 3 desktop) so good. I'll keep
you posted.

Great, thanks! As I just wrote in my reply to Ryan, I even rebased the
Cygwin DLL(*) and it runs fine for me.

I still have been getting the occasional "fork: Resource temporarily
unavailable" error with make, but I haven't seen the "unable to remap
dll" error. I have also been getting SIGABRT from throwing exceptions
across C++ DLLs, with GDB pointing to RtlUpdateClonedSRWLock() in ntdll,
but I was seeing that sometimes before this as well. (This is with
1.7.9; recent snapshots haven't been working for me but I haven't had
the time to track down why.)

Oh, please try to track it down. I'm running the latest from CVS on
W7 32 bit and 2008 R2 64 bit daily, and I don't have problems. If the
new stuff to avoid collisions works as designed, the chance to see the
fork problem should be much reduced.
Having said that, even without rebasing to large addresses I had a
strange problem a few days ago. For some reason perl was suddenly
broken. Trying to start perl generated a stackdump from within the
constructor loop in per_module::run_ctors in dll_init.cc. Rebasing
didn't help. Reinstalling helped. What could that be? I'm pretty
sure it has something to do with rebasing.

Hmm. I noticed a similar problem a while back where a statically-linked
dll loading at the wrong address caused run_dtors to call addresses
which are la-la land in the child process (sometimes even the dtor list
itself was garbage). However, we don't run_ctors inforkee, and code in
dll_list::alloc verifies that {data,bss}x{start,end} are where they belong.

If you're able to reproduce the problem, can you figure out whether it's
a bad function address being called or a valid ctor crashing? My gut
feeling is that communication with some other dll is failing, perhaps
registering debug/unwind info with libgcc_s, or some perl module phoning
home to the mothership.

Ryan

[Corinna Vinschen]

Post by Ryan Johnson

Post by Corinna Vinschen

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen

Post by Yaakov (Cygwin/X)
But I think I do. :-) I have rebased and reflagged my system
accordingly, and so far (running GNOME 3 desktop) so good. I'll keep
you posted.

Great, thanks! As I just wrote in my reply to Ryan, I even rebased the
Cygwin DLL(*) and it runs fine for me.

I still have been getting the occasional "fork: Resource temporarily
unavailable" error with make, but I haven't seen the "unable to remap
dll" error. I have also been getting SIGABRT from throwing exceptions
across C++ DLLs, with GDB pointing to RtlUpdateClonedSRWLock() in ntdll,
but I was seeing that sometimes before this as well. (This is with
1.7.9; recent snapshots haven't been working for me but I haven't had
the time to track down why.)

Oh, please try to track it down. I'm running the latest from CVS on
W7 32 bit and 2008 R2 64 bit daily, and I don't have problems. If the
new stuff to avoid collisions works as designed, the chance to see the
fork problem should be much reduced.
Having said that, even without rebasing to large addresses I had a
strange problem a few days ago. For some reason perl was suddenly
broken. Trying to start perl generated a stackdump from within the
constructor loop in per_module::run_ctors in dll_init.cc. Rebasing
didn't help. Reinstalling helped. What could that be? I'm pretty
sure it has something to do with rebasing.

Hmm. I noticed a similar problem a while back where a
statically-linked dll loading at the wrong address caused run_dtors
to call addresses which are la-la land in the child process
(sometimes even the dtor list itself was garbage). However, we don't
run_ctors inforkee, and code in dll_list::alloc verifies that
{data,bss}x{start,end} are where they belong.
If you're able to reproduce the problem, can you figure out whether
it's a bad function address being called or a valid ctor crashing?
My gut feeling is that communication with some other dll is failing,
perhaps registering debug/unwind info with libgcc_s, or some perl
module phoning home to the mothership.

Ok, I'll have a deeper look if that happens again.


Corinna

[Yaakov (Cygwin/X)]

Post by Yaakov (Cygwin/X)
I have also been getting SIGABRT from throwing exceptions
across C++ DLLs, with GDB pointing to RtlUpdateClonedSRWLock() in ntdll,
but I was seeing that sometimes before this as well. (This is with
1.7.9; recent snapshots haven't been working for me but I haven't had
the time to track down why.)

I've managed to narrow this down somewhat. With Ports' gcc-4.5.3 built
with binutils-2.21, libstdc++6 and libgcj11 break wrt throwing
exceptions if rebased *above* their natural ImageBases (IOW the ones
they were created with at link time), but not if rebased to a lower
address.

Purely speculation at this point, but I have to wonder if this is
somehow left over from a previous problem with libstdc++6, where it
completely broke if rebased at all:

http://sourceware.org/ml/binutils/2009-12/msg00080.html

Otherwise, so far so good.


Yaakov

[Corinna Vinschen]

Post by Yaakov (Cygwin/X)

Post by Yaakov (Cygwin/X)
I have also been getting SIGABRT from throwing exceptions
across C++ DLLs, with GDB pointing to RtlUpdateClonedSRWLock() in ntdll,
but I was seeing that sometimes before this as well. (This is with
1.7.9; recent snapshots haven't been working for me but I haven't had
the time to track down why.)

I've managed to narrow this down somewhat. With Ports' gcc-4.5.3 built
with binutils-2.21, libstdc++6 and libgcj11 break wrt throwing
exceptions if rebased *above* their natural ImageBases (IOW the ones
they were created with at link time), but not if rebased to a lower
address.

That sounds weird.

Post by Yaakov (Cygwin/X)
Purely speculation at this point, but I have to wonder if this is
somehow left over from a previous problem with libstdc++6, where it
http://sourceware.org/ml/binutils/2009-12/msg00080.html

Maybe somebody on the binutils list has a clue?


Corinna