RFC: Cygwin 64 bit?

[Corinna Vinschen]

Hi guys,


I'm not sure if you agree, but as far as I can see, 32 bit systems are
more and more reduced to a niche market, namely Netbooks and other very
small systems. On the Desktop, 32 bit is declining fast, in the server
segment it's practically dead.

Given this, I'm wondering how much future Cygwin has if we stick to
32 bit. I think it will be pretty limited. In fact, we're probably
rather late in the game.

Lately I read a bit more about how 32 and 64 bit processes can interact
with each other on Windows, and it seems we could come up with a 64 bit
DLL which shares all important information with its 32 bit counterpart,
so that we could run 64 and 32 bit processes in parallel on a 64 bit
system acting as a single system. This would be especially important,
given the fact that 64 bit Cygwin applications will be pretty rare in
the beginning.

As far as I can see what we have to do in about this order is

- Discuss certain basics. This is probably the most crucial step.
For instance:

- What name should the 64 bit DLL have?
- Where should 64 bit binaries and libs go?
- Do we define "long" as 32 bit or 64 bit type?
- What defines should a 64 bit Cygwin compiler define?
- What Windows headers and link libs do we use?

- Create a x86_64-pc-cygwin cross toolchain.

- Create x86_64 replacements for x86 code and in general try to make the
Cygwin code 64 bit clean where it isn't so far.

- Decide how we can integrate 64 bit stuff into the distro. Will we have
a 32 bit and a distinct 64 bit distro? Or, should we stick to a single
distro? If so, how do we separate 64 and 32 bit stuff? What is the
best package layout?

- Change setup to allow installation of 64 bit stuff on 64 bit systems.

This is a big project which can only work if we have help and support
from the community. Before we can even contemplate to start discussing,
I would like to learn:

- How much interest do you have in a 64 bit Cygwin?

- How much interest do you have to help to make 64 bit Cygwin real?

- What part of the project would most interest you to help? Coding
Cygwin? Documentation? Setup? Toolchain? You name it.


Thanks in advance,
Corinna

[JonY]

Post by Corinna Vinschen
Hi guys,
I'm not sure if you agree, but as far as I can see, 32 bit systems are
more and more reduced to a niche market, namely Netbooks and other very
small systems. On the Desktop, 32 bit is declining fast, in the server
segment it's practically dead.
Given this, I'm wondering how much future Cygwin has if we stick to
32 bit. I think it will be pretty limited. In fact, we're probably
rather late in the game.
Lately I read a bit more about how 32 and 64 bit processes can interact
with each other on Windows, and it seems we could come up with a 64 bit
DLL which shares all important information with its 32 bit counterpart,
so that we could run 64 and 32 bit processes in parallel on a 64 bit
system acting as a single system. This would be especially important,
given the fact that 64 bit Cygwin applications will be pretty rare in
the beginning.

Yes please, 64bit Cygwin will be very nice.

Post by Corinna Vinschen
As far as I can see what we have to do in about this order is
- Discuss certain basics. This is probably the most crucial step.
- What name should the 64 bit DLL have?

I think they should still use the "cyg" prefix, the libtool people was
against it when I suggested using a new prefix for 64bit mingw.

Post by Corinna Vinschen
- Where should 64 bit binaries and libs go?

Its best that if we can separate 32bit/64bit completely, so there won't
be any conflicting files.

Post by Corinna Vinschen
- Do we define "long" as 32 bit or 64 bit type?

I suggest 32bit, they'll be some awkwardness accessing w32api at the
Cygwin backend if they're 64bit.

Post by Corinna Vinschen
- What defines should a 64 bit Cygwin compiler define?

__CYGWIN__ and __CYGWIN64__? Just so to allow programmers to tell that
its cygwin and its 64bit.

Post by Corinna Vinschen
- What Windows headers and link libs do we use?

Bootstrap with mingw-w64? :)

Post by Corinna Vinschen
- Create a x86_64-pc-cygwin cross toolchain.

Yeah, I suppose newlib has to be ported first.

Post by Corinna Vinschen
- Create x86_64 replacements for x86 code and in general try to make the
Cygwin code 64 bit clean where it isn't so far.
- Decide how we can integrate 64 bit stuff into the distro. Will we have
a 32 bit and a distinct 64 bit distro? Or, should we stick to a single
distro? If so, how do we separate 64 and 32 bit stuff? What is the
best package layout?

I suggest separating them, since I don't imagine they can interact with
each other anyway.

Post by Corinna Vinschen
- Change setup to allow installation of 64 bit stuff on 64 bit systems.
This is a big project which can only work if we have help and support
from the community. Before we can even contemplate to start discussing,
- How much interest do you have in a 64 bit Cygwin?
- How much interest do you have to help to make 64 bit Cygwin real?
- What part of the project would most interest you to help? Coding
Cygwin? Documentation? Setup? Toolchain? You name it.

I'd like to help, but I don't know much on win32 API.

[Corinna Vinschen]

Post by JonY

Post by Corinna Vinschen
Hi guys,
I'm not sure if you agree, but as far as I can see, 32 bit systems are
more and more reduced to a niche market, namely Netbooks and other very
small systems. On the Desktop, 32 bit is declining fast, in the server
segment it's practically dead.
Given this, I'm wondering how much future Cygwin has if we stick to
32 bit. I think it will be pretty limited. In fact, we're probably
rather late in the game.
Lately I read a bit more about how 32 and 64 bit processes can interact
with each other on Windows, and it seems we could come up with a 64 bit
DLL which shares all important information with its 32 bit counterpart,
so that we could run 64 and 32 bit processes in parallel on a 64 bit
system acting as a single system. This would be especially important,
given the fact that 64 bit Cygwin applications will be pretty rare in
the beginning.

Yes please, 64bit Cygwin will be very nice.

Post by Corinna Vinschen
As far as I can see what we have to do in about this order is
- Discuss certain basics. This is probably the most crucial step.
- What name should the 64 bit DLL have?

I think they should still use the "cyg" prefix, the libtool people was
against it when I suggested using a new prefix for 64bit mingw.

Right, but that wasn't what I meant. Sorry for being unclear. I was
talking about the name of the Cygwin DLL. For instance, if we decide
that it must reside in the /bin directory, it must have a different name
than the 32 bit dll, for instance, cygwin64-1.dll. If we decide that
all 64 bit applications and DLLs reside in a parallel directory, it
could have the same name, for instance, /bin64/cygwin1.dll.

But let's not go into too much detail yet.

Post by JonY

Post by Corinna Vinschen
- Create a x86_64-pc-cygwin cross toolchain.

Yeah, I suppose newlib has to be ported first.

Right, I forgot about that one. But newlib works rather well for many
systems, so that shouldn't be much of a problem.

Post by JonY

Post by Corinna Vinschen
This is a big project which can only work if we have help and support
from the community. Before we can even contemplate to start discussing,
- How much interest do you have in a 64 bit Cygwin?
- How much interest do you have to help to make 64 bit Cygwin real?
- What part of the project would most interest you to help? Coding
Cygwin? Documentation? Setup? Toolchain? You name it.

I'd like to help, but I don't know much on win32 API.

That sounds kind of funny, given that you're the mingw64 maintainer :)
Anyway, the Win32 API and the native NT API are already in use just
fine. It's the porting from 32 to 64 bit which is the problem. The
calls will only marginally change, if at all.

There's also the fact that there isn't only core Cygwin to work on.
Newlib, for instance. Or any of the stuff I mentioned above. There's
a lot to do, I fear :}


Corinna

[JonY]

Post by Corinna Vinschen
Right, but that wasn't what I meant. Sorry for being unclear. I was
talking about the name of the Cygwin DLL. For instance, if we decide
that it must reside in the /bin directory, it must have a different name
than the 32 bit dll, for instance, cygwin64-1.dll. If we decide that
all 64 bit applications and DLLs reside in a parallel directory, it
could have the same name, for instance, /bin64/cygwin1.dll.
But let's not go into too much detail yet.

I was thinking that we have them totally separated, so we don't need to
deal with DLL name clashes. Eg C:\Cygwin for 32bit and C:\Cygwin64 for
64bit. No need to invent bin32 or bin64.

Post by Corinna Vinschen

Post by JonY

Post by Corinna Vinschen
- Create a x86_64-pc-cygwin cross toolchain.

Yeah, I suppose newlib has to be ported first.

Right, I forgot about that one. But newlib works rather well for many
systems, so that shouldn't be much of a problem.

There's that hairy LP64 vs LLP64 issue, personally, I'd prefer the LLP64
route since Cygwin is a translation layer and will need to communicate
with Windows at the backend, but I suspect many more will want the LP64
route for Posix software compatibility.

I suppose there could be a minimalist Cygwin fork of the win32api to
make it LP64 compatible. Maybe a thunk/translator layer will be easier.

Post by Corinna Vinschen
That sounds kind of funny, given that you're the mingw64 maintainer :)
Anyway, the Win32 API and the native NT API are already in use just
fine. It's the porting from 32 to 64 bit which is the problem. The
calls will only marginally change, if at all.
There's also the fact that there isn't only core Cygwin to work on.
Newlib, for instance. Or any of the stuff I mentioned above. There's
a lot to do, I fear :}

Well, I do work with win32 API from time to time, but I'm unfamiliar
with the using anything more complicated, eg NTDLL and its native API
friends.

I can try checking what works or what doesn't for 64bit newlib, I hope
syscall emulation is just copy and pasting.

[Corinna Vinschen]

Post by JonY
I can try checking what works or what doesn't for 64bit newlib, I hope
syscall emulation is just copy and pasting.

It's just calling functions like _open, _stat, etc. These are defined
in Cygwin as entry points for newlib. Should be rather simple.


Corinna

[Ryan Johnson]

Post by JonY

Post by Corinna Vinschen
Right, but that wasn't what I meant. Sorry for being unclear. I was
talking about the name of the Cygwin DLL. For instance, if we decide
that it must reside in the /bin directory, it must have a different name
than the 32 bit dll, for instance, cygwin64-1.dll. If we decide that
all 64 bit applications and DLLs reside in a parallel directory, it
could have the same name, for instance, /bin64/cygwin1.dll.
But let's not go into too much detail yet.

I was thinking that we have them totally separated, so we don't need to
deal with DLL name clashes. Eg C:\Cygwin for 32bit and C:\Cygwin64 for
64bit. No need to invent bin32 or bin64.

++

We'll probably have to tweak %PATH% per-app, though -- 64-bit apps would
need the Cygwin64 first and Cygwin second, with that reversed for 32-bit
apps.

Post by JonY

Post by Corinna Vinschen

Post by JonY

Post by Corinna Vinschen
- Create a x86_64-pc-cygwin cross toolchain.

Yeah, I suppose newlib has to be ported first.

Right, I forgot about that one. But newlib works rather well for many
systems, so that shouldn't be much of a problem.

There's that hairy LP64 vs LLP64 issue, personally, I'd prefer the LLP64
route since Cygwin is a translation layer and will need to communicate
with Windows at the backend, but I suspect many more will want the LP64
route for Posix software compatibility.
I suppose there could be a minimalist Cygwin fork of the win32api to
make it LP64 compatible. Maybe a thunk/translator layer will be easier.

I suspect we'll come out ahead in the end by following Linux and doing
the translator -- the number of native windows apps compiled with
cygwin-gcc (and which can't use mingw-gcc) seems a rather small fraction
of the total, and posix apps could become a royal pain to compile on
cygwin if sizeof(long) != sizeof(void*).

[JonY]

Post by Ryan Johnson

Post by JonY

Post by Corinna Vinschen
Right, but that wasn't what I meant. Sorry for being unclear. I was
talking about the name of the Cygwin DLL. For instance, if we decide
that it must reside in the /bin directory, it must have a different name
than the 32 bit dll, for instance, cygwin64-1.dll. If we decide that
all 64 bit applications and DLLs reside in a parallel directory, it
could have the same name, for instance, /bin64/cygwin1.dll.
But let's not go into too much detail yet.

I was thinking that we have them totally separated, so we don't need to
deal with DLL name clashes. Eg C:\Cygwin for 32bit and C:\Cygwin64 for
64bit. No need to invent bin32 or bin64.

++
We'll probably have to tweak %PATH% per-app, though -- 64-bit apps would
need the Cygwin64 first and Cygwin second, with that reversed for 32-bit
apps.

How does that happen for per-app?

Post by Ryan Johnson

Post by JonY

Post by Corinna Vinschen

Post by JonY

Post by Corinna Vinschen
- Create a x86_64-pc-cygwin cross toolchain.

Yeah, I suppose newlib has to be ported first.

Right, I forgot about that one. But newlib works rather well for many
systems, so that shouldn't be much of a problem.

There's that hairy LP64 vs LLP64 issue, personally, I'd prefer the LLP64
route since Cygwin is a translation layer and will need to communicate
with Windows at the backend, but I suspect many more will want the LP64
route for Posix software compatibility.
I suppose there could be a minimalist Cygwin fork of the win32api to
make it LP64 compatible. Maybe a thunk/translator layer will be easier.

I suspect we'll come out ahead in the end by following Linux and doing
the translator -- the number of native windows apps compiled with
cygwin-gcc (and which can't use mingw-gcc) seems a rather small fraction
of the total, and posix apps could become a royal pain to compile on
cygwin if sizeof(long) != sizeof(void*).

So, some sort of thunk server/client thing in between the Cygwin DLL and
system DLL?

[Ryan Johnson]

Post by JonY

Post by Ryan Johnson

Post by JonY

Post by Corinna Vinschen
Right, but that wasn't what I meant. Sorry for being unclear. I was
talking about the name of the Cygwin DLL. For instance, if we decide
that it must reside in the /bin directory, it must have a different name
than the 32 bit dll, for instance, cygwin64-1.dll. If we decide that
all 64 bit applications and DLLs reside in a parallel directory, it
could have the same name, for instance, /bin64/cygwin1.dll.
But let's not go into too much detail yet.

I was thinking that we have them totally separated, so we don't need to
deal with DLL name clashes. Eg C:\Cygwin for 32bit and C:\Cygwin64 for
64bit. No need to invent bin32 or bin64.

++
We'll probably have to tweak %PATH% per-app, though -- 64-bit apps would
need the Cygwin64 first and Cygwin second, with that reversed for 32-bit
apps.

How does that happen for per-app?

Actually, in retrospect, I think it would happen automatically. If I
understand correctly, Windows always searches the binary's own '.'
first, followed by %PATH%. That would cover loading of static libraries,
dlopen, and fork. So, if the user starts an N-bit app and then forks
some app having both 32- and 64-bit versions available, Windows would
select the N-bit version unless the file name given forces use of the
other.

Given that, the earlier suggestion to put 64-bit binaries first in the
path probably makes the most sense.

Post by JonY

Post by Ryan Johnson

Post by JonY

Post by Corinna Vinschen

Post by JonY

Post by Corinna Vinschen
- Create a x86_64-pc-cygwin cross toolchain.

Yeah, I suppose newlib has to be ported first.

Right, I forgot about that one. But newlib works rather well for many
systems, so that shouldn't be much of a problem.

There's that hairy LP64 vs LLP64 issue, personally, I'd prefer the LLP64
route since Cygwin is a translation layer and will need to communicate
with Windows at the backend, but I suspect many more will want the LP64
route for Posix software compatibility.
I suppose there could be a minimalist Cygwin fork of the win32api to
make it LP64 compatible. Maybe a thunk/translator layer will be easier.

I suspect we'll come out ahead in the end by following Linux and doing
the translator -- the number of native windows apps compiled with
cygwin-gcc (and which can't use mingw-gcc) seems a rather small fraction
of the total, and posix apps could become a royal pain to compile on
cygwin if sizeof(long) != sizeof(void*).

So, some sort of thunk server/client thing in between the Cygwin DLL and
system DLL?

Something like Wow64, I guess. Most winapi calls have a pretty narrow
interface as far as long/LONG goes, so it shouldn't be too hairy to make
the conversion. Also, for anything exotic, the target audience is
Windows programmers using cygwin, and they'll be using LONG properly
already.

If LONG != long, we'll also have to worry about the inevitable posix app
defining conflicting typedefs, but hopefully that wouldn't import the
windows headers.

Ryan

[Corinna Vinschen]

Post by JonY

Post by Ryan Johnson

Post by JonY

Post by Corinna Vinschen
Right, but that wasn't what I meant. Sorry for being unclear. I was
talking about the name of the Cygwin DLL. For instance, if we decide
that it must reside in the /bin directory, it must have a different name
than the 32 bit dll, for instance, cygwin64-1.dll. If we decide that
all 64 bit applications and DLLs reside in a parallel directory, it
could have the same name, for instance, /bin64/cygwin1.dll.
But let's not go into too much detail yet.

I was thinking that we have them totally separated, so we don't need to
deal with DLL name clashes. Eg C:\Cygwin for 32bit and C:\Cygwin64 for
64bit. No need to invent bin32 or bin64.

++
We'll probably have to tweak %PATH% per-app, though -- 64-bit apps would
need the Cygwin64 first and Cygwin second, with that reversed for 32-bit
apps.

How does that happen for per-app?

I'm wondering that too. I don't see any reason to change that at all.
On a 64 bit system,m the 64 bit binary path comes first, then the 32
bit binary path. On a 32 bit system, there won't be any 64 bit path.
Assuming we won't go single bin dir anyway.

Post by JonY

Post by Ryan Johnson

Post by JonY

Post by Corinna Vinschen

Post by JonY

Post by Corinna Vinschen
- Create a x86_64-pc-cygwin cross toolchain.

Yeah, I suppose newlib has to be ported first.

Right, I forgot about that one. But newlib works rather well for many
systems, so that shouldn't be much of a problem.

There's that hairy LP64 vs LLP64 issue, personally, I'd prefer the LLP64
route since Cygwin is a translation layer and will need to communicate
with Windows at the backend, but I suspect many more will want the LP64
route for Posix software compatibility.
I suppose there could be a minimalist Cygwin fork of the win32api to
make it LP64 compatible. Maybe a thunk/translator layer will be easier.

I suspect we'll come out ahead in the end by following Linux and doing
the translator -- the number of native windows apps compiled with
cygwin-gcc (and which can't use mingw-gcc) seems a rather small fraction
of the total, and posix apps could become a royal pain to compile on
cygwin if sizeof(long) != sizeof(void*).

So, some sort of thunk server/client thing in between the Cygwin DLL and
system DLL?

What is that good for? Cygwin apps are POSIX apps, Cygwin DLLs are
POSIX libs. They only interact with the Cygwin DLL. If they also
use Win32 API, they are on their own and they better use Win32 types
like DWORD, etc. I don't see any need to make this even a bit more
complicated than necessary. Apart from that, the only non-POSIX
DLL is the Cygwin DLL itself. If it doesn't interact cleanly with
the Win32 API it goes boom and we fix.


Corinna

[JonY]

Post by Corinna Vinschen

Post by JonY

Post by Ryan Johnson
I suspect we'll come out ahead in the end by following Linux and doing
the translator -- the number of native windows apps compiled with
cygwin-gcc (and which can't use mingw-gcc) seems a rather small fraction
of the total, and posix apps could become a royal pain to compile on
cygwin if sizeof(long) != sizeof(void*).

So, some sort of thunk server/client thing in between the Cygwin DLL and
system DLL?

What is that good for? Cygwin apps are POSIX apps, Cygwin DLLs are
POSIX libs. They only interact with the Cygwin DLL. If they also
use Win32 API, they are on their own and they better use Win32 types
like DWORD, etc. I don't see any need to make this even a bit more
complicated than necessary. Apart from that, the only non-POSIX
DLL is the Cygwin DLL itself. If it doesn't interact cleanly with
the Win32 API it goes boom and we fix.

Yeah, I meant the Cygwin DLL itself, how does it communicate/thunk the
ABI difference when talking with win32/win64 API?

Like the other post mentioned one solution might be sizeof(LONG) !=
sizeof(long).

[Corinna Vinschen]

Post by JonY

Post by Corinna Vinschen

Post by JonY

Post by Ryan Johnson
I suspect we'll come out ahead in the end by following Linux and doing
the translator -- the number of native windows apps compiled with
cygwin-gcc (and which can't use mingw-gcc) seems a rather small fraction
of the total, and posix apps could become a royal pain to compile on
cygwin if sizeof(long) != sizeof(void*).

So, some sort of thunk server/client thing in between the Cygwin DLL and
system DLL?

Why should that be necessary? This sounds a bit like a misconception.
In the 32 bit version we're just calling Win32 and native NT functions.
And we'll do the same in the 64 bit version, just that it calls the same
functions in the 64 bit Windows DLLs. I don't see where thunking or a
client/server model comes into play here.

The only important thing to keep in mind when hacking code is that
certain assumptions are dangerously wrong now. DWORD, ULONG, LONG are
no longer the same size as long and unsigned long. HANDLE is now a 64
bit type, but only the lower 32 bit are used to allow duplicating
handles between 32 and 64 bit processes.

And note to myself: Using %p to print the NT status in debug output
is wrong now. Replace with 0x%08x.

Post by JonY

Post by Corinna Vinschen
What is that good for? Cygwin apps are POSIX apps, Cygwin DLLs are
POSIX libs. They only interact with the Cygwin DLL. If they also
use Win32 API, they are on their own and they better use Win32 types
like DWORD, etc. I don't see any need to make this even a bit more
complicated than necessary. Apart from that, the only non-POSIX
DLL is the Cygwin DLL itself. If it doesn't interact cleanly with
the Win32 API it goes boom and we fix.

Yeah, I meant the Cygwin DLL itself, how does it communicate/thunk the
ABI difference when talking with win32/win64 API?

If the cygwin compiler uses the same ABI as Windows, which would make
sense, there won't be a API difference. The fact that long is != LONG
is not important for the ABI.


Corinna

[Eric Blake]

Post by JonY

Post by Corinna Vinschen
- Do we define "long" as 32 bit or 64 bit type?

I suggest 32bit, they'll be some awkwardness accessing w32api at the
Cygwin backend if they're 64bit.

I very much want LLP64 to match Linux; the _only_ software that should
be accessing w32api in the cygwin backend is the cygwin dll itself,
which can take its own precautions to do correct conversions, whereas
everything compiled against the cygwin dll will be easier to port if it
remains like Linux with sizeof(long)==sizeof(void*).

I also hope that we can use this as an opportunity to move to 64-bit
time_t and NSIG of 64, even on 32-bit cygwin, which will be an ABI
change (similar to when we moved from stat to stat64 for the off_t ABI
change).

And as long as we are considering an ABI change, should we consider
moving to 4-byte wchar_t to match Linux?

[Corinna Vinschen]

Post by Eric Blake

Post by JonY

Post by Corinna Vinschen
- Do we define "long" as 32 bit or 64 bit type?

I suggest 32bit, they'll be some awkwardness accessing w32api at the
Cygwin backend if they're 64bit.

I very much want LLP64 to match Linux; the _only_ software that should

Erm... LP? LLP is Windows...

Post by Eric Blake
be accessing w32api in the cygwin backend is the cygwin dll itself,
which can take its own precautions to do correct conversions, whereas
everything compiled against the cygwin dll will be easier to port if it
remains like Linux with sizeof(long)==sizeof(void*).
I also hope that we can use this as an opportunity to move to 64-bit
time_t and NSIG of 64, even on 32-bit cygwin, which will be an ABI
change (similar to when we moved from stat to stat64 for the off_t ABI
change).

I'm not against this but it needs somebody to do it.

Post by Eric Blake
And as long as we are considering an ABI change, should we consider
moving to 4-byte wchar_t to match Linux?

Actually I'm not considering an ABI change. A 64 bit Cygwin would
be another platform, kind of. That means a 64 bit DLL does not
change the ABI, it's just a new ABI for a platform we didn't have one
before.

Changing wchar_t is quite a beast. As much as I like to have 4 byte
wchar_t, it requires a very intrusive change in Cygwin with lots of
code changes. And you can't do that on 32 bit unless you change the
toolchain since wchar_t is defined in gcc. And if you do it on 64
bit only, you'll probably get lots and lots of #ifdefs. Very tricky,
that one.


Corinna

[Andy Koppe]

Post by Eric Blake

Post by JonY

  - Do we define "long" as 32 bit or 64 bit type?

I suggest 32bit, they'll be some awkwardness accessing w32api at the
Cygwin backend if they're 64bit.

I very much want LLP64 to match Linux; the _only_ software that should
be accessing w32api in the cygwin backend is the cygwin dll itself,

Also cygutils, XWin, rxvt, mintty, and possibly more I'm not aware of.
And users are able to mix APIs too.

But yeah, tough. They're obviously outnumbered by POSIX/Linux
software. And as Corinna pointed out, POSIX doesn't even have a
platform macro for 32-bit long combined with 64-bit pointers (at
http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/unistd.h.html).

Hence I agree with sizeof(long)==sizeof(void *).

Andy

[Yaakov (Cygwin/X)]

Post by Andy Koppe

Post by Eric Blake
I very much want LLP64 to match Linux; the _only_ software that should
be accessing w32api in the cygwin backend is the cygwin dll itself,

Also cygutils, XWin, rxvt, mintty, and possibly more I'm not aware of.
And users are able to mix APIs too.

There are a few more cases, such as libusb1.0.

Post by Andy Koppe
But yeah, tough. They're obviously outnumbered by POSIX/Linux
software. And as Corinna pointed out, POSIX doesn't even have a
platform macro for 32-bit long combined with 64-bit pointers (at
http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/unistd.h.html).
Hence I agree with sizeof(long)==sizeof(void *).

Agreed. The bulk of our software is written with POSIX/Linux in mind,
and I have the feeling that not following this would make things much
more difficult.


Yaakov

[Charles Wilson]

Post by Andy Koppe

Post by Eric Blake

Post by JonY

Post by Corinna Vinschen
- Do we define "long" as 32 bit or 64 bit type?

I suggest 32bit, they'll be some awkwardness accessing w32api at the
Cygwin backend if they're 64bit.

I very much want LLP64 to match Linux; the _only_ software that should
be accessing w32api in the cygwin backend is the cygwin dll itself,

Also cygutils, XWin, rxvt, mintty, and possibly more I'm not aware of.

Don't forget tcl/tk which is still kinda sorta not really cygwin.

And perl (at least, the perl-win32 extension), python, ruby(?),
parrot(?), ... Lots of the interpreters seem to provide a binding to the
Win32 API even when compiled "for" cygwin.

--
Chuck

[Corinna Vinschen]

Post by Charles Wilson

Post by Andy Koppe

Post by Eric Blake

Post by JonY

Post by Corinna Vinschen
- Do we define "long" as 32 bit or 64 bit type?

I suggest 32bit, they'll be some awkwardness accessing w32api at the
Cygwin backend if they're 64bit.

I very much want LLP64 to match Linux; the _only_ software that should
be accessing w32api in the cygwin backend is the cygwin dll itself,

Also cygutils, XWin, rxvt, mintty, and possibly more I'm not aware of.

Don't forget tcl/tk which is still kinda sorta not really cygwin.
And perl (at least, the perl-win32 extension), python, ruby(?),
parrot(?), ... Lots of the interpreters seem to provide a binding to the
Win32 API even when compiled "for" cygwin.

I still don't see the problem. In how far is it valid to assume that
sizeof(long) == sizeof(LONG)? long is a compiler intrinsic, LONG is
defined in a Windows header. I don't see that the SDK claims that
there's a guarantee that long == LONG. So, AFAICS, nothing speaks
against changing the w32api header to define LONG in a target dependent
way, for instance, and without gurantee for correctness:

#ifdef __X86_64__
typedef int LONG;
typedef unsigned int ULONG, DWORD;
#else
typedef long LONG;
typedef unsigned long ULONG, DWORD;
#endif


Corinna

[Charles Wilson]

Post by Corinna Vinschen
I still don't see the problem. In how far is it valid to assume that
sizeof(long) == sizeof(LONG)? long is a compiler intrinsic, LONG is
defined in a Windows header. I don't see that the SDK claims that
there's a guarantee that long == LONG. So, AFAICS, nothing speaks
against changing the w32api header to define LONG in a target dependent
#ifdef __X86_64__
typedef int LONG;
typedef unsigned int ULONG, DWORD;
#else
typedef long LONG;
typedef unsigned long ULONG, DWORD;
#endif

In principle, I agree with you. The worry is that one of the named apps
is explicitly, in its own code, currently using 'long' to hold a value,
which is passed to a w32api call where it gets converted (implicitly) to
a LONG, DWORD, or void*. Or vice versa.

That's would be a bug in the application, and it should be fixed.

But...somebody will have to do that work, for each/all of the named
apps. And that takes time and effort.

However, I agree with the consensus that we should do things the linux
way -- this extra effort will just be one of the costs of that decision,
and we should tally up these known costs, as best we can, before
committing to the L[L]P64 project. Just so we all recognize what we're
getting in to.

--
Chuck

[Ryan Johnson]

Post by Charles Wilson

Post by Corinna Vinschen
I still don't see the problem. In how far is it valid to assume that
sizeof(long) == sizeof(LONG)? long is a compiler intrinsic, LONG is
defined in a Windows header. I don't see that the SDK claims that
there's a guarantee that long == LONG. So, AFAICS, nothing speaks
against changing the w32api header to define LONG in a target dependent
#ifdef __X86_64__
typedef int LONG;
typedef unsigned int ULONG, DWORD;
#else
typedef long LONG;
typedef unsigned long ULONG, DWORD;
#endif

In principle, I agree with you. The worry is that one of the named apps
is explicitly, in its own code, currently using 'long' to hold a value,
which is passed to a w32api call where it gets converted (implicitly) to
a LONG, DWORD, or void*. Or vice versa.

Casting long <---> void* shouldn't be a problem if we go the linux way.
The vice-versa direction shouldn't be a problem for the others because
it would widen the value. That leaves long -> LONG and long -> DWORD.

Personally, I would never expect sizeof(DWORD)==8 because in x86
assembly (the only other place where I encounter dwords) it is always a
32-bit value. That leaves only long -> LONG (sample size one, YMMV, etc.)

In theory, at least, gcc should start generating warnings once long ->
LONG becomes a narrowing conversion... unless a pre-existing explicit
(LONG) cast shuts it up***. That should at least help.

*** or the function being called has been cast, or the function declared
with implicit (= int) parameters, or any number of other abuses has
occurred.

GCC is also pretty picky about format size modifiers for printf-like
functions, particularly int vs. long (even when they are the same size).
Unfortunately, LONG=long right now so people wouldn't have been getting
yelled at up to this point.

One last thought, a question, actually: given that LONG is 32 bits in
64-bit windows, and only the low 32-bits of handles are meaningful****,
are there any obvious situations where losing the high 32 bits in a
w32api call would cause unexpected behavior? The only one I can think of
right off is size parameters which are not allowed to be larger than
2^31-1, in which case wraparound would be a Bad Thing.

****how does that impact dll loading, BTW?

Thoughts?
Ryan

[Corinna Vinschen]

Post by Ryan Johnson

Post by Charles Wilson

Post by Corinna Vinschen
#ifdef __X86_64__
typedef int LONG;
typedef unsigned int ULONG, DWORD;
#else
typedef long LONG;
typedef unsigned long ULONG, DWORD;
#endif

In principle, I agree with you. The worry is that one of the named apps
is explicitly, in its own code, currently using 'long' to hold a value,
which is passed to a w32api call where it gets converted (implicitly) to
a LONG, DWORD, or void*. Or vice versa.

This might turn out to be a bad idea.

Post by Ryan Johnson
In theory, at least, gcc should start generating warnings once long
-> LONG becomes a narrowing conversion... unless a pre-existing
explicit (LONG) cast shuts it up***. That should at least help.
*** or the function being called has been cast, or the function
declared with implicit (= int) parameters, or any number of other
abuses has occurred.

I remember with some fondness the times when I switched from 32 bit
Linux to 64 bit Linux. One of these days I wrote some testcase,
business as usual:

#include <stdio.h>
#include <errno.h>

int
main ()
{
int fd = open ("foo", O_RDWR);
if (fd < 0)
{
fprintf (stderr, "open: %d <%s>\n", errno, strerror (errno));
return 1;
}
[...]
}

When open failed, the testcase did not print an error message, it
crashed. It took some time to puzzle out that strerror returned a 64
bit pointer, but only 32 bit of the pointer were given as argument to
fprintf. Why? I missed to include string.h, so strerror was treated as
a function returning an int. That was when I learned not to ignore the
gcc warning "implicit declaration of function ‘foo’".

I still don't see a problem. LONG is a Win32 type, long is a compiler
defined type. And -Wall helps a lot.

Post by Ryan Johnson
One last thought, a question, actually: given that LONG is 32 bits
in 64-bit windows, and only the low 32-bits of handles are
meaningful****, are there any obvious situations where losing the
high 32 bits in a w32api call would cause unexpected behavior?

Sign extension could cause a problem. I don't think HANDLEs are a
problem since you usually don't store HANDLEs in longs or ints. Even
though HANDLEs are defines as void *, they are not really pointers in
the usual sense. If anything, "real" pointers are a problem.

However, as my above example shows, you could get into the same kind of
problems on Linux. Developers get over it. They should be able to do
the same when calling Win32 functions.

Post by Ryan Johnson
****how does that impact dll loading, BTW?

No. Should it? Do you have an example to show what you're concerned
about?


Corinna

[Ryan Johnson]

Post by Corinna Vinschen

Post by Ryan Johnson
One last thought, a question, actually: given that LONG is 32 bits
in 64-bit windows, and only the low 32-bits of handles are
meaningful****
****how does that impact dll loading, BTW?

No. Should it? Do you have an example to show what you're concerned
about?

I just remember (somewhere in this now-enormous thread) somebody
mentioned that only the low 32 bits of handles are meaningful in order
to allow sharing of handles between 64- and 32-bit processes. It seems
that this would force dlls to load in the low 2/4GB of memory so that
their handles are shareable. Perhaps (hopefully) the sharing thing only
applies to HANDLE and not to HMODULE?

Ryan

[Corinna Vinschen]

Post by Ryan Johnson

Post by Corinna Vinschen

Post by Ryan Johnson
One last thought, a question, actually: given that LONG is 32 bits
in 64-bit windows, and only the low 32-bits of handles are
meaningful****
****how does that impact dll loading, BTW?

No. Should it? Do you have an example to show what you're concerned
about?

I just remember (somewhere in this now-enormous thread) somebody
mentioned that only the low 32 bits of handles are meaningful in
order to allow sharing of handles between 64- and 32-bit processes.
It seems that this would force dlls to load in the low 2/4GB of
memory so that their handles are shareable. Perhaps (hopefully) the
sharing thing only applies to HANDLE and not to HMODULE?

The idea of having HANDLEs with only 32 significant bit is to allow
handle sharing between 64 and 32 bit processes. The actual meaning of
the handles is something we don't care about. They are opaque values,
just like file descriptors.

HMODULE is an address value. Given that you don't share 64 bit DLLs
with 32 bit processes, they are a non-issue. And HMODULE values are not
usefully shareable anyway since they only make sense in the VM of the
given process.


Corinna

[Yaakov (Cygwin/X)]

Post by Charles Wilson

Post by Andy Koppe

Post by Eric Blake

Post by JonY

Post by Corinna Vinschen
- Do we define "long" as 32 bit or 64 bit type?

I suggest 32bit, they'll be some awkwardness accessing w32api at the
Cygwin backend if they're 64bit.

I very much want LLP64 to match Linux; the _only_ software that should
be accessing w32api in the cygwin backend is the cygwin dll itself,

Also cygutils, XWin, rxvt, mintty, and possibly more I'm not aware of.

Don't forget tcl/tk which is still kinda sorta not really cygwin.

Not in Ports it's not. :-) I thought we agreed in principle to change
that in the distro too, but for whatever reason that hasn't happened
yet.


Yaakov

[Charles Wilson]

Post by Yaakov (Cygwin/X)

Post by Charles Wilson
Don't forget tcl/tk which is still kinda sorta not really cygwin.

Not in Ports it's not. :-) I thought we agreed in principle to change
that in the distro too, but for whatever reason that hasn't happened
yet.

The holdup is (a) cgf is the maintainer, and IIRC /he/ never "agreed in
principle", (b) even if he did, there's the question of how many tuits
he has accumulated, and (c) the ripple effect on tcl/tk clients is of
unknown size, leading to some trepidation.

But that's a side issue, and should be discussed on cygwinAT or
cygwin-appsAT in a separate thread. (I'll note this in passing, tho:
somebody, I forgot who, posted a smallish patch set to one of the lists
that fully transitioned tcl/tk to cygwin internals, BUT kept the GDI
windowing calls rather than converting to X. Dunno if that's an idea
that should be discussed or not. In the separate thread.)

--
Chuck

[Charles Wilson]

Post by JonY

Post by Corinna Vinschen
- What name should the 64 bit DLL have?

I think they should still use the "cyg" prefix, the libtool people was
against it when I suggested using a new prefix for 64bit mingw.

Not /this/ libtool person. I would LIKE to be able to distinguish
between 32bit and 64bit DLLs on both cygwin and mingw. I'd support a
change to libtool for cygwin64 DLLs to have an alternate prefix
(cyg64*?), and (if it's not too late, horse/barn situation) I'd also
support a similar change for mingw64.

Post by JonY

Post by Corinna Vinschen
- Where should 64 bit binaries and libs go?

Its best that if we can separate 32bit/64bit completely, so there won't
be any conflicting files.

Mmmm...depending on the answer to the next question, maybe not. Since
we have a googleplex of applications that would need to be "ported" to
64bit, it won't all happen at once. Is the "separate" 64bit distro
going to be crippled until ALL of the projects are ported?

I prefer being able to "upgrade" slowly from 32bit to 64bit, by
intermingling the apps. But, this requires
(a) cygwin64-1.dll(?) and cygwin-1.dll being able to share data
(b) ABI/API consistency. If zlib1.dll defines an interface that takes
a (32bit) "long"...but my 64bit version of tar.exe thinks that a long is
64bits...it might break if linked to the 32bit version of zlib1.dll.
Does this mean that ALL DLLs must have 64bit implementations before ANY
client EXEs are ported to the 64bit "platform". That's another
"crippled until" non-solution...

Post by JonY

Post by Corinna Vinschen
- Do we define "long" as 32 bit or 64 bit type?

I suggest 32bit, they'll be some awkwardness accessing w32api at the
Cygwin backend if they're 64bit.

Right, but then you have sizeof(long) != sizeof(void*). That's really
bad, from what I understand.

Post by JonY

Post by Corinna Vinschen
- What defines should a 64 bit Cygwin compiler define?

__CYGWIN__ and __CYGWIN64__? Just so to allow programmers to tell that
its cygwin and its 64bit.

Agree.

Post by JonY

Post by Corinna Vinschen
- What Windows headers and link libs do we use?

Bootstrap with mingw-w64? :)

Hoo-boy. There are some issues there, which deserve their own thread --
so if you want to open this furball, please reply-to with a new subject.
Short version:

1) there are concerns that mingw64's w32api header provenance is not
IP-clean. The mingw64 folks deny this, and even have a good argument
that mingw.org's w32api provenance is not as supposedly IP-clean as
claimed. But, if cygwin (a Red Hat product) is to go in this direction,
then Red Hat legal needs to get to the bottom of this. Fortunately
(unlike mingw.org), mingw64 has corporate backing -- and a legal
department -- from Kai's employer. So, there's a possibility for some
lawyer-to-lawyer resolution here, rather than hashing stuff out on a
mailing list by non-lawyers.

2) *completely* different mingwrt/w32api headers for 32bit cygwin and
64bit cygwin just seems like a really bad idea.

Post by JonY

Post by Corinna Vinschen
- How much interest do you have in a 64 bit Cygwin?

Lots.

Post by JonY

Post by Corinna Vinschen
- How much interest do you have to help to make 64 bit Cygwin real?

interest...check. time...limited, as always. But I'll do what I can.

Post by JonY

Post by Corinna Vinschen
- What part of the project would most interest you to help? Coding
Cygwin? Documentation? Setup? Toolchain? You name it.

Toolchain, and resolving the w32api/mingwrt issues.

--
Chuck

[Corinna Vinschen]

Post by Charles Wilson

Post by JonY

Post by Corinna Vinschen
- What name should the 64 bit DLL have?

I think they should still use the "cyg" prefix, the libtool people was
against it when I suggested using a new prefix for 64bit mingw.

Not /this/ libtool person. I would LIKE to be able to distinguish
between 32bit and 64bit DLLs on both cygwin and mingw. I'd support a
change to libtool for cygwin64 DLLs to have an alternate prefix
(cyg64*?), and (if it's not too late, horse/barn situation) I'd also
support a similar change for mingw64.

cyg64 sounds like a good idea to me. It would be a good way to allow
to reside 64 and 32 bit stuff in the same /bin dir.

Post by Charles Wilson
I prefer being able to "upgrade" slowly from 32bit to 64bit, by
intermingling the apps. But, this requires
(a) cygwin64-1.dll(?) and cygwin-1.dll being able to share data

Right.

Post by Charles Wilson
(b) ABI/API consistency. If zlib1.dll defines an interface that takes
a (32bit) "long"...but my 64bit version of tar.exe thinks that a long is
64bits...it might break if linked to the 32bit version of zlib1.dll.
Does this mean that ALL DLLs must have 64bit implementations before ANY
client EXEs are ported to the 64bit "platform". That's another
"crippled until" non-solution...

No, 32 and 64 bit DLLs are not interchangable anyway. A 64 bit process
can't load 32 bit DLLs and a 32 bit process can't load 64 bit DLLs.
So, if you only have a 32 bit cygz.dll, you can only create 32 bit
executables using it. Only with a cyg64z.dll you can create a 64 bit
executable using libz. Same goes for static linking, of course.


Corinna

[Charles Wilson]

Post by Corinna Vinschen

Post by Charles Wilson
(b) ABI/API consistency. If zlib1.dll defines an interface that takes
a (32bit) "long"...but my 64bit version of tar.exe thinks that a long is
64bits...it might break if linked to the 32bit version of zlib1.dll.
Does this mean that ALL DLLs must have 64bit implementations before ANY
client EXEs are ported to the 64bit "platform". That's another
"crippled until" non-solution...

No, 32 and 64 bit DLLs are not interchangable anyway. A 64 bit process
can't load 32 bit DLLs and a 32 bit process can't load 64 bit DLLs.
So, if you only have a 32 bit cygz.dll, you can only create 32 bit
executables using it. Only with a cyg64z.dll you can create a 64 bit
executable using libz. Same goes for static linking, of course.

Oh. So we really WILL be stuck with few/no 64bit applications, until
the requisite DLLs have 64bit versions deployed. Well, ok -- there
aren't THAT many DLLs installed as part of the Base category, so at
least we will know where to focus the early efforts. :-)

Hmmm...I think I had better get busy updating our existing
gettext/libintl offerings to $current, as I'll have better luck getting
cyg64 patches upstream based on a modern gettext distribution, than on
0.17.1. :-P

--
Chuck

[Yaakov (Cygwin/X)]

Post by Corinna Vinschen

Post by Charles Wilson
Not /this/ libtool person. I would LIKE to be able to distinguish
between 32bit and 64bit DLLs on both cygwin and mingw. I'd support a
change to libtool for cygwin64 DLLs to have an alternate prefix
(cyg64*?), and (if it's not too late, horse/barn situation) I'd also
support a similar change for mingw64.

cyg64 sounds like a good idea to me.

Sorry, but please trust me, this is a BAD idea. We do NOT want to open
that proverbial can of worms.

Post by Corinna Vinschen
It would be a good way to allow to reside 64 and 32 bit stuff in the
same /bin dir.

Can *someone* tell me why this is absolutely necessary? I have yet to
hear a single reason that wouldn't be solved by supporting parallel
installations like we did with 1.5-to-1.7.


Yaakov

[Brendan Conoboy]

Post by Yaakov (Cygwin/X)
Can *someone* tell me why this is absolutely necessary? I have yet to
hear a single reason that wouldn't be solved by supporting parallel
installations like we did with 1.5-to-1.7.

Also, in 5 years when nobody is running 32 bit windows, will everybody
still be happy with all these 64s in their paths and filenames?
Coinstalled trees make sense to me, c;\cygwin and c:\cygwin64 for
example. And while assumptions are being challenged, do DLLs really
need to have a cyg prefix?

[Charles Wilson]

Post by Brendan Conoboy

Post by Yaakov (Cygwin/X)
Can *someone* tell me why this is absolutely necessary? I have yet to
hear a single reason that wouldn't be solved by supporting parallel
installations like we did with 1.5-to-1.7.

Well, given that we *can't* have a (64bit) foo.exe until all of its DLLs
have been ported to 64bit, I guess there's less worry about a partially
functioning tree than I had thought. It *will* be partially functioning
until a large number of DLLs are ported, before many apps are available
at all.

When in turn means that you'll need to keep a "working" 32bit
installation usable anyway for quite some time. What that tells me is,
even IF we supported a single-tree cyg64/32 deployment, I'd still have
to have a "pure" 32bit co-installed tree somewhere else on my system.

Post by Brendan Conoboy
Also, in 5 years when nobody is running 32 bit windows, will everybody
still be happy with all these 64s in their paths and filenames?

How many Fedora or RHE customers complain about /lib64?

Post by Brendan Conoboy
Coinstalled trees make sense to me, c;\cygwin and c:\cygwin64 for
example. And while assumptions are being challenged, do DLLs really
need to have a cyg prefix?

The cyg prefix has protected us from "corruption" by native (mingw32 or
msvc-compiled) DLLs that use the "lib" prefix. Many "native" tools have
no compunction about inserting their installation path in the front of
$PATH (hello, GIMP, I'm looking at you). Since they include a
libintl-N.dll and many other DLLs that have identical names to the
cgywin ones (modulo the prefix), this would cause any cygwin app NOT
installed in /bin to die mysteriously.

If we take one of the suggestions elsewhere in this thread, and move
cgywin DLLs into /lib and rely on solely on $PATH rather than
$dir-of-EXE to find most DLLs, the problem gets even worse: /bin/bash
would load the wrong libintl-8.dll and would coredump, before getting a
chance to run .bash_profile to set the $PATH "properly".

As much as Yaakov complains about the cyg prefix, it solves a lot more
problems than it creates, especially of the newbie
brand-new-installation-fails-to-work variety.

--
Chuck

[Thomas Wolff]

Post by Charles Wilson
...
If we take one of the suggestions elsewhere in this thread, and move
cgywin DLLs into /lib and rely on solely on $PATH rather than
$dir-of-EXE to find most DLLs, the problem gets even worse: /bin/bash
would load the wrong libintl-8.dll and would coredump, before getting a
chance to run .bash_profile to set the $PATH "properly".

Just to clarify: my suggestion to move DLLs to /lib was *not* about
leaving out the prefix; it was about dual maintenance of 32 bit and 64
bit versions. So /lib/cygfoo.dll and /lib64/cygfoo.dll would not be
harmful, I suppose.
Thomas

[Corinna Vinschen]

Post by Brendan Conoboy

Post by Yaakov (Cygwin/X)
Can *someone* tell me why this is absolutely necessary? I have yet to
hear a single reason that wouldn't be solved by supporting parallel
installations like we did with 1.5-to-1.7.

Also, in 5 years when nobody is running 32 bit windows, will
everybody still be happy with all these 64s in their paths and
filenames?

Does anybody complain about /lib64 and /usr/lib64 on Linux?

Also, I don't see a reason to disallow running non-distro 32 bit Cygwin
apps on a 64 bit system. Why does Linux allow it? Why should this be
a non-issue for Cygwin?

Post by Brendan Conoboy
Coinstalled trees make sense to me, c;\cygwin and
c:\cygwin64 for example. And while assumptions are being
challenged, do DLLs really need to have a cyg prefix?

Yes. Non-Cygwin DLLs using the same name as Cygwin DLLs exist, but they
won't work under Cygwin. For example, there's a native libz.dll. If
that's also the name of the Cygwin DLL it's just a matter of $PATH if a
Cygwin executable loading libz.dll crashes. That's why it has been
decided many years ago that Cygwin DLLs should use another prefix than
"lib" to avoid collisions with existing native DLLs.


Corinna

[Brendan Conoboy]

Post by Corinna Vinschen
Does anybody complain about /lib64 and /usr/lib64 on Linux?

No, they complain when two packages include the same headers in
/usr/include or the same support files in /usr/share. Are there going
to be architecture-dependent sub-packages that split these out? Will
there be one installer or two? If just one, how will it need to
function? Splitting the installation trees out means less
infrastructure work is required. If you want 32 and 64 bit
installations you run both. Put the PATH of the other at the end to
make sure you have access to all the commands (Particularly important
when getting all the 64 bit packages built).

Post by Corinna Vinschen
Yes. Non-Cygwin DLLs using the same name as Cygwin DLLs exist, but they
won't work under Cygwin. For example, there's a native libz.dll. If
that's also the name of the Cygwin DLL it's just a matter of $PATH if a
Cygwin executable loading libz.dll crashes. That's why it has been
decided many years ago that Cygwin DLLs should use another prefix than
"lib" to avoid collisions with existing native DLLs.

Okay, thanks.

[Yaakov (Cygwin/X)]

Post by Corinna Vinschen
Also, I don't see a reason to disallow running non-distro 32 bit Cygwin
apps on a 64 bit system. Why does Linux allow it? Why should this be
a non-issue for Cygwin?

Where are they in the first place? On Linux, there are many binary-only
packages available on the web, most of which are 32-bit only. I have
yet to see a single one of those for Cygwin. Who provides binary
packages for Cygwin except for a few of us maintainers with external
repos, which could just as well support both (Ports certainly would) or
even be part of the distro itself if we changed how the distro was
managed.


Yaakov

[Corinna Vinschen]

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen
Also, I don't see a reason to disallow running non-distro 32 bit Cygwin
apps on a 64 bit system. Why does Linux allow it? Why should this be
a non-issue for Cygwin?

Where are they in the first place? On Linux, there are many binary-only
packages available on the web, most of which are 32-bit only. I have
yet to see a single one of those for Cygwin. Who provides binary
packages for Cygwin except for a few of us maintainers with external
repos, which could just as well support both (Ports certainly would) or
even be part of the distro itself if we changed how the distro was
managed.

Customers of the Cygwin buyout license, for instance.


Corinna

[Yaakov (Cygwin/X)]

Post by Corinna Vinschen

Post by Yaakov (Cygwin/X)
Where are they in the first place? On Linux, there are many binary-only
packages available on the web, most of which are 32-bit only. I have
yet to see a single one of those for Cygwin. Who provides binary
packages for Cygwin except for a few of us maintainers with external
repos, which could just as well support both (Ports certainly would) or
even be part of the distro itself if we changed how the distro was
managed.

Customers of the Cygwin buyout license, for instance.

Alright, having not ever seen them, how many other Cygwin libraries do
those packages use besides Cygwin itself? Base? X11? GTK? Qt? It's
hard to solve a problem without knowing its scope.


Yaakov

[Corinna Vinschen]

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen

Post by Yaakov (Cygwin/X)
Where are they in the first place? On Linux, there are many binary-only
packages available on the web, most of which are 32-bit only. I have
yet to see a single one of those for Cygwin. Who provides binary
packages for Cygwin except for a few of us maintainers with external
repos, which could just as well support both (Ports certainly would) or
even be part of the distro itself if we changed how the distro was
managed.

Customers of the Cygwin buyout license, for instance.

Alright, having not ever seen them, how many other Cygwin libraries do
those packages use besides Cygwin itself? Base? X11? GTK? Qt? It's
hard to solve a problem without knowing its scope.

I don't know. We don't look how exactly a buyout customer uses the
Cygwin DLL. It was only an example anyway. I just think that there
shouldn't be any technical problems to mix 32 and 64 bit freely,
just as it works on Linux or, FWIW, Windows.


Corinna

[Andy Koppe]

Post by Corinna Vinschen

Not /this/ libtool person.  I would LIKE to be able to distinguish
between 32bit and 64bit DLLs on both cygwin and mingw.  I'd support a
change to libtool for cygwin64 DLLs to have an alternate prefix
(cyg64*?), and (if it's not too late, horse/barn situation) I'd also
support a similar change for mingw64.

cyg64 sounds like a good idea to me.

Sorry, but please trust me, this is a BAD idea.  We do NOT want to open
that proverbial can of worms.

Post by Corinna Vinschen
It would be a good way to allow to reside 64 and 32 bit stuff in the
same /bin dir.

Can *someone* tell me why this is absolutely necessary?  I have yet to
hear a single reason that wouldn't be solved by supporting parallel
installations like we did with 1.5-to-1.7.

1.7 was a complete distro from early on, due to binary backward
compatibility and the unionfs with 1.5. A parallel 1.5 wasn't needed
to run 1.5 stuff, only to build it.

This wouldn't be the case with a pure 64-bit Cygwin. As soon as you
needed any tool not yet available in 64-bit, you'd need to switch back
to 32-bit. Trying to mix them by sticking both in the path isn't gonna
work well, due to the same DLLs appearing in both distros' /bin
directories, so the wrong one is bound to be picked up somewhere.
(Incidentally, the /bin vs /bin64 scheme would have the same problem.)
Even if that did work, the separate root directories would make for a
very "interesting" user experience.

Andy

[Yaakov (Cygwin/X)]

Post by Andy Koppe
1.7 was a complete distro from early on, due to binary backward
compatibility and the unionfs with 1.5. A parallel 1.5 wasn't needed
to run 1.5 stuff, only to build it.

FWIW I rebuilt all my packages for 1.7, that's why I use that as a
reference point.

Post by Andy Koppe
This wouldn't be the case with a pure 64-bit Cygwin. As soon as you
needed any tool not yet available in 64-bit, you'd need to switch back
to 32-bit. Trying to mix them by sticking both in the path isn't gonna
work well, due to the same DLLs appearing in both distros' /bin
directories, so the wrong one is bound to be picked up somewhere.
(Incidentally, the /bin vs /bin64 scheme would have the same problem.)
Even if that did work, the separate root directories would make for a
very "interesting" user experience.

The parallel installation setup isn't meant to be a "[end-]user
experience" , its meant to allow maintainers to have a functioning
32-bit distro while working to create a 64-bit one. Nowadays I do just
about everything from Ports' GNOME desktop, and I'll still need to do
all those things while we work on building up the 64-bit platform.

Here's what I have in mind:

1) We port w32api/newlib/Cygwin and GCC to x86_64-pc-cygwin;
2) Build a x86_64-pc-cygwin cross-binutils (done?) and cross-gcc;
3) Cross-compile the basic x86_64 development platform;
4) Build the x86_64 distro natively from the bottom up.

Once (4) is well under way, then end-users can start testing and using
the x64 distro, and switch over once the packages they want are
available.


Yaakov

[Andy Koppe]

Post by Corinna Vinschen
I'm not sure if you agree, but as far as I can see, 32 bit systems are
more and more reduced to a niche market, namely Netbooks and other very
small systems.  On the Desktop, 32 bit is declining fast, in the server
segment it's practically dead.
Given this, I'm wondering how much future Cygwin has if we stick to
32 bit.  I think it will be pretty limited.  In fact, we're probably
rather late in the game.

I agree. Even though the advantages for programs that don't have much
use for 64-bit integers or pointers are modest, the '*32' in Task
Manager will increasingly be interpreted as 'legacy'.

Post by Corinna Vinschen
Lately I read a bit more about how 32 and 64 bit processes can interact
with each other on Windows, and it seems we could come up with a 64 bit
DLL which shares all important information with its 32 bit counterpart,
so that we could run 64 and 32 bit processes in parallel on a 64 bit
system acting as a single system.  This would be especially important,
given the fact that 64 bit Cygwin applications will be pretty rare in
the beginning.

That would be brilliant, because getting 64-bit versions of all the
packages looks like the biggest challenge to me. Something like the
1.5/1.7 unionfs might be helpful here.

Post by Corinna Vinschen
As far as I can see what we have to do in about this order is
- Discuss certain basics.  This is probably the most crucial step.
 - What name should the 64 bit DLL have?

That question applies not just to the Cygwin DLL but also to every
other library.

Post by Corinna Vinschen
 - Where should 64 bit binaries and libs go?

What does Linux do?

(Also, do libraries still need to go into the same directory as executables?)

Post by Corinna Vinschen
 - Do we define "long" as 32 bit or 64 bit type?

Oooh, that's a difficult one. (For anyone who doesn't know: It's 64
bits on Linux, but 32 bits on Windows, including MinGW-64.)

I suspect 64 bits would be possible, but painful, due to LONG != long.
I guess it depends on how many programs use 'long' where they should
be using 'int64_t' or 'intptr_t'. Are there other 64-bit Unixes where
long == 32 bits?

Post by Corinna Vinschen
 - What defines should a 64 bit Cygwin compiler define?

Again, what does Linux do? (Avoiding additional Cygwin-specific
defines would be nice.)

Post by Corinna Vinschen
 - What Windows headers and link libs do we use?

Meaning: can we use MinGW-64's, or do we need to create our own fork
of MinGW's 32-bit ones?

Post by Corinna Vinschen
- Decide how we can integrate 64 bit stuff into the distro.  Will we have
 a 32 bit and a distinct 64 bit distro?

I'd say yes, but wind down development of the 32-bit one as the 64-bit
one becomes stable enough, similarly to what happened with 1.5.

Packaging resources are thinly spread as it is, so maintaining two
distros in parallel might not work out well long term.

Post by Corinna Vinschen
- Change setup to allow installation of 64 bit stuff on 64 bit systems.

It would be good if setup didn't need any substantial changes (or a
64-bit version) for this.

Post by Corinna Vinschen
This is a big project which can only work if we have help and support
from the community.  Before we can even contemplate to start discussing,
- How much interest do you have in a 64 bit Cygwin?
- How much interest do you have to help to make 64 bit Cygwin real?
- What part of the project would most interest you to help?  Coding
 Cygwin?  Documentation?  Setup?  Toolchain?  You name it.

I'd try to port mintty as soon as practical, and I can chip in with
newlib as well. Unfortunately I can't promise an awful lot of time
though.

(I see there's a libc/machine/x86_64 directory in newlib. Has this
x86_64 support been used in anger?)

Andy

[Corinna Vinschen]

Post by Andy Koppe

Post by Corinna Vinschen
I'm not sure if you agree, but as far as I can see, 32 bit systems are
more and more reduced to a niche market, namely Netbooks and other very
small systems.  On the Desktop, 32 bit is declining fast, in the server
segment it's practically dead.
Given this, I'm wondering how much future Cygwin has if we stick to
32 bit.  I think it will be pretty limited.  In fact, we're probably
rather late in the game.

I agree. Even though the advantages for programs that don't have much
use for 64-bit integers or pointers are modest, the '*32' in Task
Manager will increasingly be interpreted as 'legacy'.

Yeah, WOW64 is already only an optional component of Windows 2008 R2
and can be deinstalled. Probably the next Server release requires
to install WOW64 exlicitely as "feature" to get it at all and it's
also becoming an optional component on workstations. Rumor has it
that Windows 8 is the last Windows available in a 32 bit x86 version.
Fortunately there will be an ARM version...

Post by Andy Koppe

Post by Corinna Vinschen
 - What name should the 64 bit DLL have?

That question applies not just to the Cygwin DLL but also to every
other library.

Indeed.

Post by Andy Koppe

Post by Corinna Vinschen
 - Where should 64 bit binaries and libs go?

What does Linux do?

Executables in /{s}bin, /usr/{s}bin, regardless of 32 or 64 bit.
Libs in /lib vs. /lib64, /usr/lib vs. /usr/lib64.

Post by Andy Koppe
(Also, do libraries still need to go into the same directory as executables?)

Yes, %PATH% still rulez.

Post by Andy Koppe

Post by Corinna Vinschen
 - Do we define "long" as 32 bit or 64 bit type?

Oooh, that's a difficult one. (For anyone who doesn't know: It's 64
bits on Linux, but 32 bits on Windows, including MinGW-64.)

http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/unistd.h.html
bugs me. There's no _CS_POSIX_V7_xxx definition which allows for 32
bit int, 32 bit long, 64 bit pointer and 64 bit off_t.

Post by Andy Koppe
Packaging resources are thinly spread as it is, so maintaining two
distros in parallel might not work out well long term.

That speaks for a single distro with 32 and 64 bit stuff mingling.

Post by Andy Koppe

Post by Corinna Vinschen
This is a big project which can only work if we have help and support
from the community.  Before we can even contemplate to start discussing,
- How much interest do you have in a 64 bit Cygwin?
- How much interest do you have to help to make 64 bit Cygwin real?
- What part of the project would most interest you to help?  Coding
 Cygwin?  Documentation?  Setup?  Toolchain?  You name it.

Unfortunately I can't promise an awful lot of time though.

That's what I fear most.

Post by Andy Koppe
(I see there's a libc/machine/x86_64 directory in newlib. Has this
x86_64 support been used in anger?)

In anger? I don't understand the question.


Corinna

[Andy Koppe]

Post by Andy Koppe
(I see there's a libc/machine/x86_64 directory in newlib. Has this
x86_64 support been used in anger?)

In anger?  I don't understand the question.

Is it being used on "production" systems, or is it more at a
proof-of-concept sort of level?

Andy

[Corinna Vinschen]

Post by Andy Koppe

Post by Andy Koppe
(I see there's a libc/machine/x86_64 directory in newlib. Has this
x86_64 support been used in anger?)

In anger?  I don't understand the question.

Is it being used on "production" systems, or is it more at a
proof-of-concept sort of level?

Oh, ok. I don't know, sorry. It looks like it's used, but it certainly
doesn't make a lot of effort to support fast strcpy and stuff like that.


Corinna

[Charles Wilson]

Post by Corinna Vinschen

Post by Andy Koppe
What does Linux do?

Executables in /{s}bin, /usr/{s}bin, regardless of 32 or 64 bit.
Libs in /lib vs. /lib64, /usr/lib vs. /usr/lib64.

Post by Andy Koppe
(Also, do libraries still need to go into the same directory as executables?)

Yes, %PATH% still rulez.

Which means that the 64bit dlls *must* have a different name, or we
can't have coexistence. E.g.
bin/app1.exe is not yet ported to 64, and uses (32bit) cygz-1.dll
bin/app2.exe IS ported to 64, so obviously needs the (64bit) zlib dll?
If a 64bit zlib DLL is available, then we need to have both installed,
without conflicts -- which means (a) different DLL name OR different
location for both 64bit DLLs AND EXEs, and (b) different PACKAGE name.

Oops.

So, if 64 and 32 are to coexist in the same installation, we need to
worry not just about DLL names, but also the name of packages (at least,
for delivery of DLLs).

Currently, most package sets that deliver general purpose (i.e.
non-private) DLLs do so in a separate "library" package (zlib,
ironically, being an exception):

tiff
tiff-doc
tiff-opengl
libtiff-devel

Post by Corinna Vinschen

Post by Andy Koppe

libtiff6 <<<

Now, on my (64bit) linux box, the following two packages do not conflict
(*).

libtiff-3.9.5-1.fc15.x86_64
libtiff-3.9.5-1.fc15.i686

So, the first observation is that the "package name" includes the
bitness of the binary. None of our package names do this, at present
(not counting the mingw64 cross compilers).



Now, in the linux case, the .so's themselves DO have the same name --
they just get installed into separate directories:

/usr/lib64/libtiff.so.3.9.5
/usr/lib/libtiff.so.3.9.5

However, if we only have one 'bin' directory for both 32 and 64 bit
exe/dlls, then...we need to give the DLLs different names, as well. I
like the prefix idea -- and I'll push it on libtool if necessary.


(*) Well, they do, but not with regards to the shared libraries
themselves. Both linux packages install the following:
/usr/share/doc/libtiff-3.9.5/COPYRIGHT
/usr/share/doc/libtiff-3.9.5/README
/usr/share/doc/libtiff-3.9.5/RELEASE-DATE
/usr/share/doc/libtiff-3.9.5/VERSION

Post by Corinna Vinschen

Post by Andy Koppe

- Do we define "long" as 32 bit or 64 bit type?

Oooh, that's a difficult one. (For anyone who doesn't know: It's 64
bits on Linux, but 32 bits on Windows, including MinGW-64.)

http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/unistd.h.html
bugs me. There's no _CS_POSIX_V7_xxx definition which allows for 32
bit int, 32 bit long, 64 bit pointer and 64 bit off_t.

Yep. sizeof(long) != sizeof(void*)...Bad Microsort! No cookie!

--
Chuck

[Corinna Vinschen]

Post by Charles Wilson

Post by Corinna Vinschen

Post by Andy Koppe
What does Linux do?

Executables in /{s}bin, /usr/{s}bin, regardless of 32 or 64 bit.
Libs in /lib vs. /lib64, /usr/lib vs. /usr/lib64.

Post by Andy Koppe
(Also, do libraries still need to go into the same directory as executables?)

Yes, %PATH% still rulez.

Which means that the 64bit dlls *must* have a different name, or we
can't have coexistence. E.g.
bin/app1.exe is not yet ported to 64, and uses (32bit) cygz-1.dll
bin/app2.exe IS ported to 64, so obviously needs the (64bit) zlib dll?
If a 64bit zlib DLL is available, then we need to have both installed,
without conflicts -- which means (a) different DLL name OR different
location for both 64bit DLLs AND EXEs, and (b) different PACKAGE name.
Oops.

Oops? It's basically the same as on a 64 bit Linux. 32 bit libs go
to /lib, 64 bit libs go to /lib64. It's just a bit different for DLLs
because they are in /bin. Either we invent a /bin64, or we use another
DLL prefix, like the suggested cyg64. I don't see much of a problem here.
It's just a decision to be made, isn't it?

Post by Charles Wilson
So, if 64 and 32 are to coexist in the same installation, we need to
worry not just about DLL names, but also the name of packages (at least,
for delivery of DLLs).

Well, depends on how you create packages. Another decision to be made.
I'll go with the flow, but I can easily imagine that a package maintainer
just provides 32 and 64 bit stuff in the same package and setup decides
what to do. For instance, the package is packed like this:

bin/foo.exe
bin64/foo.exe

Now setup installs bin/foo.exe unconditionally and, as soon as it
trips over bin64/foo.exe it depends on the system it's running on.
On a 64 bit system it installs bin64/foo.exe to bin/foo.exe, on a
32 bit system it simply ignores everything under bin64.

Well, it's just an idea, there are certainly better ones.

Post by Charles Wilson
Currently, most package sets that deliver general purpose (i.e.
non-private) DLLs do so in a separate "library" package (zlib,
tiff
tiff-doc
tiff-opengl
libtiff-devel

Post by Corinna Vinschen

Post by Andy Koppe

libtiff6 <<<

Now, on my (64bit) linux box, the following two packages do not conflict
(*).
libtiff-3.9.5-1.fc15.x86_64
libtiff-3.9.5-1.fc15.i686
So, the first observation is that the "package name" includes the
bitness of the binary. None of our package names do this, at present
(not counting the mingw64 cross compilers).

But there's no rule which disallows to provide another package called,
say, lib64tiff6.


Corinna

[Charles Wilson]

Post by Corinna Vinschen

Post by Charles Wilson
Which means that the 64bit dlls *must* have a different name, or we
can't have coexistence. E.g.
Oops.

Oops?
It's just a decision to be made, isn't it?

Post by Charles Wilson
So, if 64 and 32 are to coexist in the same installation, we need to
worry not just about DLL names, but also the name of packages (at least,
for delivery of DLLs).

Well, depends on how you create packages. Another decision to be made.

Right, we have to decide -- my point is there isn't an easy answer that
will have zero impact on everybody. There are basically three choices:

/bin + /bin64 -- affects $PATH, implicates package naming (*). Also
might affect script #! lines, and anything that calls
exec*() or spawn() with a full pathname.

cyg64 prefix -- affects apps which dlopen() or LoadLibrary().
Especially affects the "big stacks" like Yaakov's
gnome and kde

no multilib -- slows deployment and development of (crippled) cyg64
distro

(*) unless we combine 32 and 64 into the same package. More below.

Post by Corinna Vinschen
I'll go with the flow, but I can easily imagine that a package maintainer
just provides 32 and 64 bit stuff in the same package and setup decides
bin/foo.exe
bin64/foo.exe
Now setup installs bin/foo.exe unconditionally and, as soon as it
trips over bin64/foo.exe it depends on the system it's running on.
On a 64 bit system it installs bin64/foo.exe to bin/foo.exe, on a
32 bit system it simply ignores everything under bin64.
Well, it's just an idea, there are certainly better ones.

That could work. The downside is the maintainer's system: she is
running either 32bit or 64bit cygwin.

(1) maint=32bit: needs the native compiler to build the 32bit stuff, but
also needs a 64bit cross compiler. Build system needs to support
building the [app/lib/whatever] twice: once native and once with the
64bit toolchain, and to "know" how to package it.

(2) maint=64bit: needs a "native" 64bit compiler, and a 32bit cross
compiler. Ditto as above, but reversed.

The bulk of cygwin packages are, at present, built using cygport. It
does not (yet) have support for multiple build passes like this, but
that could be added relatively easily I think.

OTOH, if you're using gbs (or mknetrel, or Jan's tool-I-forgot-the-name)
you're looking at some hurt, I think.

Post by Corinna Vinschen

Post by Charles Wilson
Now, on my (64bit) linux box, the following two packages do not conflict
(*).
libtiff-3.9.5-1.fc15.x86_64
libtiff-3.9.5-1.fc15.i686
So, the first observation is that the "package name" includes the
bitness of the binary. None of our package names do this, at present
(not counting the mingw64 cross compilers).

But there's no rule which disallows to provide another package called,
say, lib64tiff6.

Right, but it just seems odd to me that (eventually) the "pure" 64bit
cygwin will have all these "lib64fooN" packages everywhere, and all the
requires: lines in the setup.hints will need to list those, rather (or
in addition to?) than the "libfooN" one. Doable, but...odd.

Plus, if you have 'fooapp' which depends on libfoo, and fooapp is
available in both 32 and 64bit flavors, which DLL package does its
(single?) setup.hint list as a requires:? Both of them?

--
Chuck

[Thomas Wolff]

Post by Charles Wilson

Post by Corinna Vinschen

Post by Charles Wilson
Which means that the 64bit dlls *must* have a different name, or we
can't have coexistence. E.g.
Oops.

Oops?
It's just a decision to be made, isn't it?

Post by Charles Wilson
...

Right, we have to decide -- my point is there isn't an easy answer that
/bin + /bin64 -- affects $PATH, implicates package naming (*). Also
might affect script #! lines, and anything that calls
exec*() or spawn() with a full pathname.

I have always wondered what the cyg*.dll's are doing in /bin. Not being
a dll insider..., I may be overlooking something, but maybe it's time to
move them over to /usr/lib and then add /usr/lib64, just as on Linux
systems.
Oh, and if it's about the PATH, I don't think any Windows programm will
look for a cyg*.dll, so the cygwin loading function could probably be
hard-coded to add /usr/lib (and /usr/lib64) to the path.
Thomas

[Charles Wilson]

Post by Thomas Wolff
I have always wondered what the cyg*.dll's are doing in /bin. Not being
a dll insider..., I may be overlooking something, but maybe it's time to
move them over to /usr/lib and then add /usr/lib64, just as on Linux
systems.
Oh, and if it's about the PATH, I don't think any Windows programm will
look for a cyg*.dll, so the cygwin loading function could probably be
hard-coded to add /usr/lib (and /usr/lib64) to the path.

That only helps if the new process is being launched by an existing
cygwin process. If you launch the cygwin app from windows (e.g. via a
shortcut), then you're completely at the mercy of the system $PATH
value. In many cases, you don't WANT to add the cygwin directories to
the global PATH -- what if you have two (or more) separate cygwin
installations? Which one gets the preferred system PATH entry? What if
you switch between cygwin and msys? or cygwin and interix? Having a
cygwin DLL dir in the global $PATH might interfere with the operation of
those other environments.

By putting (most) DLLs in the same directory that (most) cygwin EXEs
live in, we leverage the Windows Runtime Loader behavior that FIRST
looks in dir-of-EXE for any linked DLLs, before searching $PATH.

That's why cygwin.bat -- which starts bash.exe, which depends on
cygwin1.dll
cygintl-8.dll
cygiconv-2.dll
cyggcc_s-1.dll
cygreadline7.dll
cygncursesw-10.dll
actually works, regardless of the global $PATH settings.

--
Chuck

[Thomas Wolff]

Post by Charles Wilson

Post by Thomas Wolff
I have always wondered what the cyg*.dll's are doing in /bin. Not being
a dll insider..., I may be overlooking something, but maybe it's time to
move them over to /usr/lib and then add /usr/lib64, just as on Linux
systems.
Oh, and if it's about the PATH, I don't think any Windows programm will
look for a cyg*.dll, so the cygwin loading function could probably be
hard-coded to add /usr/lib (and /usr/lib64) to the path.

That only helps if the new process is being launched by an existing
cygwin process. If you launch the cygwin app from windows (e.g. via a
shortcut), then you're completely at the mercy of the system $PATH
value. In many cases, you don't WANT to add the cygwin directories to
the global PATH -- what if you have two (or more) separate cygwin
installations? Which one gets the preferred system PATH entry? What if
you switch between cygwin and msys? or cygwin and interix? Having a
cygwin DLL dir in the global $PATH might interfere with the operation of
those other environments.
By putting (most) DLLs in the same directory that (most) cygwin EXEs
live in, we leverage the Windows Runtime Loader behavior that FIRST
looks in dir-of-EXE for any linked DLLs, before searching $PATH.

I am aware that cygwin dirs should not appear in the global path. The
idea is... (see below).

Post by Charles Wilson
The original idea to keep the DLLs in /bin was to allow running Cygwin
applications from a non-Cygwin process without hassle. A typical
example is CMD. You can simply chdir to C:\cygwin\bin and run `ls'. If
/lib contains all the DLLs, this will break unless C:\cygwin\lib is in
%Path%.
Also, you can't just create a bash shortcut on the desktop, unless you
really know what you're doing (set the start directory in the shortcut
to C:\cygwin\lib). When moving the DLLs out of /bin, you add some new
hurdle to using Cygwin.
Without this problem, or if we decide to ignore this problem, I would be
all for moving DLLs to /lib or /lib64.

Can't cygwin compilation ensure that the cygwin1.dll is always loaded
first so it could take care of this (if only by then adding to the path)?
(I guess the answer may be no because Windows is juggling with the DLLs...)
Thomas

[Corinna Vinschen]

Post by Thomas Wolff

Post by Corinna Vinschen
The original idea to keep the DLLs in /bin was to allow running Cygwin
applications from a non-Cygwin process without hassle. A typical
example is CMD. You can simply chdir to C:\cygwin\bin and run `ls'. If
/lib contains all the DLLs, this will break unless C:\cygwin\lib is in
%Path%.
Also, you can't just create a bash shortcut on the desktop, unless you
really know what you're doing (set the start directory in the shortcut
to C:\cygwin\lib). When moving the DLLs out of /bin, you add some new
hurdle to using Cygwin.
Without this problem, or if we decide to ignore this problem, I would be
all for moving DLLs to /lib or /lib64.

Can't cygwin compilation ensure that the cygwin1.dll is always
loaded first so it could take care of this (if only by then adding
to the path)?
(I guess the answer may be no because Windows is juggling with the DLLs...)

Yes, the answer is no. At least it's very likely no. Even if Cygwin is
loaded first, I'm not aware that changing $PATH from within the DLL
entry point will influence the Windows loader for this very process.
Somehow this even sounds like a potential security problem. Otherwise,
for Cygwin distro DLLs it's pretty sure that the Cygwin DLL will be
loaded first, since every subsequent DLL depends on it.


Corinna

[Ryan Johnson]

Post by Corinna Vinschen

Post by Thomas Wolff

Post by Corinna Vinschen
The original idea to keep the DLLs in /bin was to allow running Cygwin
applications from a non-Cygwin process without hassle. A typical
example is CMD. You can simply chdir to C:\cygwin\bin and run `ls'. If
/lib contains all the DLLs, this will break unless C:\cygwin\lib is in
%Path%.
Also, you can't just create a bash shortcut on the desktop, unless you
really know what you're doing (set the start directory in the shortcut
to C:\cygwin\lib). When moving the DLLs out of /bin, you add some new
hurdle to using Cygwin.
Without this problem, or if we decide to ignore this problem, I would be
all for moving DLLs to /lib or /lib64.

Can't cygwin compilation ensure that the cygwin1.dll is always
loaded first so it could take care of this (if only by then adding
to the path)?
(I guess the answer may be no because Windows is juggling with the DLLs...)

Yes, the answer is no. At least it's very likely no. Even if Cygwin is
loaded first, I'm not aware that changing $PATH from within the DLL
entry point will influence the Windows loader for this very process.
Somehow this even sounds like a potential security problem. Otherwise,
for Cygwin distro DLLs it's pretty sure that the Cygwin DLL will be
loaded first, since every subsequent DLL depends on it.

The answer is definitely "no." Windows can map dlls in any order it
pleases, and is only semi-required to initialize them in dependency
order (because dlls are often circular dependent anyway). In particular,
dlls which suffer base address collisions tend to be loaded before
cygwin1.dll

Ryan

[Thomas Wolff]

Post by Ryan Johnson

Post by Corinna Vinschen

Post by Thomas Wolff

Post by Corinna Vinschen
The original idea to keep the DLLs in /bin was to allow running Cygwin
applications from a non-Cygwin process without hassle. A typical
example is CMD. You can simply chdir to C:\cygwin\bin and run `ls'. If
/lib contains all the DLLs, this will break unless C:\cygwin\lib is in
%Path%.
Also, you can't just create a bash shortcut on the desktop, unless you
really know what you're doing (set the start directory in the shortcut
to C:\cygwin\lib). When moving the DLLs out of /bin, you add some new
hurdle to using Cygwin.
Without this problem, or if we decide to ignore this problem, I would be
all for moving DLLs to /lib or /lib64.

Can't cygwin compilation ensure that the cygwin1.dll is always
loaded first so it could take care of this (if only by then adding
to the path)?
(I guess the answer may be no because Windows is juggling with the DLLs...)

Yes, the answer is no. At least it's very likely no. Even if Cygwin is
loaded first, I'm not aware that changing $PATH from within the DLL
entry point will influence the Windows loader for this very process.
Somehow this even sounds like a potential security problem. Otherwise,
for Cygwin distro DLLs it's pretty sure that the Cygwin DLL will be
loaded first, since every subsequent DLL depends on it.

The answer is definitely "no." Windows can map dlls in any order it
pleases, and is only semi-required to initialize them in dependency
order (because dlls are often circular dependent anyway). In
particular, dlls which suffer base address collisions tend to be
loaded before cygwin1.dll
Ryan

Last feable attempt: So why does cygwin have to let Windows load the
DLLs at all? Couldn't the programs just have a stub that uses a
dedicated version of dlopen for dynamic linking?
Thomas

[Corinna Vinschen]

Post by Thomas Wolff

Post by Ryan Johnson

Post by Thomas Wolff
Can't cygwin compilation ensure that the cygwin1.dll is always
loaded first so it could take care of this (if only by then adding
to the path)?
(I guess the answer may be no because Windows is juggling with the DLLs...)

[...]

The answer is definitely "no." Windows can map dlls in any order
it pleases, and is only semi-required to initialize them in
dependency order (because dlls are often circular dependent
anyway). In particular, dlls which suffer base address collisions
tend to be loaded before cygwin1.dll

Last feable attempt: So why does cygwin have to let Windows load the
DLLs at all? Couldn't the programs just have a stub that uses a
dedicated version of dlopen for dynamic linking?

In theory, yes. Every libfoo.dll.a would have to point back to Cygwin
and Cygwin could call LoadLibrary/GetProcAddress so that there's no
explicit load-time linking anymore as far as Cywin DLLs are concerned.
That requires support by gcc/binutils/libtool(?) when building shared
libs, though.


Corinna

[Corinna Vinschen]

Post by Thomas Wolff

Post by Charles Wilson

Post by Corinna Vinschen

Post by Charles Wilson
Which means that the 64bit dlls *must* have a different name, or we
can't have coexistence. E.g.
Oops.

Oops?
It's just a decision to be made, isn't it?

Post by Charles Wilson
...

Right, we have to decide -- my point is there isn't an easy answer that
/bin + /bin64 -- affects $PATH, implicates package naming (*). Also
might affect script #! lines, and anything that calls
exec*() or spawn() with a full pathname.

I have always wondered what the cyg*.dll's are doing in /bin. Not
being a dll insider..., I may be overlooking something, but maybe
it's time to move them over to /usr/lib and then add /usr/lib64,
just as on Linux systems.
Oh, and if it's about the PATH, I don't think any Windows programm
will look for a cyg*.dll, so the cygwin loading function could
probably be hard-coded to add /usr/lib (and /usr/lib64) to the path.

The original idea to keep the DLLs in /bin was to allow running Cygwin
applications from a non-Cygwin process without hassle. A typical
example is CMD. You can simply chdir to C:\cygwin\bin and run `ls'. If
/lib contains all the DLLs, this will break unless C:\cygwin\lib is in
%Path%.

Also, you can't just create a bash shortcut on the desktop, unless you
really know what you're doing (set the start directory in the shortcut
to C:\cygwin\lib). When moving the DLLs out of /bin, you add some new
hurdle to using Cygwin.

Without this problem, or if we decide to ignore this problem, I would be
all for moving DLLs to /lib or /lib64.


Corinna

[Daniel Colascione]

Post by Charles Wilson
Right, we have to decide -- my point is there isn't an easy answer that
/bin + /bin64 -- affects $PATH, implicates package naming (*). Also
might affect script #! lines, and anything that calls
exec*() or spawn() with a full pathname.

So if I have a foo-32bit.exe in a directory that's not /bin or /bin64,
it might segfault on startup depending on whether /bin or /bin64 comes
first in $PATH? And whichever path is correct for foo-32bit.exe will be
incorrect for foo-64bit.exe and vice versa?

[Corinna Vinschen]

Post by Daniel Colascione

Post by Charles Wilson
Right, we have to decide -- my point is there isn't an easy answer that
/bin + /bin64 -- affects $PATH, implicates package naming (*). Also
might affect script #! lines, and anything that calls
exec*() or spawn() with a full pathname.

So if I have a foo-32bit.exe in a directory that's not /bin or /bin64,
it might segfault on startup depending on whether /bin or /bin64 comes
first in $PATH? And whichever path is correct for foo-32bit.exe will be
incorrect for foo-64bit.exe and vice versa?

Now that you mention it, yes, that would be the case if the DLLs have
the same name and are using the same "cyg" prefix.

Which kind of screams for another prefix for 64 bit DLLs. Alternatively
for an entirely different way to load 64 bit DLLs.


Corinna

[Charles Wilson]

Post by Andy Koppe

Post by Corinna Vinschen
- What Windows headers and link libs do we use?

Meaning: can we use MinGW-64's, or do we need to create our own fork
of MinGW's 32-bit ones?

Well, there's no need to "fork" the mingw.org ones -- just post the
patch. The only concern the mingw.org folks would have is provenance of
any changes: are they derived from public documentation (e.g. msdn) and
provide the appropriate link. Given that, approval will be almost
instantaneous, and since the repo is hosted by sourceware, lots of
cygwin people have check-in privileges.

It's bad joss to copy stuff from the Windows SDK headers.

--
Chuck

[Chiheng Xu]

On Sun, Jun 26, 2011 at 7:45 PM, Corinna Vinschen

Post by Corinna Vinschen
As far as I can see what we have to do in about this order is
- Discuss certain basics. This is probably the most crucial step.
- What name should the 64 bit DLL have?
- Where should 64 bit binaries and libs go?
- Do we define "long" as 32 bit or 64 bit type?
- What defines should a 64 bit Cygwin compiler define?
- What Windows headers and link libs do we use?
- Create a x86_64-pc-cygwin cross toolchain.

Perhaps, you can use mingw64 cross toolchain directly, but with
cygwin/cygwin64 's own headers and libs.

mingw64 cross toolchain(32 bits and 64 bits) can be built and tested
in Cygwin environment.

Have you tried to use mingw64 cross toolchain(32 bits) to build Cygwin DLL ?

There is a problem. MinGW.org 's win32api and mingw-runtime are 2
seperate packages, but mingw64 has no seperate win32api or win64api
packages. Cygwin DLL depend on win32api or win64api packages. You
want to modify the w32api to win64api or extract win64api(the headers
, libs) from mingw64 ?

Post by Corinna Vinschen
- Create x86_64 replacements for x86 code and in general try to make the
Cygwin code 64 bit clean where it isn't so far.

Cygwin source code should really be more clean. Using some tools like
Source Insight(http://www.sourceinsight.com/) may be helpful.

Maybe you can't start Cygwin64 project untill Cygwin source code 's
cleanness reach some criteria.

If Cygwin source code is extremely clean, then it would be easy to
make it 64bit clean, I think.

Post by Corinna Vinschen
- Decide how we can integrate 64 bit stuff into the distro. Will we have
a 32 bit and a distinct 64 bit distro? Or, should we stick to a single
distro? If so, how do we separate 64 and 32 bit stuff? What is the
best package layout?

I think single distro but with seperate bin/include/lib directories is good.

In 64bit system, 64bit bin directory can precede 32bit bin directory in PATH.

This may be implemented as Cygwin64.bat, in parallel with Cygwin.bat.

32bit Cygwin will be a self sustained system, in very long time, say,
5~10 years. 32bit Cygwin known nothing about 64bit things.
But 64bit Cygwin can use 32bit Cygwin(the tools) to bootstrap itself
at its baby stage.

[JonY]

Post by Chiheng Xu
On Sun, Jun 26, 2011 at 7:45 PM, Corinna Vinschen

Post by Corinna Vinschen
As far as I can see what we have to do in about this order is
- Discuss certain basics. This is probably the most crucial step.
- What name should the 64 bit DLL have?
- Where should 64 bit binaries and libs go?
- Do we define "long" as 32 bit or 64 bit type?
- What defines should a 64 bit Cygwin compiler define?
- What Windows headers and link libs do we use?
- Create a x86_64-pc-cygwin cross toolchain.

Perhaps, you can use mingw64 cross toolchain directly, but with
cygwin/cygwin64 's own headers and libs.
mingw64 cross toolchain(32 bits and 64 bits) can be built and tested
in Cygwin environment.
Have you tried to use mingw64 cross toolchain(32 bits) to build Cygwin DLL ?
There is a problem. MinGW.org 's win32api and mingw-runtime are 2
seperate packages, but mingw64 has no seperate win32api or win64api
packages. Cygwin DLL depend on win32api or win64api packages. You
want to modify the w32api to win64api or extract win64api(the headers
, libs) from mingw64 ?

Yes, using only the win32api headers is possible, see the --disable-crt
option in the mingw-w64-headers configure. The headers are very much
LLP64 though.

Though mingw-w64-crt doesn't have this facility yet, it is possible to
use only dlltool to build the stub w32api equivalent link libraries for
win64 linking, but without mingw bits like libmingwex.

Post by Chiheng Xu

Post by Corinna Vinschen
- Create x86_64 replacements for x86 code and in general try to make the
Cygwin code 64 bit clean where it isn't so far.

Cygwin source code should really be more clean. Using some tools like
Source Insight(http://www.sourceinsight.com/) may be helpful.
Maybe you can't start Cygwin64 project untill Cygwin source code 's
cleanness reach some criteria.
If Cygwin source code is extremely clean, then it would be easy to
make it 64bit clean, I think.

The hairy parts would be the LP64/LLP64 ABI difference. For now we need
to decide which ABI to go with, the rest of the work will rest on this
decision.

Post by Chiheng Xu

Post by Corinna Vinschen
- Decide how we can integrate 64 bit stuff into the distro. Will we have
a 32 bit and a distinct 64 bit distro? Or, should we stick to a single
distro? If so, how do we separate 64 and 32 bit stuff? What is the
best package layout?

I think single distro but with seperate bin/include/lib directories is good.
In 64bit system, 64bit bin directory can precede 32bit bin directory in PATH.
This may be implemented as Cygwin64.bat, in parallel with Cygwin.bat.
32bit Cygwin will be a self sustained system, in very long time, say,
5~10 years. 32bit Cygwin known nothing about 64bit things.
But 64bit Cygwin can use 32bit Cygwin(the tools) to bootstrap itself
at its baby stage.

I was thinking that 64bit Cygwin will be installed in a completely
separate directory so there is no chance in clobbering 32bit Cygwin
DLLs, like a parallel 1.5/1.7 install.

You also don't really need 32bit Cygwin around once 64bit Cygwin is working.

[Chiheng Xu]

Post by JonY

Cygwin source code should really be more clean.  Using some tools like
Source Insight(http://www.sourceinsight.com/) may be helpful.
Maybe you can't  start Cygwin64 project untill Cygwin source code 's
cleanness reach some criteria.
If Cygwin source code is extremely clean, then it would be easy to
make it 64bit clean, I think.

The hairy parts would be the LP64/LLP64 ABI difference. For now we need
to decide which ABI to go with, the rest of the work will rest on this
decision.

2 kinds of cleanness, general cleanness(easy to read, easy to analyze,
using tools like Source Insight), and 64 bit cleanness.

Post by JonY

I think single distro but with seperate bin/include/lib directories is good.
In 64bit system, 64bit bin directory can precede 32bit  bin directory in PATH.
This may be implemented as Cygwin64.bat, in parallel with Cygwin.bat.
32bit Cygwin will be a self sustained system,  in very long time, say,
 5~10 years.  32bit Cygwin known nothing about 64bit things.
But 64bit Cygwin can use 32bit Cygwin(the tools) to bootstrap itself
at its baby stage.

I was thinking that 64bit Cygwin will be installed in a completely
separate directory so there is no chance in clobbering 32bit Cygwin
DLLs, like a parallel 1.5/1.7 install.
You also don't really need 32bit Cygwin around once 64bit Cygwin is working.

Yes, in a seperate directory, like /cygwin64/{bin,include,lib}

In Cygwin64's bootstrap stage(the coming 5~10 years), you really
need 32bit Cygwin's tools.

[Andy Koppe]

Post by Chiheng Xu

Post by JonY

Cygwin source code should really be more clean.  Using some tools like
Source Insight(http://www.sourceinsight.com/) may be helpful.
Maybe you can't  start Cygwin64 project untill Cygwin source code 's
cleanness reach some criteria.
If Cygwin source code is extremely clean, then it would be easy to
make it 64bit clean, I think.

The hairy parts would be the LP64/LLP64 ABI difference. For now we need
to decide which ABI to go with, the rest of the work will rest on this
decision.

2 kinds of cleanness, general cleanness(easy to read, easy to analyze,
using tools like Source Insight), and 64 bit cleanness.

I don't know what you find so objectionable about the Cygwin sources,
but it's certainly off-topic for this thread. And keeping on
mentioning a proprietary product that has no chance of becoming a
standard tool in the Cygwin developer's toolbox is borderline spam.

Andy

[Andy Koppe]

Post by JonY
I was thinking that 64bit Cygwin will be installed in a completely
separate directory so there is no chance in clobbering 32bit Cygwin
DLLs, like a parallel 1.5/1.7 install.

A clean cut would be ideal, but I fear it would also greatly delay the
project, because a large number of packages would need to be ported
before the 64-bit distro would become viable for users. Supporting
both 32-bit and 64-bit processes and having them interact seemlessly
could avoid that.

This doesn't meant that Cygwin 2 (to coin a handle) still has to be
installable on 32-bit systems, i.e. that 64-bit applications wouldn't
still need 32-bit versions as well.

The situation is different with libraries, where the 32-bit versions
would need to stick around as long as there are still 32-bit
applications using them. Could this be handled with ABI bumps, so for
example libncurses10 would remain 32-bit, but libncurses11 would be
64-bit? I think this would fit into the existing packaging
infrastructure.

Andy

[Corinna Vinschen]

Post by Andy Koppe

Post by JonY
I was thinking that 64bit Cygwin will be installed in a completely
separate directory so there is no chance in clobbering 32bit Cygwin
DLLs, like a parallel 1.5/1.7 install.

A clean cut would be ideal, but I fear it would also greatly delay the
project, because a large number of packages would need to be ported
before the 64-bit distro would become viable for users. Supporting
both 32-bit and 64-bit processes and having them interact seemlessly
could avoid that.

Just as on 64 bit Linux. And just as on Linux there will be
applications which won't be ported to 64 bit for quite some time.

AFAICS, the two DLLs can simply share all data and act as a unity.
There would be not much of a difference, except for running on a
different CPU, kind of. To the best of my knowledge there's no
reason to keep 64 and 32 bit separate.

Post by Andy Koppe
This doesn't meant that Cygwin 2 (to coin a handle) still has to be
installable on 32-bit systems, i.e. that 64-bit applications wouldn't
still need 32-bit versions as well.

There's still 32 bit Cygwin. I don't think we can neglect that.
And, as I said, there's no reason to do that.

Post by Andy Koppe
The situation is different with libraries, where the 32-bit versions
would need to stick around as long as there are still 32-bit
applications using them. Could this be handled with ABI bumps, so for
example libncurses10 would remain 32-bit, but libncurses11 would be
64-bit? I think this would fit into the existing packaging
infrastructure.

That's not an option, IMO. What speaks against doing it the Linux way,
keeping 32 bit libs in {/usr}/lib and 64 bit libs in {/usr}/lib64?

In addition we will probably need a {/usr}/bin64, due to the $PATH
issue, and we probably have to talk about how mount behaves.


Corinna

[Andy Koppe]

Post by Andy Koppe

Post by JonY
I was thinking that 64bit Cygwin will be installed in a completely
separate directory so there is no chance in clobbering 32bit Cygwin
DLLs, like a parallel 1.5/1.7 install.

A clean cut would be ideal, but I fear it would also greatly delay the
project, because a large number of packages would need to be ported
before the 64-bit distro would become viable for users. Supporting
both 32-bit and 64-bit processes and having them interact seemlessly
could avoid that.

Just as on 64 bit Linux.  And just as on Linux there will be
applications which won't be ported to 64 bit for quite some time.
AFAICS, the two DLLs can simply share all data and act as a unity.
There would be not much of a difference, except for running on a
different CPU, kind of.  To the best of my knowledge there's no
reason to keep 64 and 32 bit separate.

Post by Andy Koppe
This doesn't meant that Cygwin 2 (to coin a handle) still has to be
installable on 32-bit systems, i.e. that 64-bit applications wouldn't
still need 32-bit versions as well.

There's still 32 bit Cygwin.   I don't think we can neglect that.
And, as I said, there's no reason to do that.

Post by Andy Koppe
The situation is different with libraries, where the 32-bit versions
would need to stick around as long as there are still 32-bit
applications using them. Could this be handled with ABI bumps, so for
example libncurses10 would remain 32-bit, but libncurses11 would be
64-bit? I think this would fit into the existing packaging
infrastructure.

That's not an option, IMO.

I'm probably missing something obvious, but why?

What speaks against doing it the Linux way,
keeping 32 bit libs in {/usr}/lib and 64 bit libs in {/usr}/lib64?

The prefix hackery that I presume is needed for this. Granted, since
Linux does it this way, it should already exist in most cases.

In addition we will probably need a {/usr}/bin64, due to the $PATH
issue

That one seems more hairy than lib64, unless others have this as well?

Andy

[Corinna Vinschen]

Post by Andy Koppe

Post by Andy Koppe

Post by JonY
I was thinking that 64bit Cygwin will be installed in a completely
separate directory so there is no chance in clobbering 32bit Cygwin
DLLs, like a parallel 1.5/1.7 install.

A clean cut would be ideal, but I fear it would also greatly delay the
project, because a large number of packages would need to be ported
before the 64-bit distro would become viable for users. Supporting
both 32-bit and 64-bit processes and having them interact seemlessly
could avoid that.

Just as on 64 bit Linux.  And just as on Linux there will be
applications which won't be ported to 64 bit for quite some time.
AFAICS, the two DLLs can simply share all data and act as a unity.
There would be not much of a difference, except for running on a
different CPU, kind of.  To the best of my knowledge there's no
reason to keep 64 and 32 bit separate.

Post by Andy Koppe
This doesn't meant that Cygwin 2 (to coin a handle) still has to be
installable on 32-bit systems, i.e. that 64-bit applications wouldn't
still need 32-bit versions as well.

There's still 32 bit Cygwin.   I don't think we can neglect that.
And, as I said, there's no reason to do that.

Post by Andy Koppe
The situation is different with libraries, where the 32-bit versions
would need to stick around as long as there are still 32-bit
applications using them. Could this be handled with ABI bumps, so for
example libncurses10 would remain 32-bit, but libncurses11 would be
64-bit? I think this would fit into the existing packaging
infrastructure.

That's not an option, IMO.

I'm probably missing something obvious, but why?

Maybe I misunderstood you? I thought you meant that libncurses10 is
the last 32 bit version, libncurses11 is the first 64 bit version and
that's that. But now I guess you meant that libncurses10 and
libncurses11 are actually just the same thing, just for different
architectures. Same for the next version which would become
libncurses12 and libncurses13, right?

That might be possible, but it might also be a tricky burden on stuff
like libtool.

Post by Andy Koppe

What speaks against doing it the Linux way,
keeping 32 bit libs in {/usr}/lib and 64 bit libs in {/usr}/lib64?

The prefix hackery that I presume is needed for this. Granted, since
Linux does it this way, it should already exist in most cases.

It does. On a 64 bit Linux system you have a pretty distinction
between the 32 bit stuff in lib and the 64 bit stuff in lib64.

Post by Andy Koppe

In addition we will probably need a {/usr}/bin64, due to the $PATH
issue

That one seems more hairy than lib64, unless others have this as well?

They don't have to. This suggestion is based on the assumption that
we will have 32 and 64 bit DLLs using the same name, and partially
even 32 and 64 bit tools using the same name.

For instance, it would be most easy to keep the Cygwin 32bit package
as it is, with all the tools like mkpasswd, mount, getfacl, etc.
And the 64 bit package would look the same, just that it installs
into /bin64.

I don't claim that this is the way to go, far from it. It's just
some vague idea how we could mix 32 and 64 bit most easily.


Corinna

[Andy Koppe]

Post by Andy Koppe

Post by Corinna Vinschen

Post by Andy Koppe
The situation is different with libraries, where the 32-bit versions
would need to stick around as long as there are still 32-bit
applications using them. Could this be handled with ABI bumps, so for
example libncurses10 would remain 32-bit, but libncurses11 would be
64-bit? I think this would fit into the existing packaging
infrastructure.

That's not an option, IMO.

I'm probably missing something obvious, but why?

Maybe I misunderstood you?  I thought you meant that libncurses10 is
the last 32 bit version, libncurses11 is the first 64 bit version and
that's that.

I did mean that, the implication being that if a package wanted to use
the latest libncurses, it would need to be ported to 64-bit, thereby
limiting the load for library maintainers and providing gentle
pressure towards getting everything ported.

It would also mean omitting support for building 32-bit binaries, in
particular 32-bit -devel packages. Cygwin 1.7 would still be there for
people who do need to build 32-bit stuff, and Cygwin 2 would be able
to run it.

The idea here is to support 32-bit only as far as needed to avoid
having to port the whole distro to 64-bit from day one. Obviously this
doesn't make sense if you don't want to relegate 32-bit to legacy
status.

Post by Andy Koppe

Post by Corinna Vinschen
What speaks against doing it the Linux way,
keeping 32 bit libs in {/usr}/lib and 64 bit libs in {/usr}/lib64?

The prefix hackery that I presume is needed for this. Granted, since
Linux does it this way, it should already exist in most cases.

It does.  On a 64 bit Linux system you have a pretty distinction
between the 32 bit stuff in lib and the 64 bit stuff in lib64.

Post by Andy Koppe

Post by Corinna Vinschen
In addition we will probably need a {/usr}/bin64, due to the $PATH
issue

That one seems more hairy than lib64, unless others have this as well?

They don't have to.  This suggestion is based on the assumption that
we will have 32 and 64 bit DLLs using the same name, and partially
even 32 and 64 bit tools using the same name.

Of course. The point was that if no-one else does this, it's likely to
cause Cygwin-specific challenges. I don't really know what I'm talking
about here, but I think the basic issue would be install rules
assuming that binaries go into /bin, which wouldn't work if the 64-bit
DLLs are in /bin64.

For instance, it would be most easy to keep the Cygwin 32bit package
as it is, with all the tools like mkpasswd, mount, getfacl, etc.
And the 64 bit package would look the same, just that it installs
into /bin64.

With the scheme I outlined above, there could be a single 64-bit
'cygwin' package, version 2.x, much as it is now, plus a cygwin-legacy
package (or some such name) with the 32-bit DLL(s) only.

Andy

[Corinna Vinschen]

Post by Andy Koppe

Maybe I misunderstood you?  I thought you meant that libncurses10 is
the last 32 bit version, libncurses11 is the first 64 bit version and
that's that.

[...]
The idea here is to support 32-bit only as far as needed to avoid
having to port the whole distro to 64-bit from day one. Obviously this
doesn't make sense if you don't want to relegate 32-bit to legacy
status.

No chance to do that. 32 bit systems will exist for a long time yet.
There are still people buying a 32 bit W7 right now.

Post by Andy Koppe

They don't have to.  This suggestion is based on the assumption that
we will have 32 and 64 bit DLLs using the same name, and partially
even 32 and 64 bit tools using the same name.

Of course. The point was that if no-one else does this, it's likely to
cause Cygwin-specific challenges. I don't really know what I'm talking
about here, but I think the basic issue would be install rules
assuming that binaries go into /bin, which wouldn't work if the 64-bit
DLLs are in /bin64.

That's indeed a problem which has to be solved in some way.

Post by Andy Koppe

For instance, it would be most easy to keep the Cygwin 32bit package
as it is, with all the tools like mkpasswd, mount, getfacl, etc.
And the 64 bit package would look the same, just that it installs
into /bin64.

With the scheme I outlined above, there could be a single 64-bit
'cygwin' package, version 2.x, much as it is now, plus a cygwin-legacy
package (or some such name) with the 32-bit DLL(s) only.

Alternatively, we only have a single Cygwin package at all. It unpacks
a 32 bit DLL as well as a 64 bit DLL, as well as the 32 and 64 bit
utils, but only one set of headers. On all systems.


Corinna

[Daniel Colascione]

Post by Andy Koppe

Post by Corinna Vinschen
What speaks against doing it the Linux way,
keeping 32 bit libs in {/usr}/lib and 64 bit libs in {/usr}/lib64?

The prefix hackery that I presume is needed for this. Granted, since
Linux does it this way, it should already exist in most cases.

Post by Corinna Vinschen
In addition we will probably need a {/usr}/bin64, due to the $PATH
issue

Why?

Today, Cygwin DLLs all have the form cyg$SONAME.dll. In a 64-bit Cygwin,
32-bit DLLs would retain this naming convention and 64-bit DLLs would be
named cyg64$SONAME.dll. Executables and both kinds of DLL would still go
in /bin, allowing either the 32-bit or the 64-bit version of a program
to be installed, but not both. (Different word length binaries could
exist in another directory, of course.) Static libraries and import
libraries would go in /lib and /lib64 for 32- and 64-bit versions,
respectively, with the compiler choosing the correct option.

As for the choice between LP64 and LLP64 --- would it be evil to make
long 64-bit, but define DWORD, LONG, ULONG and so on to be 32 bits wide?
This way, both Windowsish and POSIXish code will continue to work fine.
Of course, this scheme breaks anything assumes sizeof(long) ==
sizeof(LONG), but every available option will break something.

[Yaakov (Cygwin/X)]

Post by Daniel Colascione
Today, Cygwin DLLs all have the form cyg$SONAME.dll. In a 64-bit Cygwin,
32-bit DLLs would retain this naming convention and 64-bit DLLs would be
named cyg64$SONAME.dll.

1) A similar naming scheme for mingw64 was already rejected.
2) Accomodating the "cyg" prefix is hard enough; using different naming
schemes for x86 and x64 would be a disaster.

I still don't see a pressing need for multilib in the first place.
Linux distros do it only with a handful of libraries needed for running
third-party binaries; that use case doesn't really apply to Cygwin, so
why would we need it?

Post by Daniel Colascione
As for the choice between LP64 and LLP64 --- would it be evil to make
long 64-bit, but define DWORD, LONG, ULONG and so on to be 32 bits wide?
This way, both Windowsish and POSIXish code will continue to work fine.

That seems to be the consensus, and it makes sense.


Yaakov

[Daniel Colascione]

Post by Yaakov (Cygwin/X)

Post by Daniel Colascione
Today, Cygwin DLLs all have the form cyg$SONAME.dll. In a 64-bit Cygwin,
32-bit DLLs would retain this naming convention and 64-bit DLLs would be
named cyg64$SONAME.dll.

1) A similar naming scheme for mingw64 was already rejected.
2) Accomodating the "cyg" prefix is hard enough; using different naming
schemes for x86 and x64 would be a disaster.

How so? The logic for the dealing with the prefix already exists.
Wouldn't it be a relatively simple matter to make the string used for
the prefix vary based on the current architecture, especially if both
variants begin with "cyg"?

[Yaakov (Cygwin/X)]

Post by Daniel Colascione

Post by Yaakov (Cygwin/X)
2) Accomodating the "cyg" prefix is hard enough; using different naming
schemes for x86 and x64 would be a disaster.

How so? The logic for the dealing with the prefix already exists.
Wouldn't it be a relatively simple matter to make the string used for
the prefix vary based on the current architecture, especially if both
variants begin with "cyg"?

The library prefix is coded in lots of places, from build systems
(libtool, cmake, qmake, waf, etc.) to module loaders (GModule, QLibrary,
libltdl, and dozens if not hundreds of programs which handle their own
module loading). Trying to change all of those to be different on two
different architectures would NOT be a "simple matter".

Besides, wrt to modules, in a multilib setup you have two different
library paths for x86 and x64, so they don't need separate prefixes
anyway (and not always do they use a prefix at all, or you have KDE4
where the module prefix must be "lib" if it exists) and making different
prefixes for shared libraries and loadable modules is out of the
question.

But this still begs the bigger question, why do we need multilib support
at all?


Yaakov

[Andy Koppe]

Post by Yaakov (Cygwin/X)
The library prefix is coded in lots of places, from build systems
(libtool, cmake, qmake, waf, etc.) to module loaders (GModule, QLibrary,
libltdl, and dozens if not hundreds of programs which handle their own
module loading).  Trying to change all of those to be different on two
different architectures would NOT be a "simple matter".
Besides, wrt to modules, in a multilib setup you have two different
library paths for x86 and x64, so they don't need separate prefixes
anyway (and not always do they use a prefix at all, or you have KDE4
where the module prefix must be "lib" if it exists) and making different
prefixes for shared libraries and loadable modules is out of the
question.

Can you comment on how difficult it would be to deal with lib64 and
particularly bin64 (due to DLLs having to go into bin*)?

Post by Yaakov (Cygwin/X)
But this still begs the bigger question, why do we need multilib support
at all?

To avoid having to port a large part of the distro before 64-bit
Cygwin becomes usable, and for more seamless compatibility with users'
existing 32-bit binaries and sources than you'd get with a separate
Cygwin32 install.

Do those benefits outweigh the costs? I don't know.

Andy

[Yaakov (Cygwin/X)]

Post by Andy Koppe
Can you comment on how difficult it would be to deal with lib64 and
particularly bin64 (due to DLLs having to go into bin*)?

lib64 is doable, the widely-used build systems support it, but a
significant number of packages still require patching or argument
overriding to use it. Fortunately, both Fedora and Gentoo
use /usr/lib64 on x86_64, so we can benefit from their work.

OTOH, bin64 is not even a FHS-compliant directory.

Post by Andy Koppe
To avoid having to port a large part of the distro before 64-bit
Cygwin becomes usable, and for more seamless compatibility with users'
existing 32-bit binaries and sources than you'd get with a separate
Cygwin32 install.
Do those benefits outweigh the costs? I don't know.

I strongly doubt it. Once you get past porting the toolchain and
lower-level stuff, everything beyond that should be more-or-less a
rebuild. (Famous last words :-)

As long as 32-bit and 64-bit installs could exist side-by-side (like we
had with the 1.5-to-1.7 transition), then I really don't see it being an
issue.


Yaakov

[Corinna Vinschen]

Post by Yaakov (Cygwin/X)

Post by Andy Koppe
To avoid having to port a large part of the distro before 64-bit
Cygwin becomes usable, and for more seamless compatibility with users'
existing 32-bit binaries and sources than you'd get with a separate
Cygwin32 install.
Do those benefits outweigh the costs? I don't know.

I strongly doubt it. Once you get past porting the toolchain and
lower-level stuff, everything beyond that should be more-or-less a
rebuild. (Famous last words :-)
As long as 32-bit and 64-bit installs could exist side-by-side (like we
had with the 1.5-to-1.7 transition), then I really don't see it being an
issue.

I have a different point of view here. As it is, we have a rather low
participation. Yes, we have a number of dedicated maintainers, but for
most of the maintainers it's just the occasional odd job. I don't think
you will get a lot of friends by asking all maintainers to create
packages for two separate distros.

This leaves two ways to handle the problem:

- A buildsystem

Sounds good, but we don't have something like this. We don't have an
infrastructure to set up a build system, not everyone uses the same
way to build packages. In short, we're not as organized as, say,
Fedora and I don't see that we have a big chance to become as
organized. Especially the buildsystem needs dedicated maintainers.

- Keep everything in a single distro.

This is the more realistic option, IMO. Some packages will stick
to 32 bit and not get a 64 bit counterpart for some time. Some
packages will get a 32 and a 64 bit part.

It's a bit early to discuss how mixed packages should exactly look
like. But *my* idea would be that the burden on the maintainers
stays low, and that a user grabs the one-and-only setup.exe and
installs Cygwin just as today.


Corinna

[Earnie]

I've been reading with much interest the posts of this thread. To help
control which libraries are used by which programs and allow one Cygwin
DLL to control both we could consider two differing entry points into
the DLL, the existing one for 32bit and a new one for 64bit. Cygwin
would then know which pointer size to use because the initialization
would be different. Would this work, I don't know but thought I would
throw the idea into the mix.

I was quite shocked when I stumbled across to variants LP64 vs LLP64 and
to learn that the long data type might represent 32 bits. That was
just, well, ignorant and I agree that the sizeof(long) == sizeof(void *)
makes sense except for the fact that the C standard states in the final
draft version in section 5.2.4.2.1 the size of data type long is 2^32 -
1 and defines long long as 2^64 - 1. It doesn't account for the
variance in the sizeof(void *) except to say that it is equivalent to
size of any other pointer of any other data type. Because of this LLP64
makes a lot of sense. But I agree that Cygwin has to bridge the two
variants since LP64 is used by Linux systems and allow for the data type
to shift from the C standard and use this model but the data type still
need to follow bit size pattern of char < short < int < long == pointer
< long long. While the native ``MinGW'' should follow the LLP64 model
for the bit size pattern of char < short < int < long < pointer == long
long.

[Eric Blake]

Post by Earnie
just, well, ignorant and I agree that the sizeof(long) == sizeof(void *)
makes sense except for the fact that the C standard states in the final
draft version in section 5.2.4.2.1 the size of data type long is 2^32 -
1 and defines long long as 2^64 - 1.

Those are minimum compliant sizes, not absolutes. A 64-bit long is
perfectly legal by the C standard. Likewise, a 16-bit int is perfectly
legal by the C standard, even though these days int is almost always
32-bits.

Post by Earnie
It doesn't account for the
variance in the sizeof(void *) except to say that it is equivalent to
size of any other pointer of any other data type. Because of this LLP64
makes a lot of sense.

In my mind, LLP64 makes absolutely zero sense. Microsoft broke lots of
programs that previously used to assume that any pointer could be
converted to a long and back. LP64 is the way to go for cygwin.

[Teemu Nätkinniemi]

Post by Eric Blake
In my mind, LLP64 makes absolutely zero sense. Microsoft broke lots of
programs that previously used to assume that any pointer could be
converted to a long and back. LP64 is the way to go for cygwin.

If LP64 is chosen as the model then for the toolchain it might be useful
to see if Interix' GCC 4.6 and Binutils could be used as a starting
point. The sources haven't been released yet as the new GCC is still in
alpha/beta phase.

For more information:

http://www.suacommunity.com/forum/tm.aspx?m=19087

http://www.suacommunity.com/forum/tm.aspx?m=18858

Teemu

[JonY]

Post by Teemu Nätkinniemi

Post by Eric Blake
In my mind, LLP64 makes absolutely zero sense. Microsoft broke lots of
programs that previously used to assume that any pointer could be
converted to a long and back. LP64 is the way to go for cygwin.

If LP64 is chosen as the model then for the toolchain it might be useful
to see if Interix' GCC 4.6 and Binutils could be used as a starting
point. The sources haven't been released yet as the new GCC is still in
alpha/beta phase.
http://www.suacommunity.com/forum/tm.aspx?m=19087
http://www.suacommunity.com/forum/tm.aspx?m=18858
Teemu

OK, who wants to start on LP64 for x86_64-pc-cygwin on trunk GCC? :)

I suppose the binutils part would be exactly similar to mingw-w64's?
Just need some more bits to recognize the new triplet.

[Yaakov (Cygwin/X)]

Post by JonY
OK, who wants to start on LP64 for x86_64-pc-cygwin on trunk GCC? :)
I suppose the binutils part would be exactly similar to mingw-w64's?
Just need some more bits to recognize the new triplet.

I think this should do the trick:

http://cygwin-ports.git.sourceforge.net/git/gitweb.cgi?p=cygwin-ports/cygwin64-binutils

This builds and seems to DTRT with reading x86_64-w64-mingw32 binaries.


Yaakov

[Corinna Vinschen]

Post by Yaakov (Cygwin/X)

Post by JonY
OK, who wants to start on LP64 for x86_64-pc-cygwin on trunk GCC? :)
I suppose the binutils part would be exactly similar to mingw-w64's?
Just need some more bits to recognize the new triplet.

http://cygwin-ports.git.sourceforge.net/git/gitweb.cgi?p=cygwin-ports/cygwin64-binutils
This builds and seems to DTRT with reading x86_64-w64-mingw32 binaries.

Oh, cool. I'll announce Cygwin64. ;)


Corinna

[Teemu Nätkinniemi]

Post by Corinna Vinschen

Post by Yaakov (Cygwin/X)

Post by JonY
OK, who wants to start on LP64 for x86_64-pc-cygwin on trunk GCC? :)
I suppose the binutils part would be exactly similar to mingw-w64's?
Just need some more bits to recognize the new triplet.

http://cygwin-ports.git.sourceforge.net/git/gitweb.cgi?p=cygwin-ports/cygwin64-binutils
This builds and seems to DTRT with reading x86_64-w64-mingw32 binaries.

Oh, cool. I'll announce Cygwin64. ;)

What will the new host triplet look like? x86_64-pc-cygwin or
x86_64-pc-cygwin64?

I just checked out GCC trunk or maybe it was unnecessary as Yaakov is
about to finish porting it as well ;)

Teemu

[Corinna Vinschen]

Post by Teemu Nätkinniemi

Post by Corinna Vinschen

Post by Yaakov (Cygwin/X)

Post by JonY
OK, who wants to start on LP64 for x86_64-pc-cygwin on trunk GCC? :)
I suppose the binutils part would be exactly similar to mingw-w64's?
Just need some more bits to recognize the new triplet.

http://cygwin-ports.git.sourceforge.net/git/gitweb.cgi?p=cygwin-ports/cygwin64-binutils
This builds and seems to DTRT with reading x86_64-w64-mingw32 binaries.

Oh, cool. I'll announce Cygwin64. ;)

What will the new host triplet look like? x86_64-pc-cygwin or
x86_64-pc-cygwin64?

x86_64-pc-cygwin

There's no cygwin64 target, just like there's no linux64 target. The
only difference is the targeted CPU.


Corinna

[Corinna Vinschen]

Post by Earnie
I've been reading with much interest the posts of this thread. To help
control which libraries are used by which programs and allow one Cygwin
DLL to control both we could consider two differing entry points into
the DLL, the existing one for 32bit and a new one for 64bit. Cygwin
would then know which pointer size to use because the initialization
would be different. Would this work, I don't know but thought I would
throw the idea into the mix.

There's no reason for this and it also wouldn't work. Windows does not
allow 64 bit processes to load 32 bit DLLs and vice versa. However,
since each Cygwin DLL is local to the process and sharing of common data
is performed using handle inheritance, shared memory, named objects, etc.,
there's no reason that a 32 bit and a 64 bit Cygwin DLL couldn't work
together and constitute a single system with shared process lists, shared
tty lists, shared mounts, etc.


Corinna

[Corinna Vinschen]

Post by Yaakov (Cygwin/X)

Post by Daniel Colascione
Today, Cygwin DLLs all have the form cyg$SONAME.dll. In a 64-bit Cygwin,
32-bit DLLs would retain this naming convention and 64-bit DLLs would be
named cyg64$SONAME.dll.

1) A similar naming scheme for mingw64 was already rejected.
2) Accomodating the "cyg" prefix is hard enough; using different naming
schemes for x86 and x64 would be a disaster.
I still don't see a pressing need for multilib in the first place.
Linux distros do it only with a handful of libraries needed for running
third-party binaries; that use case doesn't really apply to Cygwin, so
why would we need it?

On my F15 desktop system I have 124 i686 packages. It sounds like a
joke compared to the 1810 x86_64 and 445 noarch packages, but it's
still not a neglectable number.

And I still think Cygwin is different due to our low participation and
organization factor. Just imagine Cygwin had as much backing as Fedora.
For me this would mean I could drop reading the cygwin and cygwin-apps
lists, because I could stick to Cygwin development alone. I didn't have
to maintain any distro packages. I wouldn't even have to create the
cygwin package because somebody else did it for me. Wow, what a dream...


Corinna

[Charles Wilson]

Post by Yaakov (Cygwin/X)

Post by Daniel Colascione
Today, Cygwin DLLs all have the form cyg$SONAME.dll. In a 64-bit Cygwin,
32-bit DLLs would retain this naming convention and 64-bit DLLs would be
named cyg64$SONAME.dll.

1) A similar naming scheme for mingw64 was already rejected.

And I think that decision was a mistake. It means that you can't have
32bit applications and 64bit applications coexist in the same
installation tree -- and until the entire universe of applications is
ported to 64bit, you're stuck with two separate installations, one which
is "complete" but 32bit only, and one which is crippled by missing apps
and libs but -- hey, it's 64bit!

Post by Yaakov (Cygwin/X)
2) Accomodating the "cyg" prefix is hard enough; using different naming
schemes for x86 and x64 would be a disaster.

You've already made the necessary changes so that DLLPREFIX takes on the
correct #defined value based on $host. This is a relatively simple
addition since you've already done all the hard work.

Post by Yaakov (Cygwin/X)
I still don't see a pressing need for multilib in the first place.
Linux distros do it only with a handful of libraries needed for running
third-party binaries; that use case doesn't really apply to Cygwin, so
why would we need it?

Because (a) they didn't launch the 64bit version until they had
recompiled the entire distro for the new platform -- having the manpower
to do that in a relatively short time span. We don't have that luxury;
we'll be stuck without 64bit versions of lots of our distro for quite
some time. While the 64bit distro is crippled, nobody will use it, so
it will get less testing...and thus, even SLOWER development/deployment.

(b) If you DO allow multilib, we have to ensure that the correct bitness
DLLs are loaded by the exes. There are only two ways to do this: (1)
$PATH separation *and* installation separation -- /bin64/*.exe +
/bin64/*.dll, or (2) different DLL names (e.g. the cyg64 prefix thing).

One or the other. Choose. (I realize you want to pick option #3: don't
choose, by pushing a complete break from 32bit via 'no multilib' -- I
don't think that's realistic, given the inevitable long crippleness
period it will create for the new 64bit platform).

--
Chuck

[Corinna Vinschen]

Post by Charles Wilson
(b) If you DO allow multilib, we have to ensure that the correct bitness
DLLs are loaded by the exes.

Again, this is *no* problem since Windows doesn't allow to load
x86_64 DLLs from a i686 executable and vice versa.

Maybe all who are interested in a 64 bit Cygwin should read this:

http://msdn.microsoft.com/en-us/library/dd371688%28VS.85%29.aspx


Corinna

[Yaakov (Cygwin/X)]

Post by Charles Wilson
You've already made the necessary changes so that DLLPREFIX takes on the
correct #defined value based on $host. This is a relatively simple
addition since you've already done all the hard work.

No it's not, but thanks for asking. :-)

Post by Charles Wilson
Because (a) they didn't launch the 64bit version until they had
recompiled the entire distro for the new platform -- having the manpower
to do that in a relatively short time span. We don't have that luxury;
we'll be stuck without 64bit versions of lots of our distro for quite
some time. While the 64bit distro is crippled, nobody will use it, so
it will get less testing...and thus, even SLOWER development/deployment.

Right, you can't test binaries that don't exist yet. How does it help
to have 32- and 64-bit binaries in one tree instead of two when you have
exactly the same number of 64-bit binaries?

Your arguments would be equally solved with the ability to run 32- and
64-bit distros side-by-side, and I agree that that is essential. But
the use case for multilib on Linux is so that users can download and use
32-bit-only binaries from third-parties, and that really doesn't apply
to Cygwin.

As for the time it will take to get a fully operational 64-bit distro,
there are steps we can take now to help expedite things:

* Currently package maintainers are free to choose package building
systems, so we have cygport, cygbuild, g-b-s, netrel, and who knows what
else. Agreeing on one would help both maintainer transitions and also
the following points.

* Currently there is no central repository for packaging sources. Other
distros all have one place where all packaging sources are kept, so that
all maintainers can see them, help out as necessary, make an emergency
update or add a patch when security vulnerabilities arise, and take over
more easily when others leave.

* Currently only package maintainers can initiate updates or rebuilds.
For example, if ncurses has another ABI bump (and its agreed that it's
actually necessary), then someone should be able to rebuild all
ncurses-dependent packages (without making other changes) for the new
version so that we don't need to keep old ABIs for years on end.

* Currently we're only a 32-bit distro, so anyone who can use Cygwin can
contribute a package. But when we support 64-bit, we can't expect all
maintainers to have a 64-bit machine handy. Having a buildbot system
would solve this and the previous point.

There are other points as well, but I'll stop here for now.


Yaakov

[Marco atzeri]

Post by Yaakov (Cygwin/X)
* Currently we're only a 32-bit distro, so anyone who can use Cygwin can
contribute a package. But when we support 64-bit, we can't expect all
maintainers to have a 64-bit machine handy. Having a buildbot system
would solve this and the previous point.

My current build machine is a XP32bit
I don't expect to have a 64-bit windows machine in the short term
(except if my old one crash ad I am forced to buy a new one),
so a build bot will be appreciated

Post by Yaakov (Cygwin/X)
There are other points as well, but I'll stop here for now.
Yaakov

Marco

[Charles Wilson]

Post by Andy Koppe
The situation is different with libraries, where the 32-bit versions
would need to stick around as long as there are still 32-bit
applications using them. Could this be handled with ABI bumps, so for
example libncurses10 would remain 32-bit, but libncurses11 would be
64-bit? I think this would fit into the existing packaging
infrastructure.

Doing this would imply a permanent cessation of support for 32bit
ncurses. If you ever needed to bump the ABI for ncurses on
32bit...you're stuck.

So, one alternative is to "leave room" -- bump ALL ABIs by 10 or so, for
the 64bit versions.

Now go ask Yaakov if he'd like that "solution" for the entire gnome
stack -- where you have to know the DLL name, including the ABI#, for
all the various dlopen() calls...

Yaakov's already annoyed at me for the inequality between "standard" ABI
version numbers and our cygwin ones, which are sometimes different due
to historical accident or error, but we are now stuck with them.

--
Chuck

[Yaakov (Cygwin/X)]

Post by Charles Wilson
So, one alternative is to "leave room" -- bump ALL ABIs by 10 or so, for
the 64bit versions.
Now go ask Yaakov if he'd like that "solution" for the entire gnome
stack -- where you have to know the DLL name, including the ABI#, for
all the various dlopen() calls...

Thanks for saving my breath, because you're right, I'm absolutely
against this approach. I can't even begin describing how wrong it is.

Post by Charles Wilson
Yaakov's already annoyed at me for the inequality between "standard" ABI
version numbers and our cygwin ones, which are sometimes different due
to historical accident or error, but we are now stuck with them.

True to a certain extent, but there were other ways of handling it (e.g.
why did libncursesw have to inherit the mess of libncurses/narrow? It
could have been 5 or 6 instead of 10). We could even have the chance to
undo some of this by starting from scratch with x64 if we wanted to.


Yaakov

[Charles Wilson]

Post by Yaakov (Cygwin/X)

Post by Charles Wilson
Yaakov's already annoyed at me for the inequality between "standard" ABI
version numbers and our cygwin ones, which are sometimes different due
to historical accident or error, but we are now stuck with them.

True to a certain extent, but there were other ways of handling it (e.g.
why did libncursesw have to inherit the mess of libncurses/narrow? It
could have been 5 or 6 instead of 10). We could even have the chance to
undo some of this by starting from scratch with x64 if we wanted to.

With this, I agree -- mostly: if we go with Corinna's "combined package"
approach, then cygport (or other build system tools) will need the
ability to "repatch" and maybe even reautoconf between bitness
builds...icky...

Furthermore, on x64, there will only be one ncurses library and it will
support the wide interface (in addition to the narrow one). E.g.
(64)ncurses will be a 64bit version of (32)ncursesw.

--
Chuck

[Corinna Vinschen]

Post by Charles Wilson

Post by Yaakov (Cygwin/X)

Post by Charles Wilson
Yaakov's already annoyed at me for the inequality between "standard" ABI
version numbers and our cygwin ones, which are sometimes different due
to historical accident or error, but we are now stuck with them.

True to a certain extent, but there were other ways of handling it (e.g.
why did libncursesw have to inherit the mess of libncurses/narrow? It
could have been 5 or 6 instead of 10). We could even have the chance to
undo some of this by starting from scratch with x64 if we wanted to.

With this, I agree -- mostly: if we go with Corinna's "combined package"
approach, then cygport (or other build system tools) will need the
ability to "repatch" and maybe even reautoconf between bitness
builds...icky...
Furthermore, on x64, there will only be one ncurses library and it will
support the wide interface (in addition to the narrow one). E.g.
(64)ncurses will be a 64bit version of (32)ncursesw.

Erm... btw., is there a way to say in less then 100 words why ncursesw
can't be ncurses on 32 bit?


Corinna

[Thomas Wolff]

Post by Corinna Vinschen

Post by Charles Wilson

Post by Yaakov (Cygwin/X)

Post by Charles Wilson
Yaakov's already annoyed at me for the inequality between "standard" ABI
version numbers and our cygwin ones, which are sometimes different due
to historical accident or error, but we are now stuck with them.

True to a certain extent, but there were other ways of handling it (e.g.
why did libncursesw have to inherit the mess of libncurses/narrow? It
could have been 5 or 6 instead of 10). We could even have the chance to
undo some of this by starting from scratch with x64 if we wanted to.

With this, I agree -- mostly: if we go with Corinna's "combined package"
approach, then cygport (or other build system tools) will need the
ability to "repatch" and maybe even reautoconf between bitness
builds...icky...
Furthermore, on x64, there will only be one ncurses library and it will
support the wide interface (in addition to the narrow one). E.g.
(64)ncurses will be a 64bit version of (32)ncursesw.

Erm... btw., is there a way to say in less then 100 words why ncursesw
can't be ncurses on 32 bit?
Corinna

I guess because --enable-widec is a configure option of the upstream
source package, and API functions have the same names in both versions,
so they can't be merged.
See, however, the comment I'll write to that other thread in a minute
(about clearw).
Thomas

[Yaakov (Cygwin/X)]

Post by Corinna Vinschen
Erm... btw., is there a way to say in less then 100 words why ncursesw
can't be ncurses on 32 bit?

There isn't as AFAICS, as long as you provide the appropriate symlinks
for the w-less names in the devel components.


Yaakov

[Charles Wilson]

Post by Corinna Vinschen

Post by Charles Wilson
Furthermore, on x64, there will only be one ncurses library and it will
support the wide interface (in addition to the narrow one). E.g.
(64)ncurses will be a 64bit version of (32)ncursesw.

Erm... btw., is there a way to say in less then 100 words why ncursesw
can't be ncurses on 32 bit?

If you can copy cygncursesw* onto cygncurses* without breaking anything,
then I can make this happen without too much trouble.

If you can't, then...the only way to do it would be Yet Another DLL
Version Bump, and then we slowly wait for folks to undo their existing
-I/usr/include/ncursesw stuff...

The reason we currently have two, is simply historical hesitation on my
part -- I didn't want to "break" existing (narrow) apps, and just wasn't
sure we could deploy ncursesw as THE ncurses. Now, I'm less worried,
but we (are/might be) a little bit stuck.

--
Chuck

[Yaakov (Cygwin/X)]

Post by Charles Wilson
If you can copy cygncursesw* onto cygncurses* without breaking anything,
then I can make this happen without too much trouble.
If you can't, then...the only way to do it would be Yet Another DLL
Version Bump, and then we slowly wait for folks to undo their existing
-I/usr/include/ncursesw stuff...
The reason we currently have two, is simply historical hesitation on my
part -- I didn't want to "break" existing (narrow) apps, and just wasn't
sure we could deploy ncursesw as THE ncurses. Now, I'm less worried,
but we (are/might be) a little bit stuck.

The solution is *much* simpler than that:

1) Build ncurses *once* with wide char support.

2) Make sure the libncurses-devel package provides symlinks for the
w-less names:
- libncurses.{a,dll.a,la} -> libncursesw.{a,dll.a,la}
- /usr/include/ncurses -> /usr/include/ncursesw)
The headers should also be symlinked/copied into /usr/include, as it
used to be.

3) Obsolete the following packages:
- libncursesw-devel -> libncurses-devel
- ncursesw -> ncurses
- ncursesw-demo -> ncurses-demo
- libncurses10 (until dependent packages are rebuilt)


Yaakov

[Andy Koppe]

Post by Yaakov (Cygwin/X)

Post by Charles Wilson
If you can copy cygncursesw* onto cygncurses* without breaking anything,
then I can make this happen without too much trouble.
If you can't, then...the only way to do it would be Yet Another DLL
Version Bump, and then we slowly wait for folks to undo their existing
-I/usr/include/ncursesw stuff...
The reason we currently have two, is simply historical hesitation on my
part -- I didn't want to "break" existing (narrow) apps, and just wasn't
sure we could deploy ncursesw as THE ncurses.  Now, I'm less worried,
but we (are/might be) a little bit stuck.

1) Build ncurses *once* with wide char support.
2) Make sure the libncurses-devel package provides symlinks for the
  - libncurses.{a,dll.a,la} -> libncursesw.{a,dll.a,la}
  - /usr/include/ncurses -> /usr/include/ncursesw)
The headers should also be symlinked/copied into /usr/include, as it
used to be.
  - libncursesw-devel -> libncurses-devel
  - ncursesw -> ncurses
  - ncursesw-demo -> ncurses-demo
  - libncurses10 (until dependent packages are rebuilt)

Separate thread, please.

Andy

[Yaakov (Cygwin/X)]

Post by Charles Wilson
With this, I agree -- mostly: if we go with Corinna's "combined package"
approach, then cygport (or other build system tools) will need the
ability to "repatch" and maybe even reautoconf between bitness
builds...icky...

Combined packages aren't the right approach either. What we need to do
is have one set of packaging sources which are used to build two
separate 32- and 64-bit packages. cygport will of course need some
adjustments but it shouldn't bee

Post by Charles Wilson
Furthermore, on x64, there will only be one ncurses library and it will
support the wide interface (in addition to the narrow one). E.g.
(64)ncurses will be a 64bit version of (32)ncursesw.

I think that makes sense in principle.


Yaakov

[Yaakov (Cygwin/X)]

Post by Corinna Vinschen
I'm not sure if you agree, but as far as I can see, 32 bit systems are
more and more reduced to a niche market, namely Netbooks and other very
small systems. On the Desktop, 32 bit is declining fast, in the server
segment it's practically dead.
Given this, I'm wondering how much future Cygwin has if we stick to
32 bit. I think it will be pretty limited. In fact, we're probably
rather late in the game.

Compared to whom? I run very little Windows-native software anymore,
but when I look at the large open-source projects, such as Mozilla,
Google Chrome, or LibreOffice, they still don't provide 64-bit Windows
binaries. True, that's all desktop software, but I'll admit that
desktop software is my focus.

Post by Corinna Vinschen
- What name should the 64 bit DLL have?

This depends on whether there will be any multilib support on x64
systems or not, and to what extent.

Post by Corinna Vinschen
- Where should 64 bit binaries and libs go?

If no multilib, then just use /usr/bin and /usr/lib; forcing /usr/lib64
is more trouble than its worth. If there would be multilib support, I
think it depends on the extent thereof.

Post by Corinna Vinschen
- Do we define "long" as 32 bit or 64 bit type?

IIRC on Linux, x64 long is 64 bits. Going against that assumption might
make it more difficult to build other FOSS software for Cygwin x64.

Post by Corinna Vinschen
- What defines should a 64 bit Cygwin compiler define?

__CYGWIN__ and __X86_64__. I think we want to avoid __CYGWIN64__ in
case MS trademarks "Win64", if they haven't already.

Post by Corinna Vinschen
- What Windows headers and link libs do we use?

Good question. mingw.org headers do not currently support x64, but
using MinGW-w64's headers would require a lot of changes. (Trying to
build setup.exe with mingw64 certainly did.) Unless the two communities
merge or it is apparent that only one will continue long term, then I
think we choose whichever would be easier, and use those headers/implibs
for both x86 and x64.

Post by Corinna Vinschen
- Create a x86_64-pc-cygwin cross toolchain.

Binutils and GCC already have PE32+ support thanks to MinGW-w64, so that
shouldn't be too difficult, at least for C/C++.

Post by Corinna Vinschen
- Create x86_64 replacements for x86 code and in general try to make the
Cygwin code 64 bit clean where it isn't so far.

Another question: which version of Windows x64 would we start with,
Vista or 7? Yes, there was a 64-bit XP but that was a long time ago and
AFAIK never widely adopted. Starting support from a later version would
allow us to #ifndef a fair bit of code (and autoloads) needed only for
2K/XP.

Post by Corinna Vinschen
- Decide how we can integrate 64 bit stuff into the distro. Will we have
a 32 bit and a distinct 64 bit distro? Or, should we stick to a single
distro? If so, how do we separate 64 and 32 bit stuff?

Linux distros with which I am familiar provide separate repos for 32-
and 64-bit systems (i686 and x86_64/amd64), with the 64-bit repo adding
32-bit builds of certain libraries for running x86 binaries downloaded
from elsewhere. Since that use case doesn't really apply to Cygwin,
multilib support shouldn't be necessary.

Post by Corinna Vinschen
What is the best package layout?

One factor is that we would have to completely change how packages are
maintained. Currently, maintainers are responsible for building their
own packages, but what if a maintainer only has a 32-bit system? I
think we would need a buildbot-type system for building the actual
packages.

Post by Corinna Vinschen
This is a big project which can only work if we have help and support
from the community.

True, but $ome $upport from RH wouldn't hurt. :-)

Post by Corinna Vinschen
- What part of the project would most interest you to help? Coding
Cygwin? Documentation? Setup? Toolchain? You name it.

Toolchain, bootstrapping, and porting packages.


Yaakov

[Corinna Vinschen]

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen
I'm not sure if you agree, but as far as I can see, 32 bit systems are
more and more reduced to a niche market, namely Netbooks and other very
small systems. On the Desktop, 32 bit is declining fast, in the server
segment it's practically dead.
Given this, I'm wondering how much future Cygwin has if we stick to
32 bit. I think it will be pretty limited. In fact, we're probably
rather late in the game.

Compared to whom? I run very little Windows-native software anymore,
but when I look at the large open-source projects, such as Mozilla,
Google Chrome, or LibreOffice, they still don't provide 64-bit Windows
binaries. True, that's all desktop software, but I'll admit that
desktop software is my focus.

Server software is mostly 64 bit now. Latest MSFT products require
a 64 bit system (Exchange Server, for instance). Cygwin is not only
used on the desktop.

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen
- What defines should a 64 bit Cygwin compiler define?

__CYGWIN__ and __X86_64__. I think we want to avoid __CYGWIN64__ in
case MS trademarks "Win64", if they haven't already.

Good point.

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen
- What Windows headers and link libs do we use?

Good question. mingw.org headers do not currently support x64, but
using MinGW-w64's headers would require a lot of changes. (Trying to
build setup.exe with mingw64 certainly did.) Unless the two communities
merge or it is apparent that only one will continue long term, then I
think we choose whichever would be easier, and use those headers/implibs
for both x86 and x64.

Maybe we can use the current w32api with only a few changes. After all,
Cygwin only needs the kernel and socket bits and just a teenty little
bit on top of that. Shaking this out of w32api shouldn't be too hard.

Post by Yaakov (Cygwin/X)
Another question: which version of Windows x64 would we start with,
Vista or 7? Yes, there was a 64-bit XP but that was a long time ago and
AFAIK never widely adopted. Starting support from a later version would
allow us to #ifndef a fair bit of code (and autoloads) needed only for
2K/XP.

Good question. From my POV, the Cygwin code should be ported in a way
which allows to use the sources to build both versions, only depending
on which compiler you use. If we do that, there's no reason to exclude
XP 64 or Windows 2003 64 bit, since the code is in Cygwin anyway. If we
ever drop XP or 2K3 support (years from now), we drop it for 32 and 64.

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen
- Decide how we can integrate 64 bit stuff into the distro. Will we have
a 32 bit and a distinct 64 bit distro? Or, should we stick to a single
distro? If so, how do we separate 64 and 32 bit stuff?

Linux distros with which I am familiar provide separate repos for 32-
and 64-bit systems (i686 and x86_64/amd64), with the 64-bit repo adding
32-bit builds of certain libraries for running x86 binaries downloaded
from elsewhere. Since that use case doesn't really apply to Cygwin,
multilib support shouldn't be necessary.

I'm not so sure. We have a 32 bit distro but we have nothing 64
bit-wise so far. Not all package maintainers are running 64 bit systems
and/or might be interested to support two packages in parallel. As far
as I can see we will need the 32 bit parts of the distro for a long time.

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen
What is the best package layout?

One factor is that we would have to completely change how packages are
maintained. Currently, maintainers are responsible for building their
own packages, but what if a maintainer only has a 32-bit system? I
think we would need a buildbot-type system for building the actual
packages.

Who's going to maintain a build system? We don't have the man power of
Fedora.

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen
This is a big project which can only work if we have help and support
from the community.

True, but $ome $upport from RH wouldn't hurt. :-)

Well...

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen
- What part of the project would most interest you to help? Coding
Cygwin? Documentation? Setup? Toolchain? You name it.

Toolchain, bootstrapping, and porting packages.

Corinna