Resurrect discussion: Mixing 32 and 64 bit distro

[Corinna Vinschen]

Hi guys,


I slept a bit bad tonight.

As you may or may not remember, we had a discussion about how to go
forward with a 64 bit distro in 2011.

In this discussion I held vehemently to the view that we have to create
the 64 bit distro in a way which allows to mix Cygwin 64 and 32 bit
applications freely. My main point was that it may take a long time
until we get all the Cygwin 32 bit packages built for 64 bit, and
therefore have to provide a mix so that users can adopt the 64 bit
distro early without having to drop the tools they are using.

But is that really so? I'm not so sure anymore. Maybe that problem
is exaggerated or overvalued.

The communication between 32 and 64 bit processes is rather complicated.
While carefully maintained shared mem regions are not much of a problem,
the dll shared section is, since it's not shared anymore. Also, parts
of the information given to a process at spawn/execve time has to be
serialized, mainly the stuff from cygheap which needs to be inherited by
the child. Information easily available in /proc isn't that easily
available anymore when you have to share it between 342 and 64 bit
processes.

setup will also get pretty complicated, because you have to differ
between 64/32 bit apps and 64/32 bit libs and setup has to know all the
time which version it installs and to make sure to pull in the right
libs. This is easy enough with rpm, but, AFAIK, nothing of that sort is
available in setup or upset yet.

Another, more development oriented downside is the fact that we have to
introduce the cyg64 DLL prefix, which, as far as I remember from the
discussion in 2011, breaks libtool and potentially configury and/or
Makefiles of a couple of packages.

So, I'm inclined to resurrect this discussion. I'd like to hear your
point of view and why you're rating one over the other (separate
distro vs. mixed distro). Personally I'm really not sure what is
more important, a full distro right from the start, or a clean
separation.


Thanks in advance,
Corinna

[Ryan Johnson]

Post by Corinna Vinschen
Hi guys,
I slept a bit bad tonight.
As you may or may not remember, we had a discussion about how to go
forward with a 64 bit distro in 2011.
In this discussion I held vehemently to the view that we have to create
the 64 bit distro in a way which allows to mix Cygwin 64 and 32 bit
applications freely. My main point was that it may take a long time
until we get all the Cygwin 32 bit packages built for 64 bit, and
therefore have to provide a mix so that users can adopt the 64 bit
distro early without having to drop the tools they are using.
But is that really so? I'm not so sure anymore. Maybe that problem
is exaggerated or overvalued.
The communication between 32 and 64 bit processes is rather complicated.
While carefully maintained shared mem regions are not much of a problem,
the dll shared section is, since it's not shared anymore. Also, parts
of the information given to a process at spawn/execve time has to be
serialized, mainly the stuff from cygheap which needs to be inherited by
the child. Information easily available in /proc isn't that easily
available anymore when you have to share it between 342 and 64 bit
processes.
setup will also get pretty complicated, because you have to differ
between 64/32 bit apps and 64/32 bit libs and setup has to know all the
time which version it installs and to make sure to pull in the right
libs. This is easy enough with rpm, but, AFAIK, nothing of that sort is
available in setup or upset yet.
Another, more development oriented downside is the fact that we have to
introduce the cyg64 DLL prefix, which, as far as I remember from the
discussion in 2011, breaks libtool and potentially configury and/or
Makefiles of a couple of packages.
So, I'm inclined to resurrect this discussion. I'd like to hear your
point of view and why you're rating one over the other (separate
distro vs. mixed distro). Personally I'm really not sure what is
more important, a full distro right from the start, or a clean
separation.

Two thoughts:

1. How is this significantly different for the end user than the 1.5 ->
1.7 migration was? That essentially required a full upgrade as well,
with no inter-op allowed, and side-by-side installs common for a while.
I don't know how much pain it was for the package maintainers to get
their stuff working under 1.7 -- presumably at least a couple ran into
issues.

2. How much pain do we really expect the package maintainers to see
because of 32-bit -> 64-bit migration? Most open source packages have
been 64-bit capable for years now, which should rule out the majority of
subtle integer truncation bugs and leave mostly configury/#define issues
to deal with.

I'd suggest that a few brave maintainers try creating 64-bit packages
(mintty already seems to work), and if it goes relatively smoothly we
forget mixing; folks whose favorite package is missing at first can just
keep using 32-bit 1.7 until the early adopters have broken a path for them.

BTW, would it make any sense to talk about having 32-bit binaries made
for 64-bit cygwin, compiling them specially for inter-op with a 64-bit
cygwin1.dll? Kind of how most 64-bit linux default to 32-bit binaries on
a 64-bit kernel? There are fairly good reasons for small binaries
operating on small datasets to stay 32-bit, but I don't know if that's
really possible, given the issues you mention above. The 32-bit vs
64-bit .dll situation seems especially icky.

$0.02
Ryan

[Corinna Vinschen]

Post by Ryan Johnson

Post by Corinna Vinschen
Hi guys,
I slept a bit bad tonight.
[...]
So, I'm inclined to resurrect this discussion. I'd like to hear your
point of view and why you're rating one over the other (separate
distro vs. mixed distro). Personally I'm really not sure what is
more important, a full distro right from the start, or a clean
separation.

1. How is this significantly different for the end user than the 1.5
-> 1.7 migration was? That essentially required a full upgrade as
well, with no inter-op allowed, and side-by-side installs common for
a while. I don't know how much pain it was for the package
maintainers to get their stuff working under 1.7 -- presumably at
least a couple ran into issues.

The 1.5->1.7 transition did not invalidate the existing POSIX-only
binaries. They still work today. A few executables in our distro
have been built pre-1.7...

Post by Ryan Johnson
I'd suggest that a few brave maintainers try creating 64-bit
packages (mintty already seems to work), and if it goes relatively
smoothly we forget mixing; folks whose favorite package is missing
at first can just keep using 32-bit 1.7 until the early adopters
have broken a path for them.
BTW, would it make any sense to talk about having 32-bit binaries
made for 64-bit cygwin, compiling them specially for inter-op with a
64-bit cygwin1.dll? Kind of how most 64-bit linux default to 32-bit
binaries on a 64-bit kernel? There are fairly good reasons for small
binaries operating on small datasets to stay 32-bit,

I don't understand this point. A 64 bit binary usually doesn't take
much more space than a 32 bit binary, given that most data access,
jmps and calls are using 32 bit PC-relative instructions on 64 bit.
Also, int's are still 4 byte, so there's not a lot to do to keep
the data set about the same size as on a 32 bit target.

I also don't see how Linux 64 bit distros default to 32 bit binaries.
Of the 3067 binaries in my /usr/bin dir on x86_64 Fedora, only 3(!)
are 32 it binaries, and those have 64 bit counterparts.


Corinna

[Earnie Boyd]

Post by Corinna Vinschen
Hi guys,
I slept a bit bad tonight.
As you may or may not remember, we had a discussion about how to go
forward with a 64 bit distro in 2011.
In this discussion I held vehemently to the view that we have to create
the 64 bit distro in a way which allows to mix Cygwin 64 and 32 bit
applications freely. My main point was that it may take a long time
until we get all the Cygwin 32 bit packages built for 64 bit, and
therefore have to provide a mix so that users can adopt the 64 bit
distro early without having to drop the tools they are using.
But is that really so? I'm not so sure anymore. Maybe that problem
is exaggerated or overvalued.

Maybe overvalued. Would an idea that 32bit executables use the 32bit
runtime and 64bit executables use the 64bit runtime be bad? So for
the time being deliver both 32bit cygwin1.dll and cyg64w1.dll (I
forget what you called it) and allow the executables to use the
correct version?

[Corinna Vinschen]

Post by Earnie Boyd

Post by Corinna Vinschen
Hi guys,
I slept a bit bad tonight.
As you may or may not remember, we had a discussion about how to go
forward with a 64 bit distro in 2011.
In this discussion I held vehemently to the view that we have to create
the 64 bit distro in a way which allows to mix Cygwin 64 and 32 bit
applications freely. My main point was that it may take a long time
until we get all the Cygwin 32 bit packages built for 64 bit, and
therefore have to provide a mix so that users can adopt the 64 bit
distro early without having to drop the tools they are using.
But is that really so? I'm not so sure anymore. Maybe that problem
is exaggerated or overvalued.

Maybe overvalued. Would an idea that 32bit executables use the 32bit
runtime and 64bit executables use the 64bit runtime be bad? So for
the time being deliver both 32bit cygwin1.dll and cyg64w1.dll (I
forget what you called it) and allow the executables to use the
correct version?

That wasn't the question. Of course, if you mix the distros, you will
have to provide two DLLs, one for 32 and one for 64 bit. The question
is, shall the 32 and 64 bit Cygwin DLLs interact or not. Keep 32 and
64 bit distinct from each other or not. Even if you just mix them into
one /bin, you have to keep the new cyg64 DLL prefix. But then again
you would get the same result by having two distinct distros and add the
other /bin dir to $PATH, without the requirement to keep the cyg64
DLL prefix.


Corinna

[Earnie Boyd]

Post by Corinna Vinschen

Post by Earnie Boyd

Post by Corinna Vinschen
Hi guys,
I slept a bit bad tonight.
As you may or may not remember, we had a discussion about how to go
forward with a 64 bit distro in 2011.
In this discussion I held vehemently to the view that we have to create
the 64 bit distro in a way which allows to mix Cygwin 64 and 32 bit
applications freely. My main point was that it may take a long time
until we get all the Cygwin 32 bit packages built for 64 bit, and
therefore have to provide a mix so that users can adopt the 64 bit
distro early without having to drop the tools they are using.
But is that really so? I'm not so sure anymore. Maybe that problem
is exaggerated or overvalued.

Maybe overvalued. Would an idea that 32bit executables use the 32bit
runtime and 64bit executables use the 64bit runtime be bad? So for
the time being deliver both 32bit cygwin1.dll and cyg64w1.dll (I
forget what you called it) and allow the executables to use the
correct version?

That wasn't the question. Of course, if you mix the distros, you will
have to provide two DLLs, one for 32 and one for 64 bit. The question
is, shall the 32 and 64 bit Cygwin DLLs interact or not. Keep 32 and
64 bit distinct from each other or not. Even if you just mix them into
one /bin, you have to keep the new cyg64 DLL prefix. But then again
you would get the same result by having two distinct distros and add the
other /bin dir to $PATH, without the requirement to keep the cyg64
DLL prefix.

I would tend to use the cyg64 prefix. You would not have to question
which cygwin1.dll in the support arena if the dll has a differing
name. The user then has the option to have one directory for both if
he chooses. Otherwise they must be separated.

[Christopher Faylor]

Post by Corinna Vinschen

Post by Earnie Boyd

Post by Corinna Vinschen
Hi guys,
I slept a bit bad tonight.
As you may or may not remember, we had a discussion about how to go
forward with a 64 bit distro in 2011.
In this discussion I held vehemently to the view that we have to create
the 64 bit distro in a way which allows to mix Cygwin 64 and 32 bit
applications freely. My main point was that it may take a long time
until we get all the Cygwin 32 bit packages built for 64 bit, and
therefore have to provide a mix so that users can adopt the 64 bit
distro early without having to drop the tools they are using.
But is that really so? I'm not so sure anymore. Maybe that problem
is exaggerated or overvalued.

Maybe overvalued. Would an idea that 32bit executables use the 32bit
runtime and 64bit executables use the 64bit runtime be bad? So for
the time being deliver both 32bit cygwin1.dll and cyg64w1.dll (I
forget what you called it) and allow the executables to use the
correct version?

That wasn't the question. Of course, if you mix the distros, you will
have to provide two DLLs, one for 32 and one for 64 bit. The question
is, shall the 32 and 64 bit Cygwin DLLs interact or not. Keep 32 and
64 bit distinct from each other or not. Even if you just mix them into
one /bin, you have to keep the new cyg64 DLL prefix. But then again
you would get the same result by having two distinct distros and add the
other /bin dir to $PATH, without the requirement to keep the cyg64
DLL prefix.

I think two distinct distros with no explicit understanding between the
32-bit and 64-bit is the sanest approach. I hate the thought of lots of
code in 64-bit Cygwin to specifically deal with 32-bit aps.

Would it be possible to write some kind of "shim" 64-bit application
which "did something" to run a 32-bit Cygwin for people who can't
wait to have their favorite package ported to 64-bit?

cgf

[Corinna Vinschen]

Post by Christopher Faylor

Post by Corinna Vinschen

Post by Earnie Boyd

Post by Corinna Vinschen
Hi guys,
I slept a bit bad tonight.
As you may or may not remember, we had a discussion about how to go
forward with a 64 bit distro in 2011.
In this discussion I held vehemently to the view that we have to create
the 64 bit distro in a way which allows to mix Cygwin 64 and 32 bit
applications freely. My main point was that it may take a long time
until we get all the Cygwin 32 bit packages built for 64 bit, and
therefore have to provide a mix so that users can adopt the 64 bit
distro early without having to drop the tools they are using.
But is that really so? I'm not so sure anymore. Maybe that problem
is exaggerated or overvalued.

Maybe overvalued. Would an idea that 32bit executables use the 32bit
runtime and 64bit executables use the 64bit runtime be bad? So for
the time being deliver both 32bit cygwin1.dll and cyg64w1.dll (I
forget what you called it) and allow the executables to use the
correct version?

That wasn't the question. Of course, if you mix the distros, you will
have to provide two DLLs, one for 32 and one for 64 bit. The question
is, shall the 32 and 64 bit Cygwin DLLs interact or not. Keep 32 and
64 bit distinct from each other or not. Even if you just mix them into
one /bin, you have to keep the new cyg64 DLL prefix. But then again
you would get the same result by having two distinct distros and add the
other /bin dir to $PATH, without the requirement to keep the cyg64
DLL prefix.

I think two distinct distros with no explicit understanding between the
32-bit and 64-bit is the sanest approach. I hate the thought of lots of
code in 64-bit Cygwin to specifically deal with 32-bit aps.

It's worse. For a smooth operation you have to take both scenarios into
account, 32 bit processes starting 64 bit processes and vice versa.
That means, the same special code, just for the opposite target, has to
go int the 64 bit and the 32 bit DLL.

Post by Christopher Faylor
Would it be possible to write some kind of "shim" 64-bit application
which "did something" to run a 32-bit Cygwin for people who can't
wait to have their favorite package ported to 64-bit?

I'm not exactly sure what such a shim would have to cover. You don't
see much of a problem running 32 bit Cygwin processes from 64 bit
processes and vice versa in simple cases, But if you run a 32 bit
process from a 64 bit tcsh running in a 64 bit mintty, the 32 bit
process doesn't know it's running in a tty. Same goes vice versa.
I don't know how we would fix that. Also, the paths in the environment
are off, because they are converted to Windows and then back to POSIX,
but on the way Cygwin's root dir has changed.
In the first place the problem is that the processes don't use the same
shared memory, in the second it's the missing data from the cygheap.
I have the vague feeling that if you start to create such a shim,
you could just as well change the Cygwin DLL to handle the situation.


Corinna

[Corinna Vinschen]

Post by Corinna Vinschen

Post by Christopher Faylor
Would it be possible to write some kind of "shim" 64-bit application
which "did something" to run a 32-bit Cygwin for people who can't
wait to have their favorite package ported to 64-bit?

I'm not exactly sure what such a shim would have to cover. You don't
see much of a problem running 32 bit Cygwin processes from 64 bit
processes and vice versa in simple cases, But if you run a 32 bit
process from a 64 bit tcsh running in a 64 bit mintty, the 32 bit
process doesn't know it's running in a tty. Same goes vice versa.
I don't know how we would fix that. Also, the paths in the environment
are off, because they are converted to Windows and then back to POSIX,
but on the way Cygwin's root dir has changed.
In the first place the problem is that the processes don't use the same
shared memory, in the second it's the missing data from the cygheap.

FTR: ...and in the third place it's the fact that the DLLs use different
object dirs in the native NT namespace.


Corinna

[Christopher Faylor]

Post by Corinna Vinschen

Post by Christopher Faylor

Post by Corinna Vinschen

Post by Earnie Boyd

Post by Corinna Vinschen
Hi guys,
I slept a bit bad tonight.
As you may or may not remember, we had a discussion about how to go
forward with a 64 bit distro in 2011.
In this discussion I held vehemently to the view that we have to create
the 64 bit distro in a way which allows to mix Cygwin 64 and 32 bit
applications freely. My main point was that it may take a long time
until we get all the Cygwin 32 bit packages built for 64 bit, and
therefore have to provide a mix so that users can adopt the 64 bit
distro early without having to drop the tools they are using.
But is that really so? I'm not so sure anymore. Maybe that problem
is exaggerated or overvalued.

Maybe overvalued. Would an idea that 32bit executables use the 32bit
runtime and 64bit executables use the 64bit runtime be bad? So for
the time being deliver both 32bit cygwin1.dll and cyg64w1.dll (I
forget what you called it) and allow the executables to use the
correct version?

That wasn't the question. Of course, if you mix the distros, you will
have to provide two DLLs, one for 32 and one for 64 bit. The question
is, shall the 32 and 64 bit Cygwin DLLs interact or not. Keep 32 and
64 bit distinct from each other or not. Even if you just mix them into
one /bin, you have to keep the new cyg64 DLL prefix. But then again
you would get the same result by having two distinct distros and add the
other /bin dir to $PATH, without the requirement to keep the cyg64
DLL prefix.

I think two distinct distros with no explicit understanding between the
32-bit and 64-bit is the sanest approach. I hate the thought of lots of
code in 64-bit Cygwin to specifically deal with 32-bit aps.

It's worse. For a smooth operation you have to take both scenarios into
account, 32 bit processes starting 64 bit processes and vice versa.
That means, the same special code, just for the opposite target, has to
go int the 64 bit and the 32 bit DLL.

I don't recally you suggesting that the 32-bit DLL should somehow be
modified to understand 64-bit Cygwin. Ugh.

cgf

[Corinna Vinschen]

Post by Christopher Faylor

Post by Corinna Vinschen

Post by Christopher Faylor
I think two distinct distros with no explicit understanding between the
32-bit and 64-bit is the sanest approach. I hate the thought of lots of
code in 64-bit Cygwin to specifically deal with 32-bit aps.

It's worse. For a smooth operation you have to take both scenarios into
account, 32 bit processes starting 64 bit processes and vice versa.
That means, the same special code, just for the opposite target, has to
go int the 64 bit and the 32 bit DLL.

I don't recally you suggesting that the 32-bit DLL should somehow be
modified to understand 64-bit Cygwin. Ugh.

Well, that's the idea. If you want the user not to notice what CPU
the executable is running on, you need this in both directions.

Anyway, it's probably not *that* much work. It's not really a problem
as far as shared memory is concerned, as long as the DLLs are in the
same bin dir. The real problems are cygheap inheritence in spawn/exec,
the DLL's shared section and a bit of extra stuff in /proc/$PID.

But I'm really not sure if it's worth the complications...


Corinna

[Tom Honermann]

Post by Christopher Faylor

Post by Corinna Vinschen

Post by Earnie Boyd

Post by Corinna Vinschen
Hi guys,
I slept a bit bad tonight.
As you may or may not remember, we had a discussion about how to go
forward with a 64 bit distro in 2011.
In this discussion I held vehemently to the view that we have to create
the 64 bit distro in a way which allows to mix Cygwin 64 and 32 bit
applications freely. My main point was that it may take a long time
until we get all the Cygwin 32 bit packages built for 64 bit, and
therefore have to provide a mix so that users can adopt the 64 bit
distro early without having to drop the tools they are using.
But is that really so? I'm not so sure anymore. Maybe that problem
is exaggerated or overvalued.

Maybe overvalued. Would an idea that 32bit executables use the 32bit
runtime and 64bit executables use the 64bit runtime be bad? So for
the time being deliver both 32bit cygwin1.dll and cyg64w1.dll (I
forget what you called it) and allow the executables to use the
correct version?

That wasn't the question. Of course, if you mix the distros, you will
have to provide two DLLs, one for 32 and one for 64 bit. The question
is, shall the 32 and 64 bit Cygwin DLLs interact or not. Keep 32 and
64 bit distinct from each other or not. Even if you just mix them into
one /bin, you have to keep the new cyg64 DLL prefix. But then again
you would get the same result by having two distinct distros and add the
other /bin dir to $PATH, without the requirement to keep the cyg64
DLL prefix.

I think two distinct distros with no explicit understanding between the
32-bit and 64-bit is the sanest approach. I hate the thought of lots of
code in 64-bit Cygwin to specifically deal with 32-bit aps.
Would it be possible to write some kind of "shim" 64-bit application
which "did something" to run a 32-bit Cygwin for people who can't
wait to have their favorite package ported to 64-bit?

Sounds like a suggestion towards a Cygwin-On-Cygwin64 analog to WOW64.
Ie, a cygwin1.dll as a thunk/wrapper into cyg64w1.dll similar to what
wow64.dll does for WOW64 processes.

Tom.

[Christopher Faylor]

Post by Tom Honermann

Post by Christopher Faylor

Post by Corinna Vinschen

Post by Earnie Boyd

Post by Corinna Vinschen
Hi guys,
I slept a bit bad tonight.
As you may or may not remember, we had a discussion about how to go
forward with a 64 bit distro in 2011.
In this discussion I held vehemently to the view that we have to create
the 64 bit distro in a way which allows to mix Cygwin 64 and 32 bit
applications freely. My main point was that it may take a long time
until we get all the Cygwin 32 bit packages built for 64 bit, and
therefore have to provide a mix so that users can adopt the 64 bit
distro early without having to drop the tools they are using.
But is that really so? I'm not so sure anymore. Maybe that problem
is exaggerated or overvalued.

Maybe overvalued. Would an idea that 32bit executables use the 32bit
runtime and 64bit executables use the 64bit runtime be bad? So for
the time being deliver both 32bit cygwin1.dll and cyg64w1.dll (I
forget what you called it) and allow the executables to use the
correct version?

That wasn't the question. Of course, if you mix the distros, you will
have to provide two DLLs, one for 32 and one for 64 bit. The question
is, shall the 32 and 64 bit Cygwin DLLs interact or not. Keep 32 and
64 bit distinct from each other or not. Even if you just mix them into
one /bin, you have to keep the new cyg64 DLL prefix. But then again
you would get the same result by having two distinct distros and add the
other /bin dir to $PATH, without the requirement to keep the cyg64
DLL prefix.

I think two distinct distros with no explicit understanding between the
32-bit and 64-bit is the sanest approach. I hate the thought of lots of
code in 64-bit Cygwin to specifically deal with 32-bit aps.
Would it be possible to write some kind of "shim" 64-bit application
which "did something" to run a 32-bit Cygwin for people who can't
wait to have their favorite package ported to 64-bit?

Sounds like a suggestion towards a Cygwin-On-Cygwin64 analog to WOW64.
Ie, a cygwin1.dll as a thunk/wrapper into cyg64w1.dll similar to what
wow64.dll does for WOW64 processes.

Yep. That's what I was thinking. I doubt that it's worth the effort
though.

cgf

[Corinna Vinschen]

Post by Christopher Faylor

Post by Tom Honermann

Post by Christopher Faylor
[...]
I think two distinct distros with no explicit understanding between the
32-bit and 64-bit is the sanest approach. I hate the thought of lots of
code in 64-bit Cygwin to specifically deal with 32-bit aps.
Would it be possible to write some kind of "shim" 64-bit application
which "did something" to run a 32-bit Cygwin for people who can't
wait to have their favorite package ported to 64-bit?

Sounds like a suggestion towards a Cygwin-On-Cygwin64 analog to WOW64.
Ie, a cygwin1.dll as a thunk/wrapper into cyg64w1.dll similar to what
wow64.dll does for WOW64 processes.

Yep. That's what I was thinking. I doubt that it's worth the effort
though.

On second thought, I have no idea how to do that in a performant way.
Such a wrapper 32 bit DLL has the problem that it's a 32 bit executable
and as such, it won't be able to load a 64 bit DLL into its address
space.

Therefore you'd need a shim 64 bit application which interfaces with the
64 bit DLL on behalf of the 32 bit process. The 32 bit wrapper DLL has
to start the shim and open a communication channel between itself and
the shim, for instance, a pipe. Then all 32 bit calls would be
serialized and sent to the 64 bit shim via the pipe. The shim calls the
function in the 64 bit DLL and then it has to send back the results over
the pipe. That's funny if buffers are written by the function, like in
case of stat, or readlink.

Did I just hear a "Cygwin is slow" from the auditorium?


Corinna

[Yaakov (Cygwin/X)]

Post by Corinna Vinschen
As you may or may not remember, we had a discussion about how to go
forward with a 64 bit distro in 2011.
In this discussion I held vehemently to the view that we have to create
the 64 bit distro in a way which allows to mix Cygwin 64 and 32 bit
applications freely.

And I just as vehemently disagreed. :-)

To recap: on Linux, 64bit platforms (especially x86_64) provide 32bit
compatibility libraries for the benefit of some very popular 3rd party
binary-only packages, which have (at least, until recently) generally
only been 32bit. On Cygwin, those just don't exist, nor could they
without a licence buyout from RH; if any do, their user base isn't
large enough to justify this. IMHO, this would just add a lot of
work for very little gain.

If I'm wrong on that count -- and obviously I have no knowledge of who
has licence buyouts from RH and what they are used for -- then my
question is: do these binary packages depend on any other libraries
besides cygwin1.dll? If not, then *maybe* it would justify making
*only* cygwin multilib (in a lib32/lib arrangement), with the 64bit
cygwin1.dll in /usr/bin and the 32bit cygwin1.dll elsewhere or, as
cgf suggested, using a WoW64-like wrapper. But to go multilib any
further than that just isn't feasible IMO.

Post by Corinna Vinschen
My main point was that it may take a long time
until we get all the Cygwin 32 bit packages built for 64 bit, and
therefore have to provide a mix so that users can adopt the 64 bit
distro early without having to drop the tools they are using.

For some maintainers, this isn't a question of when but if; not
all maintainers has 64bit systems, and requiring them may be an
imposition for those who wish to contribute. Perhaps we should poll
current maintainers to see how big of a problem we're facing, but
either we get a build farm so that maintainers don't have to build their
packages on their own machines, or some of us are going to have to
help maintainers with their 64bit packages.

Regardless, a helpful step would be to have per-package git repos on
sourceware, like I have currently on cygwin-ports.git.sf.net. This
would open up the package maintenance process and hopefully make it
easier for others to help with, or adopt, existing packages.

Post by Corinna Vinschen
setup will also get pretty complicated, because you have to differ
between 64/32 bit apps and 64/32 bit libs and setup has to know all the
time which version it installs and to make sure to pull in the right
libs. This is easy enough with rpm, but, AFAIK, nothing of that sort is
available in setup or upset yet.

Also, setup doesn't handle file collisions well, so while it (other
problems aside) could theoretically be made to work for runtime DLLs
with libfooN and lib64fooN (with libfoo-common.noarch where necessary),
how would 64bit libdevel packages work? libfoo-devel for lib64fooN on
64bit and libfooN on 32bit? Or libfoo-devel and lib64foo-devel, whose
headers would collide? Too bad RPM isn't an option; setup/upset just
aren't up to this.

Post by Corinna Vinschen
Another, more development oriented downside is the fact that we have to
introduce the cyg64 DLL prefix, which, as far as I remember from the
discussion in 2011, breaks libtool and potentially configury and/or
Makefiles of a couple of packages.

It would break every build system used to build libraries, as well as
programs which dlopen() prefixed libraries, of which there are many.
I assure you that it's a *very* big deal.


Yaakov

[Corinna Vinschen]

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen
As you may or may not remember, we had a discussion about how to go
forward with a 64 bit distro in 2011.
In this discussion I held vehemently to the view that we have to create
the 64 bit distro in a way which allows to mix Cygwin 64 and 32 bit
applications freely.

And I just as vehemently disagreed. :-)

Yeah, and that's why I was waiting for your reply :)

Post by Yaakov (Cygwin/X)
To recap: on Linux, 64bit platforms (especially x86_64) provide 32bit
compatibility libraries for the benefit of some very popular 3rd party
binary-only packages, which have (at least, until recently) generally
only been 32bit. On Cygwin, those just don't exist, nor could they
without a licence buyout from RH; if any do, their user base isn't
large enough to justify this. IMHO, this would just add a lot of
work for very little gain.
If I'm wrong on that count -- and obviously I have no knowledge of who
has licence buyouts from RH and what they are used for -- [...]

I don't remember what I thought or wrote in 2011, but I don't think the
buyout licensees are a good argument anyway. The reason for the BL is
so that the licensees can generate proprietary projects with a Cygwin
DLL under the hood. The license is bound to the project it was
purchased for. And "proprietary" implies that the projects are
generated as closed solutions for customers of the licensee.

The existing projects have been built and packed for 32 bit. They have
nothing to do with the distro and run entirely independently. Apart
from that, we will support 32 bit Cygwin for the foreseeable future, and
nothing speaks against keeping such a project 32 bit, or providing two
separate 32 bit and 64 bit versions of the same project.

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen
My main point was that it may take a long time
until we get all the Cygwin 32 bit packages built for 64 bit, and
therefore have to provide a mix so that users can adopt the 64 bit
distro early without having to drop the tools they are using.

For some maintainers, this isn't a question of when but if; not
all maintainers has 64bit systems, and requiring them may be an
imposition for those who wish to contribute. Perhaps we should poll
current maintainers to see how big of a problem we're facing, but
either we get a build farm so that maintainers don't have to build their
packages on their own machines, or some of us are going to have to
help maintainers with their 64bit packages.

Indeed. Building packages on behalf of a maintainer only having a 32
bit system should not be much of a problem. Also, the build farm for 64
bit is a good idea, and our dept at Red Hat is inclined to provide it.

But instead of a farm, it might be even sufficient to provide a simple
mechanism for maintainers to build their package on a 64 bit machine via
mail:

Send your cygport file to cygwin-64-build-package AT redhat DOT com, and
a dedicated 64 bit Cygwin machine will try to build your package.
Results of the cygport output via mail reply. On success, packages will
be uploaded to sourceware automatically. Using this mechanism would
just require to use cygport for packaging and everybody

Post by Yaakov (Cygwin/X)
Regardless, a helpful step would be to have per-package git repos on
sourceware, like I have currently on cygwin-ports.git.sf.net. This
would open up the package maintenance process and hopefully make it
easier for others to help with, or adopt, existing packages.

Isn't adopting packages already easy enough by just fetching the
existing source package from sourceware?

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen
setup will also get pretty complicated, because you have to differ
between 64/32 bit apps and 64/32 bit libs and setup has to know all the
time which version it installs and to make sure to pull in the right
libs. This is easy enough with rpm, but, AFAIK, nothing of that sort is
available in setup or upset yet.

Also, setup doesn't handle file collisions well, so while it (other
problems aside) could theoretically be made to work for runtime DLLs
with libfooN and lib64fooN (with libfoo-common.noarch where necessary),
how would 64bit libdevel packages work? libfoo-devel for lib64fooN on
64bit and libfooN on 32bit? Or libfoo-devel and lib64foo-devel, whose
headers would collide? Too bad RPM isn't an option; setup/upset just
aren't up to this.

Our packages are missing the platform bits anyway right now, since they
were never necessary with only one supported platform. In theory the
above problem requires to separate the -devel package in -headers,
-devel32 and -devel64. But, yes, it's getting a lot more complicated.

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen
Another, more development oriented downside is the fact that we have to
introduce the cyg64 DLL prefix, which, as far as I remember from the
discussion in 2011, breaks libtool and potentially configury and/or
Makefiles of a couple of packages.

It would break every build system used to build libraries, as well as
programs which dlopen() prefixed libraries, of which there are many.
I assure you that it's a *very* big deal.

Sigh, yes, I noticed that already myself. I didn't take that serious
two years ago since it sounded easy to fix.

Ok, so far we have three voices for keeping 32 and 64 bit separate,
one voice for mixing them, and one being very unsure.

Any more input?


Corinna

[marco atzeri]

Post by Corinna Vinschen
Indeed. Building packages on behalf of a maintainer only having a 32
bit system should not be much of a problem. Also, the build farm for 64
bit is a good idea, and our dept at Red Hat is inclined to provide it.
But instead of a farm, it might be even sufficient to provide a simple
mechanism for maintainers to build their package on a 64 bit machine via
Send your cygport file to cygwin-64-build-package AT redhat DOT com, and
a dedicated 64 bit Cygwin machine will try to build your package.
Results of the cygport output via mail reply. On success, packages will
be uploaded to sourceware automatically. Using this mechanism would
just require to use cygport for packaging and everybody

two step please, building alone does not guarantee it is good
enough for upload

Post by Corinna Vinschen
Ok, so far we have three voices for keeping 32 and 64 bit separate,
one voice for mixing them, and one being very unsure.
Any more input?

I prefer the separate way. Already MS is not very good in
mixing the things, so we should avoid unstable workaround
to make 32 and 64 bit working together.

Post by Corinna Vinschen
Corinna

Marco

[Kai Tietz]

my 5 cents about this. Actual my opinion about that is a bit
indifferent. One hand there might be advantages in allowing to
support 32-bit/64-bit interacting, but there are some down-sides IMHO
too.

I don't see a really good reason to use for 64-bit a different
name-prefix as for 32-bit. I would vote for using same DLL-names for
64-bit and 32-bit. The x64 pe-loader is able to load proper DLL for
kind of process.

The down-side of letting 32-bit and 64-bit processes interacting is
IMHO pretty high. It can get very slow on process-startup and
execution, it is pretty costy in terms of effort, and I am not
convinced that this lead to sane and predictable environments, if we
allow users to mix 32-bit and 64-bit processes freely.

Kai

[Yaakov (Cygwin/X)]

Post by Corinna Vinschen

Post by Corinna Vinschen
Indeed. Building packages on behalf of a maintainer only having a 32

bit system should not be much of a problem. Also, the build farm for 64
bit is a good idea, and our dept at Red Hat is inclined to provide it.
But instead of a farm, it might be even sufficient to provide a simple
mechanism for maintainers to build their package on a 64 bit machine via
Send your cygport file to cygwin-64-build-package AT redhat DOT com, and
a dedicated 64 bit Cygwin machine will try to build your package.
Results of the cygport output via mail reply. On success, packages will
be uploaded to sourceware automatically.

Email? Three words: "git update hook" :-)

Post by Corinna Vinschen

Post by Corinna Vinschen
Regardless, a helpful step would be to have per-package git repos on
sourceware, like I have currently on cygwin-ports.git.sf.net. This
would open up the package maintenance process and hopefully make it
easier for others to help with, or adopt, existing packages.

Isn't adopting packages already easy enough by just fetching the
existing source package from sourceware?

I'm thinking more along the lines of collaborative development, history
preservation, redundant backups, all the things that (D)VCS are good
for.


Yaakov

[Corinna Vinschen]

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen

Post by Corinna Vinschen
Indeed. Building packages on behalf of a maintainer only having a 32

bit system should not be much of a problem. Also, the build farm for 64
bit is a good idea, and our dept at Red Hat is inclined to provide it.
But instead of a farm, it might be even sufficient to provide a simple
mechanism for maintainers to build their package on a 64 bit machine via
Send your cygport file to cygwin-64-build-package AT redhat DOT com, and
a dedicated 64 bit Cygwin machine will try to build your package.
Results of the cygport output via mail reply. On success, packages will
be uploaded to sourceware automatically.

Email? Three words: "git update hook" :-)

You're talking to a git idi^Williterate. I was looking for an easily
manageable solution which works if you don't have direct access to the
build machine. If you can do that with git, fine.

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen

Post by Corinna Vinschen
Regardless, a helpful step would be to have per-package git repos on
sourceware, like I have currently on cygwin-ports.git.sf.net. This
would open up the package maintenance process and hopefully make it
easier for others to help with, or adopt, existing packages.

Isn't adopting packages already easy enough by just fetching the
existing source package from sourceware?

I'm thinking more along the lines of collaborative development, history
preservation, redundant backups, all the things that (D)VCS are good
for.

Yeah, rub it in :)


Corinna

[Earnie Boyd]

Post by Corinna Vinschen

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen
Another, more development oriented downside is the fact that we have to
introduce the cyg64 DLL prefix, which, as far as I remember from the
discussion in 2011, breaks libtool and potentially configury and/or
Makefiles of a couple of packages.

It would break every build system used to build libraries, as well as
programs which dlopen() prefixed libraries, of which there are many.
I assure you that it's a *very* big deal.

Sigh, yes, I noticed that already myself. I didn't take that serious
two years ago since it sounded easy to fix.

My argument for cyg64 prefix was that there will be those that try to
do a mix and making it easily identifiable which DLL was actually
causing a problem.

Post by Corinna Vinschen
Ok, so far we have three voices for keeping 32 and 64 bit separate,
one voice for mixing them, and one being very unsure.

Again, because people will do it.

Post by Corinna Vinschen
Any more input?

Any way for the 64bit DLL to take a different code path for 32bit executables?

[Ryan Johnson]

Post by Earnie Boyd

Post by Corinna Vinschen

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen
Another, more development oriented downside is the fact that we have to
introduce the cyg64 DLL prefix, which, as far as I remember from the
discussion in 2011, breaks libtool and potentially configury and/or
Makefiles of a couple of packages.

It would break every build system used to build libraries, as well as
programs which dlopen() prefixed libraries, of which there are many.
I assure you that it's a *very* big deal.

Sigh, yes, I noticed that already myself. I didn't take that serious
two years ago since it sounded easy to fix.

My argument for cyg64 prefix was that there will be those that try to
do a mix and making it easily identifiable which DLL was actually
causing a problem.

Post by Corinna Vinschen
Ok, so far we have three voices for keeping 32 and 64 bit separate,
one voice for mixing them, and one being very unsure.

Again, because people will do it.

Post by Corinna Vinschen
Any more input?

Any way for the 64bit DLL to take a different code path for 32bit executables?

So, to get this straight, you're not so much proposing support for
inter-op as you are advocating a clear separation between 32- and 64-bit
versions to help people "see" what's wrong when they try to mix the two
inappropriately?

Doesn't the windows loader completely ignore DLLs having the wrong
bit-ness? If so, the only problem from mixing them would be failure to
load dlls that you think are available [1]. Rather than changing the
prefixes and breaking any number of tools, couldn't we just modify
ldd/cygcheck to check (if they don't already)? Then, identifying the
problematic dll(s) would be pretty easy, even if the user doesn't want
to use 'file' or 'objdump'

[1] This assumes the user didn't clobber their 32-bit install with a
64-bit one, completely replacing a subset of dlls with a new set having
the wrong bitness. Mixed 1.5/1.7 installations were never allowed AFAIK,
so I don't think 32/64-bit mixes need be.

$0.02
Ryan

[Earnie Boyd]

On Thu, Feb 14, 2013 at 8:59 AM, Ryan Johnson

[1] This assumes the user didn't clobber their 32-bit install with a 64-bit
one, completely replacing a subset of dlls with a new set having the wrong
bitness. Mixed 1.5/1.7 installations were never allowed AFAIK, so I don't
think 32/64-bit mixes need be.

This is the primary issue, this and how to communicate between the
64bit process and the 32bit process.

Mixed or separated there has to be some method for interprocess
communication and then be able to trouble shoot which is at fault when
someone complains.

[Corinna Vinschen]

Post by Earnie Boyd
On Thu, Feb 14, 2013 at 8:59 AM, Ryan Johnson

[1] This assumes the user didn't clobber their 32-bit install with a 64-bit
one, completely replacing a subset of dlls with a new set having the wrong
bitness. Mixed 1.5/1.7 installations were never allowed AFAIK, so I don't
think 32/64-bit mixes need be.

This is the primary issue, this and how to communicate between the
64bit process and the 32bit process.
Mixed or separated there has to be some method for interprocess
communication and then be able to trouble shoot which is at fault when
someone complains.

No. You're jumping to conclusions which are actually what has been
*asked* for in this thread.

Separate here means: There's no communication between 64 and 32 bit
Cygwin processes. Full stop. Calling a 32 bit process from a 64 bit
process lets the 32 bit process assume it has been started from a native
Windows process. No information about inherited handles, no common
process list, no coommon moiunt table, no common ptys.

Mixed here means: We provide additional code in the DLL so that all the
aforementioned stuff is shared, and at execve time, a 32 bit process
gets a serialized copy of the required cygheap data from the 64 bit
parent and vice versa.

If you really want, you could argue for separate directories for 32 and
64 bit distro but sharing of the aforementioned data between them. But
that, in turn, requires all the same work to put into the DLL as the
fully mixed case, and even more so, since the different mount tables
will be a big, fat problem.


Corinna

[Corinna Vinschen]

Post by Ryan Johnson

Post by Earnie Boyd

Post by Corinna Vinschen

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen
Another, more development oriented downside is the fact that we have to
introduce the cyg64 DLL prefix, which, as far as I remember from the
discussion in 2011, breaks libtool and potentially configury and/or
Makefiles of a couple of packages.

It would break every build system used to build libraries, as well as
programs which dlopen() prefixed libraries, of which there are many.
I assure you that it's a *very* big deal.

Sigh, yes, I noticed that already myself. I didn't take that serious
two years ago since it sounded easy to fix.

My argument for cyg64 prefix was that there will be those that try to
do a mix and making it easily identifiable which DLL was actually
causing a problem.

Post by Corinna Vinschen
Ok, so far we have three voices for keeping 32 and 64 bit separate,
one voice for mixing them, and one being very unsure.

Again, because people will do it.

Post by Corinna Vinschen
Any more input?

Any way for the 64bit DLL to take a different code path for 32bit executables?

So, to get this straight, you're not so much proposing support for
inter-op as you are advocating a clear separation between 32- and
64-bit versions to help people "see" what's wrong when they try to
mix the two inappropriately?
Doesn't the windows loader completely ignore DLLs having the wrong
bit-ness? If so, the only problem from mixing them would be failure
to load dlls that you think are available [1]. Rather than changing
the prefixes and breaking any number of tools, couldn't we just
modify ldd/cygcheck to check (if they don't already)? Then,
identifying the problematic dll(s) would be pretty easy, even if the
user doesn't want to use 'file' or 'objdump'
[1] This assumes the user didn't clobber their 32-bit install with a
64-bit one, completely replacing a subset of dlls with a new set
having the wrong bitness. Mixed 1.5/1.7 installations were never
allowed AFAIK, so I don't think 32/64-bit mixes need be.
$0.02
Ryan

My 2ct here: On 64 bit Linux, the 64 and 32 bit .so files have the same
name as well. If a user *seriously* thinks it's a good idea to copy a
.so file from /lib to /lib64, thus overwriting the 64 bit .so, he
deserves to suffer.

Along the same lines you could think of a user copying a cygfoo.dll to
cyg64foo.dll since that obviously changes the DLL from 32 to 64 bit.

Ultimately there's nothing you can do against cluelessness.


Corinna

[Corinna Vinschen]

Post by Ryan Johnson
Doesn't the windows loader completely ignore DLLs having the wrong
bit-ness? If so, the only problem from mixing them would be failure
to load dlls that you think are available [1]. Rather than changing
the prefixes and breaking any number of tools, couldn't we just
modify ldd/cygcheck to check (if they don't already)? Then,
identifying the problematic dll(s) would be pretty easy, even if the
user doesn't want to use 'file' or 'objdump'

I just read that again and realized that it's a question. Yaakov
tested this assumption back in 2011. Yes, if two DLLs using the
same name but for a different platform are in the DLL search path,
the Windows loader loads the right one for the given executable's
platform.

So we have five voices for keeping 64 and 32 bit separate, one for
mixing, one undecided but now leaning towards keeping it separate. I
also asked two people off-list, and both of them thought it a good idea
to keep the distros separate.

I'm inclined to revert or tweak a couple of my patches which were meant
to mix the distros. My implied question to you for each of these points
is, shall I do that or not?

1. Revert all toolchain changes which change the DLL prefix from
"cyg" to "cyg64".

This is apparently the right thing to do if we want to be able
to build 64 bit packages without having to change libtool etc.

2. Rename the Cygwin DLL back from cyg64win1.dll to cygwin1.dll.

This is probably purely a matter of taste. It has nothing to do with
point 1. We can keep the name of theCygwin DLL without compromising
the "cyg" prefix elsewhere. Actually, it even simplifies the
recognition of a 64 bit Cygwin process at spawn/exec time.

3. Revert the path to link libs from "${prefix}/lib64" to "${prefix}/lib".

I'm actually not quite sure about that. The lib64 path is in the
toolchain now and it appears to work nicely. Apparently it also
works fine for 64 bit Linux. In conjunction with point 1, if we
ever decide that we yet need interoperability with 32 bit Cygwin
processes, keeping the lib path to lib64 would help to integrate
both worlds. What is the problem with lib64 again?


Thanks for your input,
Corinna

[Yaakov (Cygwin/X)]

Post by Corinna Vinschen
1. Revert all toolchain changes which change the DLL prefix from
"cyg" to "cyg64".

Revert.

Post by Corinna Vinschen
2. Rename the Cygwin DLL back from cyg64win1.dll to cygwin1.dll.
This is probably purely a matter of taste. It has nothing to do with
point 1. We can keep the name of theCygwin DLL without compromising
the "cyg" prefix elsewhere. Actually, it even simplifies the
recognition of a 64 bit Cygwin process at spawn/exec time.

It still makes dlopen()ing the Cygwin DLL -- a technique which is used
by Mono, Python ctypes, Ruby FFI, JNA, etc., and LD_PRELOAD hacks (among
others) -- more complicated. I'd prefer to revert.

Post by Corinna Vinschen
3. Revert the path to link libs from "${prefix}/lib64" to "${prefix}/lib".
I'm actually not quite sure about that. The lib64 path is in the
toolchain now and it appears to work nicely. Apparently it also
works fine for 64 bit Linux. In conjunction with point 1, if we
ever decide that we yet need interoperability with 32 bit Cygwin
processes, keeping the lib path to lib64 would help to integrate
both worlds. What is the problem with lib64 again?

Not so sure about that first point; while ld (and w32api) wanted lib64,
gcc wouldn't recognize it, at least not with a sys-root. While
doable, it does mean adjustments to cygport and some .cygport files,
as well as patches (available in Fedora and other distros) for some
packages which aren't lib64 aware. If we don't need it, why bother?

As for the future, I think we already agreed that trying to manage a
fully multiarch distro isn't feasible with setup/upset. If we're
talking only about multiarch-ing Cygwin itself, I think a lib32/lib
combination would do.


Yaakov

[Corinna Vinschen]

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen
1. Revert all toolchain changes which change the DLL prefix from
"cyg" to "cyg64".

Revert.

Post by Corinna Vinschen
2. Rename the Cygwin DLL back from cyg64win1.dll to cygwin1.dll.
This is probably purely a matter of taste. It has nothing to do with
point 1. We can keep the name of theCygwin DLL without compromising
the "cyg" prefix elsewhere. Actually, it even simplifies the
recognition of a 64 bit Cygwin process at spawn/exec time.

It still makes dlopen()ing the Cygwin DLL -- a technique which is used
by Mono, Python ctypes, Ruby FFI, JNA, etc., and LD_PRELOAD hacks (among
others) -- more complicated. I'd prefer to revert.

Post by Corinna Vinschen
3. Revert the path to link libs from "${prefix}/lib64" to "${prefix}/lib".
I'm actually not quite sure about that. The lib64 path is in the
toolchain now and it appears to work nicely. Apparently it also
works fine for 64 bit Linux. In conjunction with point 1, if we
ever decide that we yet need interoperability with 32 bit Cygwin
processes, keeping the lib path to lib64 would help to integrate
both worlds. What is the problem with lib64 again?

Not so sure about that first point; while ld (and w32api) wanted lib64,
gcc wouldn't recognize it, at least not with a sys-root. While
doable, it does mean adjustments to cygport and some .cygport files,
as well as patches (available in Fedora and other distros) for some
packages which aren't lib64 aware. If we don't need it, why bother?
As for the future, I think we already agreed that trying to manage a
fully multiarch distro isn't feasible with setup/upset. If we're
talking only about multiarch-ing Cygwin itself, I think a lib32/lib
combination would do.

Ok, let's go the full way.

You *are* all aware that renaming the DLL back to cygwin1.dll means
that, afterwards, none of the currently existing binaries will work
anymore, right?


Corinna

[Andy Koppe]

Post by Corinna Vinschen

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen
1. Revert all toolchain changes which change the DLL prefix from
"cyg" to "cyg64".

Revert.

Post by Corinna Vinschen
2. Rename the Cygwin DLL back from cyg64win1.dll to cygwin1.dll.
This is probably purely a matter of taste. It has nothing to do with
point 1. We can keep the name of theCygwin DLL without compromising
the "cyg" prefix elsewhere. Actually, it even simplifies the
recognition of a 64 bit Cygwin process at spawn/exec time.

It still makes dlopen()ing the Cygwin DLL -- a technique which is used
by Mono, Python ctypes, Ruby FFI, JNA, etc., and LD_PRELOAD hacks (among
others) -- more complicated. I'd prefer to revert.

Post by Corinna Vinschen
3. Revert the path to link libs from "${prefix}/lib64" to "${prefix}/lib".
I'm actually not quite sure about that. The lib64 path is in the
toolchain now and it appears to work nicely. Apparently it also
works fine for 64 bit Linux. In conjunction with point 1, if we
ever decide that we yet need interoperability with 32 bit Cygwin
processes, keeping the lib path to lib64 would help to integrate
both worlds. What is the problem with lib64 again?

Not so sure about that first point; while ld (and w32api) wanted lib64,
gcc wouldn't recognize it, at least not with a sys-root. While
doable, it does mean adjustments to cygport and some .cygport files,
as well as patches (available in Fedora and other distros) for some
packages which aren't lib64 aware. If we don't need it, why bother?
As for the future, I think we already agreed that trying to manage a
fully multiarch distro isn't feasible with setup/upset. If we're
talking only about multiarch-ing Cygwin itself, I think a lib32/lib
combination would do.

Ok, let's go the full way.
You *are* all aware that renaming the DLL back to cygwin1.dll means
that, afterwards, none of the currently existing binaries will work
anymore, right?

Fine by me. It would be silly to demand binary (or source) backward
compatibilty at this point.

Andy

[Corinna Vinschen]

Post by Andy Koppe

Post by Corinna Vinschen

Post by Yaakov (Cygwin/X)

Post by Corinna Vinschen
1. Revert all toolchain changes which change the DLL prefix from
"cyg" to "cyg64".

Revert.

Post by Corinna Vinschen
2. Rename the Cygwin DLL back from cyg64win1.dll to cygwin1.dll.
This is probably purely a matter of taste. It has nothing to do with
point 1. We can keep the name of theCygwin DLL without compromising
the "cyg" prefix elsewhere. Actually, it even simplifies the
recognition of a 64 bit Cygwin process at spawn/exec time.

It still makes dlopen()ing the Cygwin DLL -- a technique which is used
by Mono, Python ctypes, Ruby FFI, JNA, etc., and LD_PRELOAD hacks (among
others) -- more complicated. I'd prefer to revert.

Post by Corinna Vinschen
3. Revert the path to link libs from "${prefix}/lib64" to "${prefix}/lib".
I'm actually not quite sure about that. The lib64 path is in the
toolchain now and it appears to work nicely. Apparently it also
works fine for 64 bit Linux. In conjunction with point 1, if we
ever decide that we yet need interoperability with 32 bit Cygwin
processes, keeping the lib path to lib64 would help to integrate
both worlds. What is the problem with lib64 again?

Not so sure about that first point; while ld (and w32api) wanted lib64,
gcc wouldn't recognize it, at least not with a sys-root. While
doable, it does mean adjustments to cygport and some .cygport files,
as well as patches (available in Fedora and other distros) for some
packages which aren't lib64 aware. If we don't need it, why bother?
As for the future, I think we already agreed that trying to manage a
fully multiarch distro isn't feasible with setup/upset. If we're
talking only about multiarch-ing Cygwin itself, I think a lib32/lib
combination would do.

Ok, let's go the full way.
You *are* all aware that renaming the DLL back to cygwin1.dll means
that, afterwards, none of the currently existing binaries will work
anymore, right?

Fine by me. It would be silly to demand binary (or source) backward
compatibilty at this point.

Right, but this means I will not be able to use 64 bit mintty and tcsh
for at least... a few hours. You don't know what you're asking!!!


Corinna

[Reini Urban]

I can only maintain 32bit packages as I have no w64.
I doubt that most packages have both.

Those who do may favor a unified approach, but practically I would prefer
if someone else will maintain the 64 bit variants for my packages, which
looks like a seperate distro approach to me.

[Earnie Boyd]

Post by Reini Urban
I can only maintain 32bit packages as I have no w64.
I doubt that most packages have both.

You could always do the cross build approach; you just won't have the
ability to test both packages.

[Corinna Vinschen]

Post by Earnie Boyd

Post by Reini Urban
I can only maintain 32bit packages as I have no w64.
I doubt that most packages have both.

You could always do the cross build approach; you just won't have the
ability to test both packages.

As for testing, if you have a 64 bit Linux available, you can just run
an evaluation version of 64 bit Windows 8, Windows 2008 R2 or Windows
2012 in a KVM:

http://technet.microsoft.com/en-US/evalcenter/hh699156.aspx
http://technet.microsoft.com/en-us/evalcenter/dd459137.aspx
http://technet.microsoft.com/en-US/evalcenter/hh670538.aspx
http://support.microsoft.com/kb/948472


HTH,
Corinna

[Reini Urban]

Post by Corinna Vinschen

Post by Earnie Boyd

Post by Reini Urban
I can only maintain 32bit packages as I have no w64.
I doubt that most packages have both.

You could always do the cross build approach; you just won't have the
ability to test both packages.

As for testing, if you have a 64 bit Linux available, you can just run
an evaluation version of 64 bit Windows 8, Windows 2008 R2 or Windows
http://technet.microsoft.com/en-US/evalcenter/hh699156.aspx
http://technet.microsoft.com/en-us/evalcenter/dd459137.aspx
http://technet.microsoft.com/en-US/evalcenter/hh670538.aspx
http://support.microsoft.com/kb/948472

I'll try that, thanks

[Reini Urban]

Post by Reini Urban

Post by Corinna Vinschen

Post by Earnie Boyd

Post by Reini Urban
I can only maintain 32bit packages as I have no w64.
I doubt that most packages have both.

You could always do the cross build approach; you just won't have the
ability to test both packages.

As for testing, if you have a 64 bit Linux available, you can just run
an evaluation version of 64 bit Windows 8, Windows 2008 R2 or Windows
http://technet.microsoft.com/en-US/evalcenter/hh699156.aspx
http://technet.microsoft.com/en-us/evalcenter/dd459137.aspx
http://technet.microsoft.com/en-US/evalcenter/hh670538.aspx
http://support.microsoft.com/kb/948472

I'll try that, thanks

I found out my colleagues still have their MSDN subscription and the images.

en_windows_8_enterprise_x64_dvd_917522.iso has Hyper-V, VHD and
RemoteFX support.
For kvm the _n_ version (without media player) is enough.

[Charles Wilson]

Post by Corinna Vinschen
So we have five voices for keeping 64 and 32 bit separate, one for
mixing, one undecided but now leaning towards keeping it separate. I
also asked two people off-list, and both of them thought it a good idea
to keep the distros separate.
I'm inclined to revert or tweak a couple of my patches which were meant
to mix the distros. My implied question to you for each of these points
is, shall I do that or not?

I realize I'm chiming in late here, but I'm going to shock everybody and
actually *agree* with Yaakov. Add another vote for keeping 64 and 32
completely separate -- I believe its actually LESS of a long-term
maintenance nightmare, because it avoids:

a) yet more upstream churn in major packages (Qt, GTK, etc...) that use
dlopen() heavily. Even if we modify dlopen to "quick, #if 64bit then
check s/lib/cyg64/ first, then s/lib/cyg/, then lib- prefix...; else
just check s/lib/cyg, then lib- prefix;" it's still going to be painful.

b) All the shim code, whether in the cygwin runtime DLL, or a shim exe,
or a shim "fake" cygwin1.dll, is a fun place for more bugs to hide.

c) All those build systems out there, including cmake (and anyone who
uses it), plain-old-makefiles (zlib, libpng), libtool, scons, etc, will
all need to be 'taught' about the cyg64 prefix.

vs.

"I can't run stuff from /cygdrive/c/cygwin32/bin when I'm in a cygwin64
shell"

Don't Do That. WJM.

and

"I ran setup64.exe to upgrade my existing (32bit) cygwin installation
and now it's all broken."

Don't Do That. WJM.

--
Chuck

[Yaakov (Cygwin/X)]

Post by Charles Wilson
I realize I'm chiming in late here, but I'm going to shock everybody and
actually *agree* with Yaakov.

WHAT?!? Nah, must be an impostor. What did you do with the real
Chuck? :-)

Post by Charles Wilson
"I can't run stuff from /cygdrive/c/cygwin32/bin when I'm in a cygwin64
shell"

Simple apps do run using the cygwin1.dll, but obviously the mount
table are different and won't IPC with 64-bit apps.

Post by Charles Wilson
"I ran setup64.exe to upgrade my existing (32bit) cygwin installation
and now it's all broken."

I think we can (and should) make sure that setup won't do that.


Yaakov