Discussion:
RFE: mount support for device UUID's
Mark O'Keefe
2012-10-04 00:47:29 UTC
Permalink
Hi all,

I'm new to this list so not entirely sure of the protocol, but I'd like to suggest an enhancement
to the mount capabilities of cygwin. I would guess I'm not the only one who plugs various
external drives into my PC and would like to predictably define where they get mounted
under cygwin. At the moment this is limited to just the drive letter (which can change). I
would like your thoughts on an enhancement which allowed the use of the drive UUID to
identify the connected drive letter, ie., make the drive letter dynamic. The intent is to add
an optional parameter to fstab to support UUID.

I have explored using libblkid to achieve this, however there are a few drawbacks to this
approach: It introduces a dependency on this library; when using UUID's, the mount
functions become slow as libblkid has to open the drives to read the UUID information.
In exploring this I discovered that libblkid uses /dev/disk/by-uuid to help with performance,
however this is not present in cygwin.

To cut to the chase, I've created a version which leverages aspects of the /proc virtual
filesystem, behaviours of Windows, blkid and the special file names of cygwin.

When a drive is first attached to Windows it creates a unique GUID for that drive. This does
not change (unless you deliberately delete these from the registry). These unique
references appear in the /proc filesystem as: /proc/sys/GLOBAL??/Volume{GUID}
These are links to the device under /proc/sys/Device

Also under the /proc filesystem are the drive letter references:
/proc/sys/GLOBAL??/[A-Z]:
These link to the device under /proc/sys/Device also.

So, you can map a unique device GUID to a drive letter dynamically using this information.
Now to get the UUID into the picture.
If you had a script which populated the UUID as a link to the unique volume you can map
the UUID to a drive letter dynamically by expanding the links at run-time.

/dev/disk/by-uuid/<UUID> => /dev/sys/GLOBAL??/Volume{<GUID>}
/dev/sys/GLOBAL??/Volume{<GUID>} => /dev/sys/Device/<device>
/dev/sys/Device/harddisk[0-9]*/partition[0-9]* => /dev/sys/Device/<device>

http://www.cygwin.com/cygwin-ug-net/using-specialnames.html
Provides the mapping of POSIX device name to Internal NT device name (3rd line above).
So, using this you can take the output of blkid and get a valid mapping to the various
components required above.

You can use a bootstrap process to create the /dev/disk/by-uuid/<UUID> link for each
of your devices the first time you attach them (manual process) and after this rely on
the /proc filesystem to reliably map a connected UUID drive to the mapped drive letter.
You can also rely on this to determine if it is connected as the /proc filesystem only shows
the links for connected devices, so while the /dev/disk/by-uuid link is permanent, the
destination only exists if the drive is mounted.

Sample directory listings:

$ ls -l /dev/disk/by-uuid
total 6.0K
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 3 14:56 022A4E072A4DF865 -> /proc/sys/GLOBAL??/Volume{07f378c0-0785-11e2-8b16-9d8354493d98}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 3 11:12 20E819DCE819B150 ->/proc/sys/GLOBAL??/Volume{33795e42-562b-11e1-9e43-806e6f6e6963}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 3 15:07 5E83-34DB ->/proc/sys/GLOBAL??/Volume{a056ffe4-c49b-11e1-9a60-e01fff2983d8}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 3 14:56 78F07EF2F07EB5CA -> /proc/sys/GLOBAL??/Volume{82620ba9-ebef-11e1-b0aa-c0aa65e5609c}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 2 17:41 BC28CC9C28CC56D4 ->/proc/sys/GLOBAL??/Volume{ff6beeb7-ef41-11e1-be0b-bb34ac766d9f}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 2 17:42 D258228358226707 ->/proc/sys/GLOBAL??/Volume{ff6beea8-ef41-11e1-be0b-bb34ac766d9f}



$ ls -l /proc/sys/GLOBAL\?\?/Volume*
lr--r--r-- 1 Administrators SYSTEM 0 Oct 4 10:22/proc/sys/GLOBAL??/Volume{33795e42-562b-11e1-9e43-806e6f6e6963} ->/proc/sys/Device/HarddiskVolume1
lr--r--r-- 1 Administrators SYSTEM 0 Oct 4 10:22/proc/sys/GLOBAL??/Volume{33795e45-562b-11e1-9e43-806e6f6e6963} ->/proc/sys/Device/CdRom0
lr--r--r-- 1 Administrators SYSTEM 0 Oct 4 10:22/proc/sys/GLOBAL??/Volume{a056ffe4-c49b-11e1-9a60-e01fff2983d8} ->/proc/sys/Device/HarddiskVolume24
lr--r--r-- 1 Administrators SYSTEM 0 Oct 4 10:22/proc/sys/GLOBAL??/Volume{ff6beea8-ef41-11e1-be0b-bb34ac766d9f} ->/proc/sys/Device/HarddiskVolume30
lr--r--r-- 1 Administrators SYSTEM 0 Oct 4 10:22/proc/sys/GLOBAL??/Volume{ff6beeb7-ef41-11e1-be0b-bb34ac766d9f} ->/proc/sys/Device/HarddiskVolume29




$ ls -l /proc/sys/GLOBAL\?\?/[A-Z]:*
lr--r--r-- 1 Administrators SYSTEM 0 Oct 4 10:23 /proc/sys/GLOBAL??/C: ->/proc/sys/Device/HarddiskVolume1
lr--r--r-- 1 Administrators SYSTEM 0 Oct 4 10:23 /proc/sys/GLOBAL??/D: ->/proc/sys/Device/CdRom0
lr--r--r-- 1 Administrators SYSTEM 0 Oct 4 10:23 /proc/sys/GLOBAL??/E: ->/proc/sys/Device/HarddiskVolume24
lr--r--r-- 1 Administrators SYSTEM 0 Oct 4 10:23 /proc/sys/GLOBAL??/F: ->/proc/sys/Device/HarddiskVolume29
lr--r--r-- 1 Administrators SYSTEM 0 Oct 4 10:23 /proc/sys/GLOBAL??/G: ->/proc/sys/Device/HarddiskVolume30


Sample fstab:
# For a description of the file format, see the Users Guide
# http://cygwin.com/cygwin-ug-net/using.html#mount-table

# This is default anyway:
# none /cygdrive cygdrive binary,posix=0,user 0 0

/slots UUID=D258228358226707 /mnt/vtapes/slots none binary,user
/slots UUID=BC28CC9C28CC56D4 /mnt/archive/slots none binary,user

/slots UUID=5E83-34DB /mnt/4Gmodem/slots none binary,user


Output from mount:
$ mount
C:/cygwin/bin on /usr/bin type ntfs (binary,auto)
C:/cygwin/lib on /usr/lib type ntfs (binary,auto)
C:/cygwin on / type ntfs (binary,auto)
E:/slots on /mnt/4Gmodem/slots type vfat (binary,user)
F:/slots on /mnt/archive/slots type ntfs (binary,user)
G:/slots on /mnt/vtapes/slots type ntfs (binary,user)
C: on /cygdrive/c type ntfs (binary,posix=0,user,noumount,auto)
E: on /cygdrive/e type vfat (binary,posix=0,user,noumount,auto)
F: on /cygdrive/f type ntfs (binary,posix=0,user,noumount,auto)
G: on /cygdrive/g type ntfs (binary,posix=0,user,noumount,auto)



Thoughts of the developer community? I have a patch that I can send to someone
if they are interested in looking at including this into cygwin.

Cheers,
Mark.
Corinna Vinschen
2012-10-09 08:53:59 UTC
Permalink
Hi Mark,
Post by Mark O'Keefe
Hi all,
I'm new to this list so not entirely sure of the protocol, but I'd like to suggest an enhancement
...not a protocol as such, but it would be nice if you could restrict
the number of chars per line to somewhat below 80 :-}
Post by Mark O'Keefe
to the mount capabilities of cygwin. I would guess I'm not the only one who plugs various
external drives into my PC and would like to predictably define where they get mounted
under cygwin. At the moment this is limited to just the drive letter (which can change). I
would like your thoughts on an enhancement which allowed the use of the drive UUID to
identify the connected drive letter, ie., make the drive letter dynamic. The intent is to add
an optional parameter to fstab to support UUID.
In contrast to a mount table on a "real" UNIX, the Cygwin mount table is
just the mapping from an arbitrary Windows path to an arbitrary POSIX
path. Devices don't count in this picture, not even really for the
cygdrive mapping since DOS drive letters are NOT devices but paths in
the first place. From this point of view, using a UUID based approach
to access drives is quite a change from the usual approach.

Having said that, I see how a new approach to access removable drives
this way could be useful.
Post by Mark O'Keefe
[...]
To cut to the chase, I've created a version which leverages aspects of the /proc virtual
filesystem, behaviours of Windows, blkid and the special file names of cygwin.
When a drive is first attached to Windows it creates a unique GUID for that drive. This does
not change (unless you deliberately delete these from the registry). These unique
references appear in the /proc filesystem as: /proc/sys/GLOBAL??/Volume{GUID}
These are links to the device under /proc/sys/Device
These link to the device under /proc/sys/Device also.
Right.
Post by Mark O'Keefe
So, you can map a unique device GUID to a drive letter dynamically using this information.
Now to get the UUID into the picture.
If you had a script which populated the UUID as a link to the unique volume you can map
the UUID to a drive letter dynamically by expanding the links at run-time.
/dev/disk/by-uuid/<UUID> => /dev/sys/GLOBAL??/Volume{<GUID>}
/dev/sys/GLOBAL??/Volume{<GUID>} => /dev/sys/Device/<device>
/dev/sys/Device/harddisk[0-9]*/partition[0-9]* => /dev/sys/Device/<device>
Uh, hang on. You meant /proc/sys/..., right? You are not suggesting
another redirection via a not yet existing /dev/sys/... I hope?
Post by Mark O'Keefe
http://www.cygwin.com/cygwin-ug-net/using-specialnames.html
Provides the mapping of POSIX device name to Internal NT device name (3rd line above).
So, using this you can take the output of blkid and get a valid mapping to the various
components required above.
You can use a bootstrap process to create the /dev/disk/by-uuid/<UUID> link for each
of your devices the first time you attach them (manual process) and after this rely on
the /proc filesystem to reliably map a connected UUID drive to the mapped drive letter.
You can also rely on this to determine if it is connected as the /proc filesystem only shows
the links for connected devices, so while the /dev/disk/by-uuid link is permanent, the
destination only exists if the drive is mounted.
...which is kind of weird, considering that Linux /dev/disk/... only
contains drive information for connected drives as well...
Post by Mark O'Keefe
$ ls -l /dev/disk/by-uuid
total 6.0K
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 3 14:56 022A4E072A4DF865 -> /proc/sys/GLOBAL??/Volume{07f378c0-0785-11e2-8b16-9d8354493d98}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 3 11:12 20E819DCE819B150 ->/proc/sys/GLOBAL??/Volume{33795e42-562b-11e1-9e43-806e6f6e6963}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 3 15:07 5E83-34DB ->/proc/sys/GLOBAL??/Volume{a056ffe4-c49b-11e1-9a60-e01fff2983d8}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 3 14:56 78F07EF2F07EB5CA -> /proc/sys/GLOBAL??/Volume{82620ba9-ebef-11e1-b0aa-c0aa65e5609c}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 2 17:41 BC28CC9C28CC56D4 ->/proc/sys/GLOBAL??/Volume{ff6beeb7-ef41-11e1-be0b-bb34ac766d9f}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 2 17:42 D258228358226707 ->/proc/sys/GLOBAL??/Volume{ff6beea8-ef41-11e1-be0b-bb34ac766d9f}
Ok. But Linux UUIDs for fixed disks are equivalent in layout to Windows
UUIDs/GUIDs. So why not just use the GUID "as is"?

lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 3 14:56 07f378c0-0785-11e2-8b16-9d8354493d98 -> /proc/sys/GLOBAL??/Volume{07f378c0-0785-11e2-8b16-9d8354493d98}

But here's an important question. What exactly have you won by
adding the redirection via /dev/disk/by-uuid? The directory does
not contain any new information which isn't already available via
/proc/sys/GLOBAL??.
Post by Mark O'Keefe
# For a description of the file format, see the Users Guide
# http://cygwin.com/cygwin-ug-net/using.html#mount-table
# none /cygdrive cygdrive binary,posix=0,user 0 0
/slots UUID=D258228358226707 /mnt/vtapes/slots none binary,user
/slots UUID=BC28CC9C28CC56D4 /mnt/archive/slots none binary,user
/slots UUID=5E83-34DB /mnt/4Gmodem/slots none binary,user
I wouldn't like to allow a change in the fstab layout which introduces
variable positions for adjacent entries. Either just

UUID=07f378c0-0785-11e2-8b16-9d8354493d98

and ignore the ability to mount subdirs of drives, or something like
UUID=07f378c0-0785-11e2-8b16-9d8354493d98/slots

This addition to fstab would already be possible if the UUID is
the same as used by Windows and the mount mechanism could do this without
having to bootstrap a /dev/disk dir.

Btw., did you know that the drive is already accessible from within
Cygwin? Try

$ cd '/proc/sys/GLOBAL??/Volume{07f378c0-0785-11e2-8b16-9d8354493d98}/'

(note the trailing slash which is required to access the root dir of
the drive)

This already allows an equivalent mechanism today, just using symlinks
rather than mount table entries:

$ ln -s '/proc/sys/GLOBAL??/Volume{07f378c0-0785-11e2-8b16-9d8354493d98}/slots' /mnt/vtapes/slots
$ cd /mnt/vtapes/slots


Corinna
--
Corinna Vinschen Please, send mails regarding Cygwin to
Cygwin Project Co-Leader cygwin AT cygwin DOT com
Red Hat
Mark O'Keefe
2012-10-10 06:42:29 UTC
Permalink
Hi Corinna,
Post by Corinna Vinschen
Hi Mark,
Post by Mark O'Keefe
Hi all,
I'm new to this list so not entirely sure of the protocol, but I'd like to suggest an enhancement
...not a protocol as such, but it would be nice if you could restrict
the number of chars per line to somewhat below 80 :-}
Sorry about that. Unfortunately my mail program doesn't support this
so I had to eyeball where 80 characters was - obviously poorly :-)
Post by Corinna Vinschen
Post by Mark O'Keefe
to the mount capabilities of cygwin. I would guess I'm not the only one who plugs various
external drives into my PC and would like to predictably define where they get mounted
under cygwin. At the moment this is limited to just the drive letter (which can change). I
would like your thoughts on an enhancement which allowed the use of the drive UUID to
identify the connected drive letter, ie., make the drive letter dynamic. The intent is to add
an optional parameter to fstab to support UUID.
In contrast to a mount table on a "real" UNIX, the Cygwin mount table is
just the mapping from an arbitrary Windows path to an arbitrary POSIX
path. Devices don't count in this picture, not even really for the
cygdrive mapping since DOS drive letters are NOT devices but paths in
the first place. From this point of view, using a UUID based approach
to access drives is quite a change from the usual approach.
Having said that, I see how a new approach to access removable drives
this way could be useful.
It would definitely be useful (IMHO). I looked at it specifically for
supporting AMANDA backup with removable drives. The idea being to
use a rotation of removable drives as the tapes.
Post by Corinna Vinschen
Post by Mark O'Keefe
[...]
To cut to the chase, I've created a version which leverages aspects of the /proc virtual
filesystem, behaviours of Windows, blkid and the special file names of cygwin.
When a drive is first attached to Windows it creates a unique GUID for that drive. This does
not change (unless you deliberately delete these from the registry). These unique
references appear in the /proc filesystem as: /proc/sys/GLOBAL??/Volume{GUID}
These are links to the device under /proc/sys/Device
These link to the device under /proc/sys/Device also.
Right.
Post by Mark O'Keefe
So, you can map a unique device GUID to a drive letter dynamically using this information.
Now to get the UUID into the picture.
If you had a script which populated the UUID as a link to the unique volume you can map
the UUID to a drive letter dynamically by expanding the links at run-time.
/dev/disk/by-uuid/<UUID> => /dev/sys/GLOBAL??/Volume{<GUID>}
/dev/sys/GLOBAL??/Volume{<GUID>} => /dev/sys/Device/<device>
/dev/sys/Device/harddisk[0-9]*/partition[0-9]* => /dev/sys/Device/<device>
Uh, hang on. You meant /proc/sys/..., right? You are not suggesting
another redirection via a not yet existing /dev/sys/... I hope?
Yes, I did mean /proc/sys. Typo. Sorry for the confusion.
Post by Corinna Vinschen
Post by Mark O'Keefe
http://www.cygwin.com/cygwin-ug-net/using-specialnames.html
Provides the mapping of POSIX device name to Internal NT device name (3rd line above).
So, using this you can take the output of blkid and get a valid mapping to the various
components required above.
You can use a bootstrap process to create the /dev/disk/by-uuid/<UUID> link for each
of your devices the first time you attach them (manual process) and after this rely on
the /proc filesystem to reliably map a connected UUID drive to the mapped drive letter.
You can also rely on this to determine if it is connected as the /proc filesystem only shows
the links for connected devices, so while the /dev/disk/by-uuid link is permanent, the
destination only exists if the drive is mounted.
...which is kind of weird, considering that Linux /dev/disk/... only
contains drive information for connected drives as well...
Agreed. It would certainly be nice to have this populated based on
what is connected, however that introduces the dependency on
libblkid. I'm not sure this is acceptable inside the core of
cygwin. Alternative would be to extract (copy) the related code
over to cygwin (not sure how viable that is and it introduces
maintenance implications). It would also be necessary to look
at how to get the best performance out of this (caching and
recognising when drives connect/disconnect). I didn't know enough
about the internals of cygwin to determine how feasible this was.
If the connect/disconnect of a drive could trigger the necessary
drive read to get the UUID and populate this as a virtual directory
then that would be a better solution that is resilient to change.
Post by Corinna Vinschen
Post by Mark O'Keefe
$ ls -l /dev/disk/by-uuid
total 6.0K
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 3 14:56 022A4E072A4DF865 -> /proc/sys/GLOBAL??/Volume{07f378c0-0785-11e2-8b16-9d8354493d98}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 3 11:12 20E819DCE819B150 ->/proc/sys/GLOBAL??/Volume{33795e42-562b-11e1-9e43-806e6f6e6963}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 3 15:07 5E83-34DB ->/proc/sys/GLOBAL??/Volume{a056ffe4-c49b-11e1-9a60-e01fff2983d8}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 3 14:56 78F07EF2F07EB5CA -> /proc/sys/GLOBAL??/Volume{82620ba9-ebef-11e1-b0aa-c0aa65e5609c}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 2 17:41 BC28CC9C28CC56D4 ->/proc/sys/GLOBAL??/Volume{ff6beeb7-ef41-11e1-be0b-bb34ac766d9f}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 2 17:42 D258228358226707 ->/proc/sys/GLOBAL??/Volume{ff6beea8-ef41-11e1-be0b-bb34ac766d9f}
Ok. But Linux UUIDs for fixed disks are equivalent in layout to Windows
UUIDs/GUIDs. So why not just use the GUID "as is"?
The Windows UUIDs/GUIDs can change (though that isn't common and I
believe does require a specific user action to cause it). Also,
they aren't consistent with the result of running blkid. If using
UUID in mount you would expect it to match the blkid output.
Post by Corinna Vinschen
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 3 14:56 07f378c0-0785-11e2-8b16-9d8354493d98 -> /proc/sys/GLOBAL??/Volume{07f378c0-0785-11e2-8b16-9d8354493d98}
But here's an important question. What exactly have you won by
adding the redirection via /dev/disk/by-uuid? The directory does
not contain any new information which isn't already available via
/proc/sys/GLOBAL??.
Actually it does. As stated above, the UUID/GUID of Windows does
not match the UUID extracted from blkid. If you run blkid on the
disk you get the correct value. This value is the same as the
value returned if you then connect that disk to a linux based
system. This value doesn't change without physical change to
that disk partitioning, unlike the UUID/GUID of Windows.
Post by Corinna Vinschen
Post by Mark O'Keefe
# For a description of the file format, see the Users Guide
# http://cygwin.com/cygwin-ug-net/using.html#mount-table
# none /cygdrive cygdrive binary,posix=0,user 0 0
/slots UUID=D258228358226707 /mnt/vtapes/slots none binary,user
/slots UUID=BC28CC9C28CC56D4 /mnt/archive/slots none binary,user
/slots UUID=5E83-34DB /mnt/4Gmodem/slots none binary,user
I wouldn't like to allow a change in the fstab layout which introduces
variable positions for adjacent entries. Either just
UUID=07f378c0-0785-11e2-8b16-9d8354493d98
and ignore the ability to mount subdirs of drives, or something like
UUID=07f378c0-0785-11e2-8b16-9d8354493d98/slots
I like this option over the previous. What I noticed was that if
you did the first, then it overrides the cygdrive mount point.
I think the second option allows for the first as a variant anyway.
It is more flexible.
Post by Corinna Vinschen
This addition to fstab would already be possible if the UUID is
the same as used by Windows and the mount mechanism could do this without
having to bootstrap a /dev/disk dir.
Btw., did you know that the drive is already accessible from within
Cygwin? Try
$ cd '/proc/sys/GLOBAL??/Volume{07f378c0-0785-11e2-8b16-9d8354493d98}/'
Yes, I did know this. I've been trawling around in there following
how things related to determine how to establish the relationships I
wanted :-)
Post by Corinna Vinschen
(note the trailing slash which is required to access the root dir of
the drive)
This already allows an equivalent mechanism today, just using symlinks
$ ln -s '/proc/sys/GLOBAL??/Volume{07f378c0-0785-11e2-8b16-9d8354493d98}/slots' /mnt/vtapes/slots
$ cd /mnt/vtapes/slots
In my case this doesn't achieve what I want as I want to be able
to switch out drives and have them map to the same location.
This would create an issue if I mounted 2 drives that want the same
location of course, but that is a user error.

Backup to drive-1. When full, swap out and put in drive-2. Backup
continues without need to change configuration. The drive bay
is acting like a tape slot. An operator getting the email to swap
drives need know nothing about the configuration, just which
drive to insert.
Post by Corinna Vinschen
Corinna
--
Corinna Vinschen Please, send mails regarding Cygwin to
Cygwin Project Co-Leader cygwin AT cygwin DOT com
Red Hat
Cheers,
Mark.
Corinna Vinschen
2012-10-10 10:20:21 UTC
Permalink
Post by Mark O'Keefe
Hi Corinna,
Post by Corinna Vinschen
Hi Mark,
Post by Mark O'Keefe
Hi all,
I'm new to this list so not entirely sure of the protocol, but I'd like to suggest an enhancement
...not a protocol as such, but it would be nice if you could restrict
the number of chars per line to somewhat below 80 :-}
Sorry about that. Unfortunately my mail program doesn't support this
so I had to eyeball where 80 characters was - obviously poorly :-)
Thanks.
Post by Mark O'Keefe
Post by Corinna Vinschen
Post by Mark O'Keefe
[...]
You can use a bootstrap process to create the /dev/disk/by-uuid/<UUID> link for each
of your devices the first time you attach them (manual process) and after this rely on
the /proc filesystem to reliably map a connected UUID drive to the mapped drive letter.
You can also rely on this to determine if it is connected as the /proc filesystem only shows
the links for connected devices, so while the /dev/disk/by-uuid link is permanent, the
destination only exists if the drive is mounted.
...which is kind of weird, considering that Linux /dev/disk/... only
contains drive information for connected drives as well...
Agreed. It would certainly be nice to have this populated based on
what is connected, however that introduces the dependency on
libblkid.
I'm not sure this is acceptable inside the core of
cygwin.
No, it isn't. The Cygwin DLL may only rely on OS-provided functionality.
Post by Mark O'Keefe
Alternative would be to extract (copy) the related code
over to cygwin (not sure how viable that is and it introduces
maintenance implications).
Licensing may be an issue. Let me see... yes, it is. LGPL is not
feasible for inclusion into Cygwin.
Post by Mark O'Keefe
It would also be necessary to look
at how to get the best performance out of this (caching and
recognising when drives connect/disconnect). I didn't know enough
about the internals of cygwin to determine how feasible this was.
If the connect/disconnect of a drive could trigger the necessary
drive read to get the UUID and populate this as a virtual directory
then that would be a better solution that is resilient to change.
Post by Corinna Vinschen
Post by Mark O'Keefe
$ ls -l /dev/disk/by-uuid
total 6.0K
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 3 14:56 022A4E072A4DF865 -> /proc/sys/GLOBAL??/Volume{07f378c0-0785-11e2-8b16-9d8354493d98}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 3 11:12 20E819DCE819B150 ->/proc/sys/GLOBAL??/Volume{33795e42-562b-11e1-9e43-806e6f6e6963}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 3 15:07 5E83-34DB ->/proc/sys/GLOBAL??/Volume{a056ffe4-c49b-11e1-9a60-e01fff2983d8}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 3 14:56 78F07EF2F07EB5CA -> /proc/sys/GLOBAL??/Volume{82620ba9-ebef-11e1-b0aa-c0aa65e5609c}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 2 17:41 BC28CC9C28CC56D4 ->/proc/sys/GLOBAL??/Volume{ff6beeb7-ef41-11e1-be0b-bb34ac766d9f}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 2 17:42 D258228358226707 ->/proc/sys/GLOBAL??/Volume{ff6beea8-ef41-11e1-be0b-bb34ac766d9f}
Ok. But Linux UUIDs for fixed disks are equivalent in layout to Windows
UUIDs/GUIDs. So why not just use the GUID "as is"?
The Windows UUIDs/GUIDs can change (though that isn't common and I
believe does require a specific user action to cause it).
The Linux UUIDs can change as well, see tune2fs(1).
Post by Mark O'Keefe
Also,
they aren't consistent with the result of running blkid. If using
UUID in mount you would expect it to match the blkid output.
I wasn't even aware of blkid in the utils-linux package so far. It's
quite a bummer that libblkid already uses a mechanism which generates
other GUIDs than the ones in the native NT namespace.

The other problem with the libblkid approach is that you have to have
admin rights to access the values. This is neither the case under
Linux, nor with the GUID values used by the Windows volume manager.
Post by Mark O'Keefe
Post by Corinna Vinschen
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 3 14:56 07f378c0-0785-11e2-8b16-9d8354493d98 -> /proc/sys/GLOBAL??/Volume{07f378c0-0785-11e2-8b16-9d8354493d98}
But here's an important question. What exactly have you won by
adding the redirection via /dev/disk/by-uuid? The directory does
not contain any new information which isn't already available via
/proc/sys/GLOBAL??.
Actually it does. As stated above, the UUID/GUID of Windows does
not match the UUID extracted from blkid. If you run blkid on the
disk you get the correct value. This value is the same as the
value returned if you then connect that disk to a linux based
system. This value doesn't change without physical change to
that disk partitioning, unlike the UUID/GUID of Windows.
I'm not sure about the latter. If the volume manager would change the
GUIDs on the fly, it would be contrary to its job. Either way, in all
cases, Windows/Linux/etc, there's a chance to change the GUIDs.
Reformatting or repartitioning of a drive, setting the UUID explicitely,
whatever. In all cases this requires a deliberate action on the user's
account. This is no good reason to disregard the method which is already
available.
Post by Mark O'Keefe
Post by Corinna Vinschen
I wouldn't like to allow a change in the fstab layout which introduces
variable positions for adjacent entries. Either just
UUID=07f378c0-0785-11e2-8b16-9d8354493d98
and ignore the ability to mount subdirs of drives, or something like
UUID=07f378c0-0785-11e2-8b16-9d8354493d98/slots
I like this option over the previous. What I noticed was that if
you did the first, then it overrides the cygdrive mount point.
It shouldn't, unless there's a bug in the code.
Post by Mark O'Keefe
I think the second option allows for the first as a variant anyway.
It is more flexible.
Right.


Corinna
--
Corinna Vinschen Please, send mails regarding Cygwin to
Cygwin Project Co-Leader cygwin AT cygwin DOT com
Red Hat
Corinna Vinschen
2012-10-10 10:27:17 UTC
Permalink
Post by Corinna Vinschen
Post by Mark O'Keefe
The Windows UUIDs/GUIDs can change (though that isn't common and I
believe does require a specific user action to cause it).
The Linux UUIDs can change as well, see tune2fs(1).
Post by Mark O'Keefe
Also,
they aren't consistent with the result of running blkid. If using
UUID in mount you would expect it to match the blkid output.
I wasn't even aware of blkid in the utils-linux package so far. It's
quite a bummer that libblkid already uses a mechanism which generates
other GUIDs than the ones in the native NT namespace.
Just for the records: The volume serial number of NTFS partitions
returned by (for instance) GetVolumeInformation is equivalent to the
lower 32 bit of the NTFS volume identifier in the partition superblock.
No idea if that may be important or come in handy at one point, though.


Corinna
--
Corinna Vinschen Please, send mails regarding Cygwin to
Cygwin Project Co-Leader cygwin AT cygwin DOT com
Red Hat
Mark O'Keefe
2012-10-10 12:00:04 UTC
Permalink
Post by Corinna Vinschen
Post by Mark O'Keefe
Hi Corinna,
Post by Corinna Vinschen
Hi Mark,
Post by Mark O'Keefe
Hi all,
I'm new to this list so not entirely sure of the protocol, but I'd like to suggest an enhancement
...not a protocol as such, but it would be nice if you could restrict
the number of chars per line to somewhat below 80 :-}
Sorry about that. Unfortunately my mail program doesn't support this
so I had to eyeball where 80 characters was - obviously poorly :-)
Thanks.
Post by Mark O'Keefe
Post by Corinna Vinschen
Post by Mark O'Keefe
[...]
You can use a bootstrap process to create the /dev/disk/by-uuid/<UUID> link for each
of your devices the first time you attach them (manual process) and after this rely on
the /proc filesystem to reliably map a connected UUID drive to the mapped drive letter.
You can also rely on this to determine if it is connected as the /proc filesystem only shows
the links for connected devices, so while the /dev/disk/by-uuid link is permanent, the
destination only exists if the drive is mounted.
...which is kind of weird, considering that Linux /dev/disk/... only
contains drive information for connected drives as well...
Agreed. It would certainly be nice to have this populated based on
what is connected, however that introduces the dependency on
libblkid.
I'm not sure this is acceptable inside the core of
cygwin.
No, it isn't. The Cygwin DLL may only rely on OS-provided functionality.
Post by Mark O'Keefe
Alternative would be to extract (copy) the related code
over to cygwin (not sure how viable that is and it introduces
maintenance implications).
Licensing may be an issue. Let me see... yes, it is. LGPL is not
feasible for inclusion into Cygwin.
Post by Mark O'Keefe
It would also be necessary to look
at how to get the best performance out of this (caching and
recognising when drives connect/disconnect). I didn't know enough
about the internals of cygwin to determine how feasible this was.
If the connect/disconnect of a drive could trigger the necessary
drive read to get the UUID and populate this as a virtual directory
then that would be a better solution that is resilient to change.
Post by Corinna Vinschen
Post by Mark O'Keefe
$ ls -l /dev/disk/by-uuid
total 6.0K
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 3 14:56 022A4E072A4DF865 -> /proc/sys/GLOBAL??/Volume{07f378c0-0785-11e2-8b16-9d8354493d98}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 3 11:12 20E819DCE819B150 ->/proc/sys/GLOBAL??/Volume{33795e42-562b-11e1-9e43-806e6f6e6963}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 3 15:07 5E83-34DB ->/proc/sys/GLOBAL??/Volume{a056ffe4-c49b-11e1-9a60-e01fff2983d8}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 3 14:56 78F07EF2F07EB5CA -> /proc/sys/GLOBAL??/Volume{82620ba9-ebef-11e1-b0aa-c0aa65e5609c}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 2 17:41 BC28CC9C28CC56D4 ->/proc/sys/GLOBAL??/Volume{ff6beeb7-ef41-11e1-be0b-bb34ac766d9f}
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 2 17:42 D258228358226707 ->/proc/sys/GLOBAL??/Volume{ff6beea8-ef41-11e1-be0b-bb34ac766d9f}
Ok. But Linux UUIDs for fixed disks are equivalent in layout to Windows
UUIDs/GUIDs. So why not just use the GUID "as is"?
The Windows UUIDs/GUIDs can change (though that isn't common and I
believe does require a specific user action to cause it).
The Linux UUIDs can change as well, see tune2fs(1).
Post by Mark O'Keefe
Also,
they aren't consistent with the result of running blkid. If using
UUID in mount you would expect it to match the blkid output.
I wasn't even aware of blkid in the utils-linux package so far. It's
quite a bummer that libblkid already uses a mechanism which generates
other GUIDs than the ones in the native NT namespace.
The other problem with the libblkid approach is that you have to have
admin rights to access the values. This is neither the case under
Linux, nor with the GUID values used by the Windows volume manager.
Post by Mark O'Keefe
Post by Corinna Vinschen
lrwxrwxrwx 1 Administrators SYSTEM 63 Oct 3 14:56 07f378c0-0785-11e2-8b16-9d8354493d98 -> /proc/sys/GLOBAL??/Volume{07f378c0-0785-11e2-8b16-9d8354493d98}
But here's an important question. What exactly have you won by
adding the redirection via /dev/disk/by-uuid? The directory does
not contain any new information which isn't already available via
/proc/sys/GLOBAL??.
Actually it does. As stated above, the UUID/GUID of Windows does
not match the UUID extracted from blkid. If you run blkid on the
disk you get the correct value. This value is the same as the
value returned if you then connect that disk to a linux based
system. This value doesn't change without physical change to
that disk partitioning, unlike the UUID/GUID of Windows.
I'm not sure about the latter. If the volume manager would change the
GUIDs on the fly, it would be contrary to its job. Either way, in all
cases, Windows/Linux/etc, there's a chance to change the GUIDs.
Reformatting or repartitioning of a drive, setting the UUID explicitely,
whatever. In all cases this requires a deliberate action on the user's
account. This is no good reason to disregard the method which is already
available.
Post by Mark O'Keefe
Post by Corinna Vinschen
I wouldn't like to allow a change in the fstab layout which introduces
variable positions for adjacent entries. Either just
UUID=07f378c0-0785-11e2-8b16-9d8354493d98
and ignore the ability to mount subdirs of drives, or something like
UUID=07f378c0-0785-11e2-8b16-9d8354493d98/slots
I like this option over the previous. What I noticed was that if
you did the first, then it overrides the cygdrive mount point.
It shouldn't, unless there's a bug in the code.
Post by Mark O'Keefe
I think the second option allows for the first as a variant anyway.
It is more flexible.
Right.
Would it be more appropriate to have a GUID instead of UUID extension
as described above? Have a new tool which provides information to
help users determine which drives map to which GUID's? Don't confuse
the issue by calling it UUID and clashing with blkid. Call it what it
is - the Windows GUID. Same concept, but different identifier. Fits
with cygwin and provides the functionality with what is there already.
Post by Corinna Vinschen
Corinna
--
Corinna Vinschen Please, send mails regarding Cygwin to
Cygwin Project Co-Leader cygwin AT cygwin DOT com
Red Hat
Corinna Vinschen
2012-10-10 15:01:01 UTC
Permalink
Post by Mark O'Keefe
Post by Corinna Vinschen
Post by Mark O'Keefe
Post by Corinna Vinschen
[...]
Ok. But Linux UUIDs for fixed disks are equivalent in layout to Windows
UUIDs/GUIDs. So why not just use the GUID "as is"?
The Windows UUIDs/GUIDs can change (though that isn't common and I
believe does require a specific user action to cause it).
The Linux UUIDs can change as well, see tune2fs(1).
Post by Mark O'Keefe
Also,
they aren't consistent with the result of running blkid. If using
UUID in mount you would expect it to match the blkid output.
I wasn't even aware of blkid in the utils-linux package so far. It's
quite a bummer that libblkid already uses a mechanism which generates
other GUIDs than the ones in the native NT namespace.
The other problem with the libblkid approach is that you have to have
admin rights to access the values. This is neither the case under
Linux, nor with the GUID values used by the Windows volume manager.
[...]
Would it be more appropriate to have a GUID instead of UUID extension
as described above? Have a new tool which provides information to
help users determine which drives map to which GUID's? Don't confuse
the issue by calling it UUID and clashing with blkid. Call it what it
is - the Windows GUID. Same concept, but different identifier. Fits
with cygwin and provides the functionality with what is there already.
Hang on. This won't go into Cygwin 1.7.17 anyway, so we still have some
time to discuss this stuff and to create a nice solution.

I just made some tests and it looks like all information is available
without requiring admin rights.

Here's what libblkid does (apart from other methods which are not
feasible on Windows). It opens the partition for reading and tries to
access the superblock info. This only works for admins on Windows.

The information taken for the UUID value is the 8 byte
VolumeSerialNumber on NTFS and the 4 byte serial number on all FAT
variations. ReFS isn't handled yet by blkid, so we're free to do as we
like, for instance, just using the serial number as on FAT. CD-ROMs and
DVDs apparently don't have a UUID, so they can be ignored.

The 8 byte NTFS VolumeSerialNumber can be read without admin rights by
using the NtFsControlFile(FSCTL_GET_NTFS_VOLUME_DATA) call. For the
returned data structure, see
http://msdn.microsoft.com/en-us/library/windows/desktop/aa365256%28v=vs.85%29.aspx

The serial number for FAT and ReFS handling can be read without admin
rights by NtQueryVolumeInformationFile(FileFsVolumeInformation), as
it's already done in fsinfo::update in mount.cc.

So, here's a vague idea for a start. If we see a UUID= entry in
the fstab file, we scan the native \Device directory for
"HarddiskVolumeXXX" devices(*). For each such device, open it with
minimal rights and call NtFsControlFile(FSCTL_GET_NTFS_VOLUME_DATA).
If this works, it's an NTFS drive and we have the UUID. If it
fails, call NtQueryVolumeInformationFile(FileFsVolumeInformation)
and use the serial number as UUID. If the UUIDs match, we have our
device path and we can create the mount point from here.

Note that the latter part, creating a mount point using the native
device path, doesn't work yet at all. This has to be implemented
as well.

(*) Alternatively we scan the \Device\HarddiskX\PartitionY entries
as we do in format_proc_partitions in fhandler_proc.cc. This also
provides a direct connection between the UUID and the Cygwin /dev/sdXY
device.

Having said that, a slightly tweaked format_proc_partitions would
also allow to implement a virtual /dev/disk/by-uuid directory with
not too much effort.


Corinna
--
Corinna Vinschen Please, send mails regarding Cygwin to
Cygwin Project Co-Leader cygwin AT cygwin DOT com
Red Hat
Loading...