Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
ZFS-FUSE and disk reordering
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
transpetaflops
Tux's lil' helper
Tux's lil' helper


Joined: 16 May 2005
Posts: 136

PostPosted: Tue Aug 03, 2010 1:01 pm    Post subject: ZFS-FUSE and disk reordering Reply with quote

Trying out ZFS-FUSE since a week back. I have 4x1,5TB (sdc, sdd, sde and sdf) in raidz. Today I rebooted the computer while a USB flashdrive was attached. The kernel chosed to label the flashdrive sdf and the last harddrive instead became sdg and the ZFS pool went into degraded mode. I simply removed the flashdrive and rebooted and everything was fine again but I wonder if I did a mistake using the device nodes for the pool? Should I perhaps instead have used the unique labels in /dev/disk/by-id or did I miss something vital during the creation of the pool? If the ids are the better solution, how do I go about to switch to them on a live pool? It's not a big deal if I have to remake it since I still have all the data on another computer too but it would be a good lesson to know how to do it live.
Regards
Morgan
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 6108
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Tue Aug 03, 2010 1:28 pm    Post subject: Reply with quote

*subscribes*

I'd be interested in that too ! :)
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa

Hardcore Gentoo Linux user since 2004 :D
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2858
Location: Bay Area, CA

PostPosted: Wed Aug 04, 2010 6:04 am    Post subject: Reply with quote

if you are using 0.6.9, just export and import the pool, no need to reboot. 'zpool import' will use the /dev/disk/by-uuid/ path in 0.6.9. You can make it force use that by using the '-d' option and that will import your pool back as well.

I typically go for a more easier approach where I wrote a udev rule to name my devices with well-known names like /dev/vertexA, /dev/wdc1tbA, /dev/seagate1tbA etc. based on the serial numbers. And then I used those to create the pool. Since, these names are not affected by plugging order on the mobo, disks coming in and going out, they always work.

PS: Note that exporting the pool means, it will unmount the FS and pool will disappear until imported back. Do not make a silly mistake of recreating the pool (someone recently did that and posted on the google-groups of zfs-fuse) if you can't access your pool. Just putting it out there.
Back to top
View user's profile Send private message
transpetaflops
Tux's lil' helper
Tux's lil' helper


Joined: 16 May 2005
Posts: 136

PostPosted: Wed Aug 04, 2010 12:34 pm    Post subject: Reply with quote

Ah, very nice! Now it did exactly what I was asking in my original post. :D However, as you can see from the output below, there are several different schemes to identify the disks in /dev/disk/by-id and zpool import didn't chose the same for the last disk, /dev/sdf, the one that got reordered. Any thought on why this occurred and how to correct it?

Regards
Morgan

Code:
# zpool status
  pool: storage
 state: ONLINE
 scrub: none requested
config:

   NAME                                               STATE     READ WRITE CKSUM
   storage                                            ONLINE       0     0     0
     raidz1-0                                         ONLINE       0     0     0
       disk/by-id/ata-SAMSUNG_HD154UI_S1XWJ1KZ118872  ONLINE       0     0     0
       disk/by-id/ata-SAMSUNG_HD154UI_S1XWJ1KZ118856  ONLINE       0     0     0
       disk/by-id/ata-SAMSUNG_HD154UI_S1XWJ1KZ118855  ONLINE       0     0     0
       disk/by-id/wwn-0x50024e9002e0c52f              ONLINE       0     0     0

errors: No known data errors
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2858
Location: Bay Area, CA

PostPosted: Wed Aug 04, 2010 4:05 pm    Post subject: Reply with quote

Why do you think it needs correction?.... :lol: :P :wink:
Back to top
View user's profile Send private message
transpetaflops
Tux's lil' helper
Tux's lil' helper


Joined: 16 May 2005
Posts: 136

PostPosted: Wed Aug 04, 2010 5:19 pm    Post subject: Reply with quote

Didn't mean to imply that the developer made a sloppy work :wink:
No, I was merely curious of why zfs chose to mix entries from two different labelling schemes and if there was some easy way to correct it. It disturbs my sense of aesthetics. :D
/Morgan
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 2511
Location: Canada

PostPosted: Wed Aug 04, 2010 5:23 pm    Post subject: Reply with quote

transpetaflops wrote:
Didn't mean to imply that the developer made a sloppy work :wink:
No, I was merely curious of why zfs chose to mix entries from two different labelling schemes and if there was some easy way to correct it. It disturbs my sense of aesthetics. :D
/Morgan


Perhaps it is not just aesthetics. That it treats the last drive, /dev/sdf, differently, is probably related to your original problem - that it was happy to insert a new flash drive in its place and shift it to /dev/sdg
Back to top
View user's profile Send private message
transpetaflops
Tux's lil' helper
Tux's lil' helper


Joined: 16 May 2005
Posts: 136

PostPosted: Wed Aug 04, 2010 5:29 pm    Post subject: Reply with quote

Well, I assume I could always detach it from the pool, erase it with dd and then insert it again, if anyone thinks that will help?
/M
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 2511
Location: Canada

PostPosted: Wed Aug 04, 2010 6:16 pm    Post subject: Reply with quote

This, by UID technique, does it cause a problem when a drive fails and you want to replace it with a new one ? Especially in hot-swap situation ?
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2858
Location: Bay Area, CA

PostPosted: Wed Aug 04, 2010 6:23 pm    Post subject: Reply with quote

You definitely want to make sure that /dev/disk/by-id/wwn-0x50024e9002e0c52f is the valid and intended device. Can you check what does it point to?

I wouldn't recommend taking it out and resilvering. Not needed and exposes you failure.

If you are worried about aesthetics, write a udev rule and name your disks with good names. Export your pool and re-import with "-d /dev". I think it uses alphanumeric sort when it looks at devices, so starting names with things less than 's' (assumption: all disks are /dev/sd* in libata world), should get you going.

Don't plug any removable drive drive when you do this.
Back to top
View user's profile Send private message
devsk
Advocate
Advocate


Joined: 24 Oct 2003
Posts: 2858
Location: Bay Area, CA

PostPosted: Wed Aug 04, 2010 6:28 pm    Post subject: Reply with quote

dmpogo wrote:
This, by UID technique, does it cause a problem when a drive fails and you want to replace it with a new one ? Especially in hot-swap situation ?
Depends on how you replace the drive. If you are using a spare and 0.6.9 code with /etc/zfs/zfs_pool_alert doing the work for you, then everything is automatic.

If you are going manual, then you need to remove a vdev and attach the new one, and path gets driven the same way. So, I don't see what UUI path will create an issue.
Back to top
View user's profile Send private message
transpetaflops
Tux's lil' helper
Tux's lil' helper


Joined: 16 May 2005
Posts: 136

PostPosted: Wed Aug 04, 2010 10:49 pm    Post subject: Reply with quote

devsk wrote:
You definitely want to make sure that /dev/disk/by-id/wwn-0x50024e9002e0c52f is the valid and intended device. Can you check what does it point to?

Yes, I did that immediately when I noticed the deviation and it does point to the correct drive. Even made a scrub to check the pool. Every drive in the system (6 identical) has the same layout in /dev/disk/by-id like shown below.
Code:
# ls -l /dev/disk/by-id/ | grep sdf
lrwxrwxrwx 1 root root  9  5 aug 00.19 ata-SAMSUNG_HD154UI_S1XWJ1KZ118854 -> ../../sdf
lrwxrwxrwx 1 root root  9  5 aug 00.19 scsi-SATA_SAMSUNG_HD154UIS1XWJ1KZ118854 -> ../../sdf
lrwxrwxrwx 1 root root  9  5 aug 00.19 wwn-0x50024e9002e0c52f -> ../../sdf

However, it's only the last drive in the zfs pool (sdf) that got linked with the wwn-label while all the others used the ata-labels. As I mentioned earlier I still have all data on other computers so I might as well recreate the pool from scratch and see if I can repeat this behaviour. Maybe it's worthy of a bug report?

devsk wrote:
If you are worried about aesthetics, write a udev rule and name your disks with good names. Export your pool and re-import with "-d /dev". I think it uses alphanumeric sort when it looks at devices, so starting names with things less than 's' (assumption: all disks are /dev/sd* in libata world), should get you going.

Well, that's beyond my abilities atm. I'd feel much better if I can use the default rules so I don't have to keep track of my own changes and how they will affect the next update. But thanks for the tip.
/M
Back to top
View user's profile Send private message
transpetaflops
Tux's lil' helper
Tux's lil' helper


Joined: 16 May 2005
Posts: 136

PostPosted: Thu Aug 05, 2010 3:49 pm    Post subject: Reply with quote

Turns out I can reproduce this. If I create the pool with the /dev/sdN device nodes and export and import it, the last drive is imported with its wwn-label instead of its ata-label. If I however create the pool with the ata-labels directly, they remain the same even after an export and import. If anyone is interested in more tests and info I'm willing to provide that but let me know within a few days before I start using this filesystem live and can't experiment with it any longer. Thanks for your help so far devsk.
Regards
Morgan

EDIT: Fixed embarrassing grammatical errors.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum