Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
trying to setup software RAID1 makes me crazy
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
lutel
Tux's lil' helper
Tux's lil' helper


Joined: 19 Oct 2003
Posts: 93
Location: Pomroczna

PostPosted: Sun Jun 15, 2014 6:15 pm    Post subject: trying to setup software RAID1 makes me crazy Reply with quote

I've been convinced to use software RAID instead of fakeraid, and I lost hours trying to boot machine from software RAID (both with lilo/grub2), here is my setup:

Code:

# cat /etc/fstab
/dev/md0      /boot           ext4    noauto,noatime  1 2
/dev/md1      /               ext4    noatime         0 1
LABEL=var       /var            ext4    noatime         0 1
LABEL=lib       /var/lib        ext4    noatime         0 1
LABEL=log       /var/log        ext4    noatime         0 1
LABEL=data      /w              ext4    noatime         0 1
/dev/cdrom      /mnt/cdrom      auto    noauto,ro       0 0
/dev/fd0        /mnt/floppy     auto    noauto          0 0

# parted /dev/sda (same for sdb, sdc)
GNU Parted 3.1
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) p
Model: ATA WDC WD2000FYYZ-0 (scsi)
Disk /dev/sda: 2000GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number  Start   End     Size    File system  Name  Flags
 1      1049kB  67.1MB  66.1MB  ext4         boot  bios_grub
 2      67.1MB  15.7GB  15.7GB               root  raid
 3      15.7GB  26.2GB  10.5GB               tmp   raid
 4      26.2GB  36.7GB  10.5GB               var   raid
 5      36.7GB  68.2GB  31.5GB               lib   raid
 6      68.2GB  78.6GB  10.5GB               log   raid
 7      78.6GB  2000GB  1922GB               data  raid

#mdadm --detail --scan
ARRAY /dev/md5 metadata=1.2 spares=1 name=livecd:5 UUID=98c62d6f:e552d93c:83f3fe37:63ca1585
ARRAY /dev/md4 metadata=1.2 spares=1 name=livecd:4 UUID=3a851e88:66872e12:d298bd0c:f2890ff7
ARRAY /dev/md3 metadata=1.2 spares=1 name=livecd:3 UUID=b1c712f8:f6aad7a9:342482f2:1fb24ff6
ARRAY /dev/md2 metadata=1.2 spares=1 name=livecd:2 UUID=fee31ef1:42726f23:514232bc:9c4bc4d1
ARRAY /dev/md1 metadata=1.2 spares=1 name=livecd:1 UUID=a724e247:4fff2180:57cd302f:42d0b052
ARRAY /dev/md0 metadata=0.90 spares=1 UUID=25a0a1e0:188a36da:cb201669:f728008a

#cat /etc/lilo.conf
compact
lba32
boot = /dev/md0
raid-extra-boot=mbr-only
menu-scheme = Wb
prompt
timeout = 50
delay = 0
root = /dev/md1

image = /boot/kernel-genkernel-x86_64-3.14.5-hardened-r2
  label = 1
  append="rootdelay=3,domdadm,real_root=a724e247-4fff2180-57cd302f-42d0b052"
  initrd=/boot/initramfs-genkernel-x86_64-3.14.5-hardened-r2

image = /boot/kernel-genkernel-x86_64-3.14.5-hardened-r2
  label = 2
  initrd=/boot/initramfs-genkernel-x86_64-3.14.5-hardened-r2


after boot from live dvd, i always have to rebuild md0, as this is the array status i see (/dev/md0 is split into two arrays), below is output before fix:

Code:
md126 : active raid1 sda1[0]
      64448 blocks [2/1] [U_]
md127 : active raid1 sdb1[1] sdc1[0]
      64448 blocks [2/2] [UU]
md1 : active raid1 sda2[0] sdc2[2](S) sdb2[1]
      15286144 blocks super 1.2 [2/2] [UU]
md2 : active raid1 sda3[0] sdc3[2](S) sdb3[1]
      10231680 blocks super 1.2 [2/2] [UU]
md3 : active raid1 sda4[0] sdc4[2](S) sdb4[1]
      10231680 blocks super 1.2 [2/2] [UU]
md4 : active raid1 sda5[0] sdc5[2](S) sdb5[1]
      30703488 blocks super 1.2 [2/2] [UU]
md5 : active raid1 sda6[0] sdc6[2](S) sdb6[1]
      10231680 blocks super 1.2 [2/2] [UU]

cat .config | egrep "(_MD_|AHCI|TMPFS|EXT4)"
CONFIG_DEVTMPFS=y
# CONFIG_DEVTMPFS_MOUNT is not set
CONFIG_SATA_AHCI=y
# CONFIG_SATA_AHCI_PLATFORM is not set
# CONFIG_SATA_ACARD_AHCI is not set
CONFIG_MD_AUTODETECT=y
# CONFIG_MD_LINEAR is not set
# CONFIG_MD_RAID0 is not set
CONFIG_MD_RAID1=y
CONFIG_MD_RAID10=y
CONFIG_MD_RAID456=y
CONFIG_MD_MULTIPATH=m
CONFIG_MD_FAULTY=m
CONFIG_EXT4_FS=y
CONFIG_EXT4_FS_POSIX_ACL=y
CONFIG_EXT4_FS_SECURITY=y
# CONFIG_EXT4_DEBUG is not set
CONFIG_TMPFS=y
CONFIG_TMPFS_POSIX_ACL=y
CONFIG_TMPFS_XATTR=y


Here is my "fix" to get /dev/md0 back (although empty)

Code:
mdadm --stop /dev/md126
mdadm --stop /dev/md127
mdadm --create -e 0.9 /dev/md0 --level=1 --raid-devices=2 /dev/sda1 /dev/sdb1 --spare-devices=1 /dev/sdc1
mkfs.ext4 -L boot -m 0 -O ^huge_file /dev/md0
mdadm --detail --scan > /mnt/gentoo/etc/mdadm.conf


Genkernel... Eventualy i'd like to have just kernel, without initrd, and compile-in all drivers nessecary. Tried couple of times - always to kernel panic, system is unable to find root.

genkernel --mdadm ----mdadm-config=/etc/mdadm.conf --install all

Tried grub2, but machine is rebooting constantly with no messages visible. After booting with lilo, system is not able to find root. When I enter shell in initrd, there are no /dev/md... entries. Here is error message I get:

Code:
Determining root device...
!! Block device 901 is not a valid root device...
!! Could not find the root block device in .


Please help, I'm out of ideas what am i doing wrong, tried grub2, lilo, I still can't boot.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43178
Location: 56N 3W

PostPosted: Sun Jun 15, 2014 8:33 pm    Post subject: Reply with quote

lutel,

Code:
 Could not find the root block device in .   
Should contain a list of block devices the kernel can see.
The list is empty, this probably means your low level dchipset dpiver is set to =M or off.

Post the output of lspci and put your kernel .config onto a pastebin please.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
lutel
Tux's lil' helper
Tux's lil' helper


Joined: 19 Oct 2003
Posts: 93
Location: Pomroczna

PostPosted: Mon Jun 16, 2014 6:24 am    Post subject: Reply with quote

NeddySeagoon,

Thanks for pointing this to this, here is my lspci and .config http://pastebin.com/8Pafaarg
I dont know which driver should I compile in for this platform, I though that with initrd it would work even if its module.
But the best would be if it can be set without initrd at all.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43178
Location: 56N 3W

PostPosted: Mon Jun 16, 2014 4:07 pm    Post subject: Reply with quote

lutel,

An initrd is just a cut down root filesystem in a file.
The can work with everything as modules ... it all depends on how the initrd is made.

I roll my own initrd, then I have a chance to fix them when they don't work.
Your kernel looks like it should have got further than it did without an initrd to help it along.



Code:
# CONFIG_SATA_AHCI_PLATFORM is not set
May be needed.
I have two different motherboards with identical AMD chip sets. One needs that option, one does not.

Code:
# CONFIG_DEVTMPFS_MOUNT is not set
may be needed too as it allows the kernel to populate the /dev filesystem with no outside help.

How did you configure your kernel?
Its not a genkernel kernel.

I have not looked at your raid and filesystem settings in any detail but you have
Code:
CONFIG_MD_AUTODETECT=y

Be warned that that only works with raid sets that have metadata version 0.90. The default for mdadm is 1.2. If you told mdadm to use 0.90 metadata, kernel auto assembly will work for you.
If not, you need an initrd to assemble your raid if you have root on raid.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
lutel
Tux's lil' helper
Tux's lil' helper


Joined: 19 Oct 2003
Posts: 93
Location: Pomroczna

PostPosted: Mon Jun 16, 2014 7:35 pm    Post subject: Reply with quote

Hi NeddySeagoon,

Thanks, indeed I have 0.90 for /boot and 1.2 for / - so as you said only option to assemble is to go initrd. So what I did, i've run

genkernel --mdadm --mdadm-config=/etc/mdadm.conf --install --menuconfig all

added CONFIG_SATA_AHCI_PLATFORM and CONFIG_DEVTMPFS_MOUNT (my new .config http://pastebin.com/T6QVHsgK), then lilo, tried to boot, still the same...

But, when i enter shell of initramfs, i see only /dev/sdx0-6, no /dev/md.... - it seems that low level drivers correctly detects my disks, but raid is not build by mdadm.

As you said, 0.90 is required to automount - if I use 0.90 metadata for all partiotns, will it be automounted and initrd wont be required? If so, whats the disadvantage of having 0.90 instead of 1.2?
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43178
Location: 56N 3W

PostPosted: Mon Jun 16, 2014 7:59 pm    Post subject: Reply with quote

lutel,

Raid 0.90 metadata only supports up to 28 drives. Thats not a problem for most people.

Swapping from 1.2 to 0.90 metadata will destroy the data on the raid but with care it can be done.
I'm not a genkernel user, so I can't help there.

You say you got the same output but if /dev listed your block devices the error
Code:
!! Could not find the root block device in .
should also list all of your block devices too.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
lutel
Tux's lil' helper
Tux's lil' helper


Joined: 19 Oct 2003
Posts: 93
Location: Pomroczna

PostPosted: Mon Jun 16, 2014 8:18 pm    Post subject: Reply with quote

I'm getting closer i think, i still get error "!! Could not find the root block device in ."
But then i can enter shell, and then, when i mdadm -As - the array is built correctly!
So the drivers are there, but still initrd is not building the array - don't know why...
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43178
Location: 56N 3W

PostPosted: Mon Jun 16, 2014 8:31 pm    Post subject: Reply with quote

lutel,

In the rescue shell, you should be able to do
Code:
cat init
and see the init script that the initrd tries to run.
If you execute the commands from the mdadm -A to the end of the script, in sequence, your system should boot.

I far prefer a hand rolled initrd to a script built one that I don't understand and can't fix.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
lutel
Tux's lil' helper
Tux's lil' helper


Joined: 19 Oct 2003
Posts: 93
Location: Pomroczna

PostPosted: Mon Jun 16, 2014 8:43 pm    Post subject: Reply with quote

Finally i found out what i did wrong, i was passing wrong parameters to kernel...

instead of

Code:
append="domdadm real_root=/dev/md1"


i've been using

Code:
append="domdadm,real_root=/dev/md1"


so stupid mistake...

But anyway, i'll try now to reinstall everything on 0.9, i'd like to get rid of initrd.

Thank you for your help so far.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43178
Location: 56N 3W

PostPosted: Mon Jun 16, 2014 8:57 pm    Post subject: Reply with quote

lutel,

Don't reinstall. Move the install over.
Do this from a liveCD

You have a working raid1
Fail one partition of /, so your raid is degraded.
Remove the failed partition from the raid set.

Using the failed partition, make a new raid1 set with a missing drive (metadata 0.90)
Make a filesystem on this raid set.

You now nave two degraded raid sets. One with your install on it one empty.
Copy your install over.

Boot the new degraded raid.
Add the partition from the orignial install to the new raid set.
Look in /proc/mdstat to see it sync.

You get to practice replacing a failed drive in a raid set too.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
lutel
Tux's lil' helper
Tux's lil' helper


Joined: 19 Oct 2003
Posts: 93
Location: Pomroczna

PostPosted: Tue Jun 17, 2014 10:31 am    Post subject: Reply with quote

Hi NeddySeagoon,

I'll give it a try as you said as soon I'll get access to it. I've got some questions though, what do you think...?

- are there any performance drawbacks from going to 0.9 metadata?
- i've made partitions with parted with type "RAID partition" - so I can't mount /dev/sd... to copy data, so i guess the only way to replace metadata with your guide is to create another array, so array numbering will change?
- i have no access to check it, but can i use md=1,/dev/sda1,/dev/sdb1 and so on to mount arrays with 1.2 metadata?
- with append md=.... - is it possible to define spare?, so something like md=1,/dev/sda1,/dev/sdb1,/dev,sdc1 (last one becoming spare)?

If spare can not be set during booting, is it possible to set it up afterwards?
Back to top
View user's profile Send private message
szatox
Veteran
Veteran


Joined: 27 Aug 2013
Posts: 1746

PostPosted: Tue Jun 17, 2014 1:14 pm    Post subject: Reply with quote

lutel, the major change going do old metadata format is the place where metadata is stored. It will be at the end of disk now, so it's size is limited to smaller value than newer format, but it's presence seems to be completly ignored by filesystem and partition table, so you can have grub on partition created on top of raid with 0.9 metadata.
Also, AFAIR kernel has direct support for auto-assembling 0.9 raid while it does not have direct support for 1.2 metadata, and that's why root on newer raid requires initramfs.

Finaly, NeddySeagoon never suggested you to mount dev/sd*. There is no reason to do that. Hey, /dev/sd* don't even exist for you anymore as they are enslaved by raid. If you have raid1 made of 2 hard drives, you have to remove one drive from array first (so you can finaly use it directly). You consider this disk free. You don't care for any data you have there, you still have working raid even though you have just crippled it a bit. So, you use that scavenged drive you have just remove from old raid to creat new raid, the one you will be using later. Yes, raid number will change. I suppose you can fix it later if you need, but it must have different number right now for you still have old raid active. Copy files from old raid to new raid, then destroy old raid, add second drive to new raid and in (most likely) a few hours DM will repair your raid. you can continue using your pc in that time though
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43178
Location: 56N 3W

PostPosted: Tue Jun 17, 2014 7:53 pm    Post subject: Reply with quote

lutel,

What szatox just said :)

Raid sets are supposed to work with a missing drive - except raid0
You are taking advantage of that.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum