Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Yet another Hybrid Raid 0 / 1 Howto for 2.6 with dmsetup
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
movrev
Tux's lil' helper
Tux's lil' helper


Joined: 07 Mar 2004
Posts: 114
Location: Berkeley, CA - USA

PostPosted: Mon Apr 04, 2005 7:36 am    Post subject: Reply with quote

So, technically, I shouldn't even need an initrd if I follow genkernel? I mean, it should be taken care of, right?
Back to top
View user's profile Send private message
garlicbread
Apprentice
Apprentice


Joined: 06 Mar 2004
Posts: 182

PostPosted: Mon Apr 04, 2005 7:59 am    Post subject: Reply with quote

If
Code:
 dmraid -ay

sets up your array perfectly then I think genkernel should be able to generate an initrd that will work with the bootup
if on the other hand you have to use mappings via dmsetup (e.g. for certain Fast Track Promise arrays or for Via with a Raid 0 cluster size other than 16K) then some manual fiddling may be in order

I think the bug of dmraid not mapping the extended partitions has been fixed in the latest versions
the only time it seems to mess up is if it gets the overall size of the array wrong
Back to top
View user's profile Send private message
flipy
Apprentice
Apprentice


Joined: 15 Jul 2004
Posts: 229

PostPosted: Mon Apr 04, 2005 8:17 am    Post subject: Reply with quote

garlicbread wrote:

if on the other hand you have to use mappings via dmsetup (e.g. for certain Fast Track Promise arrays or for Via with a Raid 0 cluster size other than 16K) then some manual fiddling may be in order

last version of dmraid also detects my via raid 0 32k cluster size and maps it ok (just a little trouble with the 2005.0-r1 amd64 cd).
so, movrev, i think you always will need an initrd to start. and it's logical. firs you boot the kernel, which detects only hardware, then you run the initrd, that just makes the nodes and maps them, and then you're ready to start booting all software stuff...
_________________
Si no entiendes algo leete detenidamente el Handbook.
Back to top
View user's profile Send private message
AlphaHeX
n00b
n00b


Joined: 17 Jan 2005
Posts: 60
Location: Warsaw, Poland

PostPosted: Thu Apr 07, 2005 7:28 am    Post subject: Creating RAID partitions under linux Reply with quote

I have 2x 120 Seagate SATA harddrives set in bios to be a RAID0 (It's intel ICH5R). I'm booting from Gen2dmraid LiveCD and after
Code:
dmraid -ay

command i have a block device is /dev/mapper/iswfsfsaf[HeX]. I know that raid was discovered by dmraid as [HeX] is a name of the raid which i've set up in the bios. Invoking
Code:
fdisk /dev/mapper/ called iswfsfsaf[HeX]
is showing me the whole raid (concerning diskspace - it's 240GB). Do i have to do partitioning before setting up raid in Linux (In Windows) or can i just make partitions under Linux using fdisk? You all are talking here that i have to do partitioning in windows because setting up partitions in Linux will overwrite RAID0 information stored at the begining and at the end of each disk ?

Last edited by AlphaHeX on Thu Apr 07, 2005 2:17 pm; edited 1 time in total
Back to top
View user's profile Send private message
garlicbread
Apprentice
Apprentice


Joined: 06 Mar 2004
Posts: 182

PostPosted: Thu Apr 07, 2005 1:14 pm    Post subject: Re: Creating RAID partitions under linux Reply with quote

AlphaHeX wrote:
Do i have to do partitioning before setting up raid in Linux (In Windows) or can i just make partitions under Linux using fdisk? You all are talking here that i have to do partitioning in windows because setting up partitions in Linux will overwrite RAID0 information stored at the begining and at the end of each disk ?


The Raid metadata for most of these types of Arrays appears to be stored at a region right at the end of the disk.
The safe easy way is just to partition the disk under windows (which would use the official drivers)
or a Dos utility such as PQMagic (which would use the BIOS).
through Dos or windows the drives in combination would simply look as if they were a little bit shorter to the partitioning utility which would prevent the partitions overlapping into that metadata region

dmraid should also do the same thing. i.e. map the array from the begining of the disk only up to where the meta data starts, that way the dev node created is the same correct length as it would be in windows (it excludes the meta data) and you should (in theory) be able to create partitions safely with sfdisk or fdisk

The bit where it gets tricky is if dmsetup is used instead to manually map out the array on the drive
Since it's difficult to know wherebouts the metadata starts at the end of the disk
if dmsetup has just been used to map out the full disk (including the metadata) then you risk setting up a partition under Linux that could overlap into that metadata region
if you've got the lengths right for the parameters to dmsetup then there shouldn't be a problem

So in short if dmraid works then there shouldn't be a problem with partitioning under Linux (in theory)
Back to top
View user's profile Send private message
makton3g
Tux's lil' helper
Tux's lil' helper


Joined: 19 Aug 2004
Posts: 88

PostPosted: Thu Apr 07, 2005 3:19 pm    Post subject: Reply with quote

Exactlly when along the installation process (by the handbook) am I supposed to setup the Raid 0? I was not about to do when setting up Hard Disks.

dmraid is not on the minimal CD and dmsetup said it had some files missing and was asking if it was installed properly. Am I supposed to make my file systems and bootstrap before I setup my Raid?
Back to top
View user's profile Send private message
garlicbread
Apprentice
Apprentice


Joined: 06 Mar 2004
Posts: 182

PostPosted: Thu Apr 07, 2005 5:58 pm    Post subject: Reply with quote

You have to setup the raid array using dmraid first (assuming dmraid recognises your array correctly)
before you write anything to the Hard disk
i.e. right at the begining of section 4 I think

assuming you've already setup / created the array in the Bios
dmraid -ay should create the dev node /dev/mapper/<whatever>
which represents the whole disk of the array

If the disk isn't partitioned yet, then you'll need to run fdisk /dev/mapper/<whatever>
on it to setup the partitions
and perhaps dmraid -ay a second time so that the partition nodes will show up
e.g.
/dev/mapper/<whatever>1
/dev/mapper/<whatever>2
etc

the <whatever> is usually pdc or via or whatever type of array yours is followed by a long number

dmraid isn't included on the official livecd yet (perhaps becuase it isn't considered stable enough I don't know)
but theres supposed to be one over here called gen2dmraid
but I've not got around to trying it out myself yet
Back to top
View user's profile Send private message
flipy
Apprentice
Apprentice


Joined: 15 Jul 2004
Posts: 229

PostPosted: Thu Apr 07, 2005 8:00 pm    Post subject: Reply with quote

garlicbread wrote:

dmraid isn't included on the official livecd yet (perhaps becuase it isn't considered stable enough I don't know)

it is included in 2005.0 (amd64 and x86, AFAIK)
_________________
Si no entiendes algo leete detenidamente el Handbook.
Back to top
View user's profile Send private message
makton3g
Tux's lil' helper
Tux's lil' helper


Joined: 19 Aug 2004
Posts: 88

PostPosted: Thu Apr 07, 2005 9:32 pm    Post subject: Reply with quote

flipy wrote:
garlicbread wrote:

dmraid isn't included on the official livecd yet (perhaps becuase it isn't considered stable enough I don't know)

it is included in 2005.0 (amd64 and x86, AFAIK)


when I type the command "dmraid" I get a "command not found". I am using the minimal install CD 2005.0. Is this or "gen2dmraid" on the minimal CD?
Back to top
View user's profile Send private message
flipy
Apprentice
Apprentice


Joined: 15 Jul 2004
Posts: 229

PostPosted: Thu Apr 07, 2005 10:03 pm    Post subject: Reply with quote

makton3g wrote:
flipy wrote:
garlicbread wrote:

dmraid isn't included on the official livecd yet (perhaps becuase it isn't considered stable enough I don't know)

it is included in 2005.0 (amd64 and x86, AFAIK)


when I type the command "dmraid" I get a "command not found". I am using the minimal install CD 2005.0. Is this or "gen2dmraid" on the minimal CD?

well, 2005.0 has dmraid on its initrd (linuxrc), and not in the livecd environtment; so, just check /dev/mapper if there is something there... if not, try gen2dmraid >=0.99a (pure udev system) and you'll be able to configure everything.
another way it's to mount the initrd, so you'll be able to execute dmraid, but this is more difficult if you're not an advance linux user...
_________________
Si no entiendes algo leete detenidamente el Handbook.
Back to top
View user's profile Send private message
mcfly.587
Tux's lil' helper
Tux's lil' helper


Joined: 29 Mar 2005
Posts: 118

PostPosted: Fri Apr 08, 2005 10:49 am    Post subject: Reply with quote

Hi everybody,

I downloaded the cd with the dmraid support from http://tienstra4.flatnet.tudelft.nl/~gerte/gen2dmraid/ but I have a problem:

On boot :
Quote:

invalid metadata checksum on /dev/sda
invalid metadata checksum on /dev/sdb
No software raid disks


Configuration :
Epox 8rda3+ with SILicon Image 3112 ;
2* Raptors 36go ;

If i test with two seagate there is no problem, all work perfect ! Have you a solution plz ?

On /dev/mapper there is nothing with the raptors ... :( no silafefazfdf ...

How can i resolve this error at boot ? Thx in advance :)
Back to top
View user's profile Send private message
garlicbread
Apprentice
Apprentice


Joined: 06 Mar 2004
Posts: 182

PostPosted: Fri Apr 08, 2005 3:27 pm    Post subject: Reply with quote

It looks as if it can see the metadata is there but thinks it's invalid
I would have thought that if it works for one set of disks then it should work for another
is this on Raid 0? (could you have unplugged then re-plugged the drives in the wrong way round?)
Have you tried deleting then re-creating the Array in the bios for those disks?
(this would destroy all data mind you)
Back to top
View user's profile Send private message
mcfly.587
Tux's lil' helper
Tux's lil' helper


Joined: 29 Mar 2005
Posts: 118

PostPosted: Fri Apr 08, 2005 3:54 pm    Post subject: Reply with quote

Yes raid 0.

I have recreated 2 times the array -> 16k,32k ... same result.

Its very strange ...
Back to top
View user's profile Send private message
garlicbread
Apprentice
Apprentice


Joined: 06 Mar 2004
Posts: 182

PostPosted: Fri Apr 08, 2005 5:18 pm    Post subject: Reply with quote

was the array on the Seagate drives created on the same motherboard / raid controller / set of Sata connections?

I remember something weird happening with mine, when I was messing about this this
I'd set up an array with a couple of disks on a pdc controller, switched the drives across to a via controller again set it up
dmraid didn't reconise the via data at the time (as it was an old version) but picked up the pdc data that was laft over from the previous array and setup the array using the pdc driver
which sort of made it work even when it shoudn't have

other than that the only other way is manually dojng it through dmsetup which is difficult to setup
Back to top
View user's profile Send private message
Erlend
Guru
Guru


Joined: 26 Dec 2004
Posts: 493

PostPosted: Sun Apr 10, 2005 9:21 pm    Post subject: Reply with quote

The kpartx script doesn't run from my initrd. Not sure if it is compatible with busybox?

Thanks,

Erlend
Back to top
View user's profile Send private message
garlicbread
Apprentice
Apprentice


Joined: 06 Mar 2004
Posts: 182

PostPosted: Mon Apr 11, 2005 11:19 am    Post subject: Reply with quote

kpartx isn't a script it's a binary included with multipath-tools that will create dev nodes for the different partitions
devmap_name is another binary also included in multipath that can identify a device-mapper's name and can be used within udev scripts to properly setup the dev nodes
since the ebuild for multipath-tools can't build static yet you'll also need to copy across any libs that kpartx would use

partition-mapper.sh is a script which does exactly the same thing as kpartx, except it uses sfdisk, awk and dmsetup (so these need to be present within the initrd if this is being used)

since dmsetup (version 1.0.20 onwards) can now do the same thing as devmap_name within the udev rules I've stopped using multi-path tools altogether for my own system and just use the partition-mapper.sh script instead
Back to top
View user's profile Send private message
Erlend
Guru
Guru


Joined: 26 Dec 2004
Posts: 493

PostPosted: Mon Apr 11, 2005 1:01 pm    Post subject: Reply with quote

Okay thanks. When I said kpartx script I was referring to your script above. I'm trying to make an initrd for mapping the partitions so that I can repartition my drive easier. After that I'll revert back to using "static mapping".

Erlend
Back to top
View user's profile Send private message
garlicbread
Apprentice
Apprentice


Joined: 06 Mar 2004
Posts: 182

PostPosted: Mon Apr 11, 2005 3:38 pm    Post subject: Reply with quote

I've tried the latest one under busybox inside an initrd and it seems to be okay
without knowing what the error or problem is I'm not sure what to recomend
I know that a pure udev system by default won't map device nodes correctly for dev-mapper so I'm assuming you've already set this up (see other Evms + Udev Howto)

I've recently managed to get evms working on top of the array as well but that involved using a loopback device, so I'm looking into patching evms as well at the moment
Back to top
View user's profile Send private message
Erlend
Guru
Guru


Joined: 26 Dec 2004
Posts: 493

PostPosted: Mon Apr 11, 2005 3:45 pm    Post subject: Reply with quote

Actually, I'm unlikely to try this myself, but does lvm2 work on the arrays?

Erlend
Back to top
View user's profile Send private message
Erlend
Guru
Guru


Joined: 26 Dec 2004
Posts: 493

PostPosted: Tue Apr 12, 2005 3:00 pm    Post subject: Reply with quote

My initrd doesn't work - sdisk isn't working. It says
Code:
"/bin/sh: /sbin/sfdisk: not found"


I think it needs a library, not sure which one? Does anybody know?

Thanks,

Erlend
Back to top
View user's profile Send private message
garlicbread
Apprentice
Apprentice


Joined: 06 Mar 2004
Posts: 182

PostPosted: Tue Apr 12, 2005 3:48 pm    Post subject: Reply with quote

The script is looking for sfdisk in /sbin/sfdisk which I think is the default location on an installed system

for my own initrd I have all the binaries located within /bin/ on the initrd
with symbolic links from /sbin /usr/bin /usr/sbin /usr/local/bin/ that all point to /bin
so depending on your initrd, ether sfdisk is not there, located in the wrong place, or there's no symlink for /sbin to the bin directory where sfdisk is located

to get a list of libs that a binary depends on you can try running ldd <path to the bin> for a list
also for sfdisk I think it's possible to get a static version by using
USE="static" ebuild /usr/portage/<path to sfdisk ebuild file> install
which should go as far as compiling / installing into /var/tmp/portage but not emerging into the main system
from there you can then copy the static version /var/tmp/portage/sfdisk/image/sbin/sfdisk (I think that's right) into your initrd
Back to top
View user's profile Send private message
Erlend
Guru
Guru


Joined: 26 Dec 2004
Posts: 493

PostPosted: Tue Apr 12, 2005 4:51 pm    Post subject: Reply with quote

Quote:
USE="static" ebuild /usr/portage/<path to sfdisk ebuild file> install

That fixed it thanks.

I'll post the linuxrc and my busybox config in this thread when I'm eventually satisfied with it. I'm sure some people will find it helpful.

Erlend
Back to top
View user's profile Send private message
Zate
n00b
n00b


Joined: 12 Apr 2005
Posts: 13

PostPosted: Wed Apr 13, 2005 12:24 am    Post subject: Reply with quote

problem i have is finding a CD with dmraid and ntfsresize on it.. i have the gen2raid livecd or knoppix. I see a bunch of links for a static linked ntfsresize about the place, but none of them work.

any ideas?.. my winXP ntfs is taking up the whole raid... so i cant resize it with a normal knoppix CD (3.7) as it doesnt see the raid.. and the gen2raid cd which see's the raid fine has no resizing tool for ntfs. :(
Back to top
View user's profile Send private message
Erlend
Guru
Guru


Joined: 26 Dec 2004
Posts: 493

PostPosted: Wed Apr 13, 2005 11:00 am    Post subject: Reply with quote

There are a few things you could try.

BootIT NG is a commercial product, with a working trial version:
http://www.bootitng.com/downloads/bootitng.zip

If you have any livecd (knoppix) with ntfsresize on it you could use that. Most livecds have dmsetup on them (which is the program dmraid calls in order to map the drives - you can call it yourself if you know the mapping. Use garlicbread's script if you're stuck.) You can actually complete the install using dmsetup (I have to use dmsetup - since dmraid doesn't work for me). If you really must have dmraid, then finding a static compiled version of that should be easy - people are always statically compiling it to put in their initrds.

In fact, you could use knoppix, but use the "Install Software" feature to install dmsetup/dmraid while you are running to CD. I think it is located: K->KNOPPIX->Utilities->Install software.

Good luck,

Erlend
Back to top
View user's profile Send private message
garlicbread
Apprentice
Apprentice


Joined: 06 Mar 2004
Posts: 182

PostPosted: Wed Apr 13, 2005 11:46 am    Post subject: Reply with quote

For info
I've noticed that an error can sometimes be genereated when setting up a mirror target for Raid 1
in the initrd it's more visible, it seems to mention something about trying to access the device outside of it's bounds (even though the map given to dmsetup isn't)
while this doesn't stop it from working, mapping / unmapping several times over (like if your experimenting with the setup) can cause a kernel oops
I think it might be something in the kernel's dm mirror source

looking at the latest Changelog for gentoo-sources-2.6.11-r6 it seems to mention something
"Removed the dm patches as they caused oopses under certain circumstances"

but I can't compile this at the moment due to some weird make error on my amd64 system
I'll need to re-compile my toolchain and world packages to try an sort this out first

EDIT
figured out the problem and tried out 2.6.11-r6, but it still has the same problem
you should be able to map then unmap a mirror target at least once with no problems, constantly re-mapping / unmapping it however will probably cause an oops (at least for mirror raid1, for raid0 there seems to be no problem)


Last edited by garlicbread on Thu Apr 14, 2005 12:07 pm; edited 1 time in total
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3, 4  Next
Page 3 of 4

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum