Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Many problems with a dmraid, LUKS, and LVM system with boot
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
KintaroBC
n00b
n00b


Joined: 15 Feb 2014
Posts: 55
Location: Australia

PostPosted: Sun Feb 09, 2020 4:44 am    Post subject: Many problems with a dmraid, LUKS, and LVM system with boot Reply with quote

I am trying to make a system which uses a kernel made by genkernel to boot and do everything it needs. I have had many issues and I can't get it to work as expected. The guides and advice out there, including the messages at the end of genkernel are only relevant to a system which uses partitions for root and swap on the LUKS drive. I however am using an LVM on top of the encrypted LUKS partition. I must use an LVM that is encrypted because the system's purpose is being a virsh KVM/QEMU cloud with guests using logical volumes.

I will explain my configuration and current problem, but I will update this thread until I have worked through all issues and have it working.

I have done the kernel and initrd with the following:
Code:

genkernel --luks --dmraid --lvm --disklabel --mdadm all


On my setup there are for disks /dev/sd[a-d] all formatted and configured with RAID, and are assembled as a RAID 5 volume. This volume appears to the admincd as /dev/md127 but shows as a long and ugly /dev/md/localhost.localdomain\:0 from the recovery shell. I could from there do cryptsetup and unlock the LUKS drives, but then I had trouble, the pvscan command was not fine and seems missing from the initrd.

I have updated the kernel boot parameters /etc/grub/default with this, in the past I used the md device I had on the admincd of /dev/md127 and have changed that to what I saw in the recovery shell trying to boot.
Code:

GRUB_CMDLINE_LINUX="dodmraid cryptdevice=/dev/md/localhost.localdomain\:0 dolvm root=/dev/mapper/master--hv-root"


I am not sure what is wrong with this setup that the commands for LVM are not in the initrd despite giving the --lvm argument to genkernel. I have everything required installed...

Code:

# emerge -pqv lvm2
[ebuild   R   ] sys-fs/lvm2-2.02.184-r5  USE="readline (selinux) thin udev -device-mapper-only -lvm2create_initrd -sanlock -static -static-libs (-systemd)"                                                                           


Code:

# emerge -pqv mdadm
[ebuild   R   ] sys-fs/mdadm-4.1  USE="-static"


Code:

# emerge -pqv cryptsetup
[ebuild   R   ] sys-fs/cryptsetup-2.2.2  USE="argon2 luks1_default nls openssl udev -gcrypt -kernel -libressl -nettle -pwquality -reencrypt -static -static-libs -urandom"                                                           

 * IMPORTANT: 7 news items need reading for repository 'gentoo'.
 * Use eselect news read to view new items.


Edited: I realised that I forgot to enable the static use flag for the packages above, but I have now done this and rebuilt these packages and their static dependencies.

At this point my issue is that for some reason genkernel has not installed the tools for LVM, and then I may still need help with other issues. Though it would be nice if I could at least use the shell I can drop to if there is no root filesystem to get the system running.

After someone can help me get it so I can do an LVM pvscan and vgscan after doing cryptsetup, I then also want it to unlock the LUKS encrypted raid with the name /etc/secured in mapper. I cannot find much of a definitive reference to kernel arguments passed by grub and how they work with the initramfs. At this point I am just tinkering and seeing what happens when I reboot.

I got this working and also discovered that from the initramfs that I must use 'lvm' in front of commands like 'pvscan' - I still need support on what to do to get the kernel parameters to boot automatically, but at least I can use the system now.

I need to get the initrd to do the following:
* Assemble the raid which I can do from the rescue shell with: mdadm --assemble --scan
* Decrypt the RAID and name the LUKS device 'secured' which I can do from rescue with: cryptsetup luksOpen /dev/md/localhost.localdomain\:0 secured
* Get the LVM running automatically, which unlike the admincd I installed with requires a series of commands, that all from the rescue shell must start with lvm and are...
Code:

lvm pvscan
lvm vgscan
lvm vgchange -a y master

After all this one can exit the rescue shell, and specify the logical volume with the root filesystem: /dev/master/hv-root (and hv stands for hypervisor).

Finally and for now I can get my system to boot, but I would still like assistance in getting the system to do it automatically.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 44945
Location: 56N 3W

PostPosted: Sun Feb 09, 2020 11:20 am    Post subject: Reply with quote

KintaroBC,

Are you confusing dmraid with mdraid?

dmraid is for BIOS created fakeraid. It looks like hardware raid in use but under the skin, is software in the BIOS.
The only excuse for using fakeraid is that Windows and Linux must both access the raw raid.
If you don't have real hardware raid and Windows is not involved, use mdraid (that's linux kernel raid) or since you will use LVM anyway, LVM can do its own raid without any help from other sources.

Why do you need LUKS?
Its only useful to protect the drive content when the LUKS volume is not unlocked. That restricts its use to portable devices.

I run several virsh KVM/QEMU systems but I don't see a use case for LUKS as none of them are portable.
A trap for the unwary is that Hosts and Guests may not share the same PV. I didn't find that out until my first bare metal install was complete.

Here's how I did it. There is no LUKS there.
Hmm, that's from 2011. Don't follow it line by line. Extract the intent, and add LUKS.

Also, building things statically for the initrd is no longer a hard requirement.
Point lddtree at the dynamic binary and include all the bits.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Goverp
l33t
l33t


Joined: 07 Mar 2007
Posts: 804

PostPosted: Sun Feb 09, 2020 12:06 pm    Post subject: Reply with quote

NeddySeagoon wrote:
...
Why do you need LUKS?
Its only useful to protect the drive content when the LUKS volume is not unlocked. That restricts its use to portable devices.
....


<Aside>Neddy, IIUC you're saying you only need to encrypt portable devices. Is there because non-portable devices are harder to steal (in which case there's still a use case for LUKS), or because there's a better solution for non-portable devices (other than big locks)?</Aside>
_________________
Greybeard
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 44945
Location: 56N 3W

PostPosted: Sun Feb 09, 2020 4:46 pm    Post subject: Reply with quote

Goverp,

If someone gains unauthorised access to my PC, I have much bigger problems than what they might find there.
If they break into a data centre and access my server, which runs 24/7 then any LUKS container will be unlocked anyway.

Anyone who targets you for data theft, will either do a USB reboot and image your RAM, so that they can recover its contents or if they have more time, pull the live RAM and image that.
DRAM can retain data for several minutes after power off.
There are several papers on both attacks and plain crypto key recovery has been demonstrated using both methods.

To my knowledge, nobody flushes RAM on power off yet, so it takes the DRAM data decay time for encryption to become effective.

Security is always about evaluating your threats then deploying countermeasures against the perceived threats.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 14971

PostPosted: Sun Feb 09, 2020 5:26 pm    Post subject: Reply with quote

I agree that encryption is far more interesting on devices that may be stolen, but I see a use case for encrypted drives on desktops: peace of mind for RMA. If I have a drive fail under warranty, and I want to exercise that warranty, I'm usually required to send the failed drive back to the manufacturer as a condition for obtaining the new drive. Some manufacturers will let you receive the new drive and then mail back the old one, but I cannot recall dealing with one that would let me just keep the failed drive for free. Depending on how exactly the drive failed, I may not be able to wipe it as fully as I would want before sending it back. If the drive is encrypted via LUKS, and I can get the drive functional long enough to clobber the LUKS header (or I'm confident enough in the LUKS setup + my chosen key that I don't feel the need to wipe the LUKS header), then I can RMA the drive and not worry about someone exploring it. The other option, of course, is to accept that you can never RMA a drive that has had sensitive information (financial records, tax documents, etc.) on it, even if the warranty is clearly still valid.
Back to top
View user's profile Send private message
Jaglover
Watchman
Watchman


Joined: 29 May 2005
Posts: 7506
Location: Saint Amant, Acadiana

PostPosted: Sun Feb 09, 2020 5:32 pm    Post subject: Reply with quote

How much can you recover from a single drive from RAID-5?
_________________
Please learn how to denote units correctly!
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 44945
Location: 56N 3W

PostPosted: Sun Feb 09, 2020 5:48 pm    Post subject: Reply with quote

Jaglover,

Not a lot.
I'm aware of a classified raid set that had its off site backups performed by cloning the raid onto a new set of drives then sending drives individually by road by to as many destinations an there were drives in the raid set.
No two drives were in transit at the same time.
IT security, who are more paranoid that most, must have been happy with the risk of losing a single drive to someone who really wanted it.

You would get the partition table, unless its fake raid, then you might not.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Jaglover
Watchman
Watchman


Joined: 29 May 2005
Posts: 7506
Location: Saint Amant, Acadiana

PostPosted: Sun Feb 09, 2020 6:51 pm    Post subject: Reply with quote

Yes, that's what I thought. I remember reading somewhere everyone who encrypts their drive will regret it at some point. :P
_________________
Please learn how to denote units correctly!
Back to top
View user's profile Send private message
rufnut
Apprentice
Apprentice


Joined: 16 May 2005
Posts: 227

PostPosted: Mon Feb 10, 2020 8:46 am    Post subject: Reply with quote

I have been looking at super-encryption (just crypto on crypto) for some of the above reasons.

I am sorry I cannot bring myself to trust wallets and third party add-ons.

For an example my new phone has "Android 10" of which they claimed was so secure companies like "Cellebrite" can not get in, yet.
However I noticed browsing the pictures I had taken with the phone and they were auto-tagged with appropriate labels!!
( I later learnt to switch this off in "app control" but the default I am sure was on!!)

The problem I visualize with multiple crypto layers is the mt-bf of drives with just "1" bit out could see a massive data block error with little or nothing recovered. (yes, test and scan those backups.)

:)


Last edited by rufnut on Mon Feb 10, 2020 9:16 am; edited 1 time in total
Back to top
View user's profile Send private message
rufnut
Apprentice
Apprentice


Joined: 16 May 2005
Posts: 227

PostPosted: Mon Feb 10, 2020 8:56 am    Post subject: Reply with quote

KintaroBC,

Sorry to hijack your thread. :(

don't forget genkernel has "/etc/genkernel.conf" for configuration.

Quote:
# Add in LVM support from static binaries if they exist on the system, or
# compile static LVM binaries if static ones do not exist.
LVM="yes"

# Add in Luks support. Needs sys-fs/cryptsetup with -dynamic installed.
LUKS="yes"



https://forums.gentoo.org/viewtopic-t-1108168-highlight-.html
might also help even though it is for dracut.


https://wiki.gentoo.org/wiki/Genkernel#Genkernel.3F_Genkernel-next.3F_Dracut.3F
keep this in mind as I seem to run an older version of genkernel-next.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 44945
Location: 56N 3W

PostPosted: Mon Feb 10, 2020 10:48 am    Post subject: Reply with quote

rufnut,

For lots of reasons, the probability of single bit errors is "infinitely improbable".
For a long time now magnetic HDD have guessed what was written.
If the drive can't recover the data, you loose a whole block. That's 4kB on magnetic media, 32kB on optical media and on SSD, it may be a whole erase block, depending on the failure mode.

Some research suggest that crypto on crypto is actually weaker than either single crypto but the theory made my head hurt.
I don't trust third party add-ons either.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
rufnut
Apprentice
Apprentice


Joined: 16 May 2005
Posts: 227

PostPosted: Mon Feb 10, 2020 11:48 am    Post subject: Reply with quote

NeddySeagoon,

Just trying to work it out to stop catastrophic failures. ( I am new to this.)
When I say crypto on crypto I should say LUKS on LUKS and different types at that, as say someone accidentally gains access to my unlocked LUKS of which I am currently on, there is another stage for more secure information they must also unlock.
i.e. I only unlock that when needed. Sort of stateful_LUKS I imagine wallets/vaults are similar.

I guess that's why I bought the android 10 thing up, is the AI on camera app that good or is the app browsing my photo's to see what they are. Perhaps I should ask Google?

:)
Back to top
View user's profile Send private message
hkmaly
n00b
n00b


Joined: 13 Jul 2006
Posts: 38

PostPosted: Thu Feb 27, 2020 2:46 am    Post subject: Reply with quote

rufnut wrote:

Just trying to work it out to stop catastrophic failures. ( I am new to this.)
When I say crypto on crypto I should say LUKS on LUKS and different types at that, as say someone accidentally gains access to my unlocked LUKS of which I am currently on, there is another stage for more secure information they must also unlock.
i.e. I only unlock that when needed. Sort of stateful_LUKS I imagine wallets/vaults are similar.
:)


Just because you need two different LUKS doesn't mean you must have one on the other. I think it would be just as safe, if not safer, to have two separate partitions, LUKS on both of them, and only mount each when needed.

NeddySeagoon wrote:

use mdraid (that's linux kernel raid) or since you will use LVM anyway, LVM can do its own raid without any help from other sources.


Does it? To be more specific, can LVM do raid with spreading reads fairly between source devices? Because last I looked, LVM's "mirror" was only reading from single drive. Granted, it's few years back so I'm seriously asking if that changed.

(I am currently running LUKS on mdraid for this reason.)
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 44945
Location: 56N 3W

PostPosted: Thu Feb 27, 2020 6:27 pm    Post subject: Reply with quote

hkmaly,

I don't know the detail ot LVM raid. When I set this system up about 11 years ago, I didn't know about it.
My only raid1 is /boot, everything else is LVM on top of mdadm raid5.

Even then, boot is raid1 to make like easy for me. My bootloader is grub-static which is not raid aware anyway.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum