Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
VM host: Maintain sourceless minimized VMs with ROOT=
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2549

PostPosted: Fri May 23, 2014 3:50 pm    Post subject: VM host: Maintain sourceless minimized VMs with ROOT= Reply with quote

Hi.

I just saw a post regarding cross compiling and setting ROOT=... for the cross compiled setup.

So here's the thing:

I'm trying to make a couple routers as VMs, and another appliance-type VM. For all of these, I want absolute minimum files on there. I don't want compilers or source, I don't want ssh clients. If I can do what I think I can, I won't even have an editor.

So it occurs to me I could cross compile (even though it's literally the same hardware) and essentially mount the root filesystem of each VM directly to the host, and then maintain a separate "cross compile" setup for each VM.

Is this rational or am I missing the point altogether?
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6228
Location: Room 101

PostPosted: Fri May 23, 2014 8:16 pm    Post subject: Re: VM host: Maintain sourceless minimized VMs with ROOT= Reply with quote

1clue wrote:
So it occurs to me I could cross compile (even though it's literally the same hardware) and essentially mount the root filesystem of each VM directly to the host, and then maintain a separate "cross compile" setup for each VM.

1clue ... if the target system is the same architecture then you don't need to cross compile, '--root=/dir' simply indicates where the target files are installed to. The '--root-deps=' parameter can be used to discard the installation of buildtime dependencies, and you can also use FEATURES="noinfo nodoc noman", profile/package.provided, and INSTALL_MASK, to further exclude things from --root.

best ... khay
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2549

PostPosted: Fri May 23, 2014 8:20 pm    Post subject: Reply with quote

Awesome!

Now I know I'm not off in left field somewhere I feel much better about tackling this.

Thanks a bunch.
Back to top
View user's profile Send private message
szatox
Veteran
Veteran


Joined: 27 Aug 2013
Posts: 1747

PostPosted: Sat May 24, 2014 11:37 am    Post subject: Reply with quote

It seems to be a good idea, with one exceptoin you don't maintain such a minimal install. You make a script building your stuff, e.g. installing packages, changing config files, and whatnot. You maintain this sript instead. And when you want to update, you run this script, boot into new image, and throw old one away.

At least it seems it has to be done this way. Those minimal images don't contain any metadata that would allow portage update them directly.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43212
Location: 56N 3W

PostPosted: Sat May 24, 2014 5:04 pm    Post subject: Reply with quote

szatox,

They can but they don't have to - thats more user choice for you.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2549

PostPosted: Sat May 24, 2014 5:23 pm    Post subject: Reply with quote

What I was thinking is that the metadata might be held in the host system, rather than the VM.

I don't really want to lose things like portage, I just don't want them to be accessible from the guest. If I can get the host to know about all these special guests -- the guests are running read only -- then there's really nothing lost, only moved from one place to another.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43212
Location: 56N 3W

PostPosted: Sat May 24, 2014 8:33 pm    Post subject: Reply with quote

1clue,

You can share things via NFS among the host and VMs, It needs a bit of thought if there are security implications.

e.g. /usr/portage and /usr/portage/distfiles can be shared.
/usr/portage/packages is sensitive to USE flag settings.
Other bits and pieces can be NFS mounted too and not mounted unless they are needed
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
szatox
Veteran
Veteran


Joined: 27 Aug 2013
Posts: 1747

PostPosted: Sun May 25, 2014 10:25 am    Post subject: Reply with quote

NeddySeagoon wrote:
szatox,

They can but they don't have to - thats more user choice for you.

Great, I like user choice. So, could you elaborate? I have found one way, but I don't mind learning about the other, perhaps it would be more efficient.

Quote:
If I can get the host to know about all these special guests (...)
A wild guess... Use run yor VM with different network mode? E.g. make it bind TAP device so you can call it over etherner?
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43212
Location: 56N 3W

PostPosted: Sun May 25, 2014 2:07 pm    Post subject: Reply with quote

szatox,

1) You can build a BINHOST, one per VM on the VM host, or anywhere else for that matter, then do binary installs.
emerge -K using the prebuilt binaries.
You do need the portage tree, or the packages can be untarred, like this stage1 fake. See Chrooting
Setting FEATURES=buildpkg uill get you the BINHOST packages, if the USE flags will not change

2) build using pump made distcc from the VMs The host does all the hard work but you can have an many /usr/portage/packages/VM1 as you need on the host to keep things separate.
Use NFS or mounting into the VMs as needed

3) Use crossdev. This should just work as its not really required but it does all the messing about with ROOT= for you.
Just as well as its intended to be used to build for other aches than the host, and you really don't want to be installing ARM binaries on your amd64 build box.

4) Some combination of any or all of the above
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2549

PostPosted: Mon May 26, 2014 2:37 am    Post subject: Reply with quote

I'm thinking 1 or 2 from NeddySeagoon's post make sense for me.

These VMs would be read-only from inside the VM, I'd put in a networked syslogger on the same host, and edit everything from the VM host.

I'm also starting to think a separate higher-end appliance which can run a full Linux distro, or maybe experiment with pfSense. These look attractive: http://store.pfsense.org/T40E4-black/ or maybe http://routerboard.com/CCR1009-8G-1S-1Splus although I'm not sure if the second can be reasonably hacked with a Linux I can actually mess with.

I'm fascinated by the Tilera cloud core processors, but Tilera is extremely closed mouthed about their products. They require me to sign an NDA to get any answers at all about even basic things like a price tag for a test system. Anyway, that's a bit off topic.
Back to top
View user's profile Send private message
szatox
Veteran
Veteran


Joined: 27 Aug 2013
Posts: 1747

PostPosted: Mon May 26, 2014 10:55 am    Post subject: Reply with quote

I like that stage1 fake howto, thanks
Does using pump do anything good on a single phisical machine? I haven't done very extensive testing, but software on VM seems to run as fast as it would on host. Also, there was a point that emerge should not be usable from within VM.

There is one more idea on "minimal VM" images. You can give your VM 2 rootd devices: Generic one that contains all the software you want on your VMs, (or perhaps full generic root), and a tiny image with changes you have to apply to your generic root to turn it particular VM's root. AUFS does that well, however you most liekly have to use it inside guest system and I'm not sure how good genkernel support for aufs is. Changing init script inside initramfs is not hard though.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2549

PostPosted: Mon May 26, 2014 2:36 pm    Post subject: Reply with quote

I'm not interested in genkernel. If I wanted something like that I could use Ubuntu just as easily. I don't mean to be snippy.

The "minimal VM" idea is to have each box have only what it needs to do the job. That means, to me:
Host:

  1. QEMU+KVM
  2. RAID0, RAID1
  3. Lvm2
  4. Utilities for managing that.
  5. IPV6 but not IPV4
  6. Provision to set up an IPV6 tunnel to a virtual device it does not own, similar to NeddySeagoon's setup.
  7. No network console software: Physical console login only.
  8. No http/whatever


Firewall:

  1. IPV6+IPV4+advanced router+VPN
  2. NO ssh client, or anything else that can be used to attack another host.
  3. Probably no DNS
  4. Read-only filesystem
  5. syslog to a network device


Internal router:

  1. Firewall settings + VLAN support
  2. DNS
  3. Hardware support for donated multi-interface NIC


Here's an idea:
Portage mirror:

  1. One VM is solely for a portage mirror.
  2. Host does not have network access to anything except the mirror.
  3. Mirrors the Gentoo repository.
  4. Host builds using this repo.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43212
Location: 56N 3W

PostPosted: Tue May 27, 2014 8:04 pm    Post subject: Reply with quote

1clue,

Thats similar to my HP Microserver setup.
The detail there covers the host, I never added the VM part but the one line thats there is 'mostly harmless'.

I have a router/firewall VM running shorewall and dhcpd. Thats all it does. I know it works with a read only root filesystem as it ran for several days when two drives failed in the raid5
I have a mailserver (qmail) VM that runs spamdyke and my rsync mirror. As this VM faces the outside world anyway, via the firewall, there is no harm in adding the rsync mirror here too.
The firewall does not permit outside access to the rsync mirror.
There is a media server VM containing 1500 DVDs, hence the raid5. It took me several months to rip that lot and I don't want to do it again.
The media server also hosts the filesystem for a diskless media player, thats redundant now I have fitted an SSD to the mediaplayer, so it still has no moving parts.
Lastly, there is a template VM which was the basis for all the others. It got pressed into service to host the root over NFS for a Raspberry Pi, which mas powered off one of the Microservers USB ports. The Pi used to run my webserver.

I can add IPv6 and I keep pestering my ISP for native IPv6 but its something else to learn, so I've not done it yet.

The router is very restrictive. The policy everywhere is deny. There are four networks,
Internet, DMZ (static IPs only), wireless and wired. Wired is permitted to contact wireless but not the other way around.

Why the HP Microserver?
There was £100 cashback when I got mine, so it was too good to resist.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2549

PostPosted: Tue May 27, 2014 8:56 pm    Post subject: Reply with quote

NeddySeagoon,

I'm using your setup as a model. We've talked about it before, and I'm trying to prevent too much repetition.

My ISP doesn't have IPV6 either in spite of my pestering, but I intend to tunnel to hurricane until such time as they (my ISP) get off their rears. It's the ONLY complaint I have with these guys.

I'm a bit old school. My DVDs are on a shelf in my living room, my CDs are on my phone though so I'm not totally hopeless there. But for every song on my phone, there exists a CD which I own from which it was scanned. I'm really fussy about that. Maybe it has something to do with my income depending on copyright law.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43212
Location: 56N 3W

PostPosted: Wed May 28, 2014 6:27 pm    Post subject: Reply with quote

1clue,

We still own all the DVDs but only have easy access to about half of them.
I did not put the music collection on the media server as after I had rippied the DVDs, my wife refused to use it because "its not the same as looking through shelves of DVDs".
She doesn't seem to mind that she can actually see and touch only about half the collection.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2549

PostPosted: Wed May 28, 2014 11:22 pm    Post subject: Reply with quote

My dvd collection is pretty small, mostly because I didn't even get a TV until I got married a year and a half ago. I have boxes and boxes of CDs though, all of which fit nicely in my pocket right now. And I found that if you get a decent phone and a really good set of headphones or earbuds make a huge difference. Like these: http://www.shure.com/americas/products/earphones-headphones/se-earphones/se535-sound-isolating-earphones
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum