Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Is it possible to donate nics from an atom host to guest?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2562

PostPosted: Sun Dec 28, 2014 5:01 am    Post subject: Is it possible to donate nics from an atom host to guest? Reply with quote

Hi,

I'm contemplating this board:

http://www.supermicro.com/products/motherboard/Atom/X10/A1SRM-LN7F-2758.cfm

The CPU is http://ark.intel.com/products/77988/Intel-Atom-Processor-C2758-4M-Cache-2_40-GHz

This is an 8-core atom board with hardware acceleration for encryption and compression (AES and QuickAssist). According to specs, it should be able to handle a multi-gigabit VPN much faster than an i7 could but will naturally fall behind on normal computing tasks.

It supports VT-x and VT-x EPT
The thing is, it doesn't have VT-d (it has VT-x and other virtualization support though) so I'm not sure if it supports donation of nics to a guest.

My intent is to run qemu and virtualize a router inside.

I have no experience with atom boards, let alone virtualization on one. Is this a real hindrance or security risk, or is the idea sound?

Thanks.
Back to top
View user's profile Send private message
Keruskerfuerst
Advocate
Advocate


Joined: 01 Feb 2006
Posts: 2288
Location: near Augsburg, Germany

PostPosted: Sun Dec 28, 2014 3:18 pm    Post subject: Reply with quote

What do you want to do exaclty?
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 44153
Location: 56N 3W

PostPosted: Sun Dec 28, 2014 4:04 pm    Post subject: Reply with quote

1clue,

There is two ways to do this.

You can donate physical interfaces te bridges. This always works but it means that the bare metal host will need a(n) IP(s) on the bridge(s).
This is what I do as the following failed for me.

You can also do PCI passthrough if all of the PCI bridge chips along the way support it.
After I bought an Intel 4 port Gigabit NIC to use PCI pass through to my KVM guest router, I discovered that there was a hardware errata in the PCI bridge chip on the card that prevents it supporting PCI pass through. As an everyday NIC, its fine.

In answer to your question - maybe, it all depends on the finer points of the hardware.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2562

PostPosted: Mon Dec 29, 2014 3:25 am    Post subject: Reply with quote

NeddySeagoon,

I've already built one box from your virtualized router model. Mine sucked because I made what I consider to be the worst possible hardware choices and had used hardware. I used 4x+1x Realtek nics instead of Intel, and it's an i7 920 and generally speaking not set up at all for that. I never trusted it to handle my Internet. This box has gone through a stack of WD green drives (which caused me to give WD a pass for a few years at least) and is just getting up there in age.

The main question is that as far as I know the donation of nics to the guest requires VT-d support, and I know this atom doesn't have it. I am just researching at this point, my parts won't be here for a week or so.

I'll check into passthrough support, I'll probably have more questions on that.

A note on pfSense comments: I started this as research for a client. They need a router distro with a gui and phone support, and IMO pfSense is probably it. I don't know anything about FreeBSD, but since most of my questions regarding this, and the choice of CPU, were acquired from the pfSense forum and research trying to deconstruct their supported firewalls, I'm referencing them a lot. I will probably wind up installing pfSense on this thing in a VM so I can give it a proof of concept at least, but I'm more familiar with Linux and for something like this I'm more comfortable with Gentoo as the host if I can do it.

Here is my desired feature list as a start:

  1. Firewall even between internal networks. One port will hook up to a SOHO wifi router for normal home use and I want to isolate that completely from everything else internal, treat it like an external network for security purposes.
  2. VLAN + trunking support.
  3. DMZ
  4. NAT on at least one network, the SOHO one.
  5. Snort for intrusion detection/prevention. I've never done this.
  6. Squid for web caching. If it doesn't help I can always remove it. I've never done this.
  7. Captive portal/multifactor authentication/radius for vpn. Would like something that hits the user's phone for validation, could be an external service. I've never done this.
  8. OpenVPN at near gigabit speeds. Not right now but within a couple years according to my ISP. I've never done a VPN on anything.
  9. Full-featured IPV6 support. I'd like to put as much of this on IPV6 as possible. I've never done that.
  10. 6to4 (I can get 200 mbps right now, but not ipv6?!!!). I've never done a whole network.
  11. Logging
  12. Route DHCP and such across networks to ISC-DHCPD.


Points of information:

  1. I know I can get all these features on a single install of Linux or pfSense, but I think that if I have layers which are separate minimal task-specific installs I believe things will be more secure. Please tell me if I'm wrong.
  2. This box is way faster than what I need for networking, technically the 2-core model is faster than I need for networking. I'm hoping to have plenty left over for at least some of the smaller VMs I need.


So, my intent: I want a setup similar in concept to what Neddy has done, only I have no experience whatsoever with Atom hardware and I'm not sure if I need VT-d for the nics.

I will have a sequence of installs:

  1. Set up Gentoo as a VM host and as a router, and a gui so I can use it as a normal workstation. This setup will test how this hardware works in my normal day job. If it's acceptable it will be much cheaper than my next workstation would be.
  2. Wipe it off and try pfSense on bare metal, get real-world performance benchmarks with the setup my client would use.
  3. Wipe it off and reinstall Gentoo from scratch with minimal VM host config.
  4. Add VM(s) to satisfy my networking requirements.
  5. Add VMs for other tasks as the hardware can handle.


Background on my research:
There is a Netgate product which comes prepackaged with pfSense with the same CPU and less impressive hardware. http://store.netgate.com/Firewall/C2758.aspx This comes with a full install of pfSense including Snort and all that. It's an 8-core Atom processor specifically designed for communications and has hardware encryption support and hardware compression support built in. That's huge for me. That it's atom implies to me that it's not in the same league as a modern i7 for normal computing tasks, but the fact that the board and CPU support 64g ECC memory makes me hope that it has some teeth. I'm hoping to find a way to transfer files from other hardware on my LAN onto a remote server and have this box handle the compression on the fly in such a way that normal software can deal with it on the other side, but that's for later.

This is brand new hardware, no real examples out there of this board being used. An earlier system with the same CPU is sold a the pfSense store as a ready-to-go firewall/router/vpn. It keeps up with multiple gigabit VPN even without the QuickAssist support.

There's also a dual-core version which is actually way more than I need for my networking needs: http://store.netgate.com/FW-7551.aspx can also handle near-gigabit VPN traffic without breaking a sweat.

The problem with pfSense IMO is that it only handles networking tasks.

Thanks.
Back to top
View user's profile Send private message
szatox
Veteran
Veteran


Joined: 27 Aug 2013
Posts: 1774

PostPosted: Mon Dec 29, 2014 7:21 pm    Post subject: Reply with quote

Quote:
You can donate physical interfaces te bridges. This always works but it means that the bare metal host will need a(n) IP(s) on the bridge(s).

Host doesn't need IP on bridge. You simply create TAP device and bind it to a VM and add tap0 and eth0 to a bridge without any kind of address assigned. Just make sure VM bound to tap0 has an IP configured on it's side.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 44153
Location: 56N 3W

PostPosted: Mon Dec 29, 2014 7:33 pm    Post subject: Reply with quote

szatox,

Thank you - I did wonder about that but I've never tested.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2562

PostPosted: Tue Dec 30, 2014 5:24 am    Post subject: Reply with quote

I'm a little lost. The best howto I've ever found for this sort of thing is NeddySeagoon's page. This is by far the most complex router/network setup I've ever tried.
I keep thinking about the entire project and I need to take baby steps. First a KVM host.

Not sure if this matters, and can't possibly find out until such time as I get all the hardware. Again, could be as late as January 7 according to the tracking info. So more information to point out about this board. Total of 8 ethernet ports. They are:

  1. 1x dedicated nic for IPMI 2.0 with virtual media over LAN and KVM-over-LAN support, Aspeed AST 2400 BMC
  2. 1x Intel nic.
  3. Dual GbE LAN w/ Intel Ethernet controller i350-AM2 - 1 pair LAN bypass
  4. Quad GbE LAN w/ Intel C2000 SoC i354 - 2 pairs LAN bypass.


So that IPMI port, I'm thinking it's a NIC all by itself anyway. According to my plan that's going to be on a network which can't be reached from outside of that network. I've never worked at a data center but I'm pretty sure this interface is designed to be on a management only network. I don't know how much functionality would be there, or if it would be enough to emerge-webrsync. I've used IPMI on exactly one device and it was the basic version of IPMI.

This board has more interfaces than I need for my router. I got it because there was some bad press about the earlier version of the board, and figured they might have fixed it in this version. I figured on 4 VLANs, maybe make that 5 with IPMI.

The hardware of this machine I had intended to only be reachable from my most private network. No way to connect inward to that VLAN, even from another internal network, even through the VPN. This network will be essentially VM hosts and IPMI, and one workstation wired in with ethernet. Probably that host will have two ethernet cables for one nic, and when I need to do something about managing these machines I'll swap cables. There's really not that much hardware here, it's mostly an unbelievable number of home-style digital cockroaches (TVs, BluRays, printers, blah blah) that will all be on a standard SOHO router, and then a couple VM hosts, and way too many raspberry pi micro-servers (e.g. dhcp+dynDNS+failover!! Stratum 1 time server!) and a couple workstations. It's all in the VMs and the separation in those.

So getting back to the point. The KVM host needs:

  1. The real driver for the nics.
  2. Tun/tap support.
  3. Patches for QuickAssist. https://01.org/packet-processing/intel®-quickassist-technology-drivers-and-patches
  4. To be able to pass all that to the guests.


So items 1 and 2 I can handle.

QuickAssist: I've done a couple kernel patches but I've never tried to make it persistent between kernels. The link above shows any number of points where the QuickAssist code needs to be present, and I'm not sure how to modify Gentoo to make it work reliably after an update.

Passing functionality to KVM guest: I have no idea. Obviously need to read some stuff.

Thanks.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 44153
Location: 56N 3W

PostPosted: Tue Dec 30, 2014 11:49 am    Post subject: Reply with quote

1clue,

Your
Code:
So getting back to the point. The KVM host needs:

    The real driver for the nics.
    Tun/tap support.


Is a "maybe" again. Thats true if the KVM host donates the NICs to bridges.
If you can do PCI passthrough, you need the stub driver in the KVM host, not the real NIC driver.
The KVM host then has no access to the NICs at all, they are passed through to the guest

Bridges are easy to set up and test. After you have created a bridge on the KVM host, virt-manager can connect guests to it.
As my hardware is buggy, I have been unable to test PCI passthrough.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2562

PostPosted: Tue Dec 30, 2014 6:17 pm    Post subject: Reply with quote

But I should be able to do passthrough on specific devices right? For sure I don't want the IPMI passed through, I want the host to keep it. And probably at first I'll keep the 1x card to the host.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2562

PostPosted: Tue Dec 30, 2014 7:08 pm    Post subject: Reply with quote

Another thing. The only place I see atom support is in x86, but this is clearly a 64-bit instruction set.

I would use amd64 right? My intent is hardened.

http://ark.intel.com/products/77988/Intel-Atom-Processor-C2758-4M-Cache-2_40-GHz
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 44153
Location: 56N 3W

PostPosted: Tue Dec 30, 2014 7:56 pm    Post subject: Reply with quote

1clue,

The original Atoms were 32 bit but this is a 64bit CPU, so you want an amd64-hardened install.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2562

PostPosted: Tue Dec 30, 2014 8:49 pm    Post subject: Reply with quote

That's a huge relief. I assumed it would be amd64 but didn't see any reference at all to atom in the amd64 handbook.

Gotta do a bunch of reading, I've never done a lot of the things I need to do here.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2562

PostPosted: Thu Jan 01, 2015 7:03 am    Post subject: Reply with quote

So if i need to patch sources for quick-assist is it better to go for gentoo sources or for upstream? How long does it take for upstream to find its way to gentoo stable?

I assume I need to check sources out from git.
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2562

PostPosted: Thu Jan 01, 2015 11:45 pm    Post subject: Reply with quote

YES!!!!!!!!

I just pulled in a new kernel with emerge-webrsync, did a quick search and it supports QuickAssist!

Gotta do more research to see what other patches might be there. This is perfect timing!
Back to top
View user's profile Send private message
1clue
Advocate
Advocate


Joined: 05 Feb 2006
Posts: 2562

PostPosted: Fri Jan 16, 2015 11:27 pm    Post subject: Reply with quote

I'm finally getting a few minutes to do some more research.

It turns out that in order to do pci passthrough I need to have VT-d support, which the atom does not have.

Is there some way to isolate the host from the nics when I don't have passthrough?

Thanks.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 44153
Location: 56N 3W

PostPosted: Sat Jan 17, 2015 12:21 pm    Post subject: Reply with quote

1clue,

See what szatox said
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum