Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
LXC containers and networking
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
Naib
Watchman
Watchman


Joined: 21 May 2004
Posts: 5785
Location: Removed by Neddy

PostPosted: Sat Jan 04, 2020 9:37 pm    Post subject: LXC containers and networking Reply with quote

I have been experimenting with containers partially to expand my knowledge but also as a testbed for something I need.

Presently I am messing around with alpine containers and can happily create, update, start ... all of this however with a network type=none (ie straight through). My end goal is to have a number of containers each providing different web services (one for mantisBT, one for dukuwiki, one for picocms ... ) each with nginx (port 10080,11080 ...) in and then the host system (on port 80) has an nginx acting as a reverse proxy to stitch all these together.

This works fine with lxc.net.0.type =none. What I would like to try is veth such that the containers were only accessible by the localhost and thus the host nginx acting as a reverse proxy stitches it all together.

I have manage to create a tap and a bridge.

Code:

# Add a MAC address to Bridge.
mac_br0="fe:00:00:00:01"


# Bridge setup
tuntap_tap0="tap"
config_tap0="null"
config_eno1="null"

# Define the Bridge (list all the interfaces for bridge). One tap[0-9] per VM
bridge_br0="eno1 tap0"

# Bridge dynamic config
config_br0="dhcp"
bridge_forward_delay_br0=0
bridge_hello_time_br0=1000

# Indicate to OpenRC that we need 'eth0' and 'vnet0' before bridge is created.
depend_br0() {
    need net.eno1
    need net.tap0
}


I think this is right (ie create a bridge out of a tap and the physical connection, let the bridge get an IP from my router . The number of tap's specified aligns to the number of containers I will use) as I get mixed results. Sometimes I lose all internet, sometime the host does (a container can still ping bbc...) sometime the container doesn't create a veth.

I am presently using alpine as they are quick to make and throwaway but if I can get 2+ containers working like this I can start working on a gentoo container.

Any advice?
_________________
The best argument against democracy is a five-minute conversation with the average voter
Great Britain is a republic, with a hereditary president, while the United States is a monarchy with an elective king
Back to top
View user's profile Send private message
lefsha
Veteran
Veteran


Joined: 30 Aug 2004
Posts: 1088
Location: Burgas, Bulgaria

PostPosted: Tue Jan 07, 2020 9:58 pm    Post subject: Reply with quote

I don't like such answers myself, therefore sorry for that, but what exactly prevents you
from running multiple web instances without any container?

I guess the containers/dockers etc are used too often when there is no single reason
for that, because by default it creates an additional burden for maintenance without clear
benefits.

Container makes sense if a different set of critical libraries used on a host and container,
because a special software has special requirements in that respect.

Another reason, may be, is the need to be able to move a container from one host to another.
That should happen quite often to justify it.

For nginx, none of above is the case.

P.S. Also one can use different IPs instead of ports for different instances if 1+ user is expected.
Asking users to remember what each port means is not the best strategy in social management.

crazyserver.com:1234
crazyserver.com:4321

are worse than

bugtracker.crazyserver.com
webmail.crazyserver.com
_________________
Lefsha
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 14925

PostPosted: Wed Jan 08, 2020 2:28 am    Post subject: Reply with quote

I don't think combining the interfaces into a bridge is the right solution for the stated problem. I think creating veth pairs will work, and that you would want to statically assign unique addresses to each veth pair, so that the nginx configuration can know them. Get LXC to create the veth pairs as part of bringing up the container. If you use veth pairs, you don't need to play games with high numbered ports. Each container will have its own address, and thus its own private port 80, on which it can listen.

Tap devices are more appropriate to full virtual machines, which is not what you are doing.
lefsha wrote:
I don't like such answers myself, therefore sorry for that, but what exactly prevents you
from running multiple web instances without any container?
Perhaps he wants the greater isolation enforced by running the applications in minimal containers. A wild read or wild write exploit is not as dangerous if the attacker finds himself in a container with no useful tools and almost everything mounted read-only.
Back to top
View user's profile Send private message
Naib
Watchman
Watchman


Joined: 21 May 2004
Posts: 5785
Location: Removed by Neddy

PostPosted: Fri Jan 10, 2020 10:59 pm    Post subject: Reply with quote

exactly, this is for security but also to learn.
If I have some port open I want to ensure if there is a vulnerability it would be contained. The only two ports I wanted to expose were 1) 22 for ssh 2) 443 for https.
Now ssh is an annoying one as this would be the host but a combination of denyhost + ssh keys should help. for https because I have a number of web services (wiki, problem report, jupyterhub, gitea) if there was a nginx vuln, a php, a mysql ... lo and behold i could have an issue.

My initial thought was lots of containers and then using tap's. I did consider veth but I have decided instead to have a container with just nginx which acts as a reverse proxy to other containers via UNIX sockets.
If the :443 container is compromised, what can they do? minimal tools in a RO container. The nginx in this container accesses some UNIX sockets (so the RW) for the other services.
_________________
The best argument against democracy is a five-minute conversation with the average voter
Great Britain is a republic, with a hereditary president, while the United States is a monarchy with an elective king
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum