Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
kdbus in the kernel
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3 ... 11, 12, 13 ... 25, 26, 27  Next  
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
Anon-E-moose
Advocate
Advocate


Joined: 23 May 2008
Posts: 4334
Location: Dallas area

PostPosted: Tue Apr 14, 2015 7:40 pm    Post subject: Reply with quote

Just read this from Andy in reply to GKH and just about spewed my drink

Quote:
I find myself comparing kdbus to win32k, and that's not a good sign...


:lol: :lol: :lol:
_________________
Asus m5a99fx, FX 8320 - nouveau, oss4, rx550 for qemu passthrough
Acer laptop E5-575, i3-7100u - i965, alsa
---both---
5.0.13 zen kernel, profile 17.1 (no-pie & modified) amd64-no-multilib
gcc 8.2.0, eudev, openrc, openbox, palemoon
Back to top
View user's profile Send private message
arnvidr
Guru
Guru


Joined: 19 Aug 2004
Posts: 599
Location: Oslo, Norway

PostPosted: Wed Apr 15, 2015 8:20 am    Post subject: Reply with quote

The discussion on the list seems clear that it cannot be merged for 4.1 at least, and they're talking about discussing it on some conference now, this summer. Because several of them are still wondering why this should be in the kernel (and why the design is so messed up).

Greg K-H's arguments seem really useless to me, "it is that way because dbus is that way". And naturally getting "fix the bad design before trying to put it in the kernel" reply.
_________________
Noone wrote:
anything
Back to top
View user's profile Send private message
steveL
Watchman
Watchman


Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery

PostPosted: Wed Apr 15, 2015 11:34 am    Post subject: Reply with quote

arnvidr wrote:
Greg K-H's arguments seem really useless to me, "it is that way because dbus is that way". And naturally getting "fix the bad design before trying to put it in the kernel" reply.

Yeah, he's beyond disingenuous, imo; he simply keeps referring people back to previous answers that haven't been given; eg: in response to this mail (which is expanded on here.) Someone tell me where he "addresses these misconceptions" please. (Note, as usual, the language implies it's all just a misunderstanding on the part of the "doubter".)

As a result the discussion just goes in circles.

Finally when everyone's had enough, we'll hear the "no more objections were made" and later it will be presented as "all technical objections were responded to and dealt with," even though the responses were woefully inadequate.

I don't hold out much hope for the "summer conference" resulting in anything other than a woozy "Greg's all right" (he bought me a pint) decision on the part of people who understandably want to believe the best of others.
So I'd guess we'll be testing the config option to disable the crapfest soon enough, irrespective of how bad a technical and design decision it might turn out to be down the road.
Back to top
View user's profile Send private message
Naib
Watchman
Watchman


Joined: 21 May 2004
Posts: 5711
Location: Removed by Neddy

PostPosted: Wed Apr 15, 2015 2:35 pm    Post subject: Reply with quote

For reference when overly complicated BS gets into kernelspace "to make intercommunication easier"

http://tech.slashdot.org/story/15/04/15/1359225/remote-code-execution-vulnerability-found-in-windows-http-stack


Windows put font raster management into the kernel "to speed it up" it then was a nasty vuln into the system. If the only valid arguement is "for speed" then well ;)
_________________
The best argument against democracy is a five-minute conversation with the average voter
Great Britain is a republic, with a hereditary president, while the United States is a monarchy with an elective king
Back to top
View user's profile Send private message
gwr
Apprentice
Apprentice


Joined: 19 Nov 2014
Posts: 194

PostPosted: Wed Apr 15, 2015 2:47 pm    Post subject: Reply with quote

steveL wrote:

Finally when everyone's had enough, we'll hear the "no more objections were made" and later it will be presented as "all technical objections were responded to and dealt with," even though the responses were woefully inadequate.


This is what I think is going to happen. Almost every thread just peters out after a long stream of objections, questions, and critiques are ignored. It's a war of attrition and eventually the kernel devs will surrender.
Back to top
View user's profile Send private message
Anon-E-moose
Advocate
Advocate


Joined: 23 May 2008
Posts: 4334
Location: Dallas area

PostPosted: Wed Apr 15, 2015 5:47 pm    Post subject: Reply with quote

The "discussion" between GKH and the kernel devs seems to be heating up. (it can all be gotten to by the lkml link)

Quote:
> Look, us kernel developers only work on one huge, multithreaded, global
> state binary. Our experience in multi-application interactions with
> shared state and permission requirements is usually quite limited. If
> you don't trust the developers of those programs outside the kernel,
> don't use them, there are still distros out there that don't require
> them.

Speak for yourself. There are a lot of us here who work and have worked
on low level messaging, on networking, on clusters and on things like
distributed shared memory, infiniband etc. I've worked on networks,
including broken stateful protocols, I've maintined and developed
internet and ISDN router code, I've worked with message passing realtime
systems.

Equally the folks who wrote dbus generally also know sweet fa about
writing a kernel and maintaining it for 25 years. Gtk is on its 3rd
completely incompatible instance (and has incompatibilities even within
major versions), Gnome is on its third major incompatible release -
closer would be to say at least the "second project with the same name",
and neither are as old as the kernel.

dbus is not an appropriate design for a kernel messaging layer for a
variety of reasons. That's not to say dbus shouldn't be able to use a
fast kernel messaging layer, or that one shouldn't exist.

dbus is basically a very large very specialized and somewhat flawed
policy engine on top of what should be simple messaging. The two need
splitting apart.

Abstract low level messaging layers are not a new concept. V7 unix had
one experimentally. It's about getting the separation right.

IMHO that probably involves getting the right people in the right place
together - dbus designers, MPI and realtime people, kernel folks and
possibly also some of the hardware messaging folk.

In filesystem terms

- stop writing a dbus only file system
- figure out what a messaging "vfs" looks like
- figure out what an clean low level kernel model looks like
- figure out what has to be where to put the policy in userspace

What might also be worth review is how much dbus traffic actually ought to
be an object store implemented say with tmpfs and inotify type
functionality (or extensions of that) so that you can
set/read/enumerate/get change notifications on properties.

Alan

_________________
Asus m5a99fx, FX 8320 - nouveau, oss4, rx550 for qemu passthrough
Acer laptop E5-575, i3-7100u - i965, alsa
---both---
5.0.13 zen kernel, profile 17.1 (no-pie & modified) amd64-no-multilib
gcc 8.2.0, eudev, openrc, openbox, palemoon
Back to top
View user's profile Send private message
Anon-E-moose
Advocate
Advocate


Joined: 23 May 2008
Posts: 4334
Location: Dallas area

PostPosted: Wed Apr 15, 2015 5:54 pm    Post subject: Reply with quote

And another kernel dev
Quote:
On Wed, Apr 15, 2015 at 01:40:36PM +0200, Greg Kroah-Hartman wrote:
>
> You have to trust someone to help make your system work together in a
> unified way. If you can't trust your distro's engineers, then either
> start your own distro, or only run busybox on top of a kernel. You
> really don't have much other "choice" than that :)

And obviously there is a lack of trust. And once kdbus is in, we must use
it, or support our own distro where we just do not have the time.

Personally, I'm fine with getting something in that will help userspace
tools work better. The issue I see, mostly from the side lines as I haven't
totally submerged myself into the dbus protocol (I think I should spend
some time to do just that), this is going too fast. Once it is in the kernel,
whatever ABI we expose is locked in stone. There's no changing it. We need
to make sure that this is well thought out. People seem to be of the impression
that the current dbus design has flaws, but because everything relies on it
we must still push it into the kernel because it mimics what is out there
in user space. I disagree.

As others have said. We do not need to follow the dbus design. If we can supply
a better transport layer than what the kernel supplies today, then tools will
eventually merge to it away from dbus. Perhaps the kernel can supply just enough
to have dbus improve its speed, but not with the entire complex solution that
kdbus is presenting today.

This isn't a case of Republicans vs Democrats pushing a health care system within
a window that was rushed. Now the US has a health care system that somewhat works
but due to politics its not being fixed (the ABI is solidified). I don't want
to have the same thing with kdbus. We are technical people here, lets solve it
with a technical solution, and not rush into things. dbus works today, what's
the rush to put something into the kernel that must be supported forever. Lets
make sure we do it right.

I'm serious about my Linux Plumbers proposal. If you can make it, and get the dbus
authors there too, and hopefully, Andy, Al and Eric can make it too. We should
really sit down and talk about it. Any other kernel developer that wants to
participate should, as a prerequisite, sit down and write a dbus interface, such
that they have an idea of how it works. I plan to. And I hope that I can learn
more about the interface and productively join in this discussion.

I'm willing to moderate the kdbus microconference. I think I'll add it now.

Thoughts?

-- Steve

_________________
Asus m5a99fx, FX 8320 - nouveau, oss4, rx550 for qemu passthrough
Acer laptop E5-575, i3-7100u - i965, alsa
---both---
5.0.13 zen kernel, profile 17.1 (no-pie & modified) amd64-no-multilib
gcc 8.2.0, eudev, openrc, openbox, palemoon
Back to top
View user's profile Send private message
Anon-E-moose
Advocate
Advocate


Joined: 23 May 2008
Posts: 4334
Location: Dallas area

PostPosted: Wed Apr 15, 2015 6:01 pm    Post subject: Reply with quote

Steve has this to say in another post

Quote:
...Is there a reason that this patch must go in this merge window? Having
something this controversial take place during the merge window
suggests its a bit premature to push in now. Especially since it
creates a new user space interface. I think we need to really think
hard and long before we add something that can not be modified at a
later date....


I think the big hurry is that GKH knows that if people take the time to
really look at it, then there is no way in hell of it getting into the kernel in
it's current form. Something they desperately want, it seems like
_________________
Asus m5a99fx, FX 8320 - nouveau, oss4, rx550 for qemu passthrough
Acer laptop E5-575, i3-7100u - i965, alsa
---both---
5.0.13 zen kernel, profile 17.1 (no-pie & modified) amd64-no-multilib
gcc 8.2.0, eudev, openrc, openbox, palemoon
Back to top
View user's profile Send private message
gwr
Apprentice
Apprentice


Joined: 19 Nov 2014
Posts: 194

PostPosted: Wed Apr 15, 2015 6:05 pm    Post subject: Reply with quote

Anon-E-moose wrote:
And another kernel dev


I'm pretty ignorant about how these things work. How much of a difference does it make that kernel devs raise flags? Do any of them have the authority to prevent kdbus from being included in the kernel?
Back to top
View user's profile Send private message
Anon-E-moose
Advocate
Advocate


Joined: 23 May 2008
Posts: 4334
Location: Dallas area

PostPosted: Wed Apr 15, 2015 6:20 pm    Post subject: Reply with quote

As of right now there are 2 NAKs for inclusion and no ACKs.

If GKH and comrades can't convince the kernel devs what are the chances of Linus accepting the code. He's even pickier than them.
_________________
Asus m5a99fx, FX 8320 - nouveau, oss4, rx550 for qemu passthrough
Acer laptop E5-575, i3-7100u - i965, alsa
---both---
5.0.13 zen kernel, profile 17.1 (no-pie & modified) amd64-no-multilib
gcc 8.2.0, eudev, openrc, openbox, palemoon
Back to top
View user's profile Send private message
Anon-E-moose
Advocate
Advocate


Joined: 23 May 2008
Posts: 4334
Location: Dallas area

PostPosted: Wed Apr 15, 2015 6:21 pm    Post subject: Reply with quote

Quote:
On Wed, Apr 15, 2015 at 10:44:40AM +0200, Greg Kroah-Hartman wrote:
> If you really don't like userspace using features the kernel provides
> you, well, there's nothing I can say that will change that odd feeling,
> sorry.

Are you even reading what people are saying?

I don't like the mandatory(!) aspect of this, which it will eventually
become. There is this thing called "choice", remember?

> Really? Who in that MAINTAINERS file entry do you not trust?

The fact that you're still pushing for this current design *in the face*
of people pointing out serious design flaws with this makes me not
really trust you.

> I don't understand what this means. If you have a technical reason
> for why this code shouldn't be merged, great, please let me know and
> we can work to address that. Andy and Al have spent time reviewing
> and giving us comments, and that's wonderful and valuable and is
> why I treat their comments seriously. If you are interested in the
> code, please review it,

Yeah, I took a brief look at the code. It is overcomplicated.

If I were to review it properly, I'd ask you to split it in small
patchsets. Hell, I'm pretty sure you would do the same for code you
don't know if you were in my shoes.

Also, considering the complexity of this patchset, it doesn't have
a single Reviewed-by by an external party. If this were any other
submission, it would've been kicked to the curb a long time ago.

--
Regards/Gruss,
Boris.


Wonder when the CoC will be invoked. :roll:
_________________
Asus m5a99fx, FX 8320 - nouveau, oss4, rx550 for qemu passthrough
Acer laptop E5-575, i3-7100u - i965, alsa
---both---
5.0.13 zen kernel, profile 17.1 (no-pie & modified) amd64-no-multilib
gcc 8.2.0, eudev, openrc, openbox, palemoon
Back to top
View user's profile Send private message
gwr
Apprentice
Apprentice


Joined: 19 Nov 2014
Posts: 194

PostPosted: Wed Apr 15, 2015 6:21 pm    Post subject: Reply with quote

Anon-E-moose wrote:
As of right now there are 2 NAKs for inclusion and no ACKs.

If GKH and comrades can't convince the kernel devs what are the chances of Linus accepting the code. He's even pickier than them.


Thanks. Is there a number of ACKs that are required, or is it a percentage of people who get say?
Back to top
View user's profile Send private message
Anon-E-moose
Advocate
Advocate


Joined: 23 May 2008
Posts: 4334
Location: Dallas area

PostPosted: Wed Apr 15, 2015 6:29 pm    Post subject: Reply with quote

This is a great read by Alan and a valid set of reasons for not letting kdbus in as it exists.

Quote:
On Wed, 15 Apr 2015 00:39:22 +0200 (CEST)
Jiri Kosina <jkosina@xxxxxxx> wrote:

> On Tue, 14 Apr 2015, Greg Kroah-Hartman wrote:
>
> > I don't understand. You can not like the D-Bus model (and accordingly
> > the X11 model),
>
> I thought that the general hatred level of the X11 "model" and the
> protocol lead to al the efforts to reimplement this properly ... in
> userspace (for example Wayland, right?).
>
> I don't think anyone was ever seriously suggesting "X11 model is broken,
> so let's push it to kernel" ... ?

The X11 model is *nothing* to do with the dbus/kdbus model. X11 does
properties by attaching them to windows. Those properties can be
monitored for changes and they can be queried. Setting them is
asynchronous, querying them is sync or with the newer event based
libraries can be async. X11 properties are network safe, handled through
the same X11 authority as everything else. Two apps can happily run on
different systems sharing a display over the network and sharing and
responding to changes in X11 properties - and it just works.

The Gnome people tried to re-invent X11 properties and embedding badly
with CORBA, then with dbus, despite the fact the Andrew system could
already do it really fast and cleanly even before Gnome was thought of.

There is no comparison between the elegance of X11 property setting and a
chunk of proposed kernel code that is half the size of a tiny X server!

The dbus model is also flawed in a load of other ways in user space
because message handling in the hands of people with no concept of
systemic performance analysis just leads to disaster. One of the big
reasons dbus is so "slow" isn't that dbus is "slow", it's that the
crapware on top of it makes *thousands* of dbus queries.

If you must do it in kernel why not use the Android binder - it's awful,
broken, and dubiously secure, but at least we'd still only have one awful,
broken dubiously secure rpc/property layer in kernel.

"It's the issue that a stateful bus is required for
applications that is the main point I'm trying to get across."

That would be the "if dbus crashes I have to reboot" design flaw of
Gnome and friends. The only state you need is beyond the endpoints. It's a
message passing system. If you think message passing needs state then I'd
take a look at the internet. State belongs in the end points.

It's telling that I can lose and recover my internet connection without
rebooting but not my desktops internal messaging.

Alan

_________________
Asus m5a99fx, FX 8320 - nouveau, oss4, rx550 for qemu passthrough
Acer laptop E5-575, i3-7100u - i965, alsa
---both---
5.0.13 zen kernel, profile 17.1 (no-pie & modified) amd64-no-multilib
gcc 8.2.0, eudev, openrc, openbox, palemoon
Back to top
View user's profile Send private message
Anon-E-moose
Advocate
Advocate


Joined: 23 May 2008
Posts: 4334
Location: Dallas area

PostPosted: Wed Apr 15, 2015 6:37 pm    Post subject: Reply with quote

From Al Viro

Quote:
On Wed, Apr 15, 2015 at 01:49:36PM +0200, Greg Kroah-Hartman wrote:
> > There is no comparison between the elegance of X11 property setting and a
> > chunk of proposed kernel code that is half the size of a tiny X server!
>
> Hey, take that up with Havoc, he made the comparison :)

Let me get it straight - you swing the reference to his posting as damn nearly
the main argument, and yet you make _this_ reply when it gets questioned?
Seriously?

There's nothing wrong with "go read $PAPER, a lot of your questions are
addressed there", but only if you are ready to answer the questions and
objections from those who have read it. "Hey, take that up with
$AUTHOR" doesn't cut it; try anything even remotely similar with e.g.
reviewers of academic paper and see where it ends up.

Havoc isn't submitting that thing; you are. If you are not qualified to
defend your design and he is, try to talk him into doing that.

Frankly, the longer it goes, the less I like the picture. It will be up to
Linus, of course, but IMO the whole situation seriously stinks. ;-/


Oh boy, I'm waiting till I see Linus' response over this :lol:
_________________
Asus m5a99fx, FX 8320 - nouveau, oss4, rx550 for qemu passthrough
Acer laptop E5-575, i3-7100u - i965, alsa
---both---
5.0.13 zen kernel, profile 17.1 (no-pie & modified) amd64-no-multilib
gcc 8.2.0, eudev, openrc, openbox, palemoon
Back to top
View user's profile Send private message
Anon-E-moose
Advocate
Advocate


Joined: 23 May 2008
Posts: 4334
Location: Dallas area

PostPosted: Wed Apr 15, 2015 8:45 pm    Post subject: Reply with quote

Linus chimes in

Quote:
On Wed, Apr 15, 2015 at 11:11 AM, Greg Kroah-Hartman
<gregkh@xxxxxxxxxxxxxxxxxxx> wrote:
> On Wed, Apr 15, 2015 at 01:33:27PM -0400, Steven Rostedt wrote:
>>
>> I'll argue that you can't fix the later one. One thing that I've observed over
>> the years of having faster computers is, as soon as you make it faster, people
>> will write slower software.
>>
>> Currently the issue is that we have thousands of dbus queries, you make dbus
>> 10x faster, I guarantee that people will write software with 10 thousand dbus
>> queries and we are no better off than we are today.
>
> Then they get to buy a faster machine :)

Is there actually a performance issue?

I've seen this claimed, but I have never seen any actual numbers. What
speeds up? By how much? is it actually measurable?

Maybe they've marched past me in this thread-from-hell. But I can't
recall having seen any (not now, not before).

That said, I think the more serious issue is that if Luto complains
about the capability-capturing code being completely broken, then
people need to take that *seriously*.

Linus


and added this in the next email

Quote:
On Wed, Apr 15, 2015 at 11:18 AM, Linus Torvalds
<torvalds@xxxxxxxxxxxxxxxxxxxx> wrote:
>
> I've seen this claimed, but I have never seen any actual numbers. What
> speeds up? By how much? is it actually measurable?

And just to clarify: by "what speeds up, and by how much", I do _not_
mean "sending a dbus message speeds up by 10x and avoids context
switches". I've seen _those_ numbers. But does it actually matter?

It was more of a "there are thousands of dbus messages during boot,
but can you actually measure the speedup?" question. That's the kind
of numbers I've not seen.

Linus

_________________
Asus m5a99fx, FX 8320 - nouveau, oss4, rx550 for qemu passthrough
Acer laptop E5-575, i3-7100u - i965, alsa
---both---
5.0.13 zen kernel, profile 17.1 (no-pie & modified) amd64-no-multilib
gcc 8.2.0, eudev, openrc, openbox, palemoon
Back to top
View user's profile Send private message
gwr
Apprentice
Apprentice


Joined: 19 Nov 2014
Posts: 194

PostPosted: Fri Apr 17, 2015 10:53 am    Post subject: Reply with quote

In answer to Linus, GKH posted this. He never did follow up on it, which is typical of all of these types of questions.
I find it inconceivable that people who have spent this much time and effort writing something for supposed performance improvements do not have any published measurements.

Quote:
Someone from BMW did the testing on one of their car systems a while ago
and posted some numbers, it was a factor of 10 faster. I'll try to dig
it up, but it was burried in a powerpoint presentation, so it might be
hard to find, give me a day.

thanks,

greg k-h
Back to top
View user's profile Send private message
ulenrich
Veteran
Veteran


Joined: 10 Oct 2010
Posts: 1389

PostPosted: Fri Apr 17, 2015 4:00 pm    Post subject: Reply with quote

While kDbus gets delayed the kernel guys tasted blood now! They want a more general approach to better achieve what steveL described as "circumventing the GPL":
Quote:
For me the biggest issue is the container problem: it's really hard to
containerise kdbus because of the stateful nature of the protocol and
the fact that it has a well known system bus. Separation into domains
works for OS containers, but application containers need more fluidity.
It's not unlike the same problem on windows: Windows application
containers are very difficult to do because the global registry means
that OLE handlers all have to run inside your container as well
(effectively making it an OS container). I'm sure, since we already
have a lot of containers people going to plumbers, that we can get them
to turn up for the discussion.

James

Quote:
We can however leave it in userspace [Alan talks about the policy part] until we understand the right small
clean way to support it and other needs. At the moment for example
cluster people can't really use this stuff because its not network aware,
and HPC people can't use it because it's got dbus hardwired into it so
can't speak MPI-3 and the like even though MPI 3 has similar concepts
around DPM, as well as having proper models for parallelism and
collective operations that are lacking in dbus.

If the userspace folks choose to continue to implement dbust over it but
the kernel layer is clean and generic then all is good, because someone
can replace dbust with something better. If its got dbust hard wired into
it then its a complete mess.

Alan

Quote:
> In filesystem terms
>
> - stop writing a dbus only file system
> - figure out what a messaging "vfs" looks like
> - figure out what an clean low level kernel model looks like
> - figure out what has to be where to put the policy in userspace
>
> What might also be worth review is how much dbus traffic actually ought to
> be an object store implemented say with tmpfs and inotify type
> functionality (or extensions of that) so that you can
> set/read/enumerate/get change notifications on properties.

FWIW, this sounds really sane and makes a lot of sense to me. I'd be
willing to give it some review cycles, as far as I can, when done this
way.

--
Regards/Gruss,
Boris.

_________________
the thread ain't easily find an end
Back to top
View user's profile Send private message
bstaletic
Apprentice
Apprentice


Joined: 05 Apr 2014
Posts: 225

PostPosted: Fri Apr 17, 2015 5:40 pm    Post subject: Reply with quote

ulenrich,

How is that circumventing GPL? It sounds, to me at least, as if kernel developers are saying "Let's leave everything as is and build a sane IPC protocol". Which is fine with me.
Back to top
View user's profile Send private message
steveL
Watchman
Watchman


Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery

PostPosted: Fri Apr 17, 2015 6:30 pm    Post subject: Reply with quote

ulenrich wrote:
They want a more general approach to better achieve what steveL described as "circumventing the GPL"

*Sigh* what nonsense.

You don't seem capable of distinguishing the argument (about "unintended", or rather unmentioned effects) from the implementation (of IPC: which is not RPC.)

By all means witter on; just keep my name out of it unless you have a clue what you're talking about.
Back to top
View user's profile Send private message
Shamus397
Apprentice
Apprentice


Joined: 03 Apr 2005
Posts: 218
Location: Ur-th

PostPosted: Fri Apr 17, 2015 9:05 pm    Post subject: Reply with quote

And (who I presume to be) Alan Cox comes in with a smackdown:

One Thousand Gnomes wrote:
> Don't make idle comments, the tty layer is far more complex and larger
> than the kdbus code, with much nastier issues and problems. And we
> handle that just fine :)

The tty layer is the way it is because of design decisions dating back 20 years that were (with hindsight) wrong coupled with the fact that POSIX took a lot of the behavioural guarantees from an armwaving claim about what Unix(tm) implemented without thinking about how to implement them (as far as I can tell - given many of the guarantees are broken in Unix!)

...

The problem I have is that every time someone points out a fundamental design issue you simply say "Why haven't you reviewed 13,000 lines of code".

I haven't given it an in depth review for the same reason as if someone posted 13,000 lines of "I've got an awesome new file system which uses a FAT and 8.3 file names". There's some more pressing concerns to sort first. The fact it's complex and hard to follow also doesn't encourage review.

And the fact Al tried to read it and is asking for help really worries me 8)

> > Having something this controversial take place during the merge window
> > suggests its a bit premature to push in now.
>
> "take place"? Have you been ignoring these patches posted numerous
> times for many months? This is the point in time to ask for code to be
> merged, just like any other code, nothing is special here.

Well - you've asked. I see two NACKs from people with great taste. So I think the next step is to defer trying to submit it and work through the fact that Al can't follow the locking, and other people don't believe the security model is maintainable.

Good to know that the kernel devs aren't so easily buffaloed as (ahem) others have been. :D

And it gets even better:

Eric W. Biederman wrote:
> Greg Kroah-Hartman <gregkh@xxxxxxxxxxxxxxxxxxx> writes:
>> Here's the kdbus pull request for 4.1-rc1.
>>
>> It's been under development for many years now, and been in linux-next
>> for many months, and has undergone loads of testing a review and even a few
>> good arguments. It comes with full documentation and tests.
>
>> There has been a few complaints about the code, notably from people who
>> don't like the use of metadata in the bus messages. That is actually
>> one of the main features here, as we can get this data in a secure and
>> reliable way, and it's something that userspace requires today. So
>> while it does look "odd" to people who are not familiar with dbus, this
>> is something that finally fixes a number of almost unfixable races in
>> the current dbus implementations.
>
> And the code that transfers the meta-data is wrong.

In fact it is worse than I thought.

With an userspace application able to give meaning to any of the bits of meta-data that are passed (capabilities, cgroup, security labels, etc) that in the fullness of time dropping in them will grant you more permissions somewhere.

Which means that it becomes impossible to change anything. Impossible to jail anything. It in fact becomes impossible to do anything right.

Which means the ultimate result of the direction kdbus is going is a world where nothing can be done without introducing a security issue or breaking userspace.

So as far as I can tell kdbus has a fundamental design flaw.

My apologies for being the bearer of bad news.


Also, for those who missed Andy's first NACK, here's a link. :D
Back to top
View user's profile Send private message
Shamus397
Apprentice
Apprentice


Joined: 03 Apr 2005
Posts: 218
Location: Ur-th

PostPosted: Fri Apr 17, 2015 10:10 pm    Post subject: Reply with quote

That thread is full of gems, and it seems that GKH just refuses to take no for an answer, no matter what anyone tries to tell him:

One Thousand Gnomes wrote:
> operating systems anymore. Those "legacy" oses provide a system bus
> that allows them to send thousands of queries just fine, but when moving
> to Linux, we don't have anything other than D-Bus, so their library is
> ported to use it, and they have to handle their old applications that
> need/want the zillions of messages.

And if you look at those systems btw many of them have a very compact, very clean very simple message passing interface, often in the hundreds not tens of thousands of lines of code.

> People take those stateless models and build stateful ones on top of
> them, yes, it's great. But you still need a stateful model somewhere in
> order to be able to achieve many things (think a shopping cart
> application).

We put the IP stack in the kernel not the shopping cart. A good shopping cart of course only has state on the client.

> Anyway, this is getting off-topic, there is very little "state" in the
> kdbus kernel code here, other than a naming database that Havoc and
> Simon explain the need for, and the normal lifecycle of kdbus "nodes"
> (new, linked, active, inactive, drained, freed).

I'm not convinced the naming data belongs in kernel beyond the simplest of "node 147". I'd offer a sort of proof by armwaving of this that if you have

/dev/dbus/014 /dev/dbus/027 etc

you can add a symlink to /dev/dbus/014 of

/dev/dbus-by-name/gnome-wombat-grooming-daemon

or whatever

and we do that today for every other naming database and static allocation we've spent the past 15 years evicting from the kernel.

That state isn't then held in a daemon that can crash nor is it invisible to debuggers, user tools and admins.


It would seem if Alan Cox is telling you that your code is broken, it would behoove you to take what he says seriously. But only if your name is not GKH, apparently. :)

But wait! Al Viro has this to say in reply to Andy:

Al Viro wrote:
On Wed, Apr 15, 2015 at 01:22:12PM -0700, Andy Lutomirski wrote:

> This leads me to a potentially interesting question: where's the
> buffering? If there's a bus with lots of untrusted clients and one of
> them broadcasts data faster than all receivers can process it, where
> does it go?
>
> At least with a userspace solution, it's clear what the OOM killer
> should kill when this happens. Unless it's PID 1. Sigh.

... and there is a PID 1 specimen that really likes to spew over dbus. A lot. I had never been able to find out _why_ does systemd feel like broadcasting all kinds of stuff from PID 1 - maybe somebody in this thread can answer that. For example, what's the point of broadcasting mount table updates, when
* it can't hope to catch all individual changes - they _can_ get lumped together, no matter what it tries.
* any process can just as easily keep track of that data on its own as it could by watching those broadcasts; parsing /proc/self/mountinfo isn't harder than parsing notifications.
* you need to start with obtaining the original state somehow, or what would you apply those updates to?
* if one insists on having a daemon doing such broadcasts, what the hell is the point of having PID 1 do that? Exact same logics would do just fine. Moreover, you could have one running in a namespace of your session, which is something PID 1 won't see.

Sure, I understand why it wants to be aware of what's mounted and where it's mounted. Just as it wants to know what time it is. Should it broadcast a dbus message every second, just to tell everyone what had it found about the time?

I'm somewhat tempted to propose AF_TWITTER - would match the style... ;-/ And frankly, this really looks like a social media braindamage - complete with status update broadcast every time a plane flies by...


AF_TWITTER! :D Seems that the kernel devs really do get it. :D
Back to top
View user's profile Send private message
ulenrich
Veteran
Veteran


Joined: 10 Oct 2010
Posts: 1389

PostPosted: Fri Apr 17, 2015 11:05 pm    Post subject: Reply with quote

Shamus397 wrote:
It would seem if Alan Cox is telling you that your code is broken, it would behoove you to take what he says seriously. But only if your name is not GKH, apparently. :)

Alan says the concept is broken. (If it only was the code there was no need for plumbers conferences)
GKH says the concept is needed and proven with dbus since years. The only problem remaining with dbus he solves with his kdbus implementation (race condition).

Perhaps the policy problem can be abstracted into something similar to a firewall: All of it (capabilities, rules, roles etc) translated into an address space of numbers and then filtered. Reading that policykit rules now you might think you'll understand the impact, because it seems to be plain english language. But the effects of inherited capabilities you won't recognize yet.
_________________
the thread ain't easily find an end
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 7198

PostPosted: Sat Apr 18, 2015 12:24 am    Post subject: Reply with quote

gwr wrote:
In answer to Linus, GKH posted this. He never did follow up on it, which is typical of all of these types of questions.
I find it inconceivable that people who have spent this much time and effort writing something for supposed performance improvements do not have any published measurements.

Quote:
Someone from BMW did the testing on one of their car systems a while ago
and posted some numbers, it was a factor of 10 faster. I'll try to dig
it up, but it was burried in a powerpoint presentation, so it might be
hard to find, give me a day.

thanks,

greg k-h


I think i'm bettter than him at googling :)
Here's the powerpoint presentation he should refer to : https://github.com/gregkh/presentation-kdbus/blob/master/stuff/EG-SI-IPC%20CommonAPI%20CPP.pptx?raw=true
Of course i'm not him, so it might not be that source, but i don't think there's that many ppt presentation made by bmw guys that shown some benchmarking results.
Look at page 21: 3.6x improvement speed, first it's not 10x (yeah, numbers climb by themselves every times someone use them, but well 3.6x isn't that bad still).
The problem is that it doesn't gave any fact, but numbers, nowhere you can see tests procedure, just "suppose true" results ; so nobody can claim this result are flaw or true anyway. And you are leaved to guess what the "guys" (bmw guys?) did to find those numbers.

Here's what we got:
1/ "measuring empty messages (no playload) to compare network performance rather than marshalling performance"
The thing i notice here: comparing "empty message" performance vs "marshalling performance" can gives real different numbers, while if dbus doesn't use any trick to improve it, systemd-libdbus may have ones (more recent code and assuming they learn from dbus) and as such could get an even more better (marshalling) performance gap between them, and it could be really 10x better or more.
But if the numbers you get aren't that good, it is just better to not speak about them... So even if we assume the tests were fairly made, they only prove systemd-libdbus is better with empty message than dbus, but the "hidden" or not done tests about real test scenario with "not empty message" can lead to a different kind of result: it could be really better, but it could also be really, really worst result.
2/ "Comparing to dbus-ping using AF-BUS"
It is strange to see two results: "libdubs, afbus, clone" vs "systemd".
Assuming clone and libdbus share same performance, i expect to get 3 results still: "libdbus, clone" "afbus" and "systemd".
It look suspect that "afbus" get the same result as "dbus", it prove AF-BUS doesn't improve performance or that if you assume everyone claims AF-BUS improve performance, how can you endup with dbus result (2074) the same as AF-BUS one?
But maybe the guys added unneed "," and the result is about "dbus + clone + AF_BUS"... or maybe indeed AF-BUS doesn't improve numbers when you test the framework only. It is not very clear.
3/ That's not that good to provide numbers without knowing the nature of those numbers: 2074 vs 7407, but 2074 what? potatoes? eggs? ms? seconds? hours???
While everyone can see indeed 7407 is greater number than 2074, if those number refer to latency, than 2074 is really a better number...
Without knowing what those numbers are, we can only assume 7407 "something" is better than 2074 "something" because the guys claims it is.
At least it prove one thing: seeing how the results are given in a real unprofessional way, it raise question about their "professional" way to do the tests too.
4/ Finally i will classify this as bullshit claims without proof, it is numbers given without procedure to check them, in a test you cannot reproduce yourself to see if the results are good or bad (even if you assume someone tests are done fairly, it is not "normal" that the procedure use to made the tests is not given, just to enable somebody else to redo the tests to confirm no mistake were done in the procedure).
5/ Now that GKH is referring to this document (if it is really this one, as remember he might not refer to this one), i see it more like the classical systemd propaganda (see common practice), Mr X do a bullshit document, now refer to it, as long as you are not Mr X it should prove in everyone minds your claims are true, as your claims are backup by another source. Of course not "everyone" will trust you because of Mr X's obscure claims were done in his document, but you don't need to convince everyone, but if you can get all the mass to get convince because they don't question your claims, you win.
So what about the remain "few" that you didn't convince?
Don't worry, the mass cannot be wrong, not only the "few" will be see as wrong, but the mass itself will do the work for you: just wait, one of the poorest minded in the mass will certainly post a message against the "few" with some "haters" tag in it ; few are haters and get discredit ; you win again! thanks to your unconditional fans.
While this is proven good tactic, it start to get weak if the mass have some brains, or if the "few" have names...
So that tactic sure works with systemd and lamba users, but when the mass are kernel devs, it start to show its limit, and it is even worst when the "few" have names like alan cox.

I will endup with this: look at page 20 in the same presentation.
It gave you another hint on something: "Kernel based to ensure security isolation".
What it gave is the real difference from kdbus vs dbus, because of the "security isolation" kernel may gives (again: no proof there, it is just a claims from systemd devs kdbus will be more secure because in kernel, i personally think with my own logic that anything in kernel is less secure, the application itself "might" be more secure as the kernel will protect it, but at a cost, and this cost is less secure kernel that would carry your insecure code. So for me, adding something in kernel to get protection from the kernel will higher your security at the cost of lowering the kernel security itself ; not really something i see as a "good deal", but i don't think it's something that someone that ask you to reboot your system to update his software really care about).

The biggest second thing to notice here is that the kdbus design is more about be in kernel than anything else.
It mean they (i'm unsure english have the same reference, but here's the french one) "sell the bear skin before killing it". If kdbus isn't in kernel, there's no kdbus concept at all. If kdbus isn't in kernel, it's a dbus++ not kdbus.
So even if its the greatest shitty concept or code made so far, you must have it in kernel, else it is dead.
It is a "designer" problem, you don't need good code or working implementation or anything, you need to have it in kernel (dot) ; that's the whole point of kdbus, be in kernel, this is how they are selling it to everyone, and this is why it must be in kernel ; at any costs...
This is something that should scared the kernel devs the most...

ps: found the link to that ppt from https://lwn.net/Articles/551969/
interesting in it is that quote
Quote:
"Kroah-Hartman's conclusion is that "if you want a faster D-Bus, rewrite the daemon, don't mess with the kernel".
Back to top
View user's profile Send private message
ulenrich
Veteran
Veteran


Joined: 10 Oct 2010
Posts: 1389

PostPosted: Sat Apr 18, 2015 11:02 am    Post subject: Reply with quote

@krinn,
that performance comparison was not about kdbus, but as stated in the power point presentation:
Quote:
Early benchmarks of systemd´s D-Bus client library on Pandaboard:
dbus-ping-systemd and dbus-test-service-systemd
Measuring empty messages (no payload) to compare framework performance rather than marshalling performance
Comparing to dbus-ping using AF-BUS

about using dbus via an optimized systemd dbuslib compared with pings using AF-Bus and then a gateway to dbus. The presentation intended to move the audience to _only_ use dbus without any gateway needed. Because the numbers perfectly fit what developers expect an outcome of performance gain with kdbus (x-times less copying the messages) this mentioning of that old presentation of 2012 is due to human remembrance short-circuit. A further hint is times, the LWN article of 2013 says abaout kdbus: "Kdbus will require a recent kernel as it uses control groups (cgroups); it also requires some patches that were only merged into 3.10-rc kernels" ... which wasn't 2012.

These kind of optimizations in 2012 motivated GKH to say: "if you want a faster D-Bus, rewrite the daemon, don't mess with the kernel". Which he mentions in the kernel mailing list thread to have said but he changed his attitude.

In the article you linked, it is explicitly written what big money stood behind kdbus development:
https://lwn.net/Articles/551969/
Quote:
The automotive industry will be particularly interested because it is used to using the QNX message passing, which it mapped to libdbus. It chose D-Bus because it is well-documented, well-understood, and is as easy to use as QNX. But, it doesn't just want a faster D-Bus (which could be achieved by rewriting it), it wants more: namespace support, audit support, SELinux, application separation, and so on.
regarding this motivation the article you linked claims:
Quote:
in the kernel, kdbus gets a number of attributes almost for free. It is namespace aware, which was easy to add because the namespace developers have made it straightforward to do so. It also integrated with the audit subystem, which is important to the enterprise distributions. For D-Bus, getting SELinux support was a lot of work, but kdbus is Linux Security Module (LSM) aware, so it got SELinux (Smack, TOMOYO, AppArmor, ...) support for free.

The article further describes the status of the kernel side in 2013, memfd was not merged:
Quote:
and then Heo came up with zero and one-copy. He would be happy if it is merged by the end of the year, but if it isn't, it shouldn't stretch much past that, and he encouraged people to start looking at kdbus for their messaging needs in the future.


Now in the kernel mailing list they are discussing distinct issues regarding kdbus. Both are worth the time to delay kdbus:
1. Alan Cox is reiterating what the LWN article mentions GKH said at the time:
Quote:
Kdbus ended up with a naming database in the kernel to track the message types and bus names, which "scared the heck out of me", Kroah-Hartman said.
This a very conceptual thing I hope some kernel developer, who will participate that planed conference in summer, has an illuminating new idea to better handle it: policy is awful complex
2. kdbus is not capable to be use beyond the local system domain. It should be splited in layers. Thus parts of it can be reused for other purpose. If kdbus gets into mainline Linux as it is now it is for limited purpose like so much dead end crap you can find in the kernel.

[edit] to not provoke the Gentoo user forum community as Neddy requested I remove my trolling in the last line of this post
_________________
the thread ain't easily find an end


Last edited by ulenrich on Sat Apr 18, 2015 1:13 pm; edited 2 times in total
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 44171
Location: 56N 3W

PostPosted: Sat Apr 18, 2015 12:32 pm    Post subject: Reply with quote

ulenrich,

If only you had not added that last line, the rest of your post might have stood on its technical merits.
That last line makes me doubt that the rest of the post is honest and unbiased.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Goto page Previous  1, 2, 3 ... 11, 12, 13 ... 25, 26, 27  Next
Page 12 of 27

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum