Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
The Underlay Overlay
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
McGruff
Tux's lil' helper
Tux's lil' helper


Joined: 28 Dec 2004
Posts: 148

PostPosted: Sat May 19, 2012 12:52 am    Post subject: The Underlay Overlay Reply with quote

A new layman overlay: the underlay overlay.

There's no particular theme for the overlay. It's simply a way to share anything I write for myself which I think might also be useful to other gentoo users.

Main item so far is gentool. Every so often an emerge will go wrong but gentool allows you to recover from that quickly and easily. It's not rocket science, but it is all packaged up in a handy little app which does all the hard work for you. A few other system-management-related features too.

The eagerly-awaited Ardour-irc allows you to control Ardour with an infra red controller. Please form an orderly queue at the download station. Both of you.

A simple but powerful command-line PVR may be added soon, and some more tools for managing an Ardour-based audio workstation.
Back to top
View user's profile Send private message
steveL
Watchman
Watchman


Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery

PostPosted: Mon May 21, 2012 8:31 am    Post subject: Reply with quote

Nice one mcgruff: gentool looks like a nice, simple approach, to snapshotting the distro.

I can't use it I'm afraid, as I have lots of split partitioning. But keep up the good work (I really should setup some audio tools soon, it's been far too long..)
Back to top
View user's profile Send private message
McGruff
Tux's lil' helper
Tux's lil' helper


Joined: 28 Dec 2004
Posts: 148

PostPosted: Mon May 21, 2012 9:20 pm    Post subject: Reply with quote

Gentool supports any number of data partitions out of the box.

It does assume that all OS & apps files are on "/" and /boot (plus some bits and pieces of data with nowhere better to go) but if you have some other kind of configuration please let me know the details.

I'd like to expand gentool to cover more use cases so anyone should feel free to request extra capabilities.
Back to top
View user's profile Send private message
steveL
Watchman
Watchman


Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery

PostPosted: Tue May 22, 2012 6:36 pm    Post subject: Reply with quote

The reason I called it "simple" was not to denigrate it, but because afaict it only allows one backup snapshot, since it's effectively just mirroring the root and /boot partitions? Sorry if I've misunderstood.

With regard to partitioning, I have separate partitions for just about everything in root that can be separate plus a few more. So /usr is split for a start (on LVM though that shouldn't matter) which definitely contains "OS files".

Regular partitions: /boot /home /tmp /opt
LVM partitions: /var /usr /usr/local /usr/src /usr/portage /usr/portage/distfiles

The ones under /usr don't matter in terms of OS snapshot (I assume you exclude src and portage anyhow, just like /tmp /home and /opt), though any solution should deal with /usr/local imo, as that is an important one for people who build their own system-wide software.

You can see the full fstab in this post if it helps any. Hmm I thought I had a separate /var/tmp: I usually set that up. Oh well, I've got lots of space so I'll sort it when I have some downtime :)

I certainly don't want to setup mirrors for each of root, /boot, /usr and /usr/local. As for /opt, which does get packages installed by portage, I guess that would be configure-in (if a separate partition) rather than default: some third-party hardware installs in there, which might be needed by the odd user (not by me though.)

I guess you'd just have a config file listing partitions and their mirror devices. If it's not listed, it doesn't get snapshotted. But having only one version is a bit of an issue, if something is broken in the previous version, which didn't get noticed; then you just end up using a standard backup solution.

FEATURES="buildpkg" is another very useful option, and deals nicely with eg a system package being broken for a couple of versions. I have PKGDIR="/var/pkg" so I suppose I'd want to back that up incrementally, as it'd be a disaster, albeit an ironic one, if a disk failure meant I lost my backup binpkgs.
Back to top
View user's profile Send private message
McGruff
Tux's lil' helper
Tux's lil' helper


Joined: 28 Dec 2004
Posts: 148

PostPosted: Tue May 22, 2012 9:42 pm    Post subject: Reply with quote

steveL wrote:
The reason I called it "simple" was not to denigrate it, but because afaict it only allows one backup snapshot, since it's effectively just mirroring the root and /boot partitions? Sorry if I've misunderstood.


You can do a...

Code:
$ gentool copy --dest="/path/to/dir"


...to copy OS & apps (ie "/" minus anything you've excluded in the config file) to any destination and hence backup multiple versions.

Of course only one backup will actually be bootable at any given moment, the one on the mirror partitions. However, I could add a capability to shuffle ordinary tar.bz2 backups on and off the mirrors. You could have as big a list of system versions as you like, and the option to choose which you want to make bootable.

It should also be fairly easy to add support for OS & apps spread around several partitions, eg in the partitioning scheme you posted. That's definitely something the program should do.
Back to top
View user's profile Send private message
hasufell
Retired Dev
Retired Dev


Joined: 29 Oct 2011
Posts: 429

PostPosted: Sun May 27, 2012 12:41 am    Post subject: Reply with quote

I had a quick look at "app-admin/gentool" and want to give a few comments on that if you don't mind:

- it is always good to follow the ebuild skeleton (/usr/portage/skel.ebuild) which you mostly do, however some newlines are common standard for readability such as:
Code:
EAPI=2

inherit eutils

DESCRIPTION=""
HOMEPAGE=""
SRC_URI=""

LICENSE=""
SLOT="0"
KEYWORDS="~amd64"
IUSE=""

RDEPEND="..."
DEPEND="..."

- I strongly encourage you to always set keywords in overlays to "unstable". Stabilization should only be done in the tree and people might get weird results when adding an overlay with several maybe even overlapping ebuilds.
- I don't see that you use any function from eutils.eclass, so you might want to drop it (we generally only inherit eclasses when we use functions from them). doenvd is an ebuild-helper not an eclass-function, see:
ls /usr/lib*/portage/bin/ebuild-helpers
- dependency, IUSE and KEYWORDS lists are mostly ordered alphabetically (this again just improves readability)
- ebuilds are basically bash scripts and we normally use bash code ( http://dev.gentoo.org/~ulm/pms/head/pms.html#x1-620006 )
- either one of dodir '/opt/gentool' or insinto '/opt/gentool' is redundant. IF you use insinto with later doins, then you don't need a dodir beforehand.
But you make use of cp here, so you need dodir. However cp should be generally avoided in favor of doins, dobin, doexe, dolib and fperms (check http://devmanual.gentoo.org/function-reference/install-functions/index.html)
doins -r also installs recursively if that is what you were missing
Also check: http://www.gentoo.org/proj/en/devrel/handbook/handbook.xml?part=2&chap=1#doc_chap4
You can use "into" in conjunction with "dobin" for example (see DESTTREE variable)
- ${S} is unqouted and ${D} too (use repoman full to check for common ebuild mistakes, developers also do)
- either provide a useful die message or none. "|| die" is just fine in most cases.
- I would use elog instead of ewarn in the last message, because those get logged and will always be re-displayed at the end of an emerge-process
- I don't get the purpose of your "ln -s" stuff. You can use dosym in src_install to create that symlink before the actual installation. dosym handles ${D} transparently. You might want to use something like qgrep -H dosym (part of app-portage/portage-utils) to see how ebuilds in the tree use it.
- you might want to drop those lines about PATH checking completely, see: grep opt/bin /etc/env.d/*
If the user tampered with that file, we can expect him to handle the situation himself.
- if you use EAPI=4 you can drop the "|| die" for helper functions (like emake, doins, doenvd). Check http://devmanual.gentoo.org/ebuild-writing/eapi/index.html beforehand. EAPI-bumps sometimes need adjustments to work.

You can visit #gentoo-dev-help and #gentoo-sunrise on freenode to ask for review/help if you like.

keep up the ebuild writing! :)
Back to top
View user's profile Send private message
McGruff
Tux's lil' helper
Tux's lil' helper


Joined: 28 Dec 2004
Posts: 148

PostPosted: Sun May 27, 2012 5:04 pm    Post subject: Reply with quote

Thanks for the tips. I'll take another look at the ebuild file and tidy it up.

At some point I'm also going to take a look at unit/acceptance testing for ebuilds. I've already made a start on a simple testing framework for bash scripts: bst.
Back to top
View user's profile Send private message
steveL
Watchman
Watchman


Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery

PostPosted: Sun May 27, 2012 9:24 pm    Post subject: Reply with quote

Nice idea mcgruff.

There's no need to force your users to type in $FILE $FUNCNAME $LINENO. The second one is an array, and you also get two other arrays, BASH_SOURCE and BASH_LINENO; the only tricky bit is that BASH_LINENO builds on top of LINENO, so indexes are one less than the corresponding FUNCNAME or BASH_SOURCE. So $BASH_LINENO (which is the same as ${BASH_LINENO[0]}) is the caller's line number, whose function is ${FUNCNAME[1]}.

So the three variables you want, to give you caller details corresponding to above, are: ${BASH_SOURCE[1]} ${FUNCNAME[1]} $BASH_LINENO and the user doesn't need to type in anything.

Yes, you can use these to get a stack-trace ;) I've done it, and it comes in handy when you're implementing complex scripts; I have an abort function I use in update which, if its first parameter is TRACE or an integer calls stackTrace 1 or 1+int, so abort 1 "unable to read input: $file" prints the stack trace from the caller's caller up, so a function called badly starts from its caller, as that's where the problem is, not in the function called.
Back to top
View user's profile Send private message
McGruff
Tux's lil' helper
Tux's lil' helper


Joined: 28 Dec 2004
Posts: 148

PostPosted: Sun May 27, 2012 9:45 pm    Post subject: Reply with quote

steveL wrote:
So the three variables you want, to give you caller details corresponding to above, are: ${BASH_SOURCE[1]} ${FUNCNAME[1]} $BASH_LINENO and the user doesn't need to type in anything.


Aha! :D I was hoping there would be something like that. I'm a bit of a bash noob, although I have done a lot of php programming.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum