Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
X86 to amd64 upgrade w/o starting from scratch
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
jesnow
Guru
Guru


Joined: 26 Apr 2006
Posts: 588

PostPosted: Thu Dec 20, 2018 1:11 am    Post subject: X86 to amd64 upgrade w/o starting from scratch Reply with quote

Boy, you guys don’t make it easy! I have several machines and all need to be migrated to 64 bit. So first I followed the recommended path and burned an install cd, copied the stage3 tarball and so forth. Hours later it’s still working its way through the first emerge -DNu world. And my machine is down and I’m on my phone. This is no good, especially at work, I can’t have a big downtime I need to work while it’s installing. So how am I going to manage the others? Here are my ideas. In each case I do have a new partition to install to:

1) install a cross compiled 64/32 bit kernel, boot my 32 bit userland with it and start the chroot to the new partition from a terminal window while I work.
2) the same using a kernel from the install cd (maybe easier).
3) boot the install cd and start my 32 bit userland from its own chroot and keep working. I got pretty far with this but it can’t seem to load the i915 video modules right for xorg from the chroot. I mount binded the lib64 on top of /lib on the chroot, butthat did not work. I think it would need compiled in drivers.
4) prepare a 64 bit bootable drive at home (once I have the install there finished) and bring it to work. I could use the world file from work and emerge -e at home so there would be minimal down time.
[edit:]
5) A stage 4 tarball extracted from my soon to be upgraded home system. Something that used to be a thing but I ha ent seen much about lately.

There really ought to be more support for more than just the classic 128K Mac solution of “just reinstall the OS from scratch”. It takes forever to get back to square zero doing that!

My laptop is a MacBook Air so at least that one will be easy. Not.

Your thoughts on which of these (or others) may be most successful and pointers to any beaten path lines with the skulls of foolish gentoo enthusiasts would be most welcome.

Cheers,

Jon.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 7127
Location: almost Mile High in the USA

PostPosted: Thu Dec 20, 2018 3:49 am    Post subject: Reply with quote

Problem is, you pretty much need to start from scratch. A half-assembled machine will not run. At best, the machine needs to be first migrated to a half multilib machine, and then the 64-bit stuff added - this mode is NOT supported.

At best you can work on a chroot with a cross compiler or run a 64-bit kernel on your existing install, but to make that new install "live" it will still require downtime, not much different than building on a home machine and migrating it in.

If you can't shutdown the machine for a complete reinstall, it's best to stay 32-bit. It still works...?

BTW, I have a remotely updated 32-bit installed machine that I would also like to move to 64-bit, alas, there's more involved if the machine is 1500 miles away...
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 13831

PostPosted: Thu Dec 20, 2018 4:41 am    Post subject: Reply with quote

Switching from x86 to amd64 is not maintained as an easy path because there is almost no value to it. People who are on x86 and cannot convert due to hardware limitations cannot benefit from such a path. People in that group who upgrade their hardware to support amd64 are relatively rare. People who started with amd64-capable hardware, but installed x86 anyway, are in an unfortunate position. People already on amd64 don't benefit. Even for those rare people who benefit, they benefit exactly once per system, and then they have an amd64 system and never need to think about it for that system again. So while yes, it is inconvenient for those people who want to switch, it's just not worth the developer time to create and maintain a migration path.
Back to top
View user's profile Send private message
Ant P.
Watchman
Watchman


Joined: 18 Apr 2009
Posts: 5761

PostPosted: Thu Dec 20, 2018 8:10 am    Post subject: Reply with quote

The traditional way to do this would be a cross-compiler and a stage1 install. But since you already have a running system, the easiest way would be to put an amd64 kernel on it and do a normal chroot stage3 from the existing OS.
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 7071

PostPosted: Thu Dec 20, 2018 1:57 pm    Post subject: Reply with quote

It's costly in time and tryouts ; you need to swtich to new /lib64;/lib structure which will put a high time cost on you, even with cross compiler.

You should:
built one 64bits env, quickpkg or emerge -B the packages your other host will need, migrate stuff you need to keep somewhere (mostly what is in /etc and other config files) then crush it with the new toolchain and all from the stage3 file.
Once done, copy back your saved config files... and then either use the other host as package provider or grab needed packages from it in order to fasten merge time.
kernel could be rebuild prior that in 64bits and it will gladly use 32bits userland tools while waiting the migration

making migration downtime reduce to only emerging binary packages, which is really low.
Back to top
View user's profile Send private message
runningnak3d
n00b
n00b


Joined: 05 Sep 2018
Posts: 57

PostPosted: Thu Dec 20, 2018 4:23 pm    Post subject: Reply with quote

Another option (if you have the disk space and are using LVM) is to create an LVM snapshot, and then use the snapshot with VirtualBox (you have to use the CLI to create the VMDK that is a pointer to the raw disk). You could use qemu / kvm just as easily though.

This way you can continue using your system, and pause the upgrade / migration if needed.

The only thing you will have to fix afterwards would be the X11 drivers -- and maybe recompile the kernel.

I converted a machine that was running Ubuntu this way, and it worked flawlessly. Granted, I was doing a fresh install, but the principal is the same -- I was able to continue to use the machine while everything emerged.

If you need the exact commands, I will be glad to provide them.

-- Brian
Back to top
View user's profile Send private message
jesnow
Guru
Guru


Joined: 26 Apr 2006
Posts: 588

PostPosted: Thu Dec 20, 2018 5:26 pm    Post subject: Reply with quote

Update: I’m now 24 hours in, got to a bootable system , copied /var/lib/portage/world, fixed various keyword and use issues and am now emerging world the second time. This may take a day or so, but this part I expected.

Thanks for the many suggestions. I really wish there had been some guidance on how to do this, it’s been very painful and frustrating. I guess it’s why they say gentoo isn’t for sissies. And maybe part of why gentoo is no longer #6 on distrowatch (does that even still exist?) which it was when I started so long ago. Why do I even stick with gentoo? Besides the usual reasons you can read anywhere, it’s what i know. And I mostly don’t need my hand held.

The main thing blocking me from doing anything but a bare metal install was my lack of a running amd64 system to prepare things on. Now that I have that, I’m going to do one of the options above for my work machine (20 miles away), not 1500 but still the building is locked and anyway it’s work) and take some notes. Cloning a world environment starting with a stage3, a world file and some etc files should not be this hard and it’s very different from a bare metal install.

Some observations:
LILO docs say you can use root=“LABEL=MyPartition”, do not do this. You will have to go old school and change lilo.conf just before booting like always. The append method does indeed work. Yes grub is better, no I won’t use it.

Mesa seems to require the wayland useflag even if you don’t use wayland. I don’t get that. I don’t have or want wayland Systemd or semantic-desktop no matter how wonderful they are and if you tell me anything starting with “just” about that, as in “just use them”, then <bad word> you and the horse you ride up on. There is no “just”.

Why does the standard install guide require deprecated hardware? I have no use for a dvd, I had to order raw disks just to do this! The standard guide should be usb based. And anyway my laptop doesn’t have one.

I think most people change environments and computers a lot, so “just” chucking the whole system when a major architecture change comes around is no problem. I remember similar levels of pain with past upgrades to glibc, gcc and the sunsetting of i386. Yes I’m that old. Those were also things you do exactly once, but lately those kinds of things hadn’t been a problem in gentoo. Well except for ruby and python versions, but let’s not speak of it.

“Just” sticking with x86 is what I was doing. But I was told that it too would be sunsetted someday and isn’t as well maintained as amd64. That makes a difference.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43178
Location: 56N 3W

PostPosted: Thu Dec 20, 2018 6:04 pm    Post subject: Reply with quote

jesnow,

Some years ago I recall a thread on the forum about migrating from 32 bit to 64 bit on AMD/Intel hardware. It did end in success.
My google foo is too weak to find it among all the reinstall advice though.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
mike155
Veteran
Veteran


Joined: 17 Sep 2010
Posts: 1294
Location: Frankfurt, Germany

PostPosted: Thu Dec 20, 2018 6:54 pm    Post subject: Reply with quote

Quote:
But I was told that it too would be sunsetted someday and isn’t as well maintained as amd64

I doubt x86 will disappear soon. Think about the embedded world. They will continue to use x86 for a long time, probably forever. There's not hurry to migrate to amd64. Stay with x86 and migrate to amd64 when you update your hardware...

I hope you were not fooled by rumors about removal of x32. x32 is completely different from x86.
Back to top
View user's profile Send private message
jesnow
Guru
Guru


Joined: 26 Apr 2006
Posts: 588

PostPosted: Thu Dec 20, 2018 7:52 pm    Post subject: Reply with quote

Success! At least partial.

It is possible once you have booted your new x86_64 partition to then chroot *back* into x86 userland and keep working while it does emerge -e world. This is good. I will call it method 6 (in my original post). The only procedure I had seen involved cross compiling the x86_64 kernel, booting it then chrooting *forward* to the new partition you just extracted the stage3 tarball into (that was method 1 in my original list before).

There's an extra step in starting the reverse chroot than usual: you must mount --bind /run/udev into place, then plasma will start.

I'm using that method now to type this! For some reason my network broke the first time I tried it -- I needed to re-link /etc/init.d/enp1s0.net in the old partition and restart the network, don't know why.

So, as a fastest practice, I am guessing now, call this method 7, which does not require a dvd player:
1) create a dedicated partition for the liveDVD, extract the image in there and boot.
2) Chroot back into the old x86 partition to keep working
3) In a virtual terminal (ctrl-alt-F2) follow the regular install guide to create your new install.

I think in my case since my home and work machines are very similar I can make a big tarfile of my current home x86_64 install and use that as a starting point instead of the stage3, this was at one time called a stage4 tarball as I mentioned earlier. [edit:] Oh look, that now exists here: https://wiki.gentoo.org/wiki/Stage_tarball but is different from what I recall, which was something you would extract from a running system.

I have often wished that the livecd kernel could be provided as a package so I could just emerge livecd-kernel, edit lilo.conf and run lilo to get that awful first boot behind me. Then I can build a kernel at my leisure. Maybe this also exists and I just didn't know it. It would save some headaches.

I think there are some crazier variants.

So please say it isn't so that I'm doing all this for nothing, I don't want to hear it at this point. LaLaLaLa I can't hear you.

Cheers,

Jon.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 7127
Location: almost Mile High in the USA

PostPosted: Thu Dec 20, 2018 8:00 pm    Post subject: Reply with quote

I have a mixture of 32 and 64 bit boxes, granted most of my 64-bit capable machines are running 64-bit. However I've noticed that the memory penalty with 64-bit versus 32-bit on some applications can be heavy (like portage/python!!), and would suggest anyone with less than 4GB RAM and especially 2GB to reconsider...

At 4GB RAM, PAE becomes an issue with 32-bit, and thus 64-bit becomes a benefit.
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?


Last edited by eccerr0r on Thu Dec 20, 2018 8:02 pm; edited 1 time in total
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43178
Location: 56N 3W

PostPosted: Thu Dec 20, 2018 8:01 pm    Post subject: Reply with quote

jesnow,

Whatever you do with Gentoo always has educational value, even if all you learn is not to do it again :)
Its never for nothing.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
jesnow
Guru
Guru


Joined: 26 Apr 2006
Posts: 588

PostPosted: Thu Dec 20, 2018 8:32 pm    Post subject: Reply with quote

So here's a relatively painless upgrade path, now that it's years past anybody needing one. Call this method 8, it's like method 1 from my original post that saves all the cross-compiling:

1) Acquire/clear a new partition to install in.
2) Acquire and install a new x86_64 kernel in the *old* x86 partition. e.g. I did manage to extract one off the livedvd image.
3) Boot that kernel in the *old* x86 partition, and back to work. Total downtime 5 minutes.
4) Now extract the amd64 stage3 tarball into the new partition, get portage snapshot, emerge a couple other things.
5) Instead of laboriously configuring everything as if it was a new system, copy over only the config files you've ever touched (eg /var/lib/portage/world), or use this handy dandy script or single line of perl code that it's beyond me to write.
6) Now you only need to edit make.conf, fstab, and lilo (or grub).conf.
7) compile and install the new real kernel in the new partition using your old .config
8) First boot into new partition and kernel, with the old one as a fallback.
9) emerge -e system && emerge -e world
10) Bob's your uncle.

By the way I noticed that emerge is much faster now in the new system. *Much* faster.

Comments?

Jon.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43178
Location: 56N 3W

PostPosted: Thu Dec 20, 2018 9:14 pm    Post subject: Reply with quote

jesnow,

Quote:
2) Acquire and install a new x86_64 kernel in the *old* x86 partition. e.g. I did manage to extract one off the livedvd image.

A word of caution an a little expansion here.

The kernel modules need to go into /lib/modules/'uname -r'/ That's the same path regardless of the kernel being 32 bit or 64 bit.
Its a very bad thing to overwrite your 32 bit modules with 64 bit ones.
You must ensure that the 'uname -r' for the 64 bit kernel is different to the 32 bit kernel. That means making sure that they are different versions.
You need the initrd too.

Not all 64 bit kernels will work. CONFIG_IA32_EMULATION must be set in the 64 bit kernel, or it won't run 32 bit code.

My preference would be to make the space for the new partition, get the stage3 onto it then build my own kernel.
That takes slightly more downtime, since you need to boot a 64 bit kernel to run the 64 bit tools to build your own kernel but all the prep can be done from the 32 bit system, including the make menuconfig.

You can bind mount your 32 bit /usr/portage into the 64 bit chroot, so you don't need to fetch a snapshot or any distfiles until later.

Thank you for a good outline guide.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 13831

PostPosted: Fri Dec 21, 2018 3:00 am    Post subject: Reply with quote

jesnow wrote:
Mesa seems to require the wayland useflag even if you don’t use wayland. I don’t get that. I don’t have or want wayland Systemd or semantic-desktop no matter how wonderful they are and if you tell me anything starting with “just” about that, as in “just use them”, then <bad word> you and the horse you ride up on. There is no “just”.
Please open a separate thread for this and post the usual diagnostics.
Back to top
View user's profile Send private message
jesnow
Guru
Guru


Joined: 26 Apr 2006
Posts: 588

PostPosted: Fri Dec 21, 2018 3:23 pm    Post subject: Reply with quote

That list is for sure skipping several steps, it's for people who know what they are doing. I will do a slightly more elaborate version of it with more warnings shortly. The dance of the hard drives (ie getting your new drive to be /dev/sda without bricking your system) is always tricky. Modern computers don't really care about this but still.

Yes, overwriting a kernel would be bad for sure. I normally keep several kernels around, and so the x86_64 kernel and modules would exist next to their 32 bit cousins. There is a way of naming kernels depending on how they're compiled, or just only use that specific kernel version in its x86_64 persona. The livecd uses an initramdisk, so that's a complication too.

In case things go wrong, you've got a fully bootable x86 partition still there, that's a nice dividend.
The variants where you boot from a livedvd extracted to its own partition, you can convert that to swap afterward. Or keep it as a rescue install. I only run stable as much as possible so I rarely need rescuing.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43178
Location: 56N 3W

PostPosted: Fri Dec 21, 2018 5:14 pm    Post subject: Reply with quote

jesnow,

Investigate using PARTUUID in your bootloader. Then HDD can came and go and it all just works.
Use UUID, PARTUUID or LABLE in fstab and again, its all unique.

Kernel device names are allocated on a first come first served order. Its possible to have a race between different drivers so that your kernel device names vary from kernel to kernel.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
jesnow
Guru
Guru


Joined: 26 Apr 2006
Posts: 588

PostPosted: Sun Dec 23, 2018 5:45 pm    Post subject: Reply with quote

Thanks NeddySeagoon!

I've now succeeded with the x86_64 conversion, which was from scratch onto a new hard drive this time. It all worked fine. What I did was chroot *back* into my x86 partition, then run startx, which is how I start the window system anyway. In order to do that, when you're mount --bind -ing stuff you have to:

Code:

# mount --bind /run/udev /mnt/old/run/udev


This step isn't well documented as far as I can tell. If you don't do that then the mouse and keyboard don't work, and unless you can ssh in you have to hard boot. Which isn't a nice thing to do to a system that's busy emerging something. In that case pulling the plug could actually mess an sdd up. I think it's better to pull the cable first (they are hot pluggable).

As you say, the dance of the hard drives isn't so necessary with using PARTUUID, still that's a little tricky with lilo. You can't use root="LABEL=NewRoot" in lilo.conf even though some of the documentation says you now can. You still need to do append="root=PARTUUID=000bbdeb-02" and this works (yes with two equals signs). Being old school I want my usual boot disk to be /dev/sda, but maybe I should just relax. Currently if I'm booting off of /dev/sdb, I install the bootloader in /dev/sda pointing at a root partition on /dev/sdb. Lilo warns about this but does it. Is it better for each hdd to have its own bootloader in the mbr? Or one bootloader for all the bootable partitions on all the drives in the system? You could accidentally overwrite your bootloader from an old lilo.conf on the wrong partition and make a mess. I'm trying to figure out whats a best practice. I think this is for a new thread.

My work machine is as far as I can get it without physically touching it. The new partition has a new stage3 and new x86_64 kernels in both old and new partitions. I should be able to reboot the old partition, chroot to the new partition and start building. For the moment I've left the step in of emerge -DNu world of the stage3 *before* copying over world and /etc/* but I don't know why it's necessary. Just you want to be at a well defined place at each step.

In retrospect, *NOT* upgrading in place also serves to remove a lot of cruft. My new x86_64 partition is half the size of my old one, with the exact same world files.

Cheers,

Jon.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum