Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
X86 to amd64 upgrade w/o starting from scratch
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
jesnow
l33t
l33t


Joined: 26 Apr 2006
Posts: 643

PostPosted: Thu Dec 20, 2018 1:11 am    Post subject: X86 to amd64 upgrade w/o starting from scratch Reply with quote

Boy, you guys don’t make it easy! I have several machines and all need to be migrated to 64 bit. So first I followed the recommended path and burned an install cd, copied the stage3 tarball and so forth. Hours later it’s still working its way through the first emerge -DNu world. And my machine is down and I’m on my phone. This is no good, especially at work, I can’t have a big downtime I need to work while it’s installing. So how am I going to manage the others? Here are my ideas. In each case I do have a new partition to install to:

1) install a cross compiled 64/32 bit kernel, boot my 32 bit userland with it and start the chroot to the new partition from a terminal window while I work.
2) the same using a kernel from the install cd (maybe easier).
3) boot the install cd and start my 32 bit userland from its own chroot and keep working. I got pretty far with this but it can’t seem to load the i915 video modules right for xorg from the chroot. I mount binded the lib64 on top of /lib on the chroot, butthat did not work. I think it would need compiled in drivers.
4) prepare a 64 bit bootable drive at home (once I have the install there finished) and bring it to work. I could use the world file from work and emerge -e at home so there would be minimal down time.
[edit:]
5) A stage 4 tarball extracted from my soon to be upgraded home system. Something that used to be a thing but I ha ent seen much about lately.

There really ought to be more support for more than just the classic 128K Mac solution of “just reinstall the OS from scratch”. It takes forever to get back to square zero doing that!

My laptop is a MacBook Air so at least that one will be easy. Not.

Your thoughts on which of these (or others) may be most successful and pointers to any beaten path lines with the skulls of foolish gentoo enthusiasts would be most welcome.

Cheers,

Jon.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 7486
Location: almost Mile High in the USA

PostPosted: Thu Dec 20, 2018 3:49 am    Post subject: Reply with quote

Problem is, you pretty much need to start from scratch. A half-assembled machine will not run. At best, the machine needs to be first migrated to a half multilib machine, and then the 64-bit stuff added - this mode is NOT supported.

At best you can work on a chroot with a cross compiler or run a 64-bit kernel on your existing install, but to make that new install "live" it will still require downtime, not much different than building on a home machine and migrating it in.

If you can't shutdown the machine for a complete reinstall, it's best to stay 32-bit. It still works...?

BTW, I have a remotely updated 32-bit installed machine that I would also like to move to 64-bit, alas, there's more involved if the machine is 1500 miles away...
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 15448

PostPosted: Thu Dec 20, 2018 4:41 am    Post subject: Reply with quote

Switching from x86 to amd64 is not maintained as an easy path because there is almost no value to it. People who are on x86 and cannot convert due to hardware limitations cannot benefit from such a path. People in that group who upgrade their hardware to support amd64 are relatively rare. People who started with amd64-capable hardware, but installed x86 anyway, are in an unfortunate position. People already on amd64 don't benefit. Even for those rare people who benefit, they benefit exactly once per system, and then they have an amd64 system and never need to think about it for that system again. So while yes, it is inconvenient for those people who want to switch, it's just not worth the developer time to create and maintain a migration path.
Back to top
View user's profile Send private message
Ant P.
Watchman
Watchman


Joined: 18 Apr 2009
Posts: 6563

PostPosted: Thu Dec 20, 2018 8:10 am    Post subject: Reply with quote

The traditional way to do this would be a cross-compiler and a stage1 install. But since you already have a running system, the easiest way would be to put an amd64 kernel on it and do a normal chroot stage3 from the existing OS.
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 7412

PostPosted: Thu Dec 20, 2018 1:57 pm    Post subject: Reply with quote

It's costly in time and tryouts ; you need to swtich to new /lib64;/lib structure which will put a high time cost on you, even with cross compiler.

You should:
built one 64bits env, quickpkg or emerge -B the packages your other host will need, migrate stuff you need to keep somewhere (mostly what is in /etc and other config files) then crush it with the new toolchain and all from the stage3 file.
Once done, copy back your saved config files... and then either use the other host as package provider or grab needed packages from it in order to fasten merge time.
kernel could be rebuild prior that in 64bits and it will gladly use 32bits userland tools while waiting the migration

making migration downtime reduce to only emerging binary packages, which is really low.
Back to top
View user's profile Send private message
runningnak3d
n00b
n00b


Joined: 05 Sep 2018
Posts: 61

PostPosted: Thu Dec 20, 2018 4:23 pm    Post subject: Reply with quote

Another option (if you have the disk space and are using LVM) is to create an LVM snapshot, and then use the snapshot with VirtualBox (you have to use the CLI to create the VMDK that is a pointer to the raw disk). You could use qemu / kvm just as easily though.

This way you can continue using your system, and pause the upgrade / migration if needed.

The only thing you will have to fix afterwards would be the X11 drivers -- and maybe recompile the kernel.

I converted a machine that was running Ubuntu this way, and it worked flawlessly. Granted, I was doing a fresh install, but the principal is the same -- I was able to continue to use the machine while everything emerged.

If you need the exact commands, I will be glad to provide them.

-- Brian
Back to top
View user's profile Send private message
jesnow
l33t
l33t


Joined: 26 Apr 2006
Posts: 643

PostPosted: Thu Dec 20, 2018 5:26 pm    Post subject: Reply with quote

Update: I’m now 24 hours in, got to a bootable system , copied /var/lib/portage/world, fixed various keyword and use issues and am now emerging world the second time. This may take a day or so, but this part I expected.

Thanks for the many suggestions. I really wish there had been some guidance on how to do this, it’s been very painful and frustrating. I guess it’s why they say gentoo isn’t for sissies. And maybe part of why gentoo is no longer #6 on distrowatch (does that even still exist?) which it was when I started so long ago. Why do I even stick with gentoo? Besides the usual reasons you can read anywhere, it’s what i know. And I mostly don’t need my hand held.

The main thing blocking me from doing anything but a bare metal install was my lack of a running amd64 system to prepare things on. Now that I have that, I’m going to do one of the options above for my work machine (20 miles away), not 1500 but still the building is locked and anyway it’s work) and take some notes. Cloning a world environment starting with a stage3, a world file and some etc files should not be this hard and it’s very different from a bare metal install.

Some observations:
LILO docs say you can use root=“LABEL=MyPartition”, do not do this. You will have to go old school and change lilo.conf just before booting like always. The append method does indeed work. Yes grub is better, no I won’t use it.

Mesa seems to require the wayland useflag even if you don’t use wayland. I don’t get that. I don’t have or want wayland Systemd or semantic-desktop no matter how wonderful they are and if you tell me anything starting with “just” about that, as in “just use them”, then <bad word> you and the horse you ride up on. There is no “just”.

Why does the standard install guide require deprecated hardware? I have no use for a dvd, I had to order raw disks just to do this! The standard guide should be usb based. And anyway my laptop doesn’t have one.

I think most people change environments and computers a lot, so “just” chucking the whole system when a major architecture change comes around is no problem. I remember similar levels of pain with past upgrades to glibc, gcc and the sunsetting of i386. Yes I’m that old. Those were also things you do exactly once, but lately those kinds of things hadn’t been a problem in gentoo. Well except for ruby and python versions, but let’s not speak of it.

“Just” sticking with x86 is what I was doing. But I was told that it too would be sunsetted someday and isn’t as well maintained as amd64. That makes a difference.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 45608
Location: 56N 3W

PostPosted: Thu Dec 20, 2018 6:04 pm    Post subject: Reply with quote

jesnow,

Some years ago I recall a thread on the forum about migrating from 32 bit to 64 bit on AMD/Intel hardware. It did end in success.
My google foo is too weak to find it among all the reinstall advice though.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
mike155
Advocate
Advocate


Joined: 17 Sep 2010
Posts: 2170
Location: Frankfurt, Germany

PostPosted: Thu Dec 20, 2018 6:54 pm    Post subject: Reply with quote

Quote:
But I was told that it too would be sunsetted someday and isn’t as well maintained as amd64

I doubt x86 will disappear soon. Think about the embedded world. They will continue to use x86 for a long time, probably forever. There's not hurry to migrate to amd64. Stay with x86 and migrate to amd64 when you update your hardware...

I hope you were not fooled by rumors about removal of x32. x32 is completely different from x86.
Back to top
View user's profile Send private message
jesnow
l33t
l33t


Joined: 26 Apr 2006
Posts: 643

PostPosted: Thu Dec 20, 2018 7:52 pm    Post subject: Reply with quote

Success! At least partial.

It is possible once you have booted your new x86_64 partition to then chroot *back* into x86 userland and keep working while it does emerge -e world. This is good. I will call it method 6 (in my original post). The only procedure I had seen involved cross compiling the x86_64 kernel, booting it then chrooting *forward* to the new partition you just extracted the stage3 tarball into (that was method 1 in my original list before).

There's an extra step in starting the reverse chroot than usual: you must mount --bind /run/udev into place, then plasma will start.

I'm using that method now to type this! For some reason my network broke the first time I tried it -- I needed to re-link /etc/init.d/enp1s0.net in the old partition and restart the network, don't know why.

So, as a fastest practice, I am guessing now, call this method 7, which does not require a dvd player:
1) create a dedicated partition for the liveDVD, extract the image in there and boot.
2) Chroot back into the old x86 partition to keep working
3) In a virtual terminal (ctrl-alt-F2) follow the regular install guide to create your new install.

I think in my case since my home and work machines are very similar I can make a big tarfile of my current home x86_64 install and use that as a starting point instead of the stage3, this was at one time called a stage4 tarball as I mentioned earlier. [edit:] Oh look, that now exists here: https://wiki.gentoo.org/wiki/Stage_tarball but is different from what I recall, which was something you would extract from a running system.

I have often wished that the livecd kernel could be provided as a package so I could just emerge livecd-kernel, edit lilo.conf and run lilo to get that awful first boot behind me. Then I can build a kernel at my leisure. Maybe this also exists and I just didn't know it. It would save some headaches.

I think there are some crazier variants.

So please say it isn't so that I'm doing all this for nothing, I don't want to hear it at this point. LaLaLaLa I can't hear you.

Cheers,

Jon.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 7486
Location: almost Mile High in the USA

PostPosted: Thu Dec 20, 2018 8:00 pm    Post subject: Reply with quote

I have a mixture of 32 and 64 bit boxes, granted most of my 64-bit capable machines are running 64-bit. However I've noticed that the memory penalty with 64-bit versus 32-bit on some applications can be heavy (like portage/python!!), and would suggest anyone with less than 4GB RAM and especially 2GB to reconsider...

At 4GB RAM, PAE becomes an issue with 32-bit, and thus 64-bit becomes a benefit.
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?


Last edited by eccerr0r on Thu Dec 20, 2018 8:02 pm; edited 1 time in total
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 45608
Location: 56N 3W

PostPosted: Thu Dec 20, 2018 8:01 pm    Post subject: Reply with quote

jesnow,

Whatever you do with Gentoo always has educational value, even if all you learn is not to do it again :)
Its never for nothing.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
jesnow
l33t
l33t


Joined: 26 Apr 2006
Posts: 643

PostPosted: Thu Dec 20, 2018 8:32 pm    Post subject: Reply with quote

So here's a relatively painless upgrade path, now that it's years past anybody needing one. Call this method 8, it's like method 1 from my original post that saves all the cross-compiling:

1) Acquire/clear a new partition to install in.
2) Acquire and install a new x86_64 kernel in the *old* x86 partition. e.g. I did manage to extract one off the livedvd image.
3) Boot that kernel in the *old* x86 partition, and back to work. Total downtime 5 minutes.
4) Now extract the amd64 stage3 tarball into the new partition, get portage snapshot, emerge a couple other things.
5) Instead of laboriously configuring everything as if it was a new system, copy over only the config files you've ever touched (eg /var/lib/portage/world), or use this handy dandy script or single line of perl code that it's beyond me to write.
6) Now you only need to edit make.conf, fstab, and lilo (or grub).conf.
7) compile and install the new real kernel in the new partition using your old .config
8) First boot into new partition and kernel, with the old one as a fallback.
9) emerge -e system && emerge -e world
10) Bob's your uncle.

By the way I noticed that emerge is much faster now in the new system. *Much* faster.

Comments?

Jon.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 45608
Location: 56N 3W

PostPosted: Thu Dec 20, 2018 9:14 pm    Post subject: Reply with quote

jesnow,

Quote:
2) Acquire and install a new x86_64 kernel in the *old* x86 partition. e.g. I did manage to extract one off the livedvd image.

A word of caution an a little expansion here.

The kernel modules need to go into /lib/modules/'uname -r'/ That's the same path regardless of the kernel being 32 bit or 64 bit.
Its a very bad thing to overwrite your 32 bit modules with 64 bit ones.
You must ensure that the 'uname -r' for the 64 bit kernel is different to the 32 bit kernel. That means making sure that they are different versions.
You need the initrd too.

Not all 64 bit kernels will work. CONFIG_IA32_EMULATION must be set in the 64 bit kernel, or it won't run 32 bit code.

My preference would be to make the space for the new partition, get the stage3 onto it then build my own kernel.
That takes slightly more downtime, since you need to boot a 64 bit kernel to run the 64 bit tools to build your own kernel but all the prep can be done from the 32 bit system, including the make menuconfig.

You can bind mount your 32 bit /usr/portage into the 64 bit chroot, so you don't need to fetch a snapshot or any distfiles until later.

Thank you for a good outline guide.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 15448

PostPosted: Fri Dec 21, 2018 3:00 am    Post subject: Reply with quote

jesnow wrote:
Mesa seems to require the wayland useflag even if you don’t use wayland. I don’t get that. I don’t have or want wayland Systemd or semantic-desktop no matter how wonderful they are and if you tell me anything starting with “just” about that, as in “just use them”, then <bad word> you and the horse you ride up on. There is no “just”.
Please open a separate thread for this and post the usual diagnostics.
Back to top
View user's profile Send private message
jesnow
l33t
l33t


Joined: 26 Apr 2006
Posts: 643

PostPosted: Fri Dec 21, 2018 3:23 pm    Post subject: Reply with quote

That list is for sure skipping several steps, it's for people who know what they are doing. I will do a slightly more elaborate version of it with more warnings shortly. The dance of the hard drives (ie getting your new drive to be /dev/sda without bricking your system) is always tricky. Modern computers don't really care about this but still.

Yes, overwriting a kernel would be bad for sure. I normally keep several kernels around, and so the x86_64 kernel and modules would exist next to their 32 bit cousins. There is a way of naming kernels depending on how they're compiled, or just only use that specific kernel version in its x86_64 persona. The livecd uses an initramdisk, so that's a complication too.

In case things go wrong, you've got a fully bootable x86 partition still there, that's a nice dividend.
The variants where you boot from a livedvd extracted to its own partition, you can convert that to swap afterward. Or keep it as a rescue install. I only run stable as much as possible so I rarely need rescuing.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 45608
Location: 56N 3W

PostPosted: Fri Dec 21, 2018 5:14 pm    Post subject: Reply with quote

jesnow,

Investigate using PARTUUID in your bootloader. Then HDD can came and go and it all just works.
Use UUID, PARTUUID or LABLE in fstab and again, its all unique.

Kernel device names are allocated on a first come first served order. Its possible to have a race between different drivers so that your kernel device names vary from kernel to kernel.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
jesnow
l33t
l33t


Joined: 26 Apr 2006
Posts: 643

PostPosted: Sun Dec 23, 2018 5:45 pm    Post subject: Reply with quote

Thanks NeddySeagoon!

I've now succeeded with the x86_64 conversion, which was from scratch onto a new hard drive this time. It all worked fine. What I did was chroot *back* into my x86 partition, then run startx, which is how I start the window system anyway. In order to do that, when you're mount --bind -ing stuff you have to:

Code:

# mount --bind /run/udev /mnt/old/run/udev


This step isn't well documented as far as I can tell. If you don't do that then the mouse and keyboard don't work, and unless you can ssh in you have to hard boot. Which isn't a nice thing to do to a system that's busy emerging something. In that case pulling the plug could actually mess an sdd up. I think it's better to pull the cable first (they are hot pluggable).

As you say, the dance of the hard drives isn't so necessary with using PARTUUID, still that's a little tricky with lilo. You can't use root="LABEL=NewRoot" in lilo.conf even though some of the documentation says you now can. You still need to do append="root=PARTUUID=000bbdeb-02" and this works (yes with two equals signs). Being old school I want my usual boot disk to be /dev/sda, but maybe I should just relax. Currently if I'm booting off of /dev/sdb, I install the bootloader in /dev/sda pointing at a root partition on /dev/sdb. Lilo warns about this but does it. Is it better for each hdd to have its own bootloader in the mbr? Or one bootloader for all the bootable partitions on all the drives in the system? You could accidentally overwrite your bootloader from an old lilo.conf on the wrong partition and make a mess. I'm trying to figure out whats a best practice. I think this is for a new thread.

My work machine is as far as I can get it without physically touching it. The new partition has a new stage3 and new x86_64 kernels in both old and new partitions. I should be able to reboot the old partition, chroot to the new partition and start building. For the moment I've left the step in of emerge -DNu world of the stage3 *before* copying over world and /etc/* but I don't know why it's necessary. Just you want to be at a well defined place at each step.

In retrospect, *NOT* upgrading in place also serves to remove a lot of cruft. My new x86_64 partition is half the size of my old one, with the exact same world files.

Cheers,

Jon.
Back to top
View user's profile Send private message
pa4wdh
Guru
Guru


Joined: 16 Dec 2005
Posts: 370

PostPosted: Tue Jul 23, 2019 1:17 pm    Post subject: Reply with quote

I just performed a similar migration from x86 to amd64 on a VPS, my approach was a bit different so I'll share it here so it might help someone.

Step 1: Get a 64bit kernel
I always compile my own kernels, without modules and only the stuff i really need. I copied the kernels .config over to a system already running 64bit and used the .config to build a kernel for the system to be migrated. Make sure to build it 64bit and enable 32bit emulation, I'll need it during the migration.
When the compiling is ready copy the resulting kernel over to the system to be migrated and add it to the bootloader as a new entry so you can always boot your old kernel if you need.
In my case the bootloader is pvgrub and it needs to match the architecture of the kernel, so in order to be able to boot the 64bit kernel i also needed to change to a 64bit pvgrub. After that was done my fresh 64bit kernel booted my x86 userland flawlessly. You can verify this by using uname -m, which should now indicate x86_64. All your usual x86 applications should be running.

Step 2: Prepare 64bit environment
Create a new partition and start a clean stage3 install as described in the handbook. Follow the handbook upto the point where you chroot. Tip: use a screen session to start the chroot in, it allows you to disconnect and go on where you left later.
Copy the portage tree, /var/lib/portage/world, /etc/portage and /etc/password, /etc/group, /etc/shadow, /etc/gshadow from your x86 to the amd64 install. Give /etc/portage/make.conf a bit of extra care, the CHOST should be different.
Now give your CPU something to work on and emerge -e world. In my case dev-libs/boost gave some problems which I solved by temporarily setting MAKEOPTS="-j1" (instead of -j3).

Step 3: Copy configuration and other data
When everything is installed you can use diff -qr to find differences in configuration between your x84 and amd64 install and fix them where needed. Beware that some configuration contains the architecture name which of course should be different.
Now copy any data from /var, /opt or /home and other locations that needs to move to the new install.

Step 4: Boot into amd64 userland
Now add a new entry to your bootloader with your 64bit kernel and the partition of your amd64 install as parameter for root=
Keep your fingers crossed and reboot into your amd64 userland, if you did everything correct you should have a booting amd64 userland which is very similar to your x86 userland.

Step 5: Clean up (still on my to-do list :))
Clean up the x86 userland when amd64 works the way you want. Since i did a no-multilib install i could also disable 32bit emulation in the kernel, i guess i need to build a amd64 initramfs before i can do that.

Total downtime: 2 reboots
Total work: Quite some hours :) Especially step 3 requires quite some manual work.
_________________
The gentoo way of bringing peace to the world:
USE="-war" emerge --newuse @world

Free as in Freedom is not limited to software only:
Music: http://www.jamendo.com
Recipes: http://www.opensourcefood.com
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum