Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Scripting parition creation
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
grant123
l33t
l33t


Joined: 23 Mar 2005
Posts: 989

PostPosted: Fri Feb 28, 2014 7:34 pm    Post subject: Scripting parition creation Reply with quote

I'm scripting partition creation like this:

Code:
fdisk /dev/sda <<EOF
d
1
d

n
p
1
2048
+64M

a
1

n
p
2
133120
250069679

w
EOF


Can this be made more dynamic? For example, how can I handle disks of various sizes and containing any number of partitions to be deleted?
Back to top
View user's profile Send private message
John R. Graham
Administrator
Administrator


Joined: 08 Mar 2005
Posts: 10275
Location: Somewhere over Atlanta, Georgia

PostPosted: Fri Feb 28, 2014 7:48 pm    Post subject: Reply with quote

Kind of dangerous but you could delete the partition table with dd before you start. Should be okay since that's your intent anyway.

- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters.
Back to top
View user's profile Send private message
frostschutz
Advocate
Advocate


Joined: 22 Feb 2005
Posts: 2971
Location: Germany

PostPosted: Fri Feb 28, 2014 10:39 pm    Post subject: Reply with quote

in fdisk? o instead of d

parted can be used for scripting as well.

disk size is blockdev --getsize64 /dev/disk
Back to top
View user's profile Send private message
grant123
l33t
l33t


Joined: 23 Mar 2005
Posts: 989

PostPosted: Fri Feb 28, 2014 10:44 pm    Post subject: Reply with quote

Quote:
Kind of dangerous but you could delete the partition table with dd before you start. Should be okay since that's your intent anyway.

Thanks, that should take care of deleting current paritions:

Code:
dd if=/dev/zero bs=1 seek=446 count=64 conv=notrunc of=dd_image_file

Any idea on choosing default (like pressing Enter) instead of 133120 and 250069679?


Last edited by grant123 on Fri Feb 28, 2014 10:50 pm; edited 1 time in total
Back to top
View user's profile Send private message
grant123
l33t
l33t


Joined: 23 Mar 2005
Posts: 989

PostPosted: Fri Feb 28, 2014 10:50 pm    Post subject: Reply with quote

Quote:
in fdisk? o instead of d

I like that better. "DOS partition table" is synonymous with "partition table" then?

Quote:
disk size is blockdev --getsize64 /dev/disk

That's strange. I get a very different number from this as compared to "End" reported in fdisk.
Back to top
View user's profile Send private message
maxim.251
Tux's lil' helper
Tux's lil' helper


Joined: 14 Jul 2012
Posts: 87

PostPosted: Sat Mar 01, 2014 12:28 am    Post subject: Reply with quote

[/code]
Code:
# Configure hard disk
echo "The following is a list of found Hard Disk devices (sda/hda):"
ls /dev/ | grep -E [s:h]da$
echo
echo "Please enter the device you would like to install to:"
read HD_DEVICE
HDD="/dev/${HD_DEVICE}"     #<--- hda or hdb or sda or sdc

# warn user last time
echo
echo "THIS IS YOUR LAST CHANCE TO CANCEL, Press CTRL+C to cancel this install or ENTER to continue"
read
echo "p" | fdisk $HDD       #<---- is wery dengerous, is deleting ewerything what you have on disk. do not use if you have system on de disk.                     #<----- denger
echo "Please specifiy the total number of partitions:"          #<---- denger
read PARTS                                                                     #<----- denger,

#TODO: add integer checking
echo -n "Generating fdisk operation file"
# Create fdisc auto file
((i = 1))
while (( i < PARTS ))
do
echo "d" >> fdisc.in
echo "$i" >> fdisc.in
((i += 1))
done
echo "d"    >> fdisc.in   # Delete last sector     
echo "n"    >> fdisc.in   # New Partiton
echo "p"    >> fdisc.in   # Primary
echo "1"    >> fdisc.in   # Partion 1
echo ""    >> fdisc.in   # default
echo "+32M"    >> fdisc.in   # 32 MB size
echo "a"    >> fdisc.in   # Set flag
echo "1"    >> fdisc.in   # bootable
echo -n "."
echo "n"    >> fdisc.in   # New Partion
echo "p"    >> fdisc.in   # Primary
echo "2"    >> fdisc.in   # Partion 2
echo ""    >> fdisc.in   # default
echo "+512M"    >> fdisc.in   # 512 MB size
echo "t"    >> fdisc.in   # Set partition type
echo "2"    >> fdisc.in   # Partition 2
echo "82"    >> fdisc.in   # 82 = SWAP
echo -n "."
echo "n"    >> fdisc.in   # New Partition
echo "p"    >> fdisc.in   # Primary
echo "3"    >> fdisc.in   # Partition 2
echo ""    >> fdisc.in   # default
echo ""    >> fdisc.in   # new Line
echo -n "."
echo "w"    >> fdisc.in   # Write partion table
echo "q"    >> fdisc.in   # Quit
echo ". Done"
# Execute file
echo "Executing fdisk script ..."
echo
fdisk $HDD < fdisc.in
#clean up
#rm -f fdisc.in 
echo ""
echo "Partions created"
echo "Applying filesystem to partitions"
mke2fs /dev/${HD_DEVICE}1
mke2fs -j /dev/${HD_DEVICE}3
mkswap /dev/${HD_DEVICE}2
echo "Activating swap partition"
swapon /dev/${HD_DEVICE}2



Be cerfull, if you do not know how is works this code, but it kan bee helpfull for you.
Back to top
View user's profile Send private message
maxim.251
Tux's lil' helper
Tux's lil' helper


Joined: 14 Jul 2012
Posts: 87

PostPosted: Sat Mar 01, 2014 12:36 am    Post subject: Reply with quote

you have an error in your
Code:
fdisk /dev/sda <<EOF
d
1
d

n
p
1
2048
+64M

a
1

n
p
2
133120
250069679

w
EOF


fdisk /dev/sda <<-EOF


youre code


My egzampel is lookt like this
Code:
passwd<<-PAS
1
1
PAS
echo "twoje hasło to 1"'

passwd maxim<<-PAS1
1
1
PAS1
Back to top
View user's profile Send private message
grant123
l33t
l33t


Joined: 23 Mar 2005
Posts: 989

PostPosted: Sat Mar 01, 2014 8:32 pm    Post subject: Reply with quote

Thank you, but where is the error in my code?
Back to top
View user's profile Send private message
fpemud
Apprentice
Apprentice


Joined: 15 Feb 2012
Posts: 276

PostPosted: Sat Mar 01, 2014 10:39 pm    Post subject: Reply with quote

I'm using python to do this kind of work in my project.
I think you can parse the output of "fdisk -l" for dynamic scenario.

Code:

class FvmUtil:

   @staticmethod
   def shell(cmd, flags=""):
      """Execute shell command"""

      assert cmd.startswith("/")

      # Execute shell command, throws exception when failed
      if flags == "":
         retcode = subprocess.Popen(cmd, shell = True).wait()
         if retcode != 0:
            raise Exception("Executing shell command \"%s\" failed, return code %d"%(cmd, retcode))
         return

      # Execute shell command, throws exception when failed, returns stdout+stderr
      if flags == "stdout":
         proc = subprocess.Popen(cmd,
                           shell = True,
                           stdout = subprocess.PIPE,
                           stderr = subprocess.STDOUT)
         out = proc.communicate()[0]
         if proc.returncode != 0:
            raise Exception("Executing shell command \"%s\" failed, return code %d, output %s"%(cmd, proc.returncode, out))
         return out

      # Execute shell command, returns (returncode,stdout+stderr)
      if flags == "retcode+stdout":
         proc = subprocess.Popen(cmd,
                           shell = True,
                           stdout = subprocess.PIPE,
                           stderr = subprocess.STDOUT)
         out = proc.communicate()[0]
         return (proc.returncode, out)

      assert False

   @staticmethod
   def shellInteractive(cmd, strInput, flags=""):
      """Execute shell command with input interaction"""

      assert cmd.startswith("/")

      # Execute shell command, throws exception when failed
      if flags == "":
         proc = subprocess.Popen(cmd,
                           shell = True,
                           stdin = subprocess.PIPE)
         proc.communicate(strInput)
         if proc.returncode != 0:
            raise Exception("Executing shell command \"%s\" failed, return code %d"%(cmd, proc.returncode))
         return

      # Execute shell command, throws exception when failed, returns stdout+stderr
      if flags == "stdout":
         proc = subprocess.Popen(cmd,
                           shell = True,
                           stdin = subprocess.PIPE,
                           stdout = subprocess.PIPE,
                           stderr = subprocess.STDOUT)
         out = proc.communicate(strInput)[0]
         if proc.returncode != 0:
            raise Exception("Executing shell command \"%s\" failed, return code %d, output %s"%(cmd, proc.returncode, out))
         return out

      # Execute shell command, returns (returncode,stdout+stderr)
      if flags == "retcode+stdout":
         proc = subprocess.Popen(cmd,
                           shell = True,
                           stdin = subprocess.PIPE,
                           stdout = subprocess.PIPE,
                           stderr = subprocess.STDOUT)
         out = proc.communicate(strInput)[0]
         return (proc.returncode, out)

      assert False


Usage example:
Code:

FvmUtil.shellInteractive('/sbin/fdisk "%s"'%(usbFile), "n\np\n\n\n\nt\n7\nw", "stdout")      # create one NTFS primary partition (system id: 7)

ret = FvmUtil.shell('/sbin/fdisk -l "%s"'%(usbFile), "stdout")
m = re.search("([0-9]+) +([0-9]+) +[^ ]+ +7", ret, re.M)            # Start(%d), End(%d), Blocks, Id(7)
if m is None:
   raise Exception("createWinUsbImg failed, fdisk failed")
startSector = int(m.group(1))
offset = int(m.group(1)) * 512
size = (int(m.group(2)) - int(m.group(1))) * 512


Caution: The code above is GPLv3 licensed. :wink:
Back to top
View user's profile Send private message
grant123
l33t
l33t


Joined: 23 Mar 2005
Posts: 989

PostPosted: Sun Mar 02, 2014 6:06 pm    Post subject: Reply with quote

Thanks fpemud.

Quote:
in fdisk? o instead of d

That actually doesn't seem to work for me. fdisk "p" looks good but I can't boot sysresccd from a USB stick if I use "o" in fdisk instead of "d".

This works:

Code:
dd if=/dev/zero bs=1 seek=446 count=64 conv=notrunc of=/dev/sd#
Back to top
View user's profile Send private message
user
Tux's lil' helper
Tux's lil' helper


Joined: 08 Feb 2004
Posts: 145

PostPosted: Sun Mar 02, 2014 8:19 pm    Post subject: Reply with quote

Again,
use parted which can take commands via command line and write a pedantic shell wrapper of you choose.
Don't fiddle with fdisk and EOF piping, this is fault-prone.
Quote:
# parted --script /dev/<disk> mklabel gpt
# parted --align=min --script /dev/<disk> mkpart primary ext4 2048s 100%
# parted --script /dev/<disk> unit s print
Back to top
View user's profile Send private message
grant123
l33t
l33t


Joined: 23 Mar 2005
Posts: 989

PostPosted: Mon Mar 03, 2014 2:43 pm    Post subject: Reply with quote

Thanks, I should probably switch to parted.
Back to top
View user's profile Send private message
srs5694
Guru
Guru


Joined: 08 Mar 2004
Posts: 434
Location: Woonsocket, RI

PostPosted: Wed Mar 05, 2014 1:17 am    Post subject: Reply with quote

I tend to agree that parted is likely to be better than fdisk for this, although sfdisk might be worth considering for some uses. (I've always found it a bit strange, but it might suit your needs.)

I'd also like to bring up another point, related to this:

grant123 wrote:
"DOS partition table" is synonymous with "partition table" then?


No. There are several different types of partition tables. The two most common are the Master Boot Record (MBR; aka MS-DOS, DOS, or BIOS) partition table and the GUID Partition Table (GPT). The MBR format has been in use for about three decades, and is a mass of kludges and workarounds. GPT is newer and not nearly as ugly. More importantly, MBR uses 32-bit pointers, which means that it tops out at 2^32 sectors -- 2TiB, given the common 512-byte sector size. Also, GPT is the preferred (and in some cases required) partition table type for computers that boot with the newer EFI/UEFI firmware. Thus, if you expect to use your script with disks that are over 2TiB or with EFI-based computers, you should either be using GPT rather than MBR in all cases or support both partition table types.

If GPT support is an issue, fdisk is a poor choice. The very latest versions of fdisk do support GPT, but only in a rudimentary way; and the version you've got installed might not even support GPT. GNU parted supports both MBR and GPT, so if your script needs to support both partition table types, parted is certainly the easiest way to go. If you decide to go GPT-only, you could instead use sgdisk (part of the gptfdisk package in Gentoo), which is designed for use by scripts.
Back to top
View user's profile Send private message
grant123
l33t
l33t


Joined: 23 Mar 2005
Posts: 989

PostPosted: Wed Mar 05, 2014 10:45 pm    Post subject: Reply with quote

Quote:
"DOS partition table" is synonymous with "partition table" then?

Quote:
No.

I found that out the hard way. This works to wipe out the partition table though:

Code:
dd if=/dev/zero bs=1 seek=446 count=64 conv=notrunc of=/dev/sd#


Quote:
There are several different types of partition tables. The two most common are the Master Boot Record (MBR; aka MS-DOS, DOS, or BIOS) partition table and the GUID Partition Table (GPT). The MBR format has been in use for about three decades, and is a mass of kludges and workarounds. GPT is newer and not nearly as ugly. More importantly, MBR uses 32-bit pointers, which means that it tops out at 2^32 sectors -- 2TiB, given the common 512-byte sector size. Also, GPT is the preferred (and in some cases required) partition table type for computers that boot with the newer EFI/UEFI firmware. Thus, if you expect to use your script with disks that are over 2TiB or with EFI-based computers, you should either be using GPT rather than MBR in all cases or support both partition table types.

If GPT support is an issue, fdisk is a poor choice. The very latest versions of fdisk do support GPT, but only in a rudimentary way; and the version you've got installed might not even support GPT. GNU parted supports both MBR and GPT, so if your script needs to support both partition table types, parted is certainly the easiest way to go. If you decide to go GPT-only, you could instead use sgdisk (part of the gptfdisk package in Gentoo), which is designed for use by scripts.

Thank you, I'm sure I'll be delving into that sooner or later.
Back to top
View user's profile Send private message
srs5694
Guru
Guru


Joined: 08 Mar 2004
Posts: 434
Location: Woonsocket, RI

PostPosted: Sat Mar 15, 2014 7:35 pm    Post subject: Reply with quote

grant123 wrote:
This works to wipe out the partition table though:

Code:
dd if=/dev/zero bs=1 seek=446 count=64 conv=notrunc of=/dev/sd#


That will wipe out the MBR partition table (and a few bytes into the following sector). This will only damage a GUID Partition Table (GPT), not destroy it. Also, you've begun the wipe at the partition table itself, leaving the BIOS-mode boot loader intact; but the boot loader is probably useless without a partition table.

A better way to wipe any partition table is to use a tool designed for the job, like parted, fdisk, or gdisk. The details depend on your start point and intentions. Since most disks have partition tables, it's likely you want to replace an old partition table with a new one.
Back to top
View user's profile Send private message
grant123
l33t
l33t


Joined: 23 Mar 2005
Posts: 989

PostPosted: Sat Mar 15, 2014 11:05 pm    Post subject: Reply with quote

Is there a way to delete all existing partitions with fdisk regardless of how many there are?

I'm basically trying to get a totally fresh start and create a new set of partitions and make filesystems without anything hanging around from the previous install.
Back to top
View user's profile Send private message
cwr
Veteran
Veteran


Joined: 17 Dec 2005
Posts: 1969

PostPosted: Sun Mar 16, 2014 3:28 pm    Post subject: Reply with quote

grant123 wrote:
Is there a way to delete all existing partitions with fdisk regardless of how many there are?

I'm basically trying to get a totally fresh start and create a new set of partitions and make filesystems without anything hanging around from the previous install.


I generally write a couple of MB of zeros over the start of the drive to do this; however, this also wipes out the MBR boot loader code, so you need a replacement MBR (or a copy of your old one) to put in its place. How this works with a GPT I've no idea.

On the whole I'd just leave things alone and edit the partition table with one of the usual editors; though it's sometimes difficult to get good alignment with the current 4K/512B block size confusion.


Will
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 13982

PostPosted: Sun Mar 16, 2014 5:30 pm    Post subject: Reply with quote

Aligning partitions to 1MB boundaries, though excessive and slightly wasteful, guarantees proper alignment regardless of whether your disk uses 512b or 4Kb sectors. As regards wiping the existing table and starting fresh, I think sfdisk and sgdisk do this by default when fed a partition table from stdin. Why are you trying to script an interactive editor when a non-interactive one is better at the job?
Back to top
View user's profile Send private message
cwr
Veteran
Veteran


Joined: 17 Dec 2005
Posts: 1969

PostPosted: Sun Mar 16, 2014 5:57 pm    Post subject: Reply with quote

Hu wrote:
Aligning partitions to 1MB boundaries, though excessive and slightly wasteful, guarantees proper alignment regardless of whether your disk uses 512b or 4Kb sectors. As regards wiping the existing table and starting fresh, I think sfdisk and sgdisk do this by default when fed a partition table from stdin. Why are you trying to script an interactive editor when a non-interactive one is better at the job?


Yes, the problem is to force alignment. Most of the current partition editors seem to do a poor job of this.

Will
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 13982

PostPosted: Sun Mar 16, 2014 6:59 pm    Post subject: Reply with quote

I think using sfdisk --force with a partition table on stdin does the right thing for MBR-style partitions even on unusual alignments.
Back to top
View user's profile Send private message
srs5694
Guru
Guru


Joined: 08 Mar 2004
Posts: 434
Location: Woonsocket, RI

PostPosted: Sun Mar 16, 2014 10:13 pm    Post subject: Reply with quote

grant123 wrote:
Is there a way to delete all existing partitions with fdisk regardless of how many there are?


Yes. The "o" option will do this, at least if it's started on an MBR disk. (If you launch fdisk on a GPT disk, it will adjust the protective MBR, but leave the bulk of the GPT data structures intact. This in turn will confuse parted.)

As others have said, though, fdisk is not the best tool for scripted use. Depending on your needs, I expect that parted, sfdisk, or maybe even sgdisk would be a better choice. See below for my specific suggestion.

cwr wrote:
I generally write a couple of MB of zeros over the start of the drive to do this; however, this also wipes out the MBR boot loader code, so you need a replacement MBR (or a copy of your old one) to put in its place. How this works with a GPT I've no idea.


I recommend breaking yourself of this habit. GPT stores its main partition table at the start of the disk and a backup at the end, so you're really only doing half the job by using dd to wipe the first MiB or two of the disk. GNU parted will interpret the result as an empty disk, but GPT fdisk won't; gdisk will ask if you want to recover the disk and sgdisk will refuse to touch it.

FWIW, a BIOS boot loader in the MBR is likely to be useless without a partition table, so the fact that your procedure wipes it out isn't a big deal.

cwr wrote:
Yes, the problem is to force alignment. Most of the current partition editors seem to do a poor job of this.


That's not really true. Current versions of libparted, GPT fdisk, and fdisk all align partitions to 2048-sector (1MiB) boundaries by default. IIRC, sfdisk requires that partitions be specified in sector units, so it's up to the user or the user's script to do this job, by design. If I'm not mistaken, cfdisk is the only major Linux tool that doesn't align partitions to a reasonable value by defaul, but should. Two or three years ago, your comment may have been accurate, but partitioning tools switched to 1MiB alignment in droves at around that time.

Getting back to the question of how to wipe a partition table, I recommend this:

Code:
parted mklabel msdos /dev/sd{x}


Change "msdos" to "gpt" if GPT is preferred, and change "/dev/sd{x}" to the appropriate device.

This command will wipe the current partition table, no matter what it is, and create a new one of the specified type. This should eliminate old GPT data, which fdisk and sfdisk will not do, and you can easily change the command to create either GPT or MBR partition tables. You can add more parted commands to create new partitions, if desired. Overall, therefore, this is the single most flexible and fool-proof way to do the job. The fdisk and sfdisk tools are unreliable because they won't wipe the backup data. Ditto for dd, unless you bother to wipe the end of the disk.

In principle, you could use sgdisk to do it, as in "sgdisk -Z /dev/sd{x}"; however, if you plan to use MBR on the disk, you might as well use an MBR-capable program for the job rather than use sgdisk to wipe the partition table.
Back to top
View user's profile Send private message
cwr
Veteran
Veteran


Joined: 17 Dec 2005
Posts: 1969

PostPosted: Tue Mar 18, 2014 4:04 pm    Post subject: Reply with quote

I tried using the current versions of fdisk and gparted a couple of months back, and got some odd results; the partitions seemed to be aligned, but there were unused gaps in the partition table entries and I couldn't find a way to build partitions of the right size to fill them. There probably is a way, but after an hour or two with several partition editors it wasn't obvious.

Will
Back to top
View user's profile Send private message
frostschutz
Advocate
Advocate


Joined: 22 Feb 2005
Posts: 2971
Location: Germany

PostPosted: Wed Mar 19, 2014 8:49 pm    Post subject: Reply with quote

in msdos logical partitions, it's normal to have 1MiB gaps for MiB-aligned partitions, since one sector is wasted for the partition information (and the alignment blows that up to a full MiB gap instead of 1-sector gap)
Back to top
View user's profile Send private message
beandog
Developer
Developer


Joined: 04 May 2003
Posts: 2066
Location: /usa/utah

PostPosted: Wed Apr 09, 2014 7:08 am    Post subject: Reply with quote

Hu wrote:
Aligning partitions to 1MB boundaries, though excessive and slightly wasteful, guarantees proper alignment regardless of whether your disk uses 512b or 4Kb sectors.


Where do you guys find these best practices for partitioning? I'm clueless, and genuinely curious.
_________________
If it ain't broke, tweak it. dvds | blurays | blog | wiki
Back to top
View user's profile Send private message
cwr
Veteran
Veteran


Joined: 17 Dec 2005
Posts: 1969

PostPosted: Wed Apr 09, 2014 2:20 pm    Post subject: Reply with quote

frostschutz wrote:
in msdos logical partitions, it's normal to have 1MiB gaps for MiB-aligned partitions, since one sector is wasted for the partition information (and the alignment blows that up to a full MiB gap instead of 1-sector gap)


Thanks - that's probably the explanation for what I saw. It's just unfortunate that cfdisk is my favorite partition editor ...

Will
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum