Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
File recovery tool that will make space efficient copy
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
LIsLinuxIsSogood
Veteran
Veteran


Joined: 13 Feb 2016
Posts: 1090

PostPosted: Tue Jul 09, 2019 8:26 am    Post subject: File recovery tool that will make space efficient copy Reply with quote

I just read this, https://wiki.gentoo.org/wiki/Ddrescue , plus the ddrescue man page and unable to find an answer to my concern anywhere. Of course someone will know the answer right away here. Can ddrescue be used to make a space efficient copy of data, or is it bit for bit for an entire partition like dd does? I want to get a bit for bit copy (sort of) but wondering if unlike dd can ddrescue work aware of directories rather than just files.

I am trying to recover files I messed up, on a NTFS partition, but the catch is while it had only <1GB of data stored the partition itself is 250GB, so I really don't want to use 250GB of space to temporarily house that information.

Obviously since that would work, I guess it is a baseline of some sort, but not a very good one I feel.

I would like some help with whatever tool will do more smart copying and not necessarily writing all the trash or unneeded stuff in the larger disk partition.

Eventually I will format an NTFS partition on usb and get this 600-700MB of data over to a windows machine to see if running chkdsk on that will fix any of the file properties leading to the input/output errors. I am aware there is a low probability that it will work, but also aware there are few linux tools that compare.

Also I considered other than dd, or ddrescue if maybe using an archiving or backup tool would be better. Please help!

Thanks!
Back to top
View user's profile Send private message
sdauth
Tux's lil' helper
Tux's lil' helper


Joined: 19 Sep 2018
Posts: 90
Location: Frankreich

PostPosted: Tue Jul 09, 2019 9:41 am    Post subject: Reply with quote

Hello,

When you says "files I messed up on a NTFS partition", what do you mean exactly ? Deleted ?
If so, you can use "photorec" from "testdisk" software (make sure to build it with "ntfs" support if "ntfs" isn't enabled globally), it's been a long time so I'm not sure it can recover directory structure.
Otherwise, to make a space-efficient copy of blocks used in filesystem, you could use "fsarchiver", which works perfectly with linux ext* partitions, but I have no experience with ntfs partitions and fsarchiver so I can't really recommend it.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 44921
Location: 56N 3W

PostPosted: Tue Jul 09, 2019 10:39 am    Post subject: Reply with quote

LIsLinuxIsSogood,

dd and ddrescue are both uninformed of any filesystem or other data structures on the input block device.
You can feed the output to your favourite compressor but then you need to decompress it to work on it.

Your logic may be faulty. You say you have corrupt files ... a directory is just a file. It contains pointer to other files and maybe other directories too.
e.g. If the root directory is missing, your data is still there (maybe) but its no longer accessible by reading the filesystem.

Make your dd copy. ddrescue is for reading volumes that have bad sectors. Tell dd to use bs=1M so it doesn't take a long time.
Once you have a copy, you can practice data recovery on one, knowing you have an undo, for when you come to a dead end.

fsck or whatever windows calls it may make the partition usable. However, it does this by guessing what should be there.
It only makes the low level metadata self consistent, so that the filesystem will mount. It says nothing about any user data that may be there.
In short, it often makes a bad situation worse, so its a last ditch thing to try. That's one of the reasons you need an undo.

-- edit --
If this is related to Filesystem Question you have bigger problems.
Code:
Jul 8 23:08:32 Reznik-CentOS ntfs-3g[5812]: Mount options: allow_other,nonempty,relatime,rw,fsname=/dev/nvme0n1p4,blkdev,blksize=4096

That's a SSD storage volume. If trim/discard was in use, (Windows does it for you) you have no assurance that any deleted data still exists on the drive.
trim/discard tells the drive it can erase the space any time it wants to.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
LIsLinuxIsSogood
Veteran
Veteran


Joined: 13 Feb 2016
Posts: 1090

PostPosted: Tue Jul 09, 2019 5:56 pm    Post subject: Reply with quote

Thank you both for the tip and I did forget to link to my other post that is related to this post. But since neddy already linked that i can skip doing it here. The options offered are good. Just to clarify there was no Windows operating system in use here, so that disk access throughout was from a linux system, at least going back for a while now it may still have been the same partition where windows was initially installed on my laptop.

Now i would like to use dd to make a copy but since this 250gb partition where am i going to write that to? I need to free an sizeable disk of equal or greater size? It doesnt make any sense to resize the disk downward since data loss might result correct?

With fsarchiver the problem becomes the same as when using any other file aware tools i get the input/output errors as it goes early and so quits out with no luck!

(edited)
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 44921
Location: 56N 3W

PostPosted: Tue Jul 09, 2019 9:19 pm    Post subject: Reply with quote

LIsLinuxIsSogood,

Correct, You can't touch the damaged filesystem. You need to read it, that's all.

The image can go anywhere you have space, even on another system.
piping over ssh works to get to the other system, its a bit slow with the crypto overhead.
nfs is another possibility.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
LIsLinuxIsSogood
Veteran
Veteran


Joined: 13 Feb 2016
Posts: 1090

PostPosted: Tue Jul 09, 2019 11:48 pm    Post subject: Reply with quote

How long should it take and does that depend on the size of used space or available space within the partition?
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 14957

PostPosted: Wed Jul 10, 2019 1:44 am    Post subject: Reply with quote

If you're copying with dd, then you get a full copy of the source partition, which you stated is 250gb. That means both that you need 250gb of free space to store the copy and that you will be reading that entire 250gb from the source partition and writing it to the destination filesystem/partition. The time required for this will depend on the read speed of the source drive (accounting for any bottlenecks due to bus limitations) and the write speed of the destination (again accounting for bottlenecks). For example, if one side was an external drive in a USB2 enclosure, your transfer would max out at USB2 transfer speed, and you would move 250gb of data at that speed. For this reason, it may well be worth a bit of extra physical setup first to ensure that both ends are connected through the fastest buses you can readily get. If you can mount the drives in the same system, that will help. (For this problem, you probably already did that, but it's worth stating explicitly for future readers.) Avoiding external USB will help. Modern USB is much better than the early protocols, but internal buses still beat it in most cases.
Back to top
View user's profile Send private message
LIsLinuxIsSogood
Veteran
Veteran


Joined: 13 Feb 2016
Posts: 1090

PostPosted: Wed Jul 10, 2019 6:45 am    Post subject: Reply with quote

Disk is inside a laptop that I have so far never had to take apart. Probably going to play things safe and just do something like make the laptop disk accessible via a net share like neddy suggested, and can handle the fact that it will be slower for the reason that i don't want to damage the outer case or any parts on an expensive lenovo laptop that I got for cheap. Thanks for the suggestion though if I could easily do it, which I still can then I would take out the hard drive and go from there. I've already recovered about 95% data and the other 5% may not be that important.

Edit: Something I'm unsure of with regards to the bottleneck issue is the network vs. USB connection, since drives that are not connected to exact same bus might be configured either way using the same OS or different machine different OS with a network share in place. Among these two options is there a clearly preferred way to run dd, I get the sense from others that network (ssh pipe) may be preferred over using a usb 3.0 connection and a hard drive enclosure. Thoughts?
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 44921
Location: 56N 3W

PostPosted: Wed Jul 10, 2019 11:05 pm    Post subject: Reply with quote

LIsLinuxIsSogood,

USB3 is close to the SATA3 bus speed, so if your USB3 drive is a spinny, you won't notice the USB3 bottleneck as the head /platter data rate will be the limit.
That's about 100MB/sec at the outside of the drive and 40MB/sec at the inside.

Do tell dd to use a 1M block size as that's where transfer speed levels off.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum