Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Btrfs deduplication using duperemove not working
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
tabanus
l33t
l33t


Joined: 11 Jun 2004
Posts: 626
Location: UK

PostPosted: Sat May 23, 2015 2:35 pm    Post subject: Btrfs deduplication using duperemove not working Reply with quote

I created a small loop device, btrfs formatted to test with a few files for deduplication.

It looks like duperemove is correctly identifying the files to deduplicate, but not actually doing it:

Code:
# duperemove -dhr /mnt/gentoo/ 
Using 128K blocks
Using hash: SHA256 
Using 4 threads for file hashing phase
csum: /mnt/gentoo//test-files/Invoice.zip       [1/24]
csum: /mnt/gentoo//test-files/Desktop.zip       [2/24]
csum: /mnt/gentoo//test-files/motherboard_manual_ga-m68sm-s2l_e.pdf     [3/24]
csum: /mnt/gentoo//test-files/zuluCrypt-4.7.2.tar.bz2   [4/24]
csum: /mnt/gentoo//test-files/files-0/Invoice.zip       [5/24]
csum: /mnt/gentoo//test-files/files-0/Desktop.zip       [6/24]
csum: /mnt/gentoo//test-files/files-0/zuluCrypt-4.7.2.tar.bz2   [7/24]
csum: /mnt/gentoo//test-files/files-0/motherboard_manual_ga-m68sm-s2l_e.pdf     [8/24]
csum: /mnt/gentoo//test-files/files-1/Invoice.zip       [9/24]
csum: /mnt/gentoo//test-files/files-1/Desktop.zip       [10/24]
csum: /mnt/gentoo//test-files/files-1/zuluCrypt-4.7.2.tar.bz2   [11/24]
csum: /mnt/gentoo//test-files/files-1/motherboard_manual_ga-m68sm-s2l_e.pdf     [12/24]
csum: /mnt/gentoo//test-files/files-2/Invoice.zip       [13/24]
csum: /mnt/gentoo//test-files/files-2/Desktop.zip       [14/24]
csum: /mnt/gentoo//test-files/files-2/zuluCrypt-4.7.2.tar.bz2   [15/24]
csum: /mnt/gentoo//test-files/files-2/motherboard_manual_ga-m68sm-s2l_e.pdf     [16/24]
csum: /mnt/gentoo//test-files/files-3/Invoice.zip       [17/24]
csum: /mnt/gentoo//test-files/files-3/Desktop.zip       [18/24]
csum: /mnt/gentoo//test-files/files-3/zuluCrypt-4.7.2.tar.bz2   [19/24]
csum: /mnt/gentoo//test-files/files-3/motherboard_manual_ga-m68sm-s2l_e.pdf     [20/24]
csum: /mnt/gentoo//test-files/files-4/Invoice.zip       [21/24]
csum: /mnt/gentoo//test-files/files-4/Desktop.zip       [22/24]
csum: /mnt/gentoo//test-files/files-4/zuluCrypt-4.7.2.tar.bz2   [23/24]
csum: /mnt/gentoo//test-files/files-4/motherboard_manual_ga-m68sm-s2l_e.pdf     [24/24]
Hashed 396 blocks, resulting in 66 unique hashes. Calculating duplicate extents - this may take some time.
[########################################]
Search completed with no errors.             
Simple read and compare of file data found 4 instances of extents that might benefit from deduplication.
Start           Length          Filename (6 extents)
0.0     384.0K  "/mnt/gentoo//test-files/zuluCrypt-4.7.2.tar.bz2"
0.0     384.0K  "/mnt/gentoo//test-files/files-0/zuluCrypt-4.7.2.tar.bz2"
0.0     384.0K  "/mnt/gentoo//test-files/files-1/zuluCrypt-4.7.2.tar.bz2"
0.0     384.0K  "/mnt/gentoo//test-files/files-2/zuluCrypt-4.7.2.tar.bz2"
0.0     384.0K  "/mnt/gentoo//test-files/files-3/zuluCrypt-4.7.2.tar.bz2"
0.0     384.0K  "/mnt/gentoo//test-files/files-4/zuluCrypt-4.7.2.tar.bz2"
Start           Length          Filename (6 extents)
0.0     384.0K  "/mnt/gentoo//test-files/Invoice.zip"
0.0     384.0K  "/mnt/gentoo//test-files/files-0/Invoice.zip"
0.0     384.0K  "/mnt/gentoo//test-files/files-1/Invoice.zip"
0.0     384.0K  "/mnt/gentoo//test-files/files-2/Invoice.zip"
0.0     384.0K  "/mnt/gentoo//test-files/files-3/Invoice.zip"
0.0     384.0K  "/mnt/gentoo//test-files/files-4/Invoice.zip"
Start           Length          Filename (6 extents)
0.0     512.0K  "/mnt/gentoo//test-files/Desktop.zip"
0.0     512.0K  "/mnt/gentoo//test-files/files-0/Desktop.zip"
0.0     512.0K  "/mnt/gentoo//test-files/files-1/Desktop.zip"
0.0     512.0K  "/mnt/gentoo//test-files/files-2/Desktop.zip"
0.0     512.0K  "/mnt/gentoo//test-files/files-3/Desktop.zip"
0.0     512.0K  "/mnt/gentoo//test-files/files-4/Desktop.zip"
Start           Length          Filename (6 extents)
0.0     7.0M    "/mnt/gentoo//test-files/motherboard_manual_ga-m68sm-s2l_e.pdf"
0.0     7.0M    "/mnt/gentoo//test-files/files-0/motherboard_manual_ga-m68sm-s2l_e.pdf"
0.0     7.0M    "/mnt/gentoo//test-files/files-1/motherboard_manual_ga-m68sm-s2l_e.pdf"
0.0     7.0M    "/mnt/gentoo//test-files/files-2/motherboard_manual_ga-m68sm-s2l_e.pdf"
0.0     7.0M    "/mnt/gentoo//test-files/files-3/motherboard_manual_ga-m68sm-s2l_e.pdf"
0.0     7.0M    "/mnt/gentoo//test-files/files-4/motherboard_manual_ga-m68sm-s2l_e.pdf"
Dedupe 5 extents with target: (0.0, 384.0K), "/mnt/gentoo//test-files/zuluCrypt-4.7.2.tar.bz2"
Dedupe 5 extents with target: (0.0, 384.0K), "/mnt/gentoo//test-files/Invoice.zip"
Dedupe 5 extents with target: (0.0, 512.0K), "/mnt/gentoo//test-files/Desktop.zip"
Dedupe 5 extents with target: (0.0, 7.0M), "/mnt/gentoo//test-files/motherboard_manual_ga-m68sm-s2l_e.pdf"
Kernel processed data (excludes target files): 41.2M
Comparison of extent info shows a net change in shared extents of: 0.0


df show the same file usage before and after running duperemove:

Code:
# df -h /dev/loop0
Filesystem      Size  Used Avail Use% Mounted on
/dev/loop0      617M   65M  553M  11% /mnt/gentoo


Using kernel 4.0.2 (gentoo-sources).

Am I doing something wrong here, or should I post a bug report?
_________________
Things you might say if you never took Physics: "I'm overweight even though I don't overeat." - Neil deGrasse Tyson
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum