Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[solved] btrfs send never terminates (nc problem)
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
jamapii
Guru
Guru


Joined: 16 Sep 2004
Posts: 582

PostPosted: Sat Aug 17, 2019 7:31 pm    Post subject: [solved] btrfs send never terminates (nc problem) Reply with quote

Hello

I'm using btrfs send/btrfs receive for backups.

I first used it on kernel 5.2-rc3 I think, with no parent (-p option), on a just created snapshot, and a btrfs receive instance connected over netcat (nc). It never terminated. I saw a lot of disk and network activity. The first time I killed it and tried again, I think that time it was a network problem. But the second time I waited a few hours, the snapshot on the receiving side looked perfectly normal and finished, and all activity had ceased.

So I did Ctrl-C on the sending side, and the receiving side finished.

Now I use the previous snapshot as a parent, and it still will never terminate. Only the layer 8 timeout can stop it. I check the receiving side if there are weird files in the snapshot (indicating ongoing activity), disk and network activity, and let a lot of time pass, before I do Ctrl-C.

At that time, I also see no btrfs process being very active.

Now with a recent kernel, latest btrfs-tools from Gentoo, it is still the same. The received snapshots seem complete. In a way, it arguably does "work", but requires manual intervention.

What could it be waiting for? It can't be looping over some corruption, because that should be visible in top. The snapshots that are sent are readonly. The receiving side does not see any problem, it just stops normally when I stop the sending side manually.


Last edited by jamapii on Sat Dec 07, 2019 10:11 am; edited 1 time in total
Back to top
View user's profile Send private message
jamapii
Guru
Guru


Joined: 16 Sep 2004
Posts: 582

PostPosted: Sat Dec 07, 2019 10:10 am    Post subject: Reply with quote

The problem was with the nc command.

If i pipe btrfs send into a simple nc command, like

btrfs send -p /snapshot-1 /snapshot-2 | nc backup-server 7000

then it won't terminate. Witn an option like "-q 100" to nc, it works:

Code:

btrfs send -p /snapshot-1 /snapshot-2 | nc -q 100 backup-server 7000
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 15329

PostPosted: Sat Dec 07, 2019 4:59 pm    Post subject: Reply with quote

It is a documented behavior of the nc command:
man nc6:
Your standard input is then sent to the host,
       and anything that comes back across the connection is sent to your standard out‐
       put.   This  continues  indefinitely,  until  the network side of the connection
       shuts down.  Note that this behavior is different from most  other  applications
       which  shut  everything down and exit after an end-of-file on the standard input
       (though this style of operation can easily be enabled).
man nc6:
       -q, --hold-timeout=SEC1[:SEC2]
                    Sets  the  hold  timeout(s)  (see "TIMEOUTS").  Specifying just one
                    value will set the hold timeout on the local endpoint, specifying a
                    second  value  will  set the hold timeout on the remote endpoint as
                    well.  Either value can be '0' (for instant timeout) or '-' (for no
                    hold timeout, eg. infinite hold).
man nc6:
       ·   The hold timeouts specify how long the connection should be held open  after
           a  given  endpoint  can  no  longer be read from (because EOF was received).
           When the hold timeout occurs for a given endpoint, no more data is read from
           the opposite endpoint.  After data read from the timed out endpoint has been
           sent, then the connection is closed and nc6 exits.

           For example, setting the hold time of the local endpoint to  5  would  cause
           reading  from the remote system to stop 5 seconds after stdin is closed (eg.
           by using ^D to send EOF).  All buffered data to be sent to the remote system
           is then sent, after which nc6 exits successfully (if no errors occurred).
Your solution appears to be correct, though you could probably use a much shorter hold timeout, since it is measured in seconds and netcat is documented to send all queued data before fully exiting.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum