[gentoo-user] Permission error when trying to rsync over nfs

[gentoo-user] Permission error when trying to rsync over nfs

[Dale]

Howdy,

I'm wanting to move a Data partition over to a new set of drives that
are encrypted.  I decided the easiest way to do this is to put the new
drives on the NAS box and mount them like I do when I backup my large
Video directory only copy the Data files instead.  So, on the NAS box, I
set up the three drives, 2 16TB and the famous 20TB drive, with LVM and
dm-setup.  Once I had that setup, I mounted in just like I would the
Video directory.  I also ran exportfs -a so it would see the newly
mounted drive set and make it available.  On my main rig, I then mounted
it using the same command I would for the Video directory for backups. 
Then I took the same command for backing up my video but just replaced
the source and target with the Data path instead of video.  Basically,
everything is set up the same, I just replaced everything with the
drives I want info copied to in both mounting drives and commands. 

This is the error I get. 


rsync: [generator] recv_generator: mkdir "/mnt/TV_Backup/Data/random
directory" failed: Permission denied (13)


I know to run the rsync command as a user, not root.  I have to do that
when updating my video backups.  I recall getting a error, could be this
one, at first but setting something to make it work.  I can't recall
what I had to do tho.  I also have no notes on this.  I'm sure I'm
missing something but no idea what.  Anyone ran into this before and
remember what to do to fix it?   Internet searches aren't helping either. 

Thanks. 

Dale

:-)  :-) 

Re: [gentoo-user] Permission error when trying to rsync over nfs

[Michael]

[-- Attachment #1: Type: text/plain, Size: 1889 bytes --]

On Monday, 19 May 2025 14:39:21 British Summer Time Dale wrote:
> Howdy,
> 
> I'm wanting to move a Data partition over to a new set of drives that
> are encrypted.  I decided the easiest way to do this is to put the new
> drives on the NAS box and mount them like I do when I backup my large
> Video directory only copy the Data files instead.  So, on the NAS box, I
> set up the three drives, 2 16TB and the famous 20TB drive, with LVM and
> dm-setup.  Once I had that setup, I mounted in just like I would the
> Video directory.  I also ran exportfs -a so it would see the newly
> mounted drive set and make it available.  On my main rig, I then mounted
> it using the same command I would for the Video directory for backups. 
> Then I took the same command for backing up my video but just replaced
> the source and target with the Data path instead of video.  Basically,
> everything is set up the same, I just replaced everything with the
> drives I want info copied to in both mounting drives and commands. 
> 
> This is the error I get. 
> 
> 
> rsync: [generator] recv_generator: mkdir "/mnt/TV_Backup/Data/random
> directory" failed: Permission denied (13)
> 
> 
> I know to run the rsync command as a user, not root.  I have to do that
> when updating my video backups.  I recall getting a error, could be this
> one, at first but setting something to make it work.  I can't recall
> what I had to do tho.  I also have no notes on this.  I'm sure I'm
> missing something but no idea what.  Anyone ran into this before and
> remember what to do to fix it?   Internet searches aren't helping either. 
> 
> Thanks. 
> 
> Dale
> 
> :-)  :-) 

OK, I am confused ...  :-/

If you want to update the contents of a fs over the network, then rsync is 
your tool.  Why is NFS coming into this at all?

Assuming the user IDs are the same across systems, add '--numeric-ids'.  
That's all.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Permission error when trying to rsync over nfs

[Dale]

Michael wrote:
> On Monday, 19 May 2025 14:39:21 British Summer Time Dale wrote:
>> Howdy,
>>
>> I'm wanting to move a Data partition over to a new set of drives that
>> are encrypted.  I decided the easiest way to do this is to put the new
>> drives on the NAS box and mount them like I do when I backup my large
>> Video directory only copy the Data files instead.  So, on the NAS box, I
>> set up the three drives, 2 16TB and the famous 20TB drive, with LVM and
>> dm-setup.  Once I had that setup, I mounted in just like I would the
>> Video directory.  I also ran exportfs -a so it would see the newly
>> mounted drive set and make it available.  On my main rig, I then mounted
>> it using the same command I would for the Video directory for backups. 
>> Then I took the same command for backing up my video but just replaced
>> the source and target with the Data path instead of video.  Basically,
>> everything is set up the same, I just replaced everything with the
>> drives I want info copied to in both mounting drives and commands. 
>>
>> This is the error I get. 
>>
>>
>> rsync: [generator] recv_generator: mkdir "/mnt/TV_Backup/Data/random
>> directory" failed: Permission denied (13)
>>
>>
>> I know to run the rsync command as a user, not root.  I have to do that
>> when updating my video backups.  I recall getting a error, could be this
>> one, at first but setting something to make it work.  I can't recall
>> what I had to do tho.  I also have no notes on this.  I'm sure I'm
>> missing something but no idea what.  Anyone ran into this before and
>> remember what to do to fix it?   Internet searches aren't helping either. 
>>
>> Thanks. 
>>
>> Dale
>>
>> :-)  :-) 
> OK, I am confused ...  :-/
>
> If you want to update the contents of a fs over the network, then rsync is 
> your tool.  Why is NFS coming into this at all?
>
> Assuming the user IDs are the same across systems, add '--numeric-ids'.  
> That's all.


Well, I mount the drives on the NAS box over the network with nfs on my
main rig.  It's just how I set it up.  For some reason, it just won't
work this time.  It works when I do my video backups tho.  :/ 

It does mount fine and everything shows up.  It's just that it won't let
me create a directory or anything to copy files over. 

You get your system fixed? 

Dale

:-)  :-) 

Re: [gentoo-user] Permission error when trying to rsync over nfs

[Michael]

[-- Attachment #1: Type: text/plain, Size: 1596 bytes --]

On Tuesday, 20 May 2025 14:27:51 British Summer Time Dale wrote:
> Michael wrote:

> > OK, I am confused ...  :-/
> > 
> > If you want to update the contents of a fs over the network, then rsync is
> > your tool.  Why is NFS coming into this at all?
> > 
> > Assuming the user IDs are the same across systems, add '--numeric-ids'.
> > That's all.
> 
> Well, I mount the drives on the NAS box over the network with nfs on my
> main rig.  It's just how I set it up.  For some reason, it just won't
> work this time.  It works when I do my video backups tho.  :/ 

NFS has other advantages, but running rsync through it ain't one of them.

Are the directories exported in the *same* way?

Do you use (rw) as an option?

Have you set up anonuid and anongid options in the exports for the IDs you 
want the remote users to acquire when connecting over NFS?


> It does mount fine and everything shows up.  It's just that it won't let
> me create a directory or anything to copy files over. 

Can you drag & drop individual files using the GUI or CLI, without rsync?

Check the above suggestions in case one of them works for you.

However, I still think you should use rsync directly, since we're talking 
about updating remote data to match your local data - it is what it was 
designed for.


> You get your system fixed? 

Almost:

>>> Installing (411 of 481) ...

It would not let me emerge python, because of missing keywords ... on an 
otherwise stable system.  I invoked emerge by prefixing it with 
ACCEPT_KEYWORDS="amd64" and it is progressing at the moment.  I'll try to sort 
out rust next.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Permission error when trying to rsync over nfs

[Dale]

[-- Attachment #1: Type: text/plain, Size: 3839 bytes --]

Michael wrote:
> On Tuesday, 20 May 2025 14:27:51 British Summer Time Dale wrote:
>> Michael wrote:
>>> OK, I am confused ...  :-/
>>>
>>> If you want to update the contents of a fs over the network, then rsync is
>>> your tool.  Why is NFS coming into this at all?
>>>
>>> Assuming the user IDs are the same across systems, add '--numeric-ids'.
>>> That's all.
>> Well, I mount the drives on the NAS box over the network with nfs on my
>> main rig.  It's just how I set it up.  For some reason, it just won't
>> work this time.  It works when I do my video backups tho.  :/ 
> NFS has other advantages, but running rsync through it ain't one of them.
>
> Are the directories exported in the *same* way?
>
> Do you use (rw) as an option?
>
> Have you set up anonuid and anongid options in the exports for the IDs you 
> want the remote users to acquire when connecting over NFS?
>
>
>> It does mount fine and everything shows up.  It's just that it won't let
>> me create a directory or anything to copy files over. 
> Can you drag & drop individual files using the GUI or CLI, without rsync?
>
> Check the above suggestions in case one of them works for you.
>
> However, I still think you should use rsync directly, since we're talking 
> about updating remote data to match your local data - it is what it was 
> designed for.
>


I had a idea.  I checked permissions of things while connected to the
new data drive set.  Then I pulled the backup drive set from the safe
and hooked it up.  I then checked the permissions of it.  There is
something different between the two.  Could this be the problem.  This
is the info, removing unneeded bits.  This is the data drive set.



root@nas ~ # mount | grep /mnt
/dev/mapper/data on /mnt/backup type ext4 (rw,relatime)
root@nas ~ # ls -al /mnt/
total 24
drwxr-xr-x  6 root root 4096 May 12 23:43 .
drwxr-xr-x 17 root root 4096 Mar 23 10:05 ..
drwxr-xr-x  2 root root 4096 May 20 09:32 backup
root@nas ~ #


root@Gentoo-1 / # mount | grep TV_Backup
10.0.0.5:/mnt/backup on /mnt/TV_Backup type nfs4
(rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,nocto,proto=tcp,nconnect=4,timeo=600,retrans=2,sec=sys,clientaddr=10.0.0.4,local_lock=none,addr=10.0.0.5)
root@Gentoo-1 / # ls /mnt/
total 68
drwxrwxr-x 17 dale users 4096 Oct 22  2024 .
drwxr-xr-x 18 root root  4096 Sep 11  2024 ..
drwxr-xr-x  2 root root  4096 May 20 09:32 TV_Backup
root@Gentoo-1 / #



This is the backup drive set which works fine. 



root@nas ~ # mount | grep /mnt
/dev/mapper/backup on /mnt/backup type ext4 (rw,relatime)
root@nas ~ # ls -al /mnt/
total 24
drwxr-xr-x  6 root root  4096 May 12 23:43 .
drwxr-xr-x 17 root root  4096 Mar 23 10:05 ..
drwxr-xr-x  4 1000 users 4096 Aug  4  2024 backup
root@nas ~ #


root@Gentoo-1 / # mount | grep TV_Backup
10.0.0.5:/mnt/backup on /mnt/TV_Backup type nfs4
(rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,nocto,proto=tcp,nconnect=4,timeo=600,retrans=2,sec=sys,clientaddr=10.0.0.4,local_lock=none,addr=10.0.0.5)
root@Gentoo-1 / # ls /mnt/
total 68
drwxrwxr-x 17 dale users 4096 Oct 22  2024 .
drwxr-xr-x 18 root root  4096 Sep 11  2024 ..
drwxr-xr-x  4 dale users 4096 Aug  4  2024 TV_Backup
root@Gentoo-1 / #


The difference is on the NAS box.  Permissions are set to 1000:users.  I
don't recall setting that.  So, I reconnected the data drive set and
changed the permissions to match the backup drive set.  As I type, it is
doing the backups. 

I wonder, why does it have to be set that way????  Oh, you can bet I put
this info in a file, for me to forget I have when I run into this
again.  :/

Dale

:-)  :-) 

P. S.  Is there a better way to do this?  I'm not worried about security
as it only goes between one box through my router to the other box.  I
just need to be able to go both ways. 

[-- Attachment #2: Type: text/html, Size: 5539 bytes --]

Re: [gentoo-user] Permission error when trying to rsync over nfs

[Michael]

[-- Attachment #1: Type: text/plain, Size: 4264 bytes --]

On Tuesday, 20 May 2025 17:30:34 British Summer Time Dale wrote:
> Michael wrote:
> > On Tuesday, 20 May 2025 14:27:51 British Summer Time Dale wrote:
> >> Michael wrote:
> >>> OK, I am confused ...  :-/
> >>> 
> >>> If you want to update the contents of a fs over the network, then rsync
> >>> is
> >>> your tool.  Why is NFS coming into this at all?
> >>> 
> >>> Assuming the user IDs are the same across systems, add '--numeric-ids'.
> >>> That's all.
> >> 
> >> Well, I mount the drives on the NAS box over the network with nfs on my
> >> main rig.  It's just how I set it up.  For some reason, it just won't
> >> work this time.  It works when I do my video backups tho.  :/
> > 
> > NFS has other advantages, but running rsync through it ain't one of them.
> > 
> > Are the directories exported in the *same* way?
> > 
> > Do you use (rw) as an option?
> > 
> > Have you set up anonuid and anongid options in the exports for the IDs you
> > want the remote users to acquire when connecting over NFS?
> > 
> >> It does mount fine and everything shows up.  It's just that it won't let
> >> me create a directory or anything to copy files over.
> > 
> > Can you drag & drop individual files using the GUI or CLI, without rsync?
> > 
> > Check the above suggestions in case one of them works for you.
> > 
> > However, I still think you should use rsync directly, since we're talking
> > about updating remote data to match your local data - it is what it was
> > designed for.
> 
> I had a idea.  I checked permissions of things while connected to the
> new data drive set.  Then I pulled the backup drive set from the safe
> and hooked it up.  I then checked the permissions of it.  There is
> something different between the two.  Could this be the problem.  This
> is the info, removing unneeded bits.  This is the data drive set.
> 
> 
> 
> root@nas ~ # mount | grep /mnt
> /dev/mapper/data on /mnt/backup type ext4 (rw,relatime)
> root@nas ~ # ls -al /mnt/
> total 24
> drwxr-xr-x  6 root root 4096 May 12 23:43 .
> drwxr-xr-x 17 root root 4096 Mar 23 10:05 ..
> drwxr-xr-x  2 root root 4096 May 20 09:32 backup
> root@nas ~ #
> 
> 
> root@Gentoo-1 / # mount | grep TV_Backup
> 10.0.0.5:/mnt/backup on /mnt/TV_Backup type nfs4
> (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,nocto,prot
> o=tcp,nconnect=4,timeo=600,retrans=2,sec=sys,clientaddr=10.0.0.4,local_lock=
> none,addr=10.0.0.5) root@Gentoo-1 / # ls /mnt/
> total 68
> drwxrwxr-x 17 dale users 4096 Oct 22  2024 .
> drwxr-xr-x 18 root root  4096 Sep 11  2024 ..
> drwxr-xr-x  2 root root  4096 May 20 09:32 TV_Backup
> root@Gentoo-1 / #
> 
> 
> 
> This is the backup drive set which works fine. 
> 
> 
> 
> root@nas ~ # mount | grep /mnt
> /dev/mapper/backup on /mnt/backup type ext4 (rw,relatime)
> root@nas ~ # ls -al /mnt/
> total 24
> drwxr-xr-x  6 root root  4096 May 12 23:43 .
> drwxr-xr-x 17 root root  4096 Mar 23 10:05 ..
> drwxr-xr-x  4 1000 users 4096 Aug  4  2024 backup
> root@nas ~ #
> 
> 
> root@Gentoo-1 / # mount | grep TV_Backup
> 10.0.0.5:/mnt/backup on /mnt/TV_Backup type nfs4
> (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,nocto,prot
> o=tcp,nconnect=4,timeo=600,retrans=2,sec=sys,clientaddr=10.0.0.4,local_lock=
> none,addr=10.0.0.5) root@Gentoo-1 / # ls /mnt/
> total 68
> drwxrwxr-x 17 dale users 4096 Oct 22  2024 .
> drwxr-xr-x 18 root root  4096 Sep 11  2024 ..
> drwxr-xr-x  4 dale users 4096 Aug  4  2024 TV_Backup
> root@Gentoo-1 / #
> 
> 
> The difference is on the NAS box.  Permissions are set to 1000:users.  I
> don't recall setting that.  So, I reconnected the data drive set and
> changed the permissions to match the backup drive set.  As I type, it is
> doing the backups. 
> 
> I wonder, why does it have to be set that way????  Oh, you can bet I put
> this info in a file, for me to forget I have when I run into this
> again.  :/
> 
> Dale
> 
> :-)  :-) 
> 
> P. S.  Is there a better way to do this?  I'm not worried about security
> as it only goes between one box through my router to the other box.  I
> just need to be able to go both ways. 

Normally, /mnt is owned by root:root.  Plain users are not allowed to create 
mountpoints at will.

UID 1000 would be the first user you set on this PC, e.g. 'dale'.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Permission error when trying to rsync over nfs

[Dale]

Michael wrote:
> On Tuesday, 20 May 2025 17:30:34 British Summer Time Dale wrote:
>>
>> I had a idea.  I checked permissions of things while connected to the
>> new data drive set.  Then I pulled the backup drive set from the safe
>> and hooked it up.  I then checked the permissions of it.  There is
>> something different between the two.  Could this be the problem.  This
>> is the info, removing unneeded bits.  This is the data drive set.
>>
>>
>>
>> root@nas ~ # mount | grep /mnt
>> /dev/mapper/data on /mnt/backup type ext4 (rw,relatime)
>> root@nas ~ # ls -al /mnt/
>> total 24
>> drwxr-xr-x  6 root root 4096 May 12 23:43 .
>> drwxr-xr-x 17 root root 4096 Mar 23 10:05 ..
>> drwxr-xr-x  2 root root 4096 May 20 09:32 backup
>> root@nas ~ #
>>
>>
>> root@Gentoo-1 / # mount | grep TV_Backup
>> 10.0.0.5:/mnt/backup on /mnt/TV_Backup type nfs4
>> (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,nocto,prot
>> o=tcp,nconnect=4,timeo=600,retrans=2,sec=sys,clientaddr=10.0.0.4,local_lock=
>> none,addr=10.0.0.5) root@Gentoo-1 / # ls /mnt/
>> total 68
>> drwxrwxr-x 17 dale users 4096 Oct 22  2024 .
>> drwxr-xr-x 18 root root  4096 Sep 11  2024 ..
>> drwxr-xr-x  2 root root  4096 May 20 09:32 TV_Backup
>> root@Gentoo-1 / #
>>
>>
>>
>> This is the backup drive set which works fine. 
>>
>>
>>
>> root@nas ~ # mount | grep /mnt
>> /dev/mapper/backup on /mnt/backup type ext4 (rw,relatime)
>> root@nas ~ # ls -al /mnt/
>> total 24
>> drwxr-xr-x  6 root root  4096 May 12 23:43 .
>> drwxr-xr-x 17 root root  4096 Mar 23 10:05 ..
>> drwxr-xr-x  4 1000 users 4096 Aug  4  2024 backup
>> root@nas ~ #
>>
>>
>> root@Gentoo-1 / # mount | grep TV_Backup
>> 10.0.0.5:/mnt/backup on /mnt/TV_Backup type nfs4
>> (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,nocto,prot
>> o=tcp,nconnect=4,timeo=600,retrans=2,sec=sys,clientaddr=10.0.0.4,local_lock=
>> none,addr=10.0.0.5) root@Gentoo-1 / # ls /mnt/
>> total 68
>> drwxrwxr-x 17 dale users 4096 Oct 22  2024 .
>> drwxr-xr-x 18 root root  4096 Sep 11  2024 ..
>> drwxr-xr-x  4 dale users 4096 Aug  4  2024 TV_Backup
>> root@Gentoo-1 / #
>>
>>
>> The difference is on the NAS box.  Permissions are set to 1000:users.  I
>> don't recall setting that.  So, I reconnected the data drive set and
>> changed the permissions to match the backup drive set.  As I type, it is
>> doing the backups. 
>>
>> I wonder, why does it have to be set that way????  Oh, you can bet I put
>> this info in a file, for me to forget I have when I run into this
>> again.  :/
>>
>> Dale
>>
>> :-)  :-) 
>>
>> P. S.  Is there a better way to do this?  I'm not worried about security
>> as it only goes between one box through my router to the other box.  I
>> just need to be able to go both ways. 
> Normally, /mnt is owned by root:root.  Plain users are not allowed to create 
> mountpoints at will.
>
> UID 1000 would be the first user you set on this PC, e.g. 'dale'.

That makes sense then.  I learned early on that I can not rsync as
root.  It just plain doesn't like that at all.  Works as a user tho.  I
guess I could set up a group, called backup perhaps, and then put the
stuff under /mnt in that group and make it writable by the group
backup.  I think that would work.  Thing is, the NAS box doesn't have
any users, only root.  It doesn't usually run much anyway, maybe a hour
or two each week. 

I tend to put things that are mounted temporarily under /mnt.  Things
that are more permanent, my video collection for example, actually
mounts to a mount point on my Desktop.  That way I can click on the
little icon and go to whatever video I want.  It may not make much sense
to anyone else but it works for me.  ;-) 

You get your OS back up and running? 

Dale

:-)  :-) 

Re: [gentoo-user] Permission error when trying to rsync over nfs

[Michael]

[-- Attachment #1: Type: text/plain, Size: 4984 bytes --]

On Wednesday, 21 May 2025 20:42:28 British Summer Time Dale wrote:
> Michael wrote:
> > On Tuesday, 20 May 2025 17:30:34 British Summer Time Dale wrote:
> >> I had a idea.  I checked permissions of things while connected to the
> >> new data drive set.  Then I pulled the backup drive set from the safe
> >> and hooked it up.  I then checked the permissions of it.  There is
> >> something different between the two.  Could this be the problem.  This
> >> is the info, removing unneeded bits.  This is the data drive set.
> >> 
> >> 
> >> 
> >> root@nas ~ # mount | grep /mnt
> >> /dev/mapper/data on /mnt/backup type ext4 (rw,relatime)
> >> root@nas ~ # ls -al /mnt/
> >> total 24
> >> drwxr-xr-x  6 root root 4096 May 12 23:43 .
> >> drwxr-xr-x 17 root root 4096 Mar 23 10:05 ..
> >> drwxr-xr-x  2 root root 4096 May 20 09:32 backup
> >> root@nas ~ #
> >> 
> >> 
> >> root@Gentoo-1 / # mount | grep TV_Backup
> >> 10.0.0.5:/mnt/backup on /mnt/TV_Backup type nfs4
> >> (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,nocto,p
> >> rot
> >> o=tcp,nconnect=4,timeo=600,retrans=2,sec=sys,clientaddr=10.0.0.4,local_l
> >> ock= none,addr=10.0.0.5) root@Gentoo-1 / # ls /mnt/
> >> total 68
> >> drwxrwxr-x 17 dale users 4096 Oct 22  2024 .
> >> drwxr-xr-x 18 root root  4096 Sep 11  2024 ..
> >> drwxr-xr-x  2 root root  4096 May 20 09:32 TV_Backup
> >> root@Gentoo-1 / #
> >> 
> >> 
> >> 
> >> This is the backup drive set which works fine.
> >> 
> >> 
> >> 
> >> root@nas ~ # mount | grep /mnt
> >> /dev/mapper/backup on /mnt/backup type ext4 (rw,relatime)
> >> root@nas ~ # ls -al /mnt/
> >> total 24
> >> drwxr-xr-x  6 root root  4096 May 12 23:43 .
> >> drwxr-xr-x 17 root root  4096 Mar 23 10:05 ..
> >> drwxr-xr-x  4 1000 users 4096 Aug  4  2024 backup
> >> root@nas ~ #
> >> 
> >> 
> >> root@Gentoo-1 / # mount | grep TV_Backup
> >> 10.0.0.5:/mnt/backup on /mnt/TV_Backup type nfs4
> >> (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,nocto,p
> >> rot
> >> o=tcp,nconnect=4,timeo=600,retrans=2,sec=sys,clientaddr=10.0.0.4,local_l
> >> ock= none,addr=10.0.0.5) root@Gentoo-1 / # ls /mnt/
> >> total 68
> >> drwxrwxr-x 17 dale users 4096 Oct 22  2024 .
> >> drwxr-xr-x 18 root root  4096 Sep 11  2024 ..
> >> drwxr-xr-x  4 dale users 4096 Aug  4  2024 TV_Backup
> >> root@Gentoo-1 / #
> >> 
> >> 
> >> The difference is on the NAS box.  Permissions are set to 1000:users.  I
> >> don't recall setting that.  So, I reconnected the data drive set and
> >> changed the permissions to match the backup drive set.  As I type, it is
> >> doing the backups.
> >> 
> >> I wonder, why does it have to be set that way????  Oh, you can bet I put
> >> this info in a file, for me to forget I have when I run into this
> >> again.  :/
> >> 
> >> Dale
> >> 
> >> :-)  :-)
> >> 
> >> P. S.  Is there a better way to do this?  I'm not worried about security
> >> as it only goes between one box through my router to the other box.  I
> >> just need to be able to go both ways.
> > 
> > Normally, /mnt is owned by root:root.  Plain users are not allowed to
> > create mountpoints at will.
> > 
> > UID 1000 would be the first user you set on this PC, e.g. 'dale'.
> 
> That makes sense then.  I learned early on that I can not rsync as
> root.  It just plain doesn't like that at all.  Works as a user tho.  I
> guess I could set up a group, called backup perhaps, and then put the
> stuff under /mnt in that group and make it writable by the group
> backup.  I think that would work.  Thing is, the NAS box doesn't have
> any users, only root.  It doesn't usually run much anyway, maybe a hour
> or two each week. 

Have a look at this (random) page which makes some security recommendations:

https://www.thegeekdiary.com/basic-nfs-security-nfs-no_root_squash-and-suid/

What is suggested may be not apply in your case, within a secure LAN and only 
one user let loose on the fs, so exercise your judgement as you see fit.


> I tend to put things that are mounted temporarily under /mnt.  Things
> that are more permanent, my video collection for example, actually
> mounts to a mount point on my Desktop.  That way I can click on the
> little icon and go to whatever video I want.  It may not make much sense
> to anyone else but it works for me.  ;-) 
> 
> You get your OS back up and running? 
> 
> Dale
> 
> :-)  :-) 

The OS is slowly coming together, another 90 packages to go.  Some of them 
rather large.  Sadly many of my USE flags are not satisfied by binhost 
packages, so I will be emerging them slowly overnight.  Should be done by 
tomorrow, unless a package fails to emerge for some reason.

What I can't understand is why I have to set explicitly 
ACCEPT_KEYWORDS="amd64" for emerge and it does not source this off the 
make.conf file.  I hope this problem will be resolved once all the packages 
which had been emerged over the last 5 weeks have been re-emerged, so both the 
software on the / fs and the portage idea of it become aligned again.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Permission error when trying to rsync over nfs

[Dale]

Michael wrote:
> On Monday, 19 May 2025 14:39:21 British Summer Time Dale wrote:
>> Howdy,
>>
>> I'm wanting to move a Data partition over to a new set of drives that
>> are encrypted.  I decided the easiest way to do this is to put the new
>> drives on the NAS box and mount them like I do when I backup my large
>> Video directory only copy the Data files instead.  So, on the NAS box, I
>> set up the three drives, 2 16TB and the famous 20TB drive, with LVM and
>> dm-setup.  Once I had that setup, I mounted in just like I would the
>> Video directory.  I also ran exportfs -a so it would see the newly
>> mounted drive set and make it available.  On my main rig, I then mounted
>> it using the same command I would for the Video directory for backups. 
>> Then I took the same command for backing up my video but just replaced
>> the source and target with the Data path instead of video.  Basically,
>> everything is set up the same, I just replaced everything with the
>> drives I want info copied to in both mounting drives and commands. 
>>
>> This is the error I get. 
>>
>>
>> rsync: [generator] recv_generator: mkdir "/mnt/TV_Backup/Data/random
>> directory" failed: Permission denied (13)
>>
>>
>> I know to run the rsync command as a user, not root.  I have to do that
>> when updating my video backups.  I recall getting a error, could be this
>> one, at first but setting something to make it work.  I can't recall
>> what I had to do tho.  I also have no notes on this.  I'm sure I'm
>> missing something but no idea what.  Anyone ran into this before and
>> remember what to do to fix it?   Internet searches aren't helping either. 
>>
>> Thanks. 
>>
>> Dale
>>
>> :-)  :-) 
> OK, I am confused ...  :-/
>
> If you want to update the contents of a fs over the network, then rsync is 
> your tool.  Why is NFS coming into this at all?
>
> Assuming the user IDs are the same across systems, add '--numeric-ids'.  
> That's all.


I did some digging.  I found some info on rsync to a networked system. 
I switched to that method.  This is the command I used. 


time rsync -uivr --progress /home/dale/Desktop/Data/*
root@10.0.0.5:/mnt/backup


My old way did act odd.  It would read a lot of data from the main rig,
transfer across the network and then write to the NAS drive.  Then
repeat.  It did not do it in a continuous way tho.  It did them one at a
time.  Read a while.  Transfer.  Then write a while.  Repeat.  Overall,
it was kinda slow, ish. 

With this new method it is doing it all at the same time in a continuous
flow but slower.  I think the NAS box is slowing things down as it
doesn't have AES built into the CPU like my other boxes do.  When it
writes to the drive, all the CPU cores max out.  I think it would be
faster if the NAS box could handle encryption better.  It would then be
faster overall as well.  The NAS box is basically the bottleneck. 

I may start doing this with my backups as well.  Then again, I got all
my scripts set up for nfs already. 

Dale

:-)  :-) 

Re: [gentoo-user] Permission error when trying to rsync over nfs

[Dale]

Dale wrote:
> I did some digging.  I found some info on rsync to a networked system. 
> I switched to that method.  This is the command I used. 
>
>
> time rsync -uivr --progress /home/dale/Desktop/Data/*
> root@10.0.0.5:/mnt/backup
>
>
> My old way did act odd.  It would read a lot of data from the main rig,
> transfer across the network and then write to the NAS drive.  Then
> repeat.  It did not do it in a continuous way tho.  It did them one at a
> time.  Read a while.  Transfer.  Then write a while.  Repeat.  Overall,
> it was kinda slow, ish. 
>
> With this new method it is doing it all at the same time in a continuous
> flow but slower.  I think the NAS box is slowing things down as it
> doesn't have AES built into the CPU like my other boxes do.  When it
> writes to the drive, all the CPU cores max out.  I think it would be
> faster if the NAS box could handle encryption better.  It would then be
> faster overall as well.  The NAS box is basically the bottleneck. 
>
> I may start doing this with my backups as well.  Then again, I got all
> my scripts set up for nfs already. 
>
> Dale
>
> :-)  :-) 
>


I found something interesting.  I bought some extra hard drives over the
past few months.  I'm moving data around.  Anyway, I have mentioned that
the old NAS box CPU doesn't support AES in the CPU like my old rig or my
new main rig.  I've often wondered just what difference that makes.  To
do one of my moves, I connected two sets of drives, both encrypted, to
the NAS box.  I'm copying the data from one drive set to the other. 
Nothing fancy, just a rsync copy.  The copy process shows data moving at
about a 70MB/sec rate.  According to htop, all CPU cores are almost
maxed out and at times are maxed out. 

So, if anyone wonders what difference it makes between a CPU with and
without AES support and encrypted drives, it makes a lot of difference. 
I've done this before on my old rig and on my new rig, CPU is almost
idle.  Transfer speed is almost the max the drives can do. 

I just wanted to add this tidbit of info.  I might add, without
encryption, the NAS box can max out the data transfer of the drive
itself.  The difference is encryption and lack of AES support in the CPU. 

Oh, the 20TB drive with the slow to respond problem is now in my main
rig.  By the time I boot up and get ready to unlock the drive, they have
had plenty of time to spin up, likely 2 or 3 minutes at least.  Oh, I
ordered another 20TB drive.  Gonna see if it does the same thing. 

Dale

:-)  :-) 

Re: [gentoo-user] Permission error when trying to rsync over nfs

[Frank Steinmetzger]

[-- Attachment #1: Type: text/plain, Size: 3912 bytes --]

Am Mon, May 26, 2025 at 05:09:23AM -0500 schrieb Dale:
> Michael wrote:
> > On Monday, 19 May 2025 14:39:21 British Summer Time Dale wrote:
> >> Howdy,
> >>
> >> I'm wanting to move a Data partition over to a new set of drives that
> >> are encrypted.  I decided the easiest way to do this is to put the new
> >> drives on the NAS box and mount them like I do when I backup my large
> >> Video directory only copy the Data files instead.  So, on the NAS box, I
> >> set up the three drives, 2 16TB and the famous 20TB drive, with LVM and
> >> dm-setup.  Once I had that setup, I mounted in just like I would the
> >> Video directory.  I also ran exportfs -a so it would see the newly
> >> mounted drive set and make it available.  On my main rig, I then mounted
> >> it using the same command I would for the Video directory for backups. 
> >> Then I took the same command for backing up my video but just replaced
> >> the source and target with the Data path instead of video.  Basically,
> >> everything is set up the same, I just replaced everything with the
> >> drives I want info copied to in both mounting drives and commands. 
> >>
> >> This is the error I get. 
> >>
> >>
> >> rsync: [generator] recv_generator: mkdir "/mnt/TV_Backup/Data/random
> >> directory" failed: Permission denied (13)
> >>
> >>
> >> I know to run the rsync command as a user, not root.

Just as you should all programs that deal with user data. With rsync, if you 
give it a --delete with the wrong destination or so, you can wipe the 
system. Or overwrite the wrong files.

> >> I have to do that
> >> when updating my video backups.  I recall getting a error, could be this
> >> one, at first but setting something to make it work.

If you copy to an NFS, I don’t quite see how running the writing process (be 
it rsync or cp) as root changes the permission situation. Unless the 
destination directory is already part of the share and has write perms only 
for root.

> > If you want to update the contents of a fs over the network, then rsync is 
> > your tool.  Why is NFS coming into this at all?

You can use rsync for local copies. In this case, local also means a mounted 
NFS share. Plus, if your CPU is as old as in Dale’s NAS, using rsync over 
ssh introduces even more overhead for SSH encryption.

> I did some digging.  I found some info on rsync to a networked system. 
> I switched to that method.  This is the command I used. 
> 
> 
> time rsync -uivr --progress /home/dale/Desktop/Data/*
> root@10.0.0.5:/mnt/backup

-i and -v are redundant. While v only shows the processed file, -i adds the 
reason to the output.

> My old way did act odd.  It would read a lot of data from the main rig,
> transfer across the network and then write to the NAS drive.  Then
> repeat.  It did not do it in a continuous way tho.  It did them one at a
> time.  Read a while.  Transfer.  Then write a while.  Repeat.  Overall,
> it was kinda slow, ish. 
> 
> With this new method it is doing it all at the same time in a continuous
> flow but slower.

There is nothing fancy about the command you posted. I, too, observe this 
read-write cycle when doing stuff over SSH onto a ZFS share and it annoys me 
sometimes. But haven’t had the urge to solve it yet. I don’t transfer huge 
volumes of files that often.

What is not very often used in my experience is the -u flag. It causes rsync 
to skip all files whose timestamp hasn’t changed. If you don’t use it, maybe 
rsync’s detection of unchanged files does not work properly over NFS, hence 
it reads and writes all files again. (Just a conjecture on my part.)

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

As long as my boss pretends to pay me adequatly,
I will pretend to work properly.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]