[gentoo-user] detox'ing files by keeping their time stamp?

[gentoo-user] detox'ing files by keeping their time stamp?

[tuxic]

Hi,

when downloading files from non-UNIX sites, they often contain
"poisonoys" characters like '#', ' ', ''' or that alike.

With the tool 'detox' those filenames could be fixed.

But detox changes the time stamp of the files, which 
filenames are altered (not all files, which are examined).

Therefore a 'detox *' in a directory will completly shuffle
the file order when listed via 'ls -rtl' (RTL is a TV broadcaster,
but this is another story... ;) )

Is there a way to either get detox not to alter the time stamp (I
looked into the manpage but found nothing) or any other neat trick
to restore the time stamp afterwards?

Thanks for any in advanced for saveing my time(stamps!) :)

Cheers
Meino

Re: [gentoo-user] detox'ing files by keeping their time stamp?

[Stroller]

> On 18 Feb 2018, at 08:21, tuxic@posteo.de wrote:
> 
> when downloading files from non-UNIX sites, they often contain
> "poisonoys" characters like '#', ' ', ''' or that alike.
> 
> With the tool 'detox' those filenames could be fixed.
> 
> But detox changes the time stamp of the files, which 
> filenames are altered (not all files, which are examined).
> 
> Is there a way to either get detox not to alter the time stamp 

I think:

  tmpfile=/tmp/foo-$RANDOM
  touch -r "$file" "$tmpfile"
  detox "$file" 
  touch -r "$tmpfile "$file"
  rm "$tmpfile"

It should be trivial to patch detox to do this itself.

Stroller

Re: [gentoo-user] detox'ing files by keeping their time stamp?

[tuxic]

On 02/18 11:38, Stroller wrote:
> 
> > On 18 Feb 2018, at 08:21, tuxic@posteo.de wrote:
> > 
> > when downloading files from non-UNIX sites, they often contain
> > "poisonoys" characters like '#', ' ', ''' or that alike.
> > 
> > With the tool 'detox' those filenames could be fixed.
> > 
> > But detox changes the time stamp of the files, which 
> > filenames are altered (not all files, which are examined).
> > 
> > Is there a way to either get detox not to alter the time stamp 
> 
> I think:
> 
>   tmpfile=/tmp/foo-$RANDOM
>   touch -r "$file" "$tmpfile"
>   detox "$file" 
>   touch -r "$tmpfile "$file"
>   rm "$tmpfile"
> 
> It should be trivial to patch detox to do this itself.
> 
> Stroller
> 
> 
> 

Hi Stroller,

will try that ! Thanks a lot!

Cheers!
Meino

Re: [gentoo-user] detox'ing files by keeping their time stamp?

[tuxic]

On 02/18 11:38, Stroller wrote:
> 
> > On 18 Feb 2018, at 08:21, tuxic@posteo.de wrote:
> > 
> > when downloading files from non-UNIX sites, they often contain
> > "poisonoys" characters like '#', ' ', ''' or that alike.
> > 
> > With the tool 'detox' those filenames could be fixed.
> > 
> > But detox changes the time stamp of the files, which 
> > filenames are altered (not all files, which are examined).
> > 
> > Is there a way to either get detox not to alter the time stamp 
> 
> I think:
> 
>   tmpfile=/tmp/foo-$RANDOM
>   touch -r "$file" "$tmpfile"
>   detox "$file" 
>   touch -r "$tmpfile "$file"
>   rm "$tmpfile"
> 
> It should be trivial to patch detox to do this itself.
> 
> Stroller
> 
> 
> 

Hi Stroller,

this seems to be an egg<->chicken problem.

I like to wrap detox with a script, which will do you magic trick.
Since I want to get rid of those evil characters (...) in the filename, 
which normally intercept shell processing, I want to use detox,
which in turn will be called by a shell script in turn, to do the 
time machine magic. To do so, I need detox, to sanitize the
filenames from the evil characters, which normally intercept.....
.....stack overflow....recursion depth failure.....process killed.

You know....

I am using zsh...

Any idea to get a chicken OR an egg instead of an scrambled egg with
feathers??? ;)

Cheers
Meino

Re: [gentoo-user] detox'ing files by keeping their time stamp?

[Floyd Anderson]

On Sun, 18 Feb 2018 13:07:33 +0100
tuxic@posteo.de wrote:
>On 02/18 11:38, Stroller wrote:
>>
>> > On 18 Feb 2018, at 08:21, tuxic@posteo.de wrote:
>> >
>> > when downloading files from non-UNIX sites, they often contain
>> > "poisonoys" characters like '#', ' ', ''' or that alike.
>> >
>> > With the tool 'detox' those filenames could be fixed.
>> >
>> > But detox changes the time stamp of the files, which
>> > filenames are altered (not all files, which are examined).
>> >
>> > Is there a way to either get detox not to alter the time stamp
>>
>> I think:
>>
>>   tmpfile=/tmp/foo-$RANDOM
>>   touch -r "$file" "$tmpfile"
>>   detox "$file"
>>   touch -r "$tmpfile "$file"
>>   rm "$tmpfile"
>>
>> It should be trivial to patch detox to do this itself.
>>
>> Stroller
>>
>>
>>
>
>Hi Stroller,
>
>this seems to be an egg<->chicken problem.
>
>I like to wrap detox with a script, which will do you magic trick.
>Since I want to get rid of those evil characters (...) in the filename,
>which normally intercept shell processing, I want to use detox,
>which in turn will be called by a shell script in turn, to do the
>time machine magic. To do so, I need detox, to sanitize the
>filenames from the evil characters, which normally intercept.....
>.....stack overflow....recursion depth failure.....process killed.
>
>You know....
>
>I am using zsh...
>
>Any idea to get a chicken OR an egg instead of an scrambled egg with
>feathers??? ;)

Go back one step and reread the manual page. It seems to be there is an 
option ‘--dry-run’ (implies ‘--verbose’) that can probably be used to 
store a list of the final new file names. Afterwards you can traverse 
this list with Stroller’s suggestion (slightly adopted of course).

Or you can try other tools which doesn’t use function rename() [1], e.g. 
perl-rename, and therefore don’t change the last modification time.

Or you can go two steps back and save the file(s) to your like when you 
download it, e.g. with curl (maybe your’re also interested in its 
‘--remote-time’ option).


References:
  - [1] <http://pubs.opengroup.org/onlinepubs/9699919799/functions/rename.html>


-- 
Regards,
floyd

Re: [gentoo-user] detox'ing files by keeping their time stamp?

[tuxic]

On 02/18 01:55, Floyd Anderson wrote:
> On Sun, 18 Feb 2018 13:07:33 +0100
> tuxic@posteo.de wrote:
> > On 02/18 11:38, Stroller wrote:
> > > 
> > > > On 18 Feb 2018, at 08:21, tuxic@posteo.de wrote:
> > > >
> > > > when downloading files from non-UNIX sites, they often contain
> > > > "poisonoys" characters like '#', ' ', ''' or that alike.
> > > >
> > > > With the tool 'detox' those filenames could be fixed.
> > > >
> > > > But detox changes the time stamp of the files, which
> > > > filenames are altered (not all files, which are examined).
> > > >
> > > > Is there a way to either get detox not to alter the time stamp
> > > 
> > > I think:
> > > 
> > >   tmpfile=/tmp/foo-$RANDOM
> > >   touch -r "$file" "$tmpfile"
> > >   detox "$file"
> > >   touch -r "$tmpfile "$file"
> > >   rm "$tmpfile"
> > > 
> > > It should be trivial to patch detox to do this itself.
> > > 
> > > Stroller
> > > 
> > > 
> > > 
> > 
> > Hi Stroller,
> > 
> > this seems to be an egg<->chicken problem.
> > 
> > I like to wrap detox with a script, which will do you magic trick.
> > Since I want to get rid of those evil characters (...) in the filename,
> > which normally intercept shell processing, I want to use detox,
> > which in turn will be called by a shell script in turn, to do the
> > time machine magic. To do so, I need detox, to sanitize the
> > filenames from the evil characters, which normally intercept.....
> > .....stack overflow....recursion depth failure.....process killed.
> > 
> > You know....
> > 
> > I am using zsh...
> > 
> > Any idea to get a chicken OR an egg instead of an scrambled egg with
> > feathers??? ;)
> 
> Go back one step and reread the manual page. It seems to be there is an
> option ‘--dry-run’ (implies ‘--verbose’) that can probably be used to store
> a list of the final new file names. Afterwards you can traverse this list
> with Stroller’s suggestion (slightly adopted of course).
> 
> Or you can try other tools which doesn’t use function rename() [1], e.g.
> perl-rename, and therefore don’t change the last modification time.
> 
> Or you can go two steps back and save the file(s) to your like when you
> download it, e.g. with curl (maybe your’re also interested in its
> ‘--remote-time’ option).
> 
> 
> References:
>  - [1] <http://pubs.opengroup.org/onlinepubs/9699919799/functions/rename.html>
> 
> 
> -- 
> Regards,
> floyd
> 
> 

Hi Floyd,

the unrenamed files are the only ones with the correct timestamp.
Therefore 'touch' has to access them.
But their filenames contain the poisonous characters.
And the circle starts right from the beginning.
The problem arises at that moment, where I need to feed the name
of a single file what program ever, since first there is the shell...
even when calling other programs.

Cheers
Meino

Re: [gentoo-user] detox'ing files by keeping their time stamp?

[Floyd Anderson]

On Sun, 18 Feb 2018 14:44:24 +0100
tuxic@posteo.de wrote:
>On 02/18 01:55, Floyd Anderson wrote:
>> On Sun, 18 Feb 2018 13:07:33 +0100
>> tuxic@posteo.de wrote:
>> > On 02/18 11:38, Stroller wrote:
>> > >
>> > > > On 18 Feb 2018, at 08:21, tuxic@posteo.de wrote:
>> > > >
>> > > > when downloading files from non-UNIX sites, they often contain
>> > > > "poisonoys" characters like '#', ' ', ''' or that alike.
>> > > >
>> > > > With the tool 'detox' those filenames could be fixed.
>> > > >
>> > > > But detox changes the time stamp of the files, which
>> > > > filenames are altered (not all files, which are examined).
>> > > >
>> > > > Is there a way to either get detox not to alter the time stamp
>> > >
>> > > I think:
>> > >
>> > >   tmpfile=/tmp/foo-$RANDOM
>> > >   touch -r "$file" "$tmpfile"
>> > >   detox "$file"
>> > >   touch -r "$tmpfile "$file"
>> > >   rm "$tmpfile"
>> > >
>> > > It should be trivial to patch detox to do this itself.
>> > >
>> > > Stroller
>> > >
>> > >
>> > >
>> >
>> > Hi Stroller,
>> >
>> > this seems to be an egg<->chicken problem.
>> >
>> > I like to wrap detox with a script, which will do you magic trick.
>> > Since I want to get rid of those evil characters (...) in the filename,
>> > which normally intercept shell processing, I want to use detox,
>> > which in turn will be called by a shell script in turn, to do the
>> > time machine magic. To do so, I need detox, to sanitize the
>> > filenames from the evil characters, which normally intercept.....
>> > .....stack overflow....recursion depth failure.....process killed.
>> >
>> > You know....
>> >
>> > I am using zsh...
>> >
>> > Any idea to get a chicken OR an egg instead of an scrambled egg with
>> > feathers??? ;)
>>
>> Go back one step and reread the manual page. It seems to be there is an
>> option ‘--dry-run’ (implies ‘--verbose’) that can probably be used to store
>> a list of the final new file names. Afterwards you can traverse this list
>> with Stroller’s suggestion (slightly adopted of course).
>>
>> Or you can try other tools which doesn’t use function rename() [1], e.g.
>> perl-rename, and therefore don’t change the last modification time.
>>
>> Or you can go two steps back and save the file(s) to your like when you
>> download it, e.g. with curl (maybe your’re also interested in its
>> ‘--remote-time’ option).
>>
>>
>> References:
>>  - [1] <http://pubs.opengroup.org/onlinepubs/9699919799/functions/rename.html>
>>
>>
>> --
>> Regards,
>> floyd
>>
>>
>
>Hi Floyd,
>
>the unrenamed files are the only ones with the correct timestamp.
>Therefore 'touch' has to access them.
>But their filenames contain the poisonous characters.

So you have to figure out why detox, that I doesn’t know and thus never 
have been used, does not rename those files. Maybe because the new file 
(after file name translation) already exists in directory as mentioned 
in the BUGS section of the manual page. So you must ensure that all 
resulting file names are unique.

A second thought is that there probably isn’t a rule for a specific 
character translation, so detox won’t change those characters until you 
define a rule first [1].

>And the circle starts right from the beginning.
>The problem arises at that moment, where I need to feed the name
>of a single file what program ever, since first there is the shell...
>even when calling other programs.

Here comes escaping and/or quoting into play but the glob `detox *`, 
you’ve specified, should work. Can you share a sample file name with 
funny characters in it?


References:
  - [1] <http://detox.sourceforge.net/detox.tbl.5.html>


-- 
Regards,
floyd

Re: [gentoo-user] detox'ing files by keeping their time stamp?

[David Haller]

Hello,

On Sun, 18 Feb 2018, Floyd Anderson wrote:
>On Sun, 18 Feb 2018 14:44:24 +0100
>tuxic@posteo.de wrote:
>> On 02/18 01:55, Floyd Anderson wrote:
>> > On Sun, 18 Feb 2018 13:07:33 +0100
>> > tuxic@posteo.de wrote:
>> > > On 02/18 11:38, Stroller wrote:
[..]
>> > > > I think:
>> > > >
>> > > >   tmpfile=/tmp/foo-$RANDOM
>> > > >   touch -r "$file" "$tmpfile"
>> > > >   detox "$file"
>> > > >   touch -r "$tmpfile "$file"
>> > > >   rm "$tmpfile"
[..]
>> > > I like to wrap detox with a script, which will do you magic trick.
>> > > Since I want to get rid of those evil characters (...) in the filename,
>> > > which normally intercept shell processing, I want to use detox,
>> > > which in turn will be called by a shell script in turn, to do the
>> > > time machine magic. To do so, I need detox, to sanitize the
>> > > filenames from the evil characters, which normally intercept.....
>> > > .....stack overflow....recursion depth failure.....process killed.
[..]
>So you have to figure out why detox, that I doesn't know and thus never have
>been used, does not rename those files. Maybe because the new file (after
>file name translation) already exists in directory as mentioned in the BUGS
>section of the manual page. So you must ensure that all resulting file names
>are unique.
[..]
>> And the circle starts right from the beginning.
>> The problem arises at that moment, where I need to feed the name
>> of a single file what program ever, since first there is the shell...
>> even when calling other programs.
>
>Here comes escaping and/or quoting into play but the glob `detox *`, you've
>specified, should work. Can you share a sample file name with funny
>characters in it?

Well, at least bash is robust enough if you quote variables correctly.

$ ls -lb
total 4
-rw-r--r-- 1 dh dh   0 Feb 18 17:23 a\ "\ b\ '\ c\ #\ d
-rw-r--r-- 1 dh dh   0 Feb 18 17:27 a"b\ c
-rw-r--r-- 1 dh dh   0 Feb 18 17:26 a'b
-rw-r----- 1 dh dh 166 Feb 18 17:26 t.sh

$ cat t.sh
#!/bin/bash
TMPF=$(mktemp "/tmp/detox_wrapper.$$.XXXXXXXX")
for f in "$@"; do
    touch -r "$f" "$TMPF"
    detox "$f"
    touch -r "$TMPF" "$f"
done
rm -f "$TMPF"

$ bash t.sh *
$ ls -lb
-rw-r--r-- 1 dh dh   0 Feb 18 17:23 a\ "\ b\ '\ c\ #\ d
-rw-r--r-- 1 dh dh   0 Feb 18 17:27 a"b\ c
-rw-r--r-- 1 dh dh   0 Feb 18 17:26 a'b
-rw-r--r-- 1 dh dh   0 Feb 18 17:26 a_b
-rw-r--r-- 1 dh dh   0 Feb 18 17:27 a_b_c
-rw-r--r-- 1 dh dh   0 Feb 18 17:23 a_b_c_#_d
-rw-r----- 1 dh dh 163 Feb 18 17:28 t.sh
$ rm *_*
$ zsh t.sh *
$ ls -lb
total 4
-rw-r--r-- 1 dh dh   0 Feb 18 17:23 a\ "\ b\ '\ c\ #\ d
-rw-r--r-- 1 dh dh   0 Feb 18 17:27 a"b\ c
-rw-r--r-- 1 dh dh   0 Feb 18 17:26 a'b
-rw-r--r-- 1 dh dh   0 Feb 18 17:26 a_b
-rw-r--r-- 1 dh dh   0 Feb 18 17:27 a_b_c
-rw-r--r-- 1 dh dh   0 Feb 18 17:23 a_b_c_#_d
-rw-r----- 1 dh dh 163 Feb 18 17:28 t.sh

So, zsh can do it too. And pdksh.

HTH,
-dnh

-- 
printk("you lose buddy boy...\n");
        linux-2.6.6/arch/sparc/kernel/traps.c

Re: [gentoo-user] detox'ing files by keeping their time stamp?

[Floyd Anderson]

Hi David,

On Sun, 18 Feb 2018 17:34:21 +0100
David Haller <gentoo@dhaller.de> wrote:
>Hello,
>
>On Sun, 18 Feb 2018, Floyd Anderson wrote:
>>On Sun, 18 Feb 2018 14:44:24 +0100
>>tuxic@posteo.de wrote:
>>> On 02/18 01:55, Floyd Anderson wrote:
>>> > On Sun, 18 Feb 2018 13:07:33 +0100
>>> > tuxic@posteo.de wrote:
>>> > > On 02/18 11:38, Stroller wrote:
>[..]
>>> > > > I think:
>>> > > >
>>> > > >   tmpfile=/tmp/foo-$RANDOM
>>> > > >   touch -r "$file" "$tmpfile"
>>> > > >   detox "$file"
>>> > > >   touch -r "$tmpfile "$file"
>>> > > >   rm "$tmpfile"
>[..]
>>> > > I like to wrap detox with a script, which will do you magic trick.
>>> > > Since I want to get rid of those evil characters (...) in the filename,
>>> > > which normally intercept shell processing, I want to use detox,
>>> > > which in turn will be called by a shell script in turn, to do the
>>> > > time machine magic. To do so, I need detox, to sanitize the
>>> > > filenames from the evil characters, which normally intercept.....
>>> > > .....stack overflow....recursion depth failure.....process killed.
>[..]
>>So you have to figure out why detox, that I doesn't know and thus never have
>>been used, does not rename those files. Maybe because the new file (after
>>file name translation) already exists in directory as mentioned in the BUGS
>>section of the manual page. So you must ensure that all resulting file names
>>are unique.
>[..]
>>> And the circle starts right from the beginning.
>>> The problem arises at that moment, where I need to feed the name
>>> of a single file what program ever, since first there is the shell...
>>> even when calling other programs.
>>
>>Here comes escaping and/or quoting into play but the glob `detox *`, you've
>>specified, should work. Can you share a sample file name with funny
>>characters in it?
>
>Well, at least bash is robust enough if you quote variables correctly.
>
>$ ls -lb
>total 4
>-rw-r--r-- 1 dh dh   0 Feb 18 17:23 a\ "\ b\ '\ c\ #\ d
>-rw-r--r-- 1 dh dh   0 Feb 18 17:27 a"b\ c
>-rw-r--r-- 1 dh dh   0 Feb 18 17:26 a'b
>-rw-r----- 1 dh dh 166 Feb 18 17:26 t.sh
>
>$ cat t.sh
>#!/bin/bash
>TMPF=$(mktemp "/tmp/detox_wrapper.$$.XXXXXXXX")
>for f in "$@"; do
>    touch -r "$f" "$TMPF"
>    detox "$f"
>    touch -r "$TMPF" "$f"
>done
>rm -f "$TMPF"

If I’m not totally wrong, the second `touch` cannot work because the 
file that "$f" holds is renamed now. That’s what I mean earlier with 
iterating a list or adapt Stroller’s suggestion.

I have no idea if:

    modtime="$(stat -c '%Y' "$f")"
    newfile="$(detox $"f")"
    touch -d "@$modtime" "$newfile"

is possible instead.


-- 
Regards,
floyd

Re: [gentoo-user] detox'ing files by keeping their time stamp?

[Stroller]

> On 18 Feb 2018, at 17:47, Floyd Anderson <f.a@31c0.net> wrote:
>> 
>> $ cat t.sh
>> #!/bin/bash
>> TMPF=$(mktemp "/tmp/detox_wrapper.$$.XXXXXXXX")
>> for f in "$@"; do
>>   touch -r "$f" "$TMPF"
>>   detox "$f"
>>   touch -r "$TMPF" "$f"
>> done
>> rm -f "$TMPF"
> 
> If I’m not totally wrong, the second `touch` cannot work because the file that "$f" holds is renamed now. That’s what I mean earlier with iterating a list or adapt Stroller’s suggestion.

How careless of me.

A solution is to use `detox -v` and capture the output.

   $ touch '1234[]'
   $ ls '1234[]'
   1234[]
   $ detox -v 1234*
   Scanning: 1234[]
   1234[] -> 1234-
   $

A bit untidy. Really, detox should be patched to check the date and apply it to the new file.

Stroller.

Re: [gentoo-user] detox'ing files by keeping their time stamp?

[Stroller]

> On 18 Feb 2018, at 11:38, Stroller <stroller@stellar.eclipse.co.uk> wrote:
>> 
>> With the tool 'detox' those filenames could be fixed.
>> 
>> But detox changes the time stamp of the files, which 
>> filenames are altered (not all files, which are examined).
>> 
>> Is there a way to either get detox not to alter the time stamp 
> 
> …
> It should be trivial to patch detox to do this itself.


BTW: `ebuild /usr/portage/app-misc/detox/detox-1.2.0-r3.ebuild unpack`

The function parse_file in file.c seems to be what does the work.

Use the utime systemcall? https://linux.die.net/man/2/utime

Stroller.

Re: [gentoo-user] detox'ing files by keeping their time stamp?

[karl]

Stroller:
> > On 18 Feb 2018, at 17:47, Floyd Anderson <f.a@31c0.net> wrote:
> >> 
> >> $ cat t.sh
> >> #!/bin/bash
> >> TMPF=$(mktemp "/tmp/detox_wrapper.$$.XXXXXXXX")
> >> for f in "$@"; do
> >>   touch -r "$f" "$TMPF"
> >>   detox "$f"
> >>   touch -r "$TMPF" "$f"
> >> done
> >> rm -f "$TMPF"
> > 
> > If I’m not totally wrong, the second `touch` cannot work because the file that "$f" holds is renamed now. That’s what I mean earlier with iterating a list or adapt Stroller’s suggestion.
> 
> How careless of me.
> 
> A solution is to use `detox -v` and capture the output.
> 
>    $ touch '1234[]'
>    $ ls '1234[]'
>    1234[]
>    $ detox -v 1234*
>    Scanning: 1234[]
>    1234[] -> 1234-
>    $
> 
> A bit untidy. Really, detox should be patched to check the date and apply it to the new file.

I must be missing something, but why don't you use plain old mv, it 
doen't change the times as long you stay in the same file system:

$ stat br.log | grep 201
Access: 2018-02-07 22:59:51.746741788 +0100
Modify: 2016-08-16 15:34:40.742454976 +0200
Change: 2018-02-07 12:30:09.737085062 +0100
$ mv br.log z.log | grep 201
$ stat z.log 
Access: 2018-02-07 22:59:51.746741788 +0100
Modify: 2016-08-16 15:34:40.742454976 +0200
Change: 2018-02-19 16:24:35.662712287 +0100


And regarding detox -v, why not use tr

ls -1 |
while read a
do
 b=`echo "$a" | tr '[]' '-'` # or whatever mapping you want
 mv "$a" "$b"
done

Regards,
/Karl Hammar

-----------------------------------------------------------------------
Aspö Data
Lilla Aspö 148
S-742 94 Östhammar
Sweden
+46 173 140 57