[gentoo-user] Cloning movie DVDs with dd - only works after accessing disk with another command?

[gentoo-user] Cloning movie DVDs with dd - only works after accessing disk with another command?

[Stroller]

Hi there,

Now I've got this new machine with all this space on it (or rather:  
now I seem to be more interested in doing stuff with this machine I  
built a while ago), I'm starting on the process of ripping all my DVDs.

In the past I tried media-video/undvd for DVD ripping, but its .mp4  
support now is broken, and probably won't be fixed.

This week I've been trying HandBrakeCLI, and it seems to work pretty  
darn lovely.

But the difference is that undvd has a simple --clone option which  
copies the DVD to disk first, and HandBrakeCLI doesn't.

To save wear & tear on my optical drive I have been cloning the DVD to  
disk before ripping, and I've noticed that it doesn't seem to work if  
I just do `dd if=/dev/cdrom of=disc.iso`. It seems like I have to  
access the disk using some other command first, then dd works just  
fine (in exactly the way I tried a moment before).

Can anyone explain this, please?

I'm guessing it has something to do with DeCSS encryption, but firstly  
I don't understand how that applies to dd, because I'd have assumed  
that treats the drive as a block device. Secondly, if this is related  
to DeCSS, I don't understand why accessing the disk with one command  
leaves it unlocked for another command to access the disk a minute  
later.

Anyone got any thoughts, please?

I haven't looked into this deeply myself, yet, so I'm not able to give  
a full analysis of how exactly the behaviour is manifesting itself. I  
tried dd'ing one disk and it didn't work, so I started undvd working  
with the --clone option and cancelled when the cloning had finished  
(and the rip began). I basically wanted to see if it worked or failed,  
and it worked just fine, so I looked through undvd's source and it  
_seems_ (I say "seems" lest I'm reading undvd's perl code wrongly)  
just to use dd itself. So next I tried `dd if=/dev/cdrom  
of=disc_manual_dd.iso` myself and not only did it work just fine, but  
the md5sums of the two images (the one produced by undvd & the one  
produced by my manual dd) were identical.

So I tried with a different disk - take a look at the attached  
cloning.txt - and dd fails repeatedly at 770kB. Then I run scandvd on  
the disk and after that dd works perfectly, "8027521024 bytes (8.0 GB)  
copied". scandvd is a tool which is packaged with undvd and it runs  
mplayer (I'm sure) on the disk, then shows the number of tracks on the  
disk in a pretty format.

This is no kind of a show-stopper for me, because I've described the  
workaround above, it's just a curiosity. I guess I'm not alone on here  
in wanting to know how these computer things work.

I'd be really interested if anyone else's DVD drives show the same  
behaviour. Does dd fail for you when you try it on a new movie? If you  
don't have undvd installed, just run `mplayer dvd://`, cancel it and  
then try dd again.

Sorry if I've been a little verbose with my explanation, BTW. It's a  
little late here, I'm a little tired, and with these weird things that  
I almost don't believe myself I always like to explain  
comprehensively, and it prolly reads like I'm blabbering.

Thanks for any thoughts,

Stroller.

Re: [gentoo-user] Cloning movie DVDs with dd - only works after accessing disk with another command?

[Stroller]

[-- Attachment #1: Type: text/plain, Size: 481 bytes --]


On 11 Aug 2009, at 05:43, Stroller wrote:
> ...
> So I tried with a different disk - take a look at the attached  
> cloning.txt - and dd fails repeatedly at 770kB. Then I run scandvd  
> on the disk and after that dd works perfectly, "8027521024 bytes  
> (8.0 GB) copied". scandvd is a tool which is packaged with undvd and  
> it runs mplayer (I'm sure) on the disk, then shows the number of  
> tracks on the disk in a pretty format.


Sorry, here's the promised attachment.


[-- Attachment #2: cloning.txt --]
[-- Type: text/plain, Size: 6357 bytes --]

~ $ cd /mnt/space/DVDrip/
/mnt/space/DVDrip $ ls
Day Of The Triffids  Heat
/mnt/space/DVDrip $ mkdir Narc
/mnt/space/DVDrip $ cd Narc/
/mnt/space/DVDrip/Narc $ dd if=/dev/cdrom of=disc.iso
dd: reading `/dev/cdrom': Input/output error
1504+0 records in
1504+0 records out
770048 bytes (770 kB) copied, 4.35681 s, 177 kB/s
/mnt/space/DVDrip/Narc $ dd if=/dev/cdrom of=disc.iso
dd: reading `/dev/cdrom': Input/output error
1504+0 records in
1504+0 records out
770048 bytes (770 kB) copied, 0.469213 s, 1.6 MB/s
/mnt/space/DVDrip/Narc $ dd if=/dev/cdrom of=disc.iso
dd: reading `/dev/cdrom': Input/output error
1504+0 records in
1504+0 records out
770048 bytes (770 kB) copied, 0.440263 s, 1.7 MB/s
/mnt/space/DVDrip/Narc $ dd if=/dev/cdrom of=disc.iso
dd: reading `/dev/cdrom': Input/output error
1504+0 records in
1504+0 records out
770048 bytes (770 kB) copied, 0.46938 s, 1.6 MB/s
/mnt/space/DVDrip/Narc $ dd if=/dev/cdrom of=disc.iso
dd: reading `/dev/cdrom': Input/output error
1504+0 records in
1504+0 records out
770048 bytes (770 kB) copied, 0.457693 s, 1.7 MB/s
/mnt/space/DVDrip/Narc $ dd if=/dev/cdrom of=disc.iso
dd: reading `/dev/cdrom': Input/output error
1504+0 records in
1504+0 records out
770048 bytes (770 kB) copied, 0.450244 s, 1.7 MB/s
/mnt/space/DVDrip/Narc $ dd if=/dev/cdrom of=disc.iso
dd: reading `/dev/cdrom': Input/output error
1504+0 records in
1504+0 records out
770048 bytes (770 kB) copied, 0.455701 s, 1.7 MB/s
/mnt/space/DVDrip/Narc $ scandvd
{( --- scandvd 0.7.5 --- )}
 * Scanning DVD for titles with lsdvd...
01  length: 00:00:00  audio: en en en en en en en en  subs: xx xx xx xx xx xx xx
 xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx xx
02  length: 00:13:20  audio: en  subs: en
03  length: 00:09:51  audio: en  subs: en
04  length: 01:40:56  audio: en en  subs: en en
05  length: 00:19:26  audio: en  subs: en
06  length: 00:12:57  audio: en  subs: en
07  length: 00:02:26  audio: en

To watch a title:
 mplayer       dvd://01     -alang en  -slang en/off
To rip titles:
 undvd         -t 01,02,03  -a en      -s en/off
/mnt/space/DVDrip/Narc $ dd if=/dev/cdrom of=disc.iso
15678752+0 records in
15678752+0 records out
8027521024 bytes (8.0 GB) copied, 1352.31 s, 5.9 MB/s
/mnt/space/DVDrip/Narc $
/mnt/space/DVDrip/Narc $ mplayer dvd://4 -dvd-device disc.iso -ao null -vo png -ss 600 -frames 1
MPlayer SVN-r29463-4.1.2 (C) 2000-2009 MPlayer Team

Playing dvd://4.
libdvdread: Using libdvdcss version 1.2.10 for DVD access
There are 7 titles on this DVD.
There are 1 angles in this DVD title.

libdvdread: Attempting to retrieve all CSS keys
libdvdread: This can take a _long_ time, please be patient

libdvdread: Get key for /VIDEO_TS/VIDEO_TS.VOB at 0x00000147
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_01_0.VOB at 0x000001b3
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_01_1.VOB at 0x00013507
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_02_0.VOB at 0x000135e2
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_02_1.VOB at 0x0001362f
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_03_0.VOB at 0x0005e0b0
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_03_1.VOB at 0x0005e0fd
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_04_0.VOB at 0x0009551b
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_04_1.VOB at 0x00095568
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_05_0.VOB at 0x002f2cc9
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_05_1.VOB at 0x002f2d16
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_06_0.VOB at 0x0035f7d4
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_06_1.VOB at 0x0035f821
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_07_0.VOB at 0x003a8051
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_07_1.VOB at 0x003a809e
libdvdread: Elapsed time 0
libdvdread: Found 7 VTS's
libdvdread: Elapsed time 0
audio stream: 0 format: ac3 (5.1) language: en aid: 128.
audio stream: 1 format: ac3 (stereo) language: en aid: 129.
number of audio channels on disk: 2.
subtitle ( sid ): 0 language: en
subtitle ( sid ): 1 language: en
number of subtitles on disk: 2
MPEG-PS file format detected.
VIDEO:  MPEG2  720x576  (aspect 3)  25.000 fps  9000.0 kbps (1125.0 kbyte/s)
==========================================================================
Opening video decoder: [mpegpes] MPEG 1/2 Video passthrough
VDec: vo config request - 720 x 576 (preferred colorspace: Mpeg PES)
Could not find matching colorspace - retrying with -vf scale...
Opening video filter: [scale]
The selected video_out device is incompatible with this codec.
Try appending the scale filter to your filter list,
e.g. -vf spp,scale instead of -vf spp.
VDecoder init failed :(
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
Unsupported PixelFormat -1
Selected video codec: [ffmpeg2] vfm: ffmpeg (FFmpeg MPEG-2)
==========================================================================
==========================================================================
Opening audio decoder: [liba52] AC3 decoding with liba52
Using SSE optimized IMDCT transform
Using MMX optimized resampler
AUDIO: 48000 Hz, 2 ch, s16le, 448.0 kbit/29.17% (ratio: 56000->192000)
Selected audio codec: [a52] afm: liba52 (AC3-liba52)
==========================================================================
AO: [null] 48000Hz 2ch s16le (2 bytes per sample)
Starting playback...
VDec: vo config request - 720 x 576 (preferred colorspace: Planar YV12)
Could not find matching colorspace - retrying with -vf scale...
Opening video filter: [scale]
VDec: using Planar YV12 as output csp (no 0)
Movie-Aspect is 1.78:1 - prescaling to correct movie aspect.
[swscaler @ 0x88861a0]using unscaled yuv420p -> bgr24 special converter
VO: [png] 720x576 => 1024x576 BGR 24-bit
[VO_PNG] Warning: compression level set to 0, compression disabled!
[VO_PNG] Info: Use -vo png:z=<n> to set compression level from 0 to 9.
[VO_PNG] Info: (0 = no compression, 1 = fastest, lowest - 9 best, slowest compre
ssion)
png: . - Output directory already exists and is writable.
A: 600.4 V: 600.5 A-V: -0.082 ct:  0.104   4/  4 ??% ??% ??,?% 0 0

Exiting... (End of file)
/mnt/space/DVDrip/Narc $

[-- Attachment #3: Type: text/plain, Size: 1 bytes --]

[gentoo-user] Re: Cloning movie DVDs with dd - only works after accessing disk with another command?

[Remy Blank]

[-- Attachment #1: Type: text/plain, Size: 393 bytes --]

This is just a wild guess, but could the decryption keys be stored in
the DVD drive? Playing the disc with mplayer retrieves the keys (using
libdvdread) and sets them in the drive. From that point, the disc is
readable by all means. When you change the disc, the keys are not valid
anymore.

As I said, this is a wild guess, I have no actual knowledge about these
things.

-- Remy


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Cloning movie DVDs with dd - only works after accessing disk with another command?

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 535 bytes --]

On Tue, 11 Aug 2009 05:43:17 +0100, Stroller wrote:

> I'm guessing it has something to do with DeCSS encryption, but firstly  
> I don't understand how that applies to dd, because I'd have assumed  
> that treats the drive as a block device.

Probably because the CSS stuff is stored in a separate area of the drive,
which is not copied by dd. If you use vobcopy with the mirror option, it
creates a decrypted copy of the DVD's contents.


-- 
Neil Bothwick

Stupidity is NOT a handicap. You'll have to park elsewhere.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Cloning movie DVDs with dd - only works after accessing disk with another command?

[Stroller]

On 11 Aug 2009, at 08:33, Neil Bothwick wrote:
> On Tue, 11 Aug 2009 05:43:17 +0100, Stroller wrote:
>
>> I'm guessing it has something to do with DeCSS encryption, but  
>> firstly
>> I don't understand how that applies to dd, because I'd have assumed
>> that treats the drive as a block device.
>
> Probably because the CSS stuff is stored in a separate area of the  
> drive,
> which is not copied by dd.

I think I may not have phrased the question clearly. Initially dd  
doesn't work, then it magically starts working. See the cloning.txt  
attached to my second post.

Stroller.

Re: [gentoo-user] Cloning movie DVDs with dd - only works after accessing disk with another command?

[Joerg Schilling]

Stroller <stroller@stellar.eclipse.co.uk> wrote:

> Now I've got this new machine with all this space on it (or rather:  
> now I seem to be more interested in doing stuff with this machine I  
> built a while ago), I'm starting on the process of ripping all my DVDs.
>
> In the past I tried media-video/undvd for DVD ripping, but its .mp4  
> support now is broken, and probably won't be fixed.
>
> This week I've been trying HandBrakeCLI, and it seems to work pretty  
> darn lovely.

You cannot simply clone a DVD with dd.

1)	Use dvdcopy with deCSS and create a copied directory tree.
	Make sure to retain kapital letters in filenames.

2)	Use mkisofs -dvd-video to create a new ifs image

3)	Use cdrecord -atip tp read the layerbreak

4)	Use cdrecord -dao -v driveropts=layerbreak=# xxx.iso
	to write the data to a DVD+R/DL

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

[gentoo-user] Bash script inquiry

[Richard Marza]

I'm trying to run a command in a loop. I have a counter device set...the 
number that the counter generates is supposed to go inside the command in 
the loop after every successive iteration of the loop. This is all really to 
get a general idea I've attached a snippet below.

FILE=`cat filename.txt`
TICK=`cat filename.txt | wc -l'
TOCK="0"

while [ $TICK != $TOCK ] ; do
            let $TOCK=$TOCK+1
            Var1= `cat FirstWordOfFirstColumnOfFirstLine` (This I actually 
achieved with sed and awk)
            Var2=`cat FirstFloatOfFirstLine`   (The problem lies here; it's 
my inability to come up with a way of implementing a variable that changes 
along with the counter.  so that the second time this is run it doesn't do 
the first line but moved to the second line and the third line and so on...)

done

exit 0



My file is like so:


Variable        Sys1    Sys2            Sys3            Sys4      Sys5


Dbase1          5.0       4.6              5.6              6. 6       .004

Re: [gentoo-user] Bash script inquiry

[Etaoin Shrdlu]

On Tuesday 11 August 2009 09:07:31 Richard Marza wrote:

> I'm trying to run a command in a loop. I have a counter device set...the
> number that the counter generates is supposed to go inside the command in
> the loop after every successive iteration of the loop. This is all really
> to get a general idea I've attached a snippet below.
>
> FILE=`cat filename.txt`
> TICK=`cat filename.txt | wc -l'
> TOCK="0"
>
> while [ $TICK != $TOCK ] ; do
>             let $TOCK=$TOCK+1
>             Var1= `cat FirstWordOfFirstColumnOfFirstLine` (This I actually
> achieved with sed and awk)
>             Var2=`cat FirstFloatOfFirstLine`   (The problem lies here; it's
> my inability to come up with a way of implementing a variable that changes
> along with the counter.  so that the second time this is run it doesn't do
> the first line but moved to the second line and the third line and so
> on...)

I'm not sure I understand what you want, but you can probably reimplement the 
whole thing in awk or sed only much more efficiently. To get more help on 
that, post a more complete description of the problem and the result you want 
to achieve. 
If you really want to keep the current logic, you can save the nth line of the 
file at iteration n and process it how many times you want:

# line number is in N

nthline=`sed -n "$N{p;q;}" file.txt`
var1=`echo "$nthline" | .....`
var2=`echo "$nthline" | .....`

Re: [gentoo-user] Bash script inquiry

[Alex Schuster]

Richard Marza writes:

> FILE=`cat filename.txt`
> TICK=`cat filename.txt | wc -l'
> TOCK="0"
>
> while [ $TICK != $TOCK ] ; do
>             let $TOCK=$TOCK+1

Or, simpler, as we are using bash: (( TOCK++ ))

>             Var1= `cat FirstWordOfFirstColumnOfFirstLine` (This I
> actually achieved with sed and awk)
>             Var2=`cat FirstFloatOfFirstLine`   (The problem lies here;
> it's my inability to come up with a way of implementing a variable that
> changes along with the counter.  so that the second time this is run it
> doesn't do the first line but moved to the second line and the third line
> and so on...)
>
> done
>
> exit 0

What should Var1 contain - "Dbase1" or the content of the file "Dbase1"? 
What should Var2 contain - "5.0"    or the content of the file "5.0"? 
Because you are using cat in the assignment.
If you just need the values in a variable, do it like this:

file=filename.txt
Var1=( $( cat "$file" | awk '{ print $1 }' ) ) # creates an array variable
Var2=( $( cat "$file" | awk '{ print $2 }' ) )

The $() notation does the same as backticks, but is more readable. Using 
foo=( ... ) will create foo as an array. I assume there is no whitespace in 
your data, that is Var1 will never contain something like "Dbase 1".

${#Var1[@]} will contain the number of elements (your $TICK). To access the 
5th element (for example), use ${Var1[4]}.

Oh, an please don't hijack threads by replying to an existing one, but start 
a new one. This one appears inside the "Cloning movie DVDs" thread.

And feel free to ask more questions, maybe I got it all wrong.

	Wonko

Re: [gentoo-user] Bash script inquiry

[Richard Marza]

----- Original Message ----- 
From: "Alex Schuster" <wonko@wonkology.org>
To: <gentoo-user@lists.gentoo.org>
Sent: Tuesday, August 11, 2009 4:59 AM
Subject: Re: [gentoo-user] Bash script inquiry


> Richard Marza writes:
>
>> FILE=`cat filename.txt`
>> TICK=`cat filename.txt | wc -l'
>> TOCK="0"
>>
>> while [ $TICK != $TOCK ] ; do
>>             let $TOCK=$TOCK+1
>
> Or, simpler, as we are using bash: (( TOCK++ ))
>
>>             Var1= `cat FirstWordOfFirstColumnOfFirstLine` (This I
>> actually achieved with sed and awk)
>>             Var2=`cat FirstFloatOfFirstLine`   (The problem lies here;
>> it's my inability to come up with a way of implementing a variable that
>> changes along with the counter.  so that the second time this is run it
>> doesn't do the first line but moved to the second line and the third line
>> and so on...)
>>
>> done
>>
>> exit 0
>
> What should Var1 contain - "Dbase1" or the content of the file "Dbase1"?
> What should Var2 contain - "5.0"    or the content of the file "5.0"?
> Because you are using cat in the assignment.
> If you just need the values in a variable, do it like this:
>
> file=filename.txt
> Var1=( $( cat "$file" | awk '{ print $1 }' ) ) # creates an array variable
> Var2=( $( cat "$file" | awk '{ print $2 }' ) )
>
> The $() notation does the same as backticks, but is more readable. Using
> foo=( ... ) will create foo as an array. I assume there is no whitespace 
> in
> your data, that is Var1 will never contain something like "Dbase 1".
>
> ${#Var1[@]} will contain the number of elements (your $TICK). To access 
> the
> 5th element (for example), use ${Var1[4]}.
>
> Oh, an please don't hijack threads by replying to an existing one, but 
> start
> a new one. This one appears inside the "Cloning movie DVDs" thread.
>
> And feel free to ask more questions, maybe I got it all wrong.
>
> Wonko
>

I did not intend to hijack. Next time I will be more cautious. Your 
information is useful. I have used awk. The array is very useful here.

Think of the file I'm using as a spreadsheet. The headers(column names) are 
on top and the values are below them. Each line has an item with multiple 
values under different systems.


Item   System1 System3 System4 ...

nio        5.0        5.5        5.0            (these are individual 
values. They are nothing more than what they represent. The item (nio) and 
the float or integer representing its value under the different 
system...It's just a file)

My goal is to take "nio" and figure out which system has a different price 
than the others. So if the script were to run through 200 lines of similar 
text it should definitely kick-out: "nio has discrepancy in System3; price 
5.5 in line 1"


Another thing, all systems can have different prices. This must also kick 
out. This is really a script to report discrepancies. 

Re: [gentoo-user] Bash script inquiry

[Peter Humphrey]

On Tuesday 11 August 2009 09:07:31 Richard Marza wrote:
> I'm trying to run a command in a loop. I have a counter device set...the
> number that the counter generates is supposed to go inside the command in
> the loop after every successive iteration of the loop.

In what sense is this a reply to Neil?

Please do not hijack threads.

-- 
Rgds
Peter

Re: [gentoo-user] Bash script inquiry

[Etaoin Shrdlu]

On Tuesday 11 August 2009 10:27:26 Richard Marza wrote:

> Think of the file I'm using as a spreadsheet. The headers(column names) are
> on top and the values are below them. Each line has an item with multiple
> values under different systems.
>
>
> Item   System1 System3 System4 ...
>
> nio        5.0        5.5        5.0            (these are individual
> values. They are nothing more than what they represent. The item (nio) and
> the float or integer representing its value under the different
> system...It's just a file)
>
> My goal is to take "nio" and figure out which system has a different price
> than the others. So if the script were to run through 200 lines of similar
> text it should definitely kick-out: "nio has discrepancy in System3; price
> 5.5 in line 1"
>
>
> Another thing, all systems can have different prices. This must also kick
> out. This is really a script to report discrepancies.

As I suspected, you can do the whole thing in awk only. Basically, I'm going 
to assume you want to report items which don't have all the same values on all 
systems. It's easy to spot those, but it might not be as easy to determine 
which values are the "normal" ones and which are the discrepant ones, 
especially if for example each system has a different value. You have to 
provide additional logic to tell the "discrepant" values from the others. For 
the moment, the script just prints out the lines where all the columns don't 
have the same value.

awk 'NR==1{print;next}{for(i=3;i<=NF;i++){if($i!=$2){print;break}}}' file.txt  

Re: [gentoo-user] Bash script inquiry

[Richard Marza]

----- Original Message ----- 
From: "Etaoin Shrdlu" <shrdlu@unlimitedmail.org>
To: <gentoo-user@lists.gentoo.org>
Sent: Tuesday, August 11, 2009 5:55 AM
Subject: Re: [gentoo-user] Bash script inquiry


> On Tuesday 11 August 2009 10:27:26 Richard Marza wrote:
>
>> Think of the file I'm using as a spreadsheet. The headers(column names) 
>> are
>> on top and the values are below them. Each line has an item with multiple
>> values under different systems.
>>
>>
>> Item   System1 System3 System4 ...
>>
>> nio        5.0        5.5        5.0            (these are individual
>> values. They are nothing more than what they represent. The item (nio) 
>> and
>> the float or integer representing its value under the different
>> system...It's just a file)
>>
>> My goal is to take "nio" and figure out which system has a different 
>> price
>> than the others. So if the script were to run through 200 lines of 
>> similar
>> text it should definitely kick-out: "nio has discrepancy in System3; 
>> price
>> 5.5 in line 1"
>>
>>
>> Another thing, all systems can have different prices. This must also kick
>> out. This is really a script to report discrepancies.
>
> As I suspected, you can do the whole thing in awk only. Basically, I'm 
> going
> to assume you want to report items which don't have all the same values on 
> all
> systems. It's easy to spot those, but it might not be as easy to determine
> which values are the "normal" ones and which are the discrepant ones,
> especially if for example each system has a different value. You have to
> provide additional logic to tell the "discrepant" values from the others. 
> For
> the moment, the script just prints out the lines where all the columns 
> don't
> have the same value.
>
> awk 'NR==1{print;next}{for(i=3;i<=NF;i++){if($i!=$2){print;break}}}' 
> file.txt
>
>
This is great. But it is important that I find which system has a mismatch 
for each item. I guess this is where loops and if statements come in. I 
believe I have sufficient information. Although, more discussion is welcome. 
Thank you all.

Re: [gentoo-user] Cloning movie DVDs with dd - only works after accessing disk with another command?

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 857 bytes --]

On Tue, 11 Aug 2009 08:47:58 +0100, Stroller wrote:

> > Probably because the CSS stuff is stored in a separate area of the  
> > drive,
> > which is not copied by dd.  
> 
> I think I may not have phrased the question clearly. Initially dd  
> doesn't work, then it magically starts working. See the cloning.txt  
> attached to my second post.

Probably because of Remy's explanation, that you have forced the keys to
be read. I'd still us vobcopy as it does the decrypting once and stored a
decrypted copy of the disc's contents, and handbrake can work with a
VIDEO_TS directory instead of a DVD or image file.

Handbrake looks interesting, I'm trying to rip some DVDs to play on my
Eee during a long flight next week, but the ebuild from b.g.o fails
during compilation here.


-- 
Neil Bothwick

Portable: Survives system reboot.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] Bash script inquiry

[Etaoin Shrdlu]

On Tuesday 11 August 2009 11:17:12 Richard Marza wrote:

> > For the moment, the script just prints out the lines where all the columns
> > don't have the same value.
> >
> > awk 'NR==1{print;next}{for(i=3;i<=NF;i++){if($i!=$2){print;break}}}'
> > file.txt
>
> This is great. But it is important that I find which system has a mismatch
> for each item. I guess this is where loops and if statements come in. I
> believe I have sufficient information. Although, more discussion is
> welcome. Thank you all.

If you have a way to tell "regular" values from discrepant ones, it's easy to 
modify the program to account for that and print only the values of discrepant 
columns. One way could be to look for the value that appears most times in a 
row, and consider that value normal and all the others discrepant. But again, 
you could have two values repeated the same number of times, so that would not 
help. That's why it's important to have an unambiguous rule.

[gentoo-user] Re: Cloning movie DVDs with dd - only works after accessing disk with another command?

[James]

Stroller <stroller <at> stellar.eclipse.co.uk> writes:


> Now I've got this new machine with all this space on it (or rather:  
> now I seem to be more interested in doing stuff with this machine I  
> built a while ago), I'm starting on the process of ripping all my DVDs.


QDVDauthor is very cool. Version 2.0 is soon to be released. When it is,
I'm going to ask the dev for a version bump.  The current release is 1.11.0
Ebuild is currently at 1.2.0


If you can wait a few weeks, QDVDauthor 2.0 is suppose to be very cool.
You can beta test it now, on gentoo, as Varol, the main dev is really
cool to work with.


  http://qdvdauthor.sourceforge.net/


HTH,
James

Re: [gentoo-user] Cloning movie DVDs with dd - only works after accessing disk with another command?

[Stroller]

On 11 Aug 2009, at 11:24, Neil Bothwick wrote:
> On Tue, 11 Aug 2009 08:47:58 +0100, Stroller wrote:
>
>>> Probably because the CSS stuff is stored in a separate area of the
>>> drive,
>>> which is not copied by dd.
>>
>> I think I may not have phrased the question clearly. Initially dd
>> doesn't work, then it magically starts working. See the cloning.txt
>> attached to my second post.
>
> Probably because of Remy's explanation, that you have forced the  
> keys to
> be read.

I can only think - this occurred to me after I posted this morning -  
that the keys are somehow stored in the drive's memory. But  
nevertheless I'm copying the encrypted image of the disk, so why would  
the keys make a difference? Besides, the drive is RPC-1, so whatever I  
run on it is doing the DeCSS in software - does the key get uploaded  
to the drive? Maybe it's a more primitive mechanism that's being  
unlocked. I'd just like to understand it.

> I'd still us vobcopy as it does the decrypting once and stored a
> decrypted copy of the disc's contents, and handbrake can work with a
> VIDEO_TS directory instead of a DVD or image file.

HandbrakeCLI appears _here_ to be working fine with the encrypted  
image produced with dd (I've only tried 2 disks so far). Certainly if  
you rip with mplayer / mencoder it sees that the stream is encrypted,  
cracks the DeCSS in a moment & has no problems with it.

I guess I just prefer this encrypted.dd.iso image because it's a  
single file to work with, rather than a directory containing a mess  
of .vob files. Or, at least, the mess of .vob files are hidden from  
me. ;)  I guess I've just gotten used to doing it this way.

> Handbrake looks interesting, I'm trying to rip some DVDs to play on my
> Eee during a long flight next week, but the ebuild from b.g.o fails
> during compilation here.

I think I used this one:
http://gentoo-overlays.zugaina.org/voyageur/portage/media-video/handbrake/handbrake-0.9.3.ebuild

I think that you can get this using layman, but I just copied it into / 
usr/local/portage/...

I set -gtk in package.use, because I only want to use the command line  
version, and it compiled fine.

I get the impression that Gentoo's devs find it difficult to reconcile  
Handbrake's build system with Portage. But I'd really love to see it  
in the tree, because I don't think there's anything better, easier &  
"more complete" for ripping at the command line.

undvd is now broken for mp4s, and its avi files are no good to me. To  
playback on the PS3, rips really need to be .mp4. In the past (maybe  
this is fixed now) the avi files produced by undvd wouldn't work on my  
Mac (I think because they combined h264 with mp3 audio, an "invalid"  
combination)

I found another integrated command-line ripper (lxdvdrip?) which has  
an "interactive" interface. It didn't seem to take --arguments (as I  
prefer) and I found it quite unintuitive.

The dependencies of media-video/shrip are really just too onerous if  
you only want a command-line ripper. Emerging it, or one of those  
dependencies, required I change the USE flags for a package & remerge  
that; I gave up when it required a second package's USE flags to be  
changed.

Simply `HandbrakeCLI -o file.mp4 -b 1500 -i $input` seems to be  
producing really nice rips here, and I just find that a really easy  
method.

Stroller.

Re: [gentoo-user] Cloning movie DVDs with dd - only works after accessing disk with another command?

[Stroller]

On 11 Aug 2009, at 08:50, Joerg Schilling wrote:
> Stroller <stroller@stellar.eclipse.co.uk> wrote:
>
>> Now I've got this new machine with all this space on it (or rather:
>> now I seem to be more interested in doing stuff with this machine I
>> built a while ago), I'm starting on the process of ripping all my  
>> DVDs.
>>
>> In the past I tried media-video/undvd for DVD ripping, but its .mp4
>> support now is broken, and probably won't be fixed.
>>
>> This week I've been trying HandBrakeCLI, and it seems to work pretty
>> darn lovely.
>
> You cannot simply clone a DVD with dd.

Did you read the cloning.txt I attached to my previous (second) message?

Initially running `dd` on the disk doesn't work.

Then I open the disk with a DVD reading script (scandvd - it calls  
mplayer, I think).

Then running `dd` on the disk produces an 8gb file which is readable  
by mplayer (using libdvdread / libdvdcss).

If you carefully read the cloning.txt I previously attached you will  
see I run mplayer using the argument "-dvd-device disc.iso". The  
screenshot I produced by that command is at <http://linux.stroller.uk.eu.org/dd.dvd.screenshot.png 
 >

I am NOT trying to clone a movie from a commerical DVD to DVD-R in  
this way. I am merely trying to "clone" a copy of the disk onto my  
hard-drive as a single file. I am sorry if my language has been  
unclear, but the cloning.txt console log quite readable. I am not,  
incidentally, faking these results.

I'm just really curious why `dd` works perfectly fine the last time,  
but not the first.


> 1)	Use dvdcopy with deCSS and create a copied directory tree.
> 	Make sure to retain kapital letters in filenames.
>
> 2)	Use mkisofs -dvd-video to create a new ifs image
>
> 3)	Use cdrecord -atip tp read the layerbreak
>
> 4)	Use cdrecord -dao -v driveropts=layerbreak=# xxx.iso
> 	to write the data to a DVD+R/DL

These instructions may be useful in the future. I'm afraid that I do  
find burning to somewhat elude my understanding when it comes to  
dealing with the layerbreak. However right now I don't want to burn to  
disk - only to grab an image of the disk for ripping to file.mp4 type  
movie files.

Stroller.
  

Re: [gentoo-user] Re: Cloning movie DVDs with dd - only works after accessing disk with another command?

[Stroller]

On 11 Aug 2009, at 16:28, James wrote:
> Stroller <stroller <at> stellar.eclipse.co.uk> writes:
>
>> Now I've got this new machine with all this space on it (or rather:
>> now I seem to be more interested in doing stuff with this machine I
>> built a while ago), I'm starting on the process of ripping all my  
>> DVDs.
>
> QDVDauthor is very cool. Version 2.0 is soon to be released. When it  
> is,
> I'm going to ask the dev for a version bump.  The current release is  
> 1.11.0
> Ebuild is currently at 1.2.0
>
> If you can wait a few weeks, QDVDauthor 2.0 is suppose to be very  
> cool.
> You can beta test it now, on gentoo, as Varol, the main dev is really
> cool to work with.

It looks jolly, but it's waaay more than I need. The machine is a  
server with lot of disk space in it, but it's headless and I interact  
with it by ssh. It's no problem to walk through & pop a disk in it,  
but I prefer to just run a single `myrippingalias filename.mp4` in a  
screen session, rather than to have to open a window & check boxes.

I have no need to author DVDs right now. I think I have *once* made a  
DVD-R of a movie "to prevent the kids from scratching the original"  
and in such circumstances I was quite happy with the video quality of  
a single layer, so was able to do that pretty easily at the command  
line, too.

Stroller.

Re: [gentoo-user] Cloning movie DVDs with dd - only works after accessing disk with another command?

[Paul Hartman]

On Tue, Aug 11, 2009 at 2:16 PM, Stroller<stroller@stellar.eclipse.co.uk> wrote:
> However right now I don't want to burn to disk - only to grab an image of
> the disk for ripping to file.mp4 type movie files.

For this purpose I highly recommend dvd::rip. There are many things
involved beyond simply copying the files from the disc. Deinterlacing,
audio sync, cropping and scaling, etc. dvd::rip handles it with
relative ease. It even supports clustering so if you have more than 1
computer they can combine power to transcode your movies faster.

[gentoo-user] Re: Cloning movie DVDs with dd - only works after accessing disk with another command?

[Grant Edwards]

On 2009-08-11, Stroller <stroller@stellar.eclipse.co.uk> wrote:

> I'm just really curious why `dd` works perfectly fine the last time,  
> but not the first.

Hasn't this been answered several times already?

Presumably it works the second time but not the first time
because between the two attempts you've run a program that has
written the decryption keys to the optical drive.

If you disagree with that answer, please esplain why rather
than just re-asking the question again and again.

-- 
Grant Edwards                   grante             Yow! I want a WESSON OIL
                                  at               lease!!
                               visi.com            

Re: [gentoo-user] Cloning movie DVDs with dd - only works after accessing disk with another command?

[Stroller]

On 11 Aug 2009, at 20:31, Paul Hartman wrote:

> On Tue, Aug 11, 2009 at 2:16 PM, Stroller<stroller@stellar.eclipse.co.uk 
> > wrote:
>> However right now I don't want to burn to disk - only to grab an  
>> image of
>> the disk for ripping to file.mp4 type movie files.
>
> For this purpose I highly recommend dvd::rip. There are many things
> involved beyond simply copying the files from the disc. Deinterlacing,
> audio sync, cropping and scaling, etc. dvd::rip handles it with
> relative ease. It even supports clustering so if you have more than 1
> computer they can combine power to transcode your movies faster.

Have you tried HandbrakeCLI?

Is it possible to use dvd::rip completely without the GUI?

Stroller.

Re: [gentoo-user] Cloning movie DVDs with dd - only works after accessing disk with another command?

[Paul Hartman]

On Tue, Aug 11, 2009 at 2:36 PM, Stroller<stroller@stellar.eclipse.co.uk> wrote:
>
> On 11 Aug 2009, at 20:31, Paul Hartman wrote:
>
>> On Tue, Aug 11, 2009 at 2:16 PM, Stroller<stroller@stellar.eclipse.co.uk>
>> wrote:
>>>
>>> However right now I don't want to burn to disk - only to grab an image of
>>> the disk for ripping to file.mp4 type movie files.
>>
>> For this purpose I highly recommend dvd::rip. There are many things
>> involved beyond simply copying the files from the disc. Deinterlacing,
>> audio sync, cropping and scaling, etc. dvd::rip handles it with
>> relative ease. It even supports clustering so if you have more than 1
>> computer they can combine power to transcode your movies faster.
>
> Have you tried HandbrakeCLI?

Nope, I had never even heard of it before this thread. I just Googled
it and it looks interesting. It appears to have a lot of presets that
would eliminate the need to manually select codecs and filters etc.
Try it and let us know how the videos look. :)

> Is it possible to use dvd::rip completely without the GUI?

Yes, though you might want (or need) to use the GUI to determine which
settings you want to put into the configuration file. There are a lot
of options to control container format, deinterlacing, video and audio
codecs, bitrate, framerate, encoding passes, cropping, scaling,
normalizing, audio channels, subtitles, etc.

If one of the presets in Handbrake is satisfactory to you, then it
seems like that is probably the far easier path to take for your
desires. I think they are both just wrappers for the usual set of
tools so let whatever works best win.

Re: [gentoo-user] Re: Cloning movie DVDs with dd - only works after accessing disk with another command?

[Stroller]

On 11 Aug 2009, at 20:31, Grant Edwards wrote:
> On 2009-08-11, Stroller <stroller@stellar.eclipse.co.uk> wrote:
>
>> I'm just really curious why `dd` works perfectly fine the last time,
>> but not the first.
>
> Hasn't this been answered several times already?


No, it hasn't.

> Presumably it works the second time but not the first time
> because between the two attempts you've run a program that has
> written the decryption keys to the optical drive.

I'm just looking for a better answer than ones prefaced with  
"presumably" and which treat DeCSS encryption like it's black magic.

I don't say that to offend anyone, because I'm sure none of the other  
posters doing so are claiming to be experts on the subject, either.

Presumably anyone replying saying "oh, it must be something like this"  
is interested in discussing their conjecture.

> If you disagree with that answer, please esplain why rather
> than just re-asking the question again and again.


Well, in the case of the message you quoted, the poster seemed not to  
have read the cloning.txt console log I posted. He seemed to be  
telling me that what I have done is "impossible" and I was correcting  
his misunderstanding.

You will see that I already explained in my message of 11 August 2009  
19:58:11 BST some aspects of this still confuse me.

If you'd like me to clarify that post further then I guess the best  
way I can explain it is: if I've "run a program that has written the  
decryption keys to the optical drive" (your words), how come mplayer  
still has to retrieve the CSS keys (see cloning.txt) when it's run on  
the disc.iso file? (using "-dvd-device" argument)  The answer to that  
is surely "because the movie is still encrypted", so (to me) that begs  
the question "why's the movie file still encrypted if I've written the  
decryption keys to the optical drive?"

This seems to me to be a logical loop, and I'd really be genuinely  
glad for someone to explain where I'm looking at it wrong (but please  
ignore this message if the subject is bothering you).

I suspect someone who really understands what's going on here doesn't  
need the additional clarification of my previous 2 paragraphs. There's  
probably a really simple explanation for what's going on here.

Stroller.

[gentoo-user] HandbrakeCLI vs dvd::rip was: Cloning movie DVDs with dd

[Stroller]

>
> On 11 Aug 2009, at 21:02, Paul Hartman wrote:
> On Tue, Aug 11, 2009 at 2:36 PM, Stroller<stroller@stellar.eclipse.co.uk 
> > wrote:
>> On 11 Aug 2009, at 20:31, Paul Hartman wrote:
>>> On Tue, Aug 11, 2009 at 2:16 PM, Stroller<stroller@stellar.eclipse.co.uk 
>>> >
>>> wrote:
>>>>
>>>> However right now I don't want to burn to disk - only to grab an  
>>>> image of
>>>> the disk for ripping to file.mp4 type movie files.
>>>
>>> For this purpose I highly recommend dvd::rip. There are many things
>>> involved beyond simply copying the files from the disc.  
>>> Deinterlacing,
>>> audio sync, cropping and scaling, etc. dvd::rip handles it with
>>> relative ease. It even supports clustering so if you have more  
>>> than 1
>>> computer they can combine power to transcode your movies faster.
>>
>> Have you tried HandbrakeCLI?
>
> Nope, I had never even heard of it before this thread. I just Googled
> it and it looks interesting. It appears to have a lot of presets that
> would eliminate the need to manually select codecs and filters etc.
> Try it and let us know how the videos look. :)
> ...
> If one of the presets in Handbrake is satisfactory to you, then it
> seems like that is probably the far easier path to take for your
> desires.

So far HandBrakeCLI is looking pretty good indeed.

At the moment I'm working on some caveats that may only apply to PS3  
playback of the video files. Cropping of files is alleged to break PS3  
playback, but I haven't found that with the two movies I've tried so  
far.

Running Handbrake on a trailer for the movie Heat using just  
`HandbrakeCLI -2 -o file.mp4 -b $bitrate -i $input` (this autocrops) I  
can't see any difference between bitrates of 1500 & 2500.

Using undvd and a target size of about 1.2gig there were visible  
differences between the original DVD & the .mp4 rip of this full  
movie. So I'll have to try viewing that in the next day or two - I  
recall now that it was the opening scenes that I compared between the  
DVD & undvd's rip, not just the trailer. There are some nighttime  
scenes in this movie with large dark areas in the picture which showed  
some pixellation or sorta "shallow colour-depth jpeg-iness" on undvd's  
rip.

Correction to my earlier post (11 August 2009 19:58:11 BST) in reply  
to Neil Bothwick: I'm using the "-2" argument to HandBrakeCLI to  
perform two-pass encoding. You probably know that this gives better  
quality for relative to the file size / bitrate, but takes longer to  
encode.

Stroller.

Re: [gentoo-user] Re: Cloning movie DVDs with dd - only works after accessing disk with another command?

[Paul Hartman]

On Tue, Aug 11, 2009 at 3:22 PM, Stroller<stroller@stellar.eclipse.co.uk> wrote:
> I'm just looking for a better answer than ones prefaced with "presumably"
> and which treat DeCSS encryption like it's black magic.

I can't give you a definitive answer to your original question but I
can hopefully shed some light on how CSS works, no black magic. Let me
say also that I'm no expert and this is all "as I understand it".

First, DeCSS is actually one (the first?) decryption method, and is
very old and mostly obsolete. What you probably meant to say was CSS
(Content-Scrambling System) which is the encryption method. Most
likely whatever programs you're running nowadays are using libdvdcss,
not DeCSS.

DeCSS used a compromised player key (gleamed from Xing player on
Windows, IIRC) which was revoked and invalid on newer discs, while
libdvdcss uses a computed lookup table of every possible player key
(400 or so) as well as falling back to brute force in the event of a
disc with a new set of keys so that it can decrypt nearly everything.

CSS is handled in a series of keys, and the hardware of the DVD drive
is involved in at least some of the key authentication. On the DVD,
there are keys at the sector, title and disc level. The disc key is
needed to get the title key, and the title key is needed to decrypt
the sector keys to get to the actual movie data. The 400+ variations
of the disc key (one for each possible player key) stored on the DVD
itself in abnormal locations (possibly unreadable to dd?) such as disc
lead-in. The DVD player software gives the drive a player key and it
cycles through all of the disc keys on the DVD until it finds a match
(or doesn't). Even if you made a 100% perfect read of the original,
nearly all consumer-grade DVD burners are forbidden from writing the
CSS key data to DVD+/-R discs (and I believe DVD-R discs even
physically lack the area where they would go).

I can't give you a definitive answer on why dd gives you different
results, but I would tend to agree with the presumption of the other
people who replied. My suspicion is that a valid player key was given
to the drive by the other program you used and the "door was left
open" so to speak, causing the drive to then output valid data when
you used dd on it the second time. I'm not sure how long that player
key lasts in the DVD drive (if it does at all). I guess you could look
at the sources of libdvdread and libdvdcss to see exactly how it's
done.

Typically, if you want to reduce wear and tear on your DVD drive,
you'd use vobcopy which will handle the decryption and copying of the
data in one motion. Then you should be able to transcode from those
files to the format you want without needing the disc in the drive.
Since transcoding is usually the slow part, if you've got a lot of
disc space on this new beast of a machine you might want to do all of
the copying first and then just transcode them in a huge batch
(perhaps that was already your plan from the beginning).

As CSS was rendered obsolete 10 years ago, finding any technical info
is kind of hard since it seems people have moved onto more challenging
pursuits. Googling mainly results in a billion pages about copying
movies on Windows.

Re: [gentoo-user] Re: Cloning movie DVDs with dd - only works after accessing disk with another command?

[Stroller]

Hi Paul,

Many thanks for the details synopsis. Some of this was known to me,  
some not. Comments below.

On 11 Aug 2009, at 23:49, Paul Hartman wrote:
> On Tue, Aug 11, 2009 at 3:22 PM, Stroller<stroller@stellar.eclipse.co.uk 
> > wrote:
>> I'm just looking for a better answer than ones prefaced with  
>> "presumably"
>> and which treat DeCSS encryption like it's black magic.
>
> ...
> First, DeCSS is actually one (the first?) decryption method, and is
> very old and mostly obsolete. What you probably meant to say was CSS
> (Content-Scrambling System) which is the encryption method. Most
> likely whatever programs you're running nowadays are using libdvdcss,
> not DeCSS.

Yes, indeed. My apologies.

> DeCSS used a compromised player key (gleamed from Xing player on
> Windows, IIRC) which was revoked and invalid on newer discs, while
> libdvdcss uses a computed lookup table of every possible player key
> (400 or so) as well as falling back to brute force in the event of a
> disc with a new set of keys so that it can decrypt nearly everything.

I didn't realise this. I assumed there was one key for each region,  
and had not realised that keys had actually been revoked (how does  
that work with old players?)

> ... On the DVD,
> there are keys at the sector, title and disc level. The disc key is
> needed to get the title key, and the title key is needed to decrypt
> the sector keys to get to the actual movie data. The 400+ variations
> of the disc key (one for each possible player key) stored on the DVD
> itself in abnormal locations (possibly unreadable to dd?) such as disc
> lead-in. The DVD player software gives the drive a player key and it
> cycles through all of the disc keys on the DVD until it finds a match
> (or doesn't).

I had not been aware of these multiple levels, either.

The behaviour makes a lot more sense to me knowing that there are  
multiple keys per disk. So it's possible that one key has been  
provided, yet other parts of the data stream are left encrypted. I can  
relate that much more easily to the results I'm seeing.

> Even if you made a 100% perfect read of the original,
> nearly all consumer-grade DVD burners are forbidden from writing the
> CSS key data to DVD+/-R discs (and I believe DVD-R discs even
> physically lack the area where they would go).

Yes, I was aware of this, thanks.

> Typically, if you want to reduce wear and tear on your DVD drive,
> you'd use vobcopy which will handle the decryption and copying of the
> data in one motion.

It's not really much odds to me one way or the other. For the sake of  
running a single command on the drive (scandvd in the cloning.txt log)  
I'd rather have the single image file created by dd, even if it's  
still encrypted (rather than a clutterful directory of .vob files).

> Then you should be able to transcode from those
> files to the format you want without needing the disc in the drive.

Yes, I can do that, anyway - working from the (still encrypted)  
"disc.iso" created by `dd`. That's why I left the `mplayer` command in  
the cloning.txt log - you can see it finds the titles in the disc.iso  
image and gets a key for each vob file. The screencap is clear.

> Since transcoding is usually the slow part, if you've got a lot of
> disc space on this new beast of a machine you might want to do all of
> the copying first and then just transcode them in a huge batch
> (perhaps that was already your plan from the beginning).

The transcoding is, so far, taking 12 - 24 hours per title.

With undvd (using mencoder) a movie was pretty consistently at c 12  
hours, IIRC. Maybe 11 - 14 depending on the runtime. That was a  
different machine - probably very slightly faster, but not enough to  
make an appreciable difference. Both are olde Pentium 4s.

Today HandBrakeCLI seems to be doing 25-minute TV episodes in about 6  
hours each. It's averaging about 3.6 fps, but halve that because I'm  
doing two pass encoding.

I believe that both mencoder & handbrake may both rely on some of the  
same libraries, but that they're basically different applications  
built on top of those. Obviously I'm hoping that this apparently  
slower ripping means that handbrake will give better quality. ;)  It  
looks like HandBrake defaults to higher bit rate / larger file sizes  
than undvd, however.

Anyway, 12 or 24 hours doesn't matter - I'm plenty happy with one  
movie ripped per day. It's quite a luxury to have a whole TB of free  
space, and just to be able to copy the DVD to the hard-drive whenever  
I want and rip at leisure. The machine on which I was previously  
ripping has 6gig free on one drive & 23gig free on another, but that's  
just at the sort of limits where I was finding that - experimenting  
with quality settings & stuff - I couldn't keep as many images around  
as I wanted to, or had to store them on the "wrong" drive and move  
them to the other & stuff like that.

I have been using `for title in $(seq 1 6) ; do ...` to rip these  
individual episodes, but if we're generally talking about a main  
feature film then generally I don't mind checking up on the rip every  
couple of hours and starting a new one when necessary, or loading the  
drive once per day & making the disk image. I don't know whether it's  
the switch to HandBrake or whether it's the extra space or what, but I  
feel quite liberated, and that the regime I was attempting previously  
somehow constrained me from ripping more. Hopefully I'll get into a  
regular routine & have my whole collection done within a few weeks now.

> As CSS was rendered obsolete 10 years ago, finding any technical info
> is kind of hard since it seems people have moved onto more challenging
> pursuits. Googling mainly results in a billion pages about copying
> movies on Windows.

Yeah, I've found this in the past. It's really a bit of a swine that  
all this encryption nonsense was (pretty much) imposed on us. In the  
context of encryption & stuff, we're really pretty lucky with CSS that  
it was so hacked so quickly.

Stroller.

Re: [gentoo-user] Re: Cloning movie DVDs with dd - only works after accessing disk with another command?

[Richard Marza]

----- Original Message ----- 
From: "Stroller" <stroller@stellar.eclipse.co.uk>
To: <gentoo-user@lists.gentoo.org>
Sent: Tuesday, August 11, 2009 4:22 PM
Subject: Re: [gentoo-user] Re: Cloning movie DVDs with dd - only works after 
accessing disk with another command?


>
> On 11 Aug 2009, at 20:31, Grant Edwards wrote:
>> On 2009-08-11, Stroller <stroller@stellar.eclipse.co.uk> wrote:
>>
>>> I'm just really curious why `dd` works perfectly fine the last time,
>>> but not the first.
>>
>> Hasn't this been answered several times already?
>
>
> No, it hasn't.
>
>> Presumably it works the second time but not the first time
>> because between the two attempts you've run a program that has
>> written the decryption keys to the optical drive.
>
> I'm just looking for a better answer than ones prefaced with
> "presumably" and which treat DeCSS encryption like it's black magic.
>
> I don't say that to offend anyone, because I'm sure none of the other
> posters doing so are claiming to be experts on the subject, either.
>
> Presumably anyone replying saying "oh, it must be something like this"
> is interested in discussing their conjecture.
>
>> If you disagree with that answer, please esplain why rather
>> than just re-asking the question again and again.
>
>
> Well, in the case of the message you quoted, the poster seemed not to
> have read the cloning.txt console log I posted. He seemed to be
> telling me that what I have done is "impossible" and I was correcting
> his misunderstanding.
>
> You will see that I already explained in my message of 11 August 2009
> 19:58:11 BST some aspects of this still confuse me.
>
> If you'd like me to clarify that post further then I guess the best
> way I can explain it is: if I've "run a program that has written the
> decryption keys to the optical drive" (your words), how come mplayer
> still has to retrieve the CSS keys (see cloning.txt) when it's run on
> the disc.iso file? (using "-dvd-device" argument)  The answer to that
> is surely "because the movie is still encrypted", so (to me) that begs
> the question "why's the movie file still encrypted if I've written the
> decryption keys to the optical drive?"
>
> This seems to me to be a logical loop, and I'd really be genuinely
> glad for someone to explain where I'm looking at it wrong (but please
> ignore this message if the subject is bothering you).
>
> I suspect someone who really understands what's going on here doesn't
> need the additional clarification of my previous 2 paragraphs. There's
> probably a really simple explanation for what's going on here.
>
> Stroller.
>
>
To copy dvd to your p.c. just use mplayer/mencoder...It's not that 
difficult. It shouldn't be too difficult to get the dump back onto DVD disk. 
It's not that big of a deal. dd probably worked because the disk wasn't 
encrypted. Run the history  command and investigate. This could have been a 
fluke. It's nothing to ramble on about.

Re: [gentoo-user] Cloning movie DVDs with dd - only works after accessing disk with another command?

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 1581 bytes --]

On Tue, 11 Aug 2009 19:58:11 +0100, Stroller wrote:

> I guess I just prefer this encrypted.dd.iso image because it's a  
> single file to work with, rather than a directory containing a mess  
> of .vob files. Or, at least, the mess of .vob files are hidden from  
> me. ;)  I guess I've just gotten used to doing it this way.

If you're only encoding one title from each DVD, you could use mplayer to
rip the title to a single file.

mplayer dvd://1 -dumpstream -dumpfile somefilm.mpeg

> > Handbrake looks interesting, I'm trying to rip some DVDs to play on my
> > Eee during a long flight next week, but the ebuild from b.g.o fails
> > during compilation here.  
> 
> I think I used this one:
> http://gentoo-overlays.zugaina.org/voyageur/portage/media-video/handbrake/handbrake-0.9.3.ebuild

I've since grabbed the latest from SVN, which builds with the standard 
./configure && make && make install.

However, it did download and compile a bunch of libraries that I already
have. Programs using their own copies of libraries is a bit Windowsy for
my liking.
 
> I get the impression that Gentoo's devs find it difficult to reconcile  
> Handbrake's build system with Portage. But I'd really love to see it  
> in the tree, because I don't think there's anything better, easier &  
> "more complete" for ripping at the command line.

I suspect 0.9.4 will have more luck, as it uses a more standard build
system. I'm also trying undvd, which seems simplicity itself.


-- 
Neil Bothwick

And on the seventh day God said :wq and then make

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[gentoo-user] HandBrakeCLI & undvd. was: Cloning movie DVDs with dd

[Stroller]

On 12 Aug 2009, at 12:05, Neil Bothwick wrote:
> On Tue, 11 Aug 2009 19:58:11 +0100, Stroller wrote:
>
>> I guess I just prefer this encrypted.dd.iso image because it's a
>> single file to work with, rather than a directory containing a mess
>> of .vob files. Or, at least, the mess of .vob files are hidden from
>> me. ;)  I guess I've just gotten used to doing it this way.
>
> If you're only encoding one title from each DVD, you could use  
> mplayer to
> rip the title to a single file.
>
> mplayer dvd://1 -dumpstream -dumpfile somefilm.mpeg

No point though, really, is there? I mean, I'm happy with it this way,  
I'm not short of space, and to me that just seems to be complicating  
things.

>>> Handbrake looks interesting, I'm trying to rip some DVDs to play  
>>> on my
>>> Eee during a long flight next week, but the ebuild from b.g.o fails
>>> during compilation here.
>>
>> I think I used this one:
>> http://gentoo-overlays.zugaina.org/voyageur/portage/media-video/handbrake/handbrake-0.9.3.ebuild
>
> I've since grabbed the latest from SVN, which builds with the standard
> ./configure && make && make install.
>
> However, it did download and compile a bunch of libraries that I  
> already
> have. Programs using their own copies of libraries is a bit Windowsy  
> for
> my liking.

I know. I find it more "Macintoshy" than Windowsy myself, but whatever  
- it just feels a little dirty. I can only guess the Handbrake devs  
did it this ways because the latest versions of media libs like these  
aren't so well tracked by many distros, so this saves them manually  
updating from SVN (or are the versions pinned?). Also, I think they  
develop for Windows & that HandBrakeCLI will compile & work on that  
platform.

>
>> I get the impression that Gentoo's devs find it difficult to  
>> reconcile
>> Handbrake's build system with Portage. ...
>
> I suspect 0.9.4 will have more luck, as it uses a more standard build
> system.

I really nope so. If I had more time I might look at the SVN & see if  
I could hack an ebuild, but 0.9.3 is working just fine for me right  
now. I'm fairly happy to tolerate it having done it's own rude thing  
wrt lib downloads, at least since I've installed using an ebuild, so  
`emerge -C` should remove all files installed by it, when removal or  
upgrading becomes necessary or desirable.

> I'm also trying undvd, which seems simplicity itself.


undvd is really nice. There's some stuff the author has got really  
right: coloured output, nice display / layout, just the right amount  
of options, help display is not too long, split onto two pages  
(standard & advanced is accessed with "-z"). To me it seems just right  
for casual use at a terminal, like the author has focussed on that,  
and it's an aspect which the alternatives neglect to some extent or  
another.

Depending on how much time is available to you for ripping, and upon  
your optical prescription &/or tolerance for video quality, you  
probably want to look at undvd's "-2" argument & its target size  
option. I think you can set the bitrate in undvd, but I didn't find it  
obvious what bitrate it's using by default - thus it's not obvious  
what bitrate to choose in order to improve quality (or by default does  
it try to make a file that'll fit on a CD-R? I can't recall). Setting  
a target video size is the easy way to improve video quality in undvd  
& on a number of movies I found a 1.2gig rip indistinguishable from  
the original DVD.

The downside of undvd, as I've said & I'll keep saying, is that the  
video files it produces don't play so nice on other platforms. The  
author is a bit of a Linux evangelist, and doesn't really care about  
that, since mplayer (at least; I don't know about vlc or other Linux  
players) is really forgiving, and makes extra efforts to overcome  
video files' shortcomings. For me, however, this is just an  
insurmountable snag, and TBH I think undvd should neither be in the  
tree, nor promoted the way it is on its sourceforge page (he should  
admit it's just a personal project that "works for him"). I can't be  
the only person who wants to rip movies in Linux to play back on the  
Mac or PS3, and if I've wasted this much time on undvd then surely  
others will, too.

If you just want the movies to play back on a Linux laptop whilst on a  
holiday trip then this probably won't bother you at all, but if you  
want an "archive" of your movie collection which you'll keep for  
playback into the future then undvd isn't the best ripper. It seems to  
me that HandBrakeCLI takes that prize, in the command-line category,  
at least.

Stroller.

Re: [gentoo-user] HandBrakeCLI & undvd. was: Cloning movie DVDs with dd

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 3066 bytes --]

On Wed, 12 Aug 2009 21:05:02 +0100, Stroller wrote:

> > mplayer dvd://1 -dumpstream -dumpfile somefilm.mpeg
> 
> No point though, really, is there? I mean, I'm happy with it this way,  
> I'm not short of space, and to me that just seems to be complicating  
> things.

Well it's a one line command, no more complex than the dd you use, except
it works first time ;-)

> > However, it did download and compile a bunch of libraries that I  
> > already
> > have. Programs using their own copies of libraries is a bit Windowsy  
> > for
> > my liking.
> 
> I know. I find it more "Macintoshy" than Windowsy myself, but whatever  
> - it just feels a little dirty. I can only guess the Handbrake devs  
> did it this ways because the latest versions of media libs like these  
> aren't so well tracked by many distros, so this saves them manually  
> updating from SVN (or are the versions pinned?). Also, I think they  
> develop for Windows & that HandBrakeCLI will compile & work on that  
> platform.

There's also the way the ffmpeg API keeps changing, breaking everything
built against it :(

> > I suspect 0.9.4 will have more luck, as it uses a more standard build
> > system.
> 
> I really nope so. If I had more time I might look at the SVN & see if  
> I could hack an ebuild, but 0.9.3 is working just fine for me right  
> now.

I didn't bother with an ebuild, it only installs one executable if you
don't build the GUI.

> Depending on how much time is available to you for ripping, and upon  
> your optical prescription &/or tolerance for video quality, you  
> probably want to look at undvd's "-2" argument & its target size  
> option. I think you can set the bitrate in undvd, but I didn't find it  
> obvious what bitrate it's using by default - thus it's not obvious  
> what bitrate to choose in order to improve quality (or by default does  
> it try to make a file that'll fit on a CD-R? I can't recall). Setting  
> a target video size is the easy way to improve video quality in undvd  
> & on a number of movies I found a 1.2gig rip indistinguishable from  
> the original DVD.

I think it uses 900 as the default bitrate, I read that somewhere in the
docs.

> If you just want the movies to play back on a Linux laptop whilst on a  
> holiday trip then this probably won't bother you at all, but if you  
> want an "archive" of your movie collection which you'll keep for  
> playback into the future then undvd isn't the best ripper. It seems to  
> me that HandBrakeCLI takes that prize, in the command-line category,  
> at least.

At the moment that's all I want, but that may not always be true, so I'd
rather find one program that addresses my current and possible future
needs, instead of having to learn a second later on. Undvd is very simple
to use, but after some playing it turns out to be too simple, I'm a
Gentoo user and therefore a control freak.


-- 
Neil Bothwick

the sum of all human intelligence is constant, only the number of humans
increases.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-user] HandBrakeCLI & undvd. was: Cloning movie DVDs with dd

[Stroller]

On 12 Aug 2009, at 21:43, Neil Bothwick wrote:
> ...
>> If you just want the movies to play back on a Linux laptop whilst  
>> on a
>> holiday trip then this probably won't bother you at all, but if you
>> want an "archive" of your movie collection which you'll keep for
>> playback into the future then undvd isn't the best ripper. It seems  
>> to
>> me that HandBrakeCLI takes that prize, in the command-line category,
>> at least.
>
> At the moment that's all I want, but that may not always be true, so  
> I'd
> rather find one program that addresses my current and possible future
> needs, instead of having to learn a second later on. Undvd is very  
> simple
> to use, but after some playing it turns out to be too simple, I'm a
> Gentoo user and therefore a control freak.

What don't you like about undvd, please?

What settings do you want to change, that it prevents you?


The reason I'm asking is that HandBrakeCLI has a bug WRT cropping &  
PS3 playback, see:
http://forum.handbrake.fr/viewtopic.php?f=23&t=8032&start=0#p45390

I would prefer to be able to crop, because ISTM that the uncropped  
setting means I encode big black borders around the sides, resulting  
in larger files.

I don't believe that undvd (relying upon mplayer) has this problem, so  
I am tempted to have another crack at hacking on undvd's source &  
porting its call upon mpeg4ip (which has been depreciated from the  
tree) to instead work with MP4Box (part of media-video/gpac).

I already had a little go at this, but I'm not an expert coder -  
certainly not in perl - and got a little stuck. I think I can probably  
manage to overcome the difficulty I encountered, but I'd be grateful  
to hear of any other shortcomings of undvd that I may not have  
considered, before I invest any more time in it.

Thanks in advance for your thoughts,

Stroller.