[gentoo-amd64] USB Scanner Problems with Newer Kernels/Libusb

[gentoo-amd64] USB Scanner Problems with Newer Kernels/Libusb

[Frank Peters]

I do not make scans very often, but when I do my scanners work very well
with Linux and Sane -- or at least they used to.

Checking the file dates on my last scans I discover that I last used my
scanner during January of 2012.  This was about the time of the release
of kernel 3.5.  The significance of this is that with kernel 3.5 the usbfs
file system was eliminated.

Previously, libusb was used to read/write to the scanner from user space.
Libusb is still used by Sane but since usbfs has been eliminated it no longer
detects my scanner.

When I plug in the scanner the kernel log shows this message:

Kernel: usb 2-1.4: new full-speed USB device number 4 using ehci-pci

The kernel can detect the device, but the user-space libusb, and hence Sane,
cannot access the scanner.

In fact, sane-find-scanner, which should detect the scanner, fails with this
output:

# Could not initialize libusb-1.0, error -99
# Skipping libusb devices

From what I've manged to find through searches I believe that udev is used
to create devices "on the fly" when a scanner is detected.  However, I've
never used udev before and never had problems with libusb detecting the scanner.
But since usbfs is gone I am now unable to scan due to a failure to detect
the scanner.

The utility usb-devices, which uses libusb, can produce data on the
scanner, but lsusb, which also uses libusb but depends on usbfs, fails.

Does anyone know to resolve such a problem?  How can I get my scanner
to be utilized by libusb and sane?

Frank Peters

[gentoo-amd64] Re: USB Scanner Problems with Newer Kernels/Libusb

[Duncan]

Frank Peters posted on Fri, 08 Nov 2013 22:25:53 -0500 as excerpted:

> The utility usb-devices, which uses libusb, can produce data on the
> scanner, but lsusb, which also uses libusb but depends on usbfs, fails.
> 
> Does anyone know to resolve such a problem?  How can I get my scanner to
> be utilized by libusb and sane?

I haven't had a scanner in awhile, but based on what I've read verified 
by what I see for other usb devices, udev creates usb devices in 
/dev/bus/usb.  Each usb port is a (three-digit sequentially numbered) 
directory, with the individual device functions on that port appearing as 
(again three-digit sequentially numbered) device nodes.  Usually there'll 
only be one such function per dir, but some devices, such as my Logitech 
USB mouse/keyboard, have two or more.

There's also /sys/bus/usb/, which like most /sys/ dirs, contains trees 
with information, in this case about usb devices.  I believe that's all 
kernel level, however, with udev in fact using some of the information 
found there to populate its udev device tree.

Libusb, meanwhile, has been updated to work with the /dev/bus/usb/ tree, 
so AFAIK that's what you need to create... somehow.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-amd64] Re: USB Scanner Problems with Newer Kernels/Libusb

[Frank Peters]

On Sat, 9 Nov 2013 12:29:49 +0000 (UTC)
Duncan <1i5t5.duncan@cox.net> wrote:

> 
> Libusb, meanwhile, has been updated to work with the /dev/bus/usb/ tree, 
> so AFAIK that's what you need to create... somehow.
> 

Thanks for this synopsis.  I've been slowly piecing together essentially
the same picture from many different sources but was still unsure about
how libusb fits into the scheme.

Creating, somehow, the /dev/bus/usb devices is also the conclusion I thought
would be necessary, but I don't believe that it is possible without udev.
Yet it seems that it *should* be possible.  Everything, at least according
to my limited understanding, within the /dev tree is just an interface
to the kernel using major and minor numbers.  The necessary modules,
namely ehci-pci and uhci-pci, are already integrated into the kernel.

Scanners used to work in the same way that USB printers or USB mass
storage devices *still* work.  That is, an appropriate module is either
built or separately loaded into the kernel which will allow the use
of certain interfaces in the /dev tree (e.g. /dev/usb/lp0).  My USB
modem also functions in the same way using a module, cdc-acm, and
/dev/usb/ttyACM0.  These /dev interfaces could be created independently
of udev.  Why not with /dev/bus/usb?

In fact, everything classed as USB on my system, i.e. keyboard, mouse,
printer, mass storage, modem, external hard drives, can be straightforwardly
controlled using this module/dev method -- with the sole exception
of USB scanners.  Why this crazy distinction?

Of course I could always jump on the udev bandwagon, like most everyone
else, but I still very much enjoy the ability to control, in a simple
manner, the operation of my own system.  I don't like the idea of another
system daemon doing things without my knowledge or approval.

Frank Peters

Re: [gentoo-amd64] Re: USB Scanner Problems with Newer Kernels/Libusb

[Barry Schwartz]

Frank Peters <frank.peters@comcast.net> skribis:
> Of course I could always jump on the udev bandwagon, like most everyone
> else, but I still very much enjoy the ability to control, in a simple
> manner, the operation of my own system.  I don't like the idea of another
> system daemon doing things without my knowledge or approval.

I don’t blame you, because it is quite a disaster, and is becoming all
rolled up in the attempt to turn ‘Linux’ from a kernel into a
‘vertically integrated’ variant of MacOS. I’m actually using eudev,
myself, but, regardless, never know when an update will render things
like scanners non-working or at least give them incorrect access
controls.

I suppose it should be _possible_ to create the devices but would not
be surprised if the task is non-trivial. Which is very, very un-Unix,
and I think that really ought to be the end of it. I don’t mind fancy
stuff but do mind when it is done in an extremely un-Unixy fashion.

Re: [gentoo-amd64] Re: USB Scanner Problems with Newer Kernels/Libusb

[Frank Peters]

On Sat, 9 Nov 2013 14:40:21 -0600
Barry Schwartz <chemoelectric@chemoelectric.org> wrote:

>
> I suppose it should be _possible_ to create the devices but would not
> be surprised if the task is non-trivial.
>

Right now I am investigating if I can employ the udev daemon to create
the necessary devices, when needed, in the /dev tree.  IOW, I want to find
out if I can just start the daemon, plug in the scanner, and have udev create
the things I need.  Then, when I am done scanning, I can just shut off the
daemon.  This method should allow me to use udev (or eudev) only when
I need to use it.  Otherwise it would not be running.

Hopefully, this will work.  It may seem awkward, but with some custom
scripts it should be quite painless.

I can't understand why libusb will not just pull its information from
/sys/bus/usb as a substitute for the former usbfs.

> 
>  because it is quite a disaster, and is becoming all
> rolled up in the attempt to turn ‘Linux’ from a kernel into a
> ‘vertically integrated’ variant of MacOS.
>

IMHO, those folks at freedesktop.org are going to destroy the simplicity,
beauty, and diverse utility of Linux.

I downloaded and examined sytemd to try to learn more about the udev
process.  What a convoluted mess!  Compared to the simple and straightforward
boot-up and device system which I have implemented on my machines, systemd
is a confusing morass.  For what reason?  I can boot up and configure my
entire machine using a single bash script of 155 lines (including the comments).
Why would I want to replace that comfort and ease with the expansive cacophony
that is systemd?

The freedesktop folks can do as they please, but I will never want to
join that party.

Frank Peters

Re: [gentoo-amd64] Re: USB Scanner Problems with Newer Kernels/Libusb

[Barry Schwartz]

Frank Peters <frank.peters@comcast.net> skribis:
> Right now I am investigating if I can employ the udev daemon to create
> the necessary devices, when needed, in the /dev tree.  IOW, I want to find
> out if I can just start the daemon, plug in the scanner, and have udev create
> the things I need.  Then, when I am done scanning, I can just shut off the
> daemon.  This method should allow me to use udev (or eudev) only when
> I need to use it.  Otherwise it would not be running.

Maybe you can run it once even as a non-daemon just to create the
device, similarly to running rescan-scsi-bus.

> >  because it is quite a disaster, and is becoming all
> > rolled up in the attempt to turn ‘Linux’ from a kernel into a
> > ‘vertically integrated’ variant of MacOS.
> 
> IMHO, those folks at freedesktop.org are going to destroy the simplicity,
> beauty, and diverse utility of Linux.

Yes. Particularly distressing because they are completely neglecting
GNUstep, which is an existing _thoughtful_ attempt to reconcile
Unix/GNU with an integrated desktop non-Unix folk would find familiar.

> I downloaded and examined sytemd to try to learn more about the udev
> process.  What a convoluted mess!  Compared to the simple and straightforward
> boot-up and device system which I have implemented on my machines, systemd
> is a confusing morass.  For what reason?  I can boot up and configure my
> entire machine using a single bash script of 155 lines (including the comments).
> Why would I want to replace that comfort and ease with the expansive cacophony
> that is systemd?

I like baselayout but as a former Slackware user also have a strong
appreciation of a BSD-style write-a-script-for-each-runlevel. Nothing
could be simpler. I have never investigated systemd too deeply, but
simply know that I am lost trying to play with runlevels on an
Ubuntu/Debian system. (There is also the Hurd approach of not needing
such scripts in the first place, but, alas, that is not transferable
to Linux.)

Re: [gentoo-amd64] Re: USB Scanner Problems with Newer Kernels/Libusb

[Mark Knecht]

Hi,

On Sat, Nov 9, 2013 at 2:11 PM, Barry Schwartz
<chemoelectric@chemoelectric.org> wrote:
> Frank Peters <frank.peters@comcast.net> skribis:
<SNIP>
>> IMHO, those folks at freedesktop.org are going to destroy the simplicity,
>> beauty, and diverse utility of Linux.
>
> Yes. Particularly distressing because they are completely neglecting
> GNUstep, which is an existing _thoughtful_ attempt to reconcile
> Unix/GNU with an integrated desktop non-Unix folk would find familiar.


There's no question that the two of you have forgotten more about
Linux than I'll even know but I don't really worry too much about
Frank's original worry. I sort of feel that the kernel will always be
the kernel and everything that gets piled on top sort of seems to take
care of itself in the end. If freedesktop.org goes down a path where
it gets too complicated then I believe\think\hope some other group
will step forward quietly, fork that portion of our beloved world and
go do the right thing.

Then again, I'm probably naive, but I can at least hope... :-)

- Mark

Re: [gentoo-amd64] Re: USB Scanner Problems with Newer Kernels/Libusb

[Barry Schwartz]

Mark Knecht <markknecht@gmail.com> skribis:
> If freedesktop.org goes down a path where
> it gets too complicated then I believe\think\hope some other group
> will step forward quietly, fork that portion of our beloved world and
> go do the right thing.

That’s already happening, right here at Gentoo, with eudev and the
general effort to keep a totally systemd-free Gentoo as an option.

(I have run into trouble, though, due to systemd being in the 32-bit
compatibility binaries and installing pkg-config files. I suppose I
could just have emerge remove those files via a Portage hook.)

Re: [gentoo-amd64] Re: USB Scanner Problems with Newer Kernels/Libusb

[Tanstaafl]

On 2013-11-09 6:10 PM, Barry Schwartz <chemoelectric@chemoelectric.org> 
wrote:
> (I have run into trouble, though, due to systemd being in the 32-bit
> compatibility binaries and installing pkg-config files. I suppose I
> could just have emerge remove those files via a Portage hook.)

Isn't this what

INSTALL_MASK="/usr/lib/systemd/"

in /etc/portage/make.conf is for?

Re: [gentoo-amd64] Re: USB Scanner Problems with Newer Kernels/Libusb

[Barry Schwartz]

Tanstaafl <tanstaafl@libertytrek.org> skribis:
> On 2013-11-09 6:10 PM, Barry Schwartz <chemoelectric@chemoelectric.org> 
> wrote:
> > (I have run into trouble, though, due to systemd being in the 32-bit
> > compatibility binaries and installing pkg-config files. I suppose I
> > could just have emerge remove those files via a Portage hook.)
> 
> Isn't this what
> 
> INSTALL_MASK="/usr/lib/systemd/"
> 
> in /etc/portage/make.conf is for?
> 

Maybe. Portage is very complicated and full of redundancies. :)

Re: [gentoo-amd64] Re: USB Scanner Problems with Newer Kernels/Libusb

[Frank Peters]

On Sat, 9 Nov 2013 14:57:26 -0800
Mark Knecht <markknecht@gmail.com> wrote:

> I sort of feel that the kernel will always be
> the kernel and everything that gets piled on top sort of seems to take
> care of itself in the end. If freedesktop.org goes down a path where
> it gets too complicated then I believe\think\hope some other group
> will step forward quietly, fork that portion
>

Yes, the kernel is fine.  But I need/want a GUI, not a DE, just a GUI --
and that's the problem because freedesktop.org is the only game in town
and whatever they do everyone must do.  Coupled with the fact that graphics
processors are all tightly controlled by Nvidia, Radeon, etc. we are
destined for an extremely monolithic Linux graphical environment in
the future.

Fork?  Who's gonna take on that challenge?  It seems that a handful
of developers, with their own special and exclusionary agenda, will control
the Linux GE for all time.

Frank Peters

Re: [gentoo-amd64] Re: USB Scanner Problems with Newer Kernels/Libusb

[Barry Schwartz]

Frank Peters <frank.peters@comcast.net> skribis:
> Fork?  Who's gonna take on that challenge?  It seems that a handful
> of developers, with their own special and exclusionary agenda, will control
> the Linux GE for all time.

Forking the bulk of this stuff isn’t very productive, because it’s not
good enough to improve upon significantly. At best what can be done by
forking is stuff like eudev, hacks to try to minimize the damage.

It’s just not competent software, as a whole. What am I to make of it,
that I have to use OpenBSD’s pkg-config clone, because the one from
freedesktop, the most basic piece of software in the system, does not
work correctly? I think they haven’t even done something as simple as
properly maintain the serial number of pkg.m4.

Re: [gentoo-amd64] Re: USB Scanner Problems with Newer Kernels/Libusb [Solved]

[Frank Peters]

On Sat, 9 Nov 2013 12:29:49 +0000 (UTC)
Duncan <1i5t5.duncan@cox.net> wrote:

>  
>  udev creates usb devices in /dev/bus/usb.  Each usb port is a (three-digit
> sequentially numbered) directory,
> Libusb, meanwhile, has been updated to work with the /dev/bus/usb/ tree, 
> so AFAIK that's what you need to create... somehow.
>

On Sat, 9 Nov 2013 16:11:48 -0600
Barry Schwartz <chemoelectric@chemoelectric.org> wrote:

> 
> Maybe you can run it once even as a non-daemon just to create the
> device, similarly to running rescan-scsi-bus.
> 

Well, it works!  That is, I can start udev as a daemon, plug in my
scanner, and udev will tell me what the major and minor numbers of the
scanner device node.  Then I can use my scanner with libusb and SANE.

I thought that udev would also create the device node, but for some
reason it does not.  However, I can create the device manually in the /dev
tree and then SANE is able to detect the scanner.

Thanks especially to Duncan for his info on the /dev/bus/usb/001 ...
location.  I can manually create, for example, /dev/bus/usb/001
using the major/minor numbers reported by udev.  Then I am in
business for scanning.

Actually, I can just create a whole series of nodes under /dev/bus/usb
with the same major number and an incremental minor number.  This
will eliminate the need to use udev entirely.  Whenever I plug in my
scanner one of those nodes will be relevant.

But the whole process illustrates that udev is not really necessary.
The scanner is accessed through a device node in the /dev tree
and there is no reason why udev must create this node.  As long
as the major/minor numbers are known (and the char or block characteristic)
then anything can create the node.  IOW, the kernel access is the
same as always.  Only these user space programs impose these node
constraints.

I wonder if I can determine the major/minor numbers in some other way.
Maybe the /sys tree has that information.  If so, then I could
completely script the process without using udev.

Thanks to all for the contribution.

Frank Peters

Re: [gentoo-amd64] Re: USB Scanner Problems with Newer Kernels/Libusb [Solved]

[Barry Schwartz]

Frank Peters <frank.peters@comcast.net> skribis:
> Actually, I can just create a whole series of nodes under /dev/bus/usb
> with the same major number and an incremental minor number.  This
> will eliminate the need to use udev entirely.  Whenever I plug in my
> scanner one of those nodes will be relevant.

I figured as much. Please let us know if you come up with a good
technique that can be adapted to a MAKEDEV script or such. It is nice
to know how to dump e?udev, just in case; and I also use a scanner.

Re: [gentoo-amd64] Re: USB Scanner Problems with Newer Kernels/Libusb [Solved]

[Frank Peters]

On Sat, 9 Nov 2013 19:53:02 -0600
Barry Schwartz <chemoelectric@chemoelectric.org> wrote:

> 
> I figured as much. Please let us know if you come up with a good
> technique that can be adapted to a MAKEDEV script or such. It is nice
> to know how to dump e?udev, just in case; and I also use a scanner.
> 

As soon as I plugged in the scanner (after starting the udev daemon)
an entry was created in /run/udev/data for c189:131.  This obviously
refers to a device node for a character (c) file with a major number
of 189 and a minor number of 131.  This is all that is needed to access
the device through the kernel.

However, with udev and libusb there are user-space conventions.
A utility such as libusb expects device nodes to be in a certain
location, i.e. /dev/bus/usb, and to be named a certain name, i.e.
001, 002, etc.  If not for these conventions the device node could
be placed anywhere and named anything.

So, to stick to conventions, create the node in /dev/bus/usb:

mknod --mode=0666 /dev/bus/usb/001 c 189 131

That's it.  Now the scanner can be detected by libusb and SANE.
Furthermore, this is a *permanent* device node.  It will not
disappear upon reboot.  The udev daemon is needed only once
to get the major/minor numbers.

However, if the device is unplugged and then plugged back in again
the minor number usually jumps.  So just create a sequence of
nodes to allow for this:

mknod --mode=0666 /dev/bus/usb/002 c 189 132
mknod --mode=0666 /dev/bus/usb/003 c 189 133 ...

Upon reboot, the node will reset back to the first minor number.

If you check devices.txt in the kernel source documentation, these
particular major/minor numbers are not listed for any device.  I don't
know where they come from, but as long as they work I don't care.
However, this would preclude using something like MAKEDEV to pre-build
these nodes.  Also, user-space programs expect certain node naming
conventions and I don't know where such conventions are listed.

A new issue is determining how udev gets its device info.  The kernel
sends out a notice when it detects a new device and the udev daemon
gets the notice and then reads the new kernel device parameters
from somewhere.  But from where?  If this were known then any script
could read the same parameters.

In short, if udev reads data from the kernel then a simple custom
program or script could accomplish the same thing.

Frank Peters

Re: [gentoo-amd64] Re: USB Scanner Problems with Newer Kernels/Libusb [Solved]

[Frank Peters]

On Sat, 9 Nov 2013 19:53:02 -0600
Barry Schwartz <chemoelectric@chemoelectric.org> wrote:

> 
> I figured as much. Please let us know if you come up with a good
> technique that can be adapted to a MAKEDEV script or such. It is nice
> to know how to dump e?udev, just in case; and I also use a scanner.
> 

There are some difficulties with my previous response, but now I have
found the final answer.

Here are the steps.  Udev (or equivalent) is *not* necessary.

1) Plug in the scanner.  Check kernel log.  Mine shows this message:

kernal: usb 2-1.4 ...

2) Based on kernel message, check matching entry in the /sys tree.

/sys/bus/usb/devices/2-1.4/uevent

Mine shows this:

MAJOR=189
MINOR=130
DEVNAME=bus/usb/002/003
DEVTYPE=usb_device
DRIVER=usb
PRODUCT=4b8/110/110
TYPE=255/255/255
BUSNUM=002
DEVNUM=003

3) Based on this information, create the device node:

mknod --mode=0666 /dev/bus/usb/002/003 c 189 130


Now it works well.  This is certainly the correct device node.
I can scan with SANE without problems.

Also, this can be scripted using bash to read the /sys file.
But there may be a much better way to get the kernel output
other than using the kernel log. I need to research this step.

Frank Peters

Re: [gentoo-amd64] Re: USB Scanner Problems with Newer Kernels/Libusb [Solved]

[Frank Peters]

On Sat, 9 Nov 2013 19:53:02 -0600
Barry Schwartz <chemoelectric@chemoelectric.org> wrote:

>
> It is nice to know how to dump e?udev, just in case; and I also use a scanner.
> 

I just want to add a parting shot to this thread.

I have several scanners and laser printers, all USB, that have not
been supported under Microsoft Windows for some time.  No MS drivers are
available for these devices.  On various MS Windows forums many people
ask in vain about obtaining such drivers.

But under Linux all my scanners and printers continue to work without problem,
and most likely will continue to work for many years to come.

This just underscores the difference between proprietary and open source
software.

Frank Peters

[gentoo-amd64] Re: Re: USB Scanner Problems with Newer Kernels/Libusb [Solved]

[Jörg Schaible]

Frank Peters wrote:

> On Sat, 9 Nov 2013 19:53:02 -0600
> Barry Schwartz <chemoelectric@chemoelectric.org> wrote:
> 
>>
>> It is nice to know how to dump e?udev, just in case; and I also use a
>> scanner.
>> 
> 
> I just want to add a parting shot to this thread.
> 
> I have several scanners and laser printers, all USB, that have not
> been supported under Microsoft Windows for some time.  No MS drivers are
> available for these devices.  On various MS Windows forums many people
> ask in vain about obtaining such drivers.
> 
> But under Linux all my scanners and printers continue to work without
> problem, and most likely will continue to work for many years to come.
> 
> This just underscores the difference between proprietary and open source
> software.

Yeah, but don't forget to mention the other side of the coin. I had the case 
where my new USB printer/scanner did not work under Linux (at least the 
scanning part) while if worked flawlessly under Windows. 6 months later the 
support for the scanner finally arrived. Thanks to the community - no 
complaints, but it can be annoying if your new hardware is simply not yet 
Linux ready...

- Jörg

Re: [gentoo-amd64] Re: Re: USB Scanner Problems with Newer Kernels/Libusb [Solved]

[Paul Jewell]

Sent from my iPhone

> On 12 Nov 2013, at 23:18, Jörg Schaible <joerg.schaible@gmx.de> wrote:
> 
> Frank Peters wrote:
> 
>> On Sat, 9 Nov 2013 19:53:02 -0600
>> Barry Schwartz <chemoelectric@chemoelectric.org> wrote:
>> 
>> I just want to add a parting shot to this thread.
>> 
>> I have several scanners and laser printers, all USB, that have not
>> been supported under Microsoft Windows for some time.  No MS drivers are
>> available for these devices.  On various MS Windows forums many people
>> ask in vain about obtaining such drivers.
>> 
>> But under Linux all my scanners and printers continue to work without
>> problem, and most likely will continue to work for many years to come.
>> 
>> This just underscores the difference between proprietary and open source
>> software.
> 
> Yeah, but don't forget to mention the other side of the coin. I had the case 
> where my new USB printer/scanner did not work under Linux (at least the 
> scanning part) while if worked flawlessly under Windows. 6 months later the 
> support for the scanner finally arrived. Thanks to the community - no 
> complaints, but it can be annoying if your new hardware is simply not yet 
> Linux ready...
> 
> - Jörg
> 
> 
I still prefer the Linux situation. Aware of the risk you identify, I check for Linux capability before buying new equipment. Support some time soon, is better than support never, or worse still support being dropped. 

Don't even start me on apple and AirPrint printers (now happily emulated on a raspberry pi here for my wife's iPad).....

Paul