Scanning with Brother MFC is down and to the left, and too large

[Poikilos]

Describe the bug
Scanning with Brother MFC is down and to the left, and too large (all 4 numbers specifying area are too large).

To Reproduce
Steps to reproduce the behavior:

1. Click Preview

2. Choose an area:
   

3. Scan

4. Click "Files."

5. You get (blurred for privacy):
   

If I manually choose "Flatbed" instead of "Automatic Document Feeder", there is no change (scanservjs or the scanner used the flatbed automatically regardless, which is fine).

Expected behavior
I was able to do a scan from the command line and get the correct portion:

1. I scanned a preview image as follows:

SCAN_NAME="ScanImage_`date '+%Y-%m-%d_%H..%m..%S'`.png"
SCAN_PATH=~/$SCAN_NAME
/usr/bin/scanimage -d 'brother4:bus9;dev1' --source FlatBed --format=png --resolution 300 --output-file $SCAN_PATH

2. I cropped the image in Gimp to the top left corner of myself in the group photo, and used the Print Size window to get the size in millimeters not including the portion I wanted. Then I did undo, then cropped including the portion I wanted, got that print size in millimeters, then calculated the difference:

# crop (mm):
# x and y are sizes not coordinates. t and l are top and left.
# 119.55,51.65 to 143.43,70.10
# would become:
# -l 119.55 -t 51.65 -x 23.88 -y 18.45
# so:
/usr/bin/scanimage -d 'brother4:bus9;dev1' --resolution 1200 -l 119.55 -t 51.65 -x 23.88 -y 18.45 --source FlatBed --format=png --output-file $SCAN_PATH

The scanner rounds it "in steps of 0.0999908" according to /usr/bin/scanimage -d 'brother4:bus9;dev1' -A, but that rounding seemed to occur automatically and I didn't have to do any changes to the coordinates. There was a message saying it was rounded, but the scanned portion with the manual command was correct whereas scanning via the web interface was not.

• Notice that the correct numbers differ significantly from the numbers shown it the web interface when that same area is selected visually. I expected them to be slightly different since the GIMP mouse work was more accurate, but the numbers were significantly different (too far off to work correctly):

Source	Left	Top	Width	Height
scannerjs	143	60	34	27
correct	119.55	51.65	23.88	18.45

Client (please complete the following information):

• Browser Firefox
• Version 84

Server (please complete the following information):

• OS: Server is Ubuntu 20.04
• Node version 12.20.0
• NPM version 6.14.8
• Docker version (N/A: manual install)

[sbs20]

Can you upload the logs please?

Something like this should work:

sudo journalctl | grep server.js > scanservjs.log.installed.txt

[Poikilos]

Here you go:
scanservjs.log.installed.txt

[Poikilos]

The log may help with #102 as well, and may help with the docker part if the docker version uses the same log file.

[sbs20]

What's different about this scanner is that the x/y ratio is not an ISO standard - i.e. it's not A4 / A5. The ratio of this scanner is 355/215 (rather than 297/215). The ratio of the two ratios is 355/297 which happens to be the ratio of scanservjs/correct above (143/119.5). I don't think this is a coincidence.

I had thought that the UI (which translates the pixels to mm) was getting confused because of the different aspect ratio (which is a bug) and sending the wrong values for scanning. And, the evidence suggests this is likely the case. What I can't figure out is quite why or where this is happening - every attempt I have made to reproduce this has failed.

Out of interest, if you just reload the page now, does it fix itself? (it may be a bug in reloading the preview image)

[Poikilos]

I tested it a few times so I don't think reloading the preview helps. I'll try the fix, thanks!

[Poikilos]

Ok. After upgrading, the y axis is now pretty good (seems down a little--see pictures), but the x axis is still off (In this example, I intentionally moved the selection to the left to complete my scan after seeing the approximate offset of the failed scans):

[toxic0berliner]

I got the same issue today with a Brother MFC 7320.
I attached the logs since the last start of scanservjs service, the 2 attached pictures come from the last scan only (I did some other tests before)
I just installed scanservjs today manually on a fresh ubuntu machine.


scanservjs.log.installed.txt

[sbs20]

I wonder if this is something to do with the source. Does it get closer when set to Flatbed?

[sbs20]

Is this issue still present? I made some changes recently for #118 which may make a difference to this. Not being able to reproduce myself, makes it hard to test!

[toxic0berliner]

Sorry, I didn't see your answer from February.
Will save my devices.conf and try to remember how to update and will report back after testing.
To be honest this issue is a bit nitpicking, I never crop in using the scanner, I always scan the full A4 page and if needed edit the jpg later on...
But this soft is so aweso6I wanted to test everything and pitch in as I can.
Anyway, will report test results before end of week.

[sbs20]

Will save my devices.conf and try to remember how to update and will report back after testing.

If you're doing a direct install (not docker) then there is now a one line install which should preseve all your existing config for you. That said, devices.json will get recreated as it's a new version.

To be honest this issue is a bit nitpicking, I never crop in using the scanner, I always scan the full A4 page and if needed edit the jpg later on...

Yeah, that's mostly what I do too!

But this soft is so aweso6I wanted to test everything and pitch in as I can.

Thank you - very kind!!

[toxic0berliner]

First : love the oneline installer that preserved my config.logal.js (my PDF+jpg profile that I like, having both the pdf and jpg for archiving...) Moved sucessfully from 2.6.0 to 2.9.2

Trying it again, it did fix it partly, it feels now left-right is correctly aligned, but top-bottom seems still a bit off :

Result:

A hint is maybe in the fact that what looks like almost a suare on the preview turns into a quite high rectangle...

But again, realy not an issue for me, almost never using this...

[sbs20]

First : love the oneline installer that preserved my config.logal.js

😄

A hint is maybe in the fact that...

You're right. That is very interesting.

So, I'm wondering if I can ask you to run a few tests for me and share the logs of what happens - not urgent, and completely understand if you don't have time. But I want to see where and why a square is turning into a rectangle.

• Clear and request a new preview
• Perform a square scan on ADF - and include screenshot (I want to see what width and height appear in the UI)
• Perform a full page scan on ADF
• Perform a square scan on Flatbed
• Perform a full page scan on Flatbed

Thank you!

[toxic0berliner]

Seems you identified one of the issues here...
A square in the gui is not a squer on the preview :
Using flatbed and setting the size as a square in the GUI :

Result :

Then clear and preview, try to select something that looks like a squer. Gui:

Result:

Clear+Reset+SetPageSize to A4. Then realized I left source to ADF while in reality the document on the flatbed ! So changed source and tried this :

Result:

Doesn-t seem to make a difference.
Now full page scan using FB before switching to ADF (also changed for a similag page but with a border all around to better see where the page is) :

As you can see, A4 preview seems it will cut of the end...
Result is not cropped though, perfect scan of full A4 page:

Now with ADF putting the document in ADF this time ;) :
Preview :

Full Page scan looks fine, cropped a little on top and added a little white on the bottom, but nothing bad, I'm not even sure the official brother MFC7320 software does better :

Re-preview and selecting something that looks like a square :

Looks a bit better 34/35... (don't mind I fed the document not perfectly right it seems)
The result :

Now ADF selecting a 36/36 square :

And the matching result

[toxic0berliner]

Hope it'll be useful, sorry, I changed the document inbeween, and left ADF selected but my scanner uses FB instead since it senses there is nothing in FB...

I'm happy to help if you like this, just don't expect snappy answers outside weekends due to time-consuming day-job on weekdays...

[sbs20]

Thank you so much. Same here for weekdays! One other thing, any chance of the logs? The final piece of the puzzle!

[toxic0berliner]

Hmm.. Where can I find the logs ?

[toxic0berliner]

I did this, shoudl contain what you need
journalctl -xe |grep server\.js > /tmp/scanservjs.log
scanservjs.log

There are quite a few scans/preview I failed and did not paste above, sorry if it's a bit of a mess to find which logs corresponds to which scan, already deleted the files that could have contained the hour:minute...

[sbs20]

I usually do sudo journalctl -e -u scanservjs but yours looks good!

[sbs20]

Ok.

Observations:

• The horizontal alignment with the ADF looks closer (but still wrong) on left aligned than `centrally aligned

Some questions for when you next get a chance to look at this.

• What are the actual physical dimensions of the Flatbed? The scanner is reporting 215 x 355
• Can you post the result of scanimage -A?

Hypothesis

• The scanner is reporting its height as 355mm. In reality it appears from the preview and the scans that it's more like 297mm (or probably 300).
• The scanner dimensions for the flatbed are actually (roughly) A4.
• The scanner dimensions for the ADF are larger.
• Scanimage / SANE is getting bogus dimensions (height from the ADF and width from the flatbed) - or scanservjs is reading those settings incorrectly.

I have also found a bug which may be masking this issue. The preview code ignores the source (so it's always using the ADF setting).

[toxic0berliner]

Will come back with scanimage and exact dimensions later, but in fact the pane of glass of the flatbed is indeed a bit larger and quite a bit higher than standard A4 paper, 355mm sounds right, will check though. But I've never really used it to scan anything higher than an A4 sheet of paper and I always got the feeling that somehow it tends to "snap" onto A4 size, especially with ADF I remember having scanned several sheets of A4 stuck together by the short size to make like a rolled paper and feeding it through the ADF, it will scan the first page and then continue unrolling until the end of the paper before wanting to scan the next page.

[sbs20]

For whenever you get a chance...

New hypothesis:

• There was a bug Add source to preview #203 which meant that preview always used the default source
• The default source in these cases is the ADF
• I suspect (but don't know) that some kind of magic happens with the ADF which trims the size of the image to the real size of the physical input (e.g. A4) - this matches your thinking too.
• But scanimagejs thinks that the aspect ratio is something else (215 x 355) - and this causes the cropping arithmetic to be incorrect.

Tests:

• Post the results of scanimage -A
• Use the latest version (v2.10.0) to preview using the Flatbed. I am hoping that this results in a preview which has an aspect ratio of 215 x 355 - a screenshot of the full UI after the preview would be great. (I also fixed a bug which meant that the preview wasn't resized correctly all the time)
• Then attempt to crop
• Then do the same with the ADF

My suspicion (hope) is that the flatbed will be ok, but the ADF will not.

Thank you so much for your help with this. And no need to hurry! 🙏

[toxic0berliner]

Physical measured dimensions of my flatbed on MFC-7320 : 22.9cm width by 30.3cm height, on top of the flatbed is a windows for the ADS same width but 2cm height and it's separated fom the flatbed by a 2.2cm piece of plastic.

Updated to 2.10.0

scanimage -A

printer@printer:~ $scanimage -A
Output format is not set, using pnm as a default.
scanimage: open of device brother3:bus2;dev1 failed: Invalid argument
#damn, this again, trying as root...
root@printer:/home/printer $scanimage -A
Output format is not set, using pnm as a default.
scanimage: rounded value of br-x from 215.9 to 215.88
scanimage: rounded value of br-y from 355.6 to 355.567

All options specific to device `brother3:bus2;dev1':
  Mode:
    --mode Black & White|Gray[Error Diffusion]|True Gray|24bit Color [24bit Color]
        Select the scan mode
    --resolution 100|150|200|300|400|600|1200|2400|4800|9600dpi [200]
        Sets the resolution of the scanned image.
    --source FlatBed|Automatic Document Feeder(left aligned)|Automatic Document Feeder(centrally aligned) [Automatic Document Feeder(left aligned)]
        Selects the scan source (such as a document-feeder).
    --brightness -50..50% (in steps of 1) [inactive]
        Controls the brightness of the acquired image.
    --contrast -50..50% (in steps of 1) [inactive]
        Controls the contrast of the acquired image.
  Geometry:
    -l 0..215.9mm (in steps of 0.0999908) [0]
        Top-left x position of scan area.
    -t 0..355.6mm (in steps of 0.0999908) [0]
        Top-left y position of scan area.
    -x 0..215.9mm (in steps of 0.0999908) [215.88]
        Width of scan-area.
    -y 0..355.6mm (in steps of 0.0999908) [355.567]
        Height of scan-area.

The full preview with FB selected :

Trying to crop to something square-ish the coordinates still seem off:

And the result is still off:

Reset, then the same with ADF:

Croping looks to be a square :

But the result is still to the right :

I think my first attempt with Flatbed was a cache issue in firefox since the preview window changed size when I hit reset...
Trying again the reset+FB :

It "snaps" to A4 size again, while after a reset the "empty" preview window with the nice nature picture does look higher than A4...
Cropping:

Result:

When I hit reset :

It does seem to have proper aspect ratio.

My guess is that you do have the proper size in the preview when I hit reset, but the preview returns a picture that is only A4 and never the full width and height of the flatbed... If I preview, crop, but then move the paper to the top-left of the flatbed and launch the scan, I get the following result :

So thit time it's to the left and somehow still down... Maybe my guess was wrong...

With the paper alignment needing to be in the middle even fot the FB, I think a picture of the device is the easyest way for you to understand :

I think the empty space on 3 sides of the paper is causing the issue, probably the scanner is messing around reporting it's realy physical size but only scanning A4 size unless maybe some specific params are given...

Hope that helps.
I'll be quite busy for the next month and not at home to test physically with the scanner, but if you want more tests I should be back in june..
And again, from my point of view, this is nit-picking as I'm always scanning full page A4 paper, so the software is already doing all I need ! ;)

[sbs20]

Thank you so much for all your help!! I think this gives me enough to guess what's happening but I need time to write it up. Thanks again!

[sbs20]

Ok, as you hinted at, I think the clue is in:

I think my first attempt with Flatbed was a cache issue in firefox since the preview window changed size when I hit reset...
Trying again the reset+FB :

The preview image in the screenshot has an approximate ratio of 1.41 which is the same as the ISO (e.g. A4) ratio. If the preview was the size we'd expect (from the scanner dimensions) it would have a ratio of more like 1.65 (355/215). Therefore I'm willing to guess that running the following command line will result in an A4 scan:

/usr/bin/scanimage -d 'brother3:bus2;dev1' --mode '24bit Color' --source 'FlatBed' --resolution 200 -l 0 -t 0 -x 215 -y 355 --format 'tiff' > ~/ratio-test.tif

Assuming that is the case then something between and including scanimage, the sane front end and the sane backend or scanner is magically cropping the image. Since this only seems to happen with Brother scanners, my guess is it's in the driver or hardware. I guess I'd be interested to see what happened if you put an A5 document in there.

Once we assume that the auto-cropping is happening, everything else follows... the software uses the aspect ratio of the scanner (not the image), then the cropping will be off - as I'd expect.

If it's the case that the scanner is doing this, then as far as I can see, there is no reasonable fix and I will close this issue.

[toxic0berliner]

$sudo /usr/bin/scanimage -d 'brother3:bus2;dev1' --mode '24bit Color' --source 'FlatBed' --resolution 200 -l 0 -t 0 -x 215 -y 355 --format 'tiff' > ~/ratio-test.tif

So yes, something is cropping to A4 regardless...
That was Flatbed with an A5 piece of paper in there...
Got the samee with an A6 paper sidewise in the ADF : scan results in A4 output.

So I'm perfectly fine with this issue being closed since the issue is realy in the scanner not doing what it's being told.

I'm simply going to trust you that relying on the scanner reported dimensions is the right way and not analyzing the file it returns.

Nonetheless, now that I've been able to test it under windows, I can confirm, despite the physical size of the scanner being larger than A4, there is no way in which I got the scanner to scan anything larger than A4, even in preview mode or anything, and using the manufacturer's tools or windows integrated tools...

So maybe as a hack, do you have any idea on how to trick the tools in believing the scanner reports a size of A4 paper an no bigger ? Or is it something hardcoded in the driver and we can't change it ? I'm in the process of editing the devices.conf to fake the dimensions, will report back if any sucess.

[sbs20]

Bizarre! Ok cool.

So maybe as a hack, do you have any idea on how to trick the tools in believing the scanner reports a size of A4 paper an no bigger ?

Yes. Create / edit a config.local.js and add something like below... (compare to what's in config.default.js). Please note, I haven't actually run or tried this code so sorry if there's an error - but I've done similar things for testing, so the principle will work. This gets call after scanimage -A but before it gets used anywhere... so it will always overwrite the supplied values from the scanner. You may wish to adjust the 215/297 according to what actually gets scanned.

  afterDevices(devices) {
    const device = devices.filter(d => d.id.includes('brother'))[0];
    if (device) {
      device.features['-l'].limits = [0, 215];
      device.features['-t'].limits = [0, 297];
      device.features['-x'].default = 215;
      device.features['-x'].limits = [0, 215];
      device.features['-y'].default = 297;
      device.features['-y'].limits = [0, 297];
    }
  }

[toxic0berliner]

I did edit the devices.json manually since I knew I did it in the past, probably a bad idea since I realize now that the device gets a new address each time I unplug/replug it even on the same port...

Nonetheless, I got better results :

Result:

This one is perfect but not quite easy to do, as you can see the preview window is not completely filled, this is, I think, because the preview window is using the size I put in devices.conf for it's overall size, but the picture in it is the result of a SCAN and not of the preview, so it fills only what the scanne is capable of doing.

Doing it with the preview I am slightly off still:

We can see it already, doing a preview it keeps the position of the correct scan and we see somehow it's not aligned :

I'm not sure I'm making sense here, this last picture is a resulf of : full scan, then select the proper circle, then click preview.

I'm pretty sure now that it's because the scanner is cropping to even less than a real A4 paper, so I'll have to check the number of pixels, use the resolution to go to inches and then to milimeters and try more precise values...

Will try the config.default.js thing once I get good values ;)

An idea thought, since I see that going to the preview page when the last picture was a scan, in this case the preview windows is not completely filled, I imaging that somewhere you "stretch" the preview to the preview window size. Maybe if you did not "stretch" it and just put it the way you put the last scan in the preview window, then the issue would completely disappear, and only consequence would be a preview window that can never be fully filled.

[toxic0berliner]

Also... Week after the fact, I see that the scanner reports to be able to scan in a lot of resulutions that it is in fact not capable of, it tops out at 1200dpi in real life, and scanservjs sees it having a max at 9600dpi... Any proposal for device.features['resolution'].max=1200 ?

Also, I'm not seeing A4 in the proposed paper sizes in the preview window. 5Sorry, I'm turning this issue into a bit of a mess).

Going back to testing the new size and config.local.js

Edit: I'm trying to scan at 1200dpi to get more pixels and have a better calculation of the real scan size, so I noticed that the scanner is taking so long that it seems scanservjs is giving up on it and does not show the scanned file... I'm doing this in console to see what it does, the tiff is already at 200MB and still growing, maybe that's an issue for scanservjs ;)

[sbs20]

Also... Week after the fact, I see that the scanner reports to be able to scan in a lot of resulutions that it is in fact not capable of

Also add the following to the local config to control the resolutions (use your own values)

      device.features['--resolution'].default = 150;
      device.features['--resolution'].options = [75, 150, 300, 600];

Also, I'm not seeing A4 in the proposed paper sizes in the preview window.

Paper sizes greater than the device dimensions are excluded - my guess is that the dimensions have at least one axis less than the equivalent A4 axis. Note that there's a bug pre v2.11.0 where A4 is the incorrect size.

Sorry, I'm turning this issue into a bit of a mess).

Haha - don't worry - this is all very helpful!

I noticed that the scanner is taking so long that it seems scanservjs is giving up on it and does not show the scanned file

Yeah, I have seen than before. I'm more inclined to limit the resolutions than do anything else.

[toxic0berliner]

so, your options to limit the resolution looks fine and works for me too !

As you guessed, the real size turned out to be below A4 paper size in my calculations.

But now I'm giving up, since I believe the scanner is a lost cause, in fact it crops quite a bit into a real a4 paper, but when I try setting the real size or get too close to it I hit some weird artifacts where the lost bottom of the page repeats in a vertical band on the right of the paper.... That magically disappeared when I set source to FlatBed...

I've got it setup to be able to select A4 paper size in case someone wants to use it, and with a size set to 210x297 I'm now not too far off, just by what the scanner realy crops... I have the workaround to scan then crop, for a feature I almost never use, that'll be fine...

Anyway, I pimped up my config.local.js quite a bit:

  afterDevices(devices) {
    const device = devices.filter(d => d.id.includes('brother'))[0];
    if (device) {
      device.features['-l'].limits = [0, 215];
      device.features['-t'].limits = [0, 297];
      device.features['-x'].default = 215;
      device.features['-x'].limits = [0, 215];
      device.features['-y'].default = 297;
      device.features['-y'].limits = [0, 297];
      device.features['--resolution'].default = 200;
      device.features['--resolution'].options = [75, 100, 150, 200, 300, 400, 600];
      device.features['--mode'].default = '24bit Color';
      //device.features['--source'].default = 'Automatic Document Feeder(centrally aligned)';
      device.features['--source'].default = 'FlatBed';
    }
  }

But also fixed up my prefered pipelines and added them on top of the list:

const myPipelines = [
    {
        extension: 'pdf',
        description: 'OCR | PDF & JPG (JPG | Low quality)',
        commands: [
            'convert @- -quality 50 scan-%d.jpg && ls scan-*.jpg',
            `/usr/bin/tesseract -l fra -c stream_filelist=true - - pdf > scan-0001.pdf`,
            'ls scan-*.*'
        ]
    },
    {
        extension: 'pdf',
        description: 'OCR | PDF & JPG (JPG | Medium quality)',
        commands: [
            'convert @- -quality 75 scan-%d.jpg && ls scan-*.jpg',
            `/usr/bin/tesseract -l fra -c stream_filelist=true - - pdf > scan-0001.pdf`,
            'ls scan-*.*'
        ]
    },
    {
        extension: 'pdf',
        description: 'OCR | PDF & JPG (JPG | High quality)',
        commands: [
            'convert @- -quality 92 scan-%d.jpg && ls scan-*.jpg',
            `/usr/bin/tesseract -l fra -c stream_filelist=true - - pdf > scan-0001.pdf`,
            'ls scan-*.*'
        ]
    }
    ];

    //myPipelines=myPipelines.reverse()
    config.pipelines=config.pipelines.reverse();
    config.pipelines.push(myPipelines[0]);
    config.pipelines.push(myPipelines[1]);
    config.pipelines.push(myPipelines[2]);
    config.pipelines=config.pipelines.reverse();

Too lazy to search for the length of my pipelines and make a loop, it's been too long since I last did some javascript...

I do like this pipeline that give both the pdf with OCR and the jpg thought, I use this for archiving !

Maybe one day I'll get around to making a pdf-a since pdf+jpg is probably not the best for archiving, pdf-a might be preferrable...

Anyway, that's now in a state where I'm very satisfied with what it does ;) Scanning from my phone is a breeze ! Thanks again !

[toxic0berliner]

Maybe one last thing : is there an easy way for me to "disable" the Auto(Reverse) batch mode since I only use Auto(Collate) ?

[sbs20]

Right. I'm going to close this.

Root issue is mismatch between SANE reported values and what actually happens. Addressed with documentation here

Too lazy to search for the length of my pipelines and make a loop, it's been too long since I last did some javascript...

This recipe might help!

Maybe one day I'll get around to making a pdf-a since pdf+jpg is probably not the best for archiving, pdf-a might be preferrable...

I've raised #223 for you.

Maybe one last thing : is there an easy way for me to "disable" the Auto(Reverse) batch mode since I only use Auto(Collate) ?

Done. See #222 (I also disabled that mode by default since I doubt anyone uses it)

[toxic0berliner]

Awesome! Thanks a lot and congrats again on the amazing soft!