V 1.0.1 Crash at start of scan with HP Laserjet

[roechr]

When I try to start scanning on a HP Color Laserjet MFP M283fdw the app crashes (closes itself and the homescreen of android is shown).
After restarting the app a message is shown: "Absturzprotokoll gefunden"
Details I will add in the other sections ...

Steps to reproduce the behavior:

1. Choose the printer connection
2. Click on 'Scannen'
3. The printer starts the physical scan process
4. See error

Expected behavior
The scanned document should be displayed in the app.

Screenshots
Will add them later if necessary

Smartphone (please complete the following information):
Device: Motorola Edge 50 pro
Android version: 14
ScanBridge version: 1.0.1 (1000001, 42f9281 ) nightly

Additional context

nl.adaptivity.xmlutil.serialization.UnknownXmlFieldException: Could not find a field for name (io.github.chrisimx.esclkt.JobInfo) {http://schemas.hp.com/imaging/escl/2011/05/03}JobInfo/{http://schemas.hp.com/imaging/escl/2011/05/03}TransferRetryCount (Element)
  candidates: {http://www.pwg.org/schemas/2010/12/sm}JobStateReasons (Element), {http://www.pwg.org/schemas/2010/12/sm}ImagesCompleted (Element), {http://www.pwg.org/schemas/2010/12/sm}JobState (Element), {http://www.pwg.org/schemas/2010/12/sm}JobUuid (Element), {http://www.pwg.org/schemas/2010/12/sm}ImagesToTransfer (Element), {http://www.pwg.org/schemas/2010/12/sm}JobUri (Element), {http://schemas.hp.com/imaging/escl/2011/05/03}Age (Element) at position 2:433
	at nl.adaptivity.xmlutil.serialization.XmlConfig.DEFAULT_UNKNOWN_CHILD_HANDLER$lambda$9(XmlConfig.kt:672)
	at nl.adaptivity.xmlutil.serialization.XmlConfig.$r8$lambda$4h_-389lwYU9b2M8ciqUTcz2BD8(Unknown Source:0)
	at nl.adaptivity.xmlutil.serialization.XmlConfig$$ExternalSyntheticLambda1.handleUnknownChildRecovering(D8$$SyntheticClass:0)
	at nl.adaptivity.xmlutil.serialization.DefaultXmlSerializationPolicy.handleUnknownContentRecovering(XmlSerializationPolicy.kt:647)
	at nl.adaptivity.xmlutil.serialization.XmlDecoderBase$TagDecoderBase.indexOf(XMLDecoder.kt:1059)
	at nl.adaptivity.xmlutil.serialization.XmlDecoderBase$TagDecoderBase.decodeElementIndex(XMLDecoder.kt:1298)
	at nl.adaptivity.xmlutil.serialization.XmlDecoderBase$TagDecoderBase.decodeElementIndex(XMLDecoder.kt:1128)
	at io.github.chrisimx.esclkt.JobInfo$$serializer.deserialize(ScannerStatus.kt:87)
	at io.github.chrisimx.esclkt.JobInfo$$serializer.deserialize(ScannerStatus.kt:87)
	at nl.adaptivity.xmlutil.serialization.XmlDecoderBase.deserializeSafe(XMLDecoder.kt:111)
	at nl.adaptivity.xmlutil.serialization.XmlDecoderBase$AnonymousListDecoder.decodeSerializableElement(XMLDecoder.kt:1701)
	at kotlinx.serialization.encoding.CompositeDecoder.decodeSerializableElement$default(Decoding.kt:539)
	at kotlinx.serialization.internal.CollectionLikeSerializer.readElement(CollectionSerializers.kt:80)
	at kotlinx.serialization.internal.AbstractCollectionSerializer.readElement$default(CollectionSerializers.kt:51)
	at kotlinx.serialization.internal.AbstractCollectionSerializer.merge(CollectionSerializers.kt:36)
	at nl.adaptivity.xmlutil.serialization.XmlDecoderBase$TagDecoderBase.decodeSerializableElement(XMLDecoder.kt:900)
	at io.github.chrisimx.esclkt.Jobs$$serializer.deserialize(ScannerStatus.kt:80)
	at io.github.chrisimx.esclkt.Jobs$$serializer.deserialize(ScannerStatus.kt:80)
	at nl.adaptivity.xmlutil.serialization.XmlDecoderBase$TagDecoderBase.decodeNullableSerializableElement(XMLDecoder.kt:978)
	at io.github.chrisimx.esclkt.ScannerStatus$$serializer.deserialize(ScannerStatus.kt:63)
	at io.github.chrisimx.esclkt.ScannerStatus$$serializer.deserialize(ScannerStatus.kt:63)
	at nl.adaptivity.xmlutil.serialization.XmlDecoderBase.deserializeSafe(XMLDecoder.kt:111)
	at nl.adaptivity.xmlutil.serialization.XmlDecoderBase.deserializeSafe$default(XMLDecoder.kt:99)
	at nl.adaptivity.xmlutil.serialization.XmlDecoderBase$XmlDecoder.decodeSerializableValue(XMLDecoder.kt:345)
	at nl.adaptivity.xmlutil.serialization.XML.decodeFromReader(XML.kt:485)
	at nl.adaptivity.xmlutil.serialization.XML.decodeFromReader$default(XML.kt:450)
	at nl.adaptivity.xmlutil.serialization.XML.decodeFromString(XML.kt:390)
	at io.github.chrisimx.esclkt.ESCLRequestClient.getScannerStatus(ESCLRequestClient.kt:165)
	at io.github.chrisimx.esclkt.ScanJob.getJobStatus(ScanJob.kt:41)
	at io.github.chrisimx.scanbridge.ScanningScreenKt.doScan$lambda$12(ScanningScreen.kt:399)
	at io.github.chrisimx.scanbridge.ScanningScreenKt.$r8$lambda$Tih5VPO2HbOBEwVApYcFrk5SjZw(Unknown Source:0)
	at io.github.chrisimx.scanbridge.ScanningScreenKt$$ExternalSyntheticLambda3.invoke(D8$$SyntheticClass:0)
	at kotlin.concurrent.ThreadsKt$thread$thread$1.run(Thread.kt:30)

[Chrisimx]

The eSCL specification is sadly not written very well. Therefore, implementations might diverge from one scanner to another and of course, I can't test it with every device. This was the source of the last error and probably is also the source of this bug.

Can you redo the scan with the app, let it crash and then open http://[scanner address]/eSCL/ScannerStatus in a web browser and send the XML to me? I will meanwhile take care of making sure that the app doesn't crash in such cases and instead just aborts scanning and shows an error. In the best case, it should also provide the ScannerStatus which led to the error for easy copying (#38).

[roechr]

Here is the result of ScannerStatus:

<!--
 THIS DATA SUBJECT TO DISCLAIMER(S) INCLUDED WITH THE PRODUCT OF ORIGIN. 
-->
<scan:ScannerStatus xsi:schemaLocation="http://schemas.hp.com/imaging/escl/2011/05/03 eSCL.xsd">
<pwg:Version>2.63</pwg:Version>
<pwg:State>Processing</pwg:State>
<scan:AdfState>ScannerAdfEmpty</scan:AdfState>
<scan:Jobs>
<scan:JobInfo>
<pwg:JobUri>/eSCL/ScanJobs/d73dfsas-y7p0-080o-1016-ulzm4fap</pwg:JobUri>
<pwg:JobUuid>d73dfsas-y7p0-080o-1016-ulzm4fap</pwg:JobUuid>
<scan:Age>11</scan:Age>
<pwg:ImagesCompleted>1</pwg:ImagesCompleted>
<pwg:ImagesToTransfer>1</pwg:ImagesToTransfer>
<scan:TransferRetryCount>0</scan:TransferRetryCount>
<pwg:JobState>Processing</pwg:JobState>
<pwg:JobStateReasons>
<pwg:JobStateReason>JobScanning</pwg:JobStateReason>
</pwg:JobStateReasons>
</scan:JobInfo>
<scan:JobInfo>
<pwg:JobUri>/eSCL/ScanJobs/d73dfsas-lgvh-ch0k-1015-f7xh8fge</pwg:JobUri>
<pwg:JobUuid>d73dfsas-lgvh-ch0k-1015-f7xh8fge</pwg:JobUuid>
<scan:Age>2294</scan:Age>
<pwg:ImagesCompleted>1</pwg:ImagesCompleted>
<pwg:ImagesToTransfer>1</pwg:ImagesToTransfer>
<scan:TransferRetryCount>0</scan:TransferRetryCount>
<pwg:JobState>Aborted</pwg:JobState>
<pwg:JobStateReasons>
<pwg:JobStateReason>JobCanceledAtDevice</pwg:JobStateReason>
</pwg:JobStateReasons>
</scan:JobInfo>
</scan:Jobs>
</scan:ScannerStatus>

[Chrisimx]

That's kind of interesting. With <scan:ScannerStatus xsi:schemaLocation="http://schemas.hp.com/imaging/escl/2011/05/03 eSCL.xsd"> there is no namespace declaration for the scan and pwg namespace. With all scanners I tested it looked like this:

<scan:ScannerStatus
        xmlns:scan="http://schemas.hp.com/imaging/escl/2011/05/03"
        xmlns:pwg="http://www.pwg.org/schemas/2010/12/sm">

The used XML parser is very strict on namespaces and it's difficult to change that. It's a pretty stupid fix but the simplest way to fix this will be to search for the scan:ScannerStatus tag and just replace it with the above no matter what other declarations are given or not given.

[Chrisimx]

I've now found out that surprisingly the web browser just removes the namespaces declarations from the XML output. Your scanner however does luckily send them correctly. The actual problem was the TransferRetryCount element that was unknown to eSCLKt. Can you please test if the nightly build works with your scanner now?

[roechr]

No 100% success for now, but we are getting closer:

Pressing the "Scannen" button now starts the scanner, but shows:
"NotSuccessfulCode(responseCode=503)"

[Chrisimx]

Hm, that's pretty difficult to diagnose. Your scanner says it's temporarily unavailable for some reason. With my scanner this sometimes happened when ScanBridge would abort retrieving pages because of an error/crash. The scanner would then see the job as unfinished. After a while the scanner gave up on the job and was available again.

Try completely rebooting the scanner device. If it still happens, can you send the ScannerStatus before and after trying to scan? Maybe there is some insightful information there.
If this doesn't bring the necessary info: Would it be feasible to take an Android logcat of ScanBridge and filter for the tag "ScanningScreen"? Do you know how to do that?
And if that still doesn't help maybe an recording with Wireshark would bring the necessary information, but for that you would need to run ScanBridge in an emulator on your computer. That's pretty cumbersome. If it's necessary I would probably write a minimal app which records all available information instead.

[roechr]

Rebooting the Scanner did not bring any progress.

So here come the outputs of ScannerStatus before and after the 503 error:

<!--
 THIS DATA SUBJECT TO DISCLAIMER(S) INCLUDED WITH THE PRODUCT OF ORIGIN. 
-->
<scan:ScannerStatus xsi:schemaLocation="http://schemas.hp.com/imaging/escl/2011/05/03 eSCL.xsd">
<pwg:Version>2.63</pwg:Version>
<pwg:State>Idle</pwg:State>
<scan:AdfState>ScannerAdfEmpty</scan:AdfState>
</scan:ScannerStatus>

<!--
 THIS DATA SUBJECT TO DISCLAIMER(S) INCLUDED WITH THE PRODUCT OF ORIGIN. 
-->
<scan:ScannerStatus xsi:schemaLocation="http://schemas.hp.com/imaging/escl/2011/05/03 eSCL.xsd">
<pwg:Version>2.63</pwg:Version>
<pwg:State>Processing</pwg:State>
<scan:AdfState>ScannerAdfEmpty</scan:AdfState>
<scan:Jobs>
<scan:JobInfo>
<pwg:JobUri>/eSCL/ScanJobs/k4o8psqg-quid-gmvf-1001-a3bv4bts</pwg:JobUri>
<pwg:JobUuid>k4o8psqg-quid-gmvf-1001-a3bv4bts</pwg:JobUuid>
<scan:Age>51</scan:Age>
<pwg:ImagesCompleted>0</pwg:ImagesCompleted>
<pwg:ImagesToTransfer>0</pwg:ImagesToTransfer>
<scan:TransferRetryCount>0</scan:TransferRetryCount>
<pwg:JobState>Processing</pwg:JobState>
<pwg:JobStateReasons>
<pwg:JobStateReason>JobScanning</pwg:JobStateReason>
</pwg:JobStateReasons>
</scan:JobInfo>
</scan:Jobs>
</scan:ScannerStatus>

<!--
 THIS DATA SUBJECT TO DISCLAIMER(S) INCLUDED WITH THE PRODUCT OF ORIGIN. 
-->
<scan:ScannerStatus xsi:schemaLocation="http://schemas.hp.com/imaging/escl/2011/05/03 eSCL.xsd">
<pwg:Version>2.63</pwg:Version>
<pwg:State>Idle</pwg:State>
<scan:AdfState>ScannerAdfEmpty</scan:AdfState>
<scan:Jobs>
<scan:JobInfo>
<pwg:JobUri>/eSCL/ScanJobs/k4o8psqg-quid-gmvf-1001-a3bv4bts</pwg:JobUri>
<pwg:JobUuid>k4o8psqg-quid-gmvf-1001-a3bv4bts</pwg:JobUuid>
<scan:Age>7</scan:Age>
<pwg:ImagesCompleted>0</pwg:ImagesCompleted>
<pwg:ImagesToTransfer>0</pwg:ImagesToTransfer>
<scan:TransferRetryCount>0</scan:TransferRetryCount>
<pwg:JobState>Aborted</pwg:JobState>
<pwg:JobStateReasons>
<pwg:JobStateReason>JobCanceledAtDevice</pwg:JobStateReason>
</pwg:JobStateReasons>
</scan:JobInfo>
</scan:Jobs>
</scan:ScannerStatus>

With getting logcat output on android and deepdive with wireshark I have no experience.
Would it be possible to implement a "debug" checkbox in the settings of ScanBridge with the effect, that all helpful output gets written to an extra file?

[roechr]

Just tried the brand new official 1.1.0 release, but no better :-(

[Chrisimx]

With getting logcat output on android and deepdive with wireshark I have no experience.
Would it be possible to implement a "debug" checkbox in the settings of ScanBridge with the effect, that all helpful output gets written to an extra file?

That's a very good idea, also for making future bug reports easier. I've implemented this now but I'll have to review everything again before I can merge it into the main branch (#41). Meanwhile, you can download a prerelease APK here: https://fireamp.eu/static/ScanBridge-debug.apk (GitHub doesn't allow uploading APKs here, so I had to put it on a private server)

To use the debugging log feature navigate into the settings and activate the debug log setting. Then do what you normally would do to cause the problem. Afterwards, go into the settings again and export the log file. You can then send it to me and I will have a look.

[Chrisimx]

I've taken a look at the debug log. Unfortunately, it is not very conclusive. Your scanner doesn't send any reason why the NextDocument is temporarily unavailable. But I've made some interesting/weird discoveries:

• Your scanner says, that the actual eSCL endpoint is at port 8080 and not 80. I never thought that this would be the case for a scanner, so this is not implemented, but this could theoretically be the source of the problem. However, more likely its just a bug in your scanners firmware. Try to access http://[IPv4 address]:8080/eSCL/ScannerCapabilities and tell me if it works
• Does the error 503 show before the scanner has completed scanning? If that's the case it could mean your scanner wants to tell that ScanBridge should try to get the document multiple times until it's available. Most scanners start to scan when NextDocument is accessed and then the response only comes when the document is scanned (they let the requesting client wait until the document is available). But maybe yours is different and requires polling. Again, eSCL is sadly not well specified but I think this is the might be the most likely reason for the problem at this moment. I also read that this is the case for some scanners
• Is there a paper on the flatbed glass? Some scanners detect this and do not return something until something is on the glass. But really this is much more a thing with ADFs (did you try to something in the ADF and use this as the input source?)
• Your scanner seams weird in a couple ways. Maybe something's wrong in the settings. I assume it has a web panel? With my scanner there is a panel just for protocols. Does something similar exist with yours?:

• To determine if something is wrong with ScanBridge you might want to try another scanning software using eSCL. On Linux you can use eSCL-sane with Skanpage as frontend. On Windows I sadly don't know of any eSCL frontends

[Chrisimx]

<!--
 THIS DATA SUBJECT TO DISCLAIMER(S) INCLUDED WITH THE PRODUCT OF ORIGIN. 
-->
<scan:ScannerStatus xsi:schemaLocation="http://schemas.hp.com/imaging/escl/2011/05/03 eSCL.xsd">
<pwg:Version>2.63</pwg:Version>
<pwg:State>Idle</pwg:State>
<scan:AdfState>ScannerAdfEmpty</scan:AdfState>
<scan:Jobs>
<scan:JobInfo>
<pwg:JobUri>/eSCL/ScanJobs/k4o8psqg-quid-gmvf-1001-a3bv4bts</pwg:JobUri>
<pwg:JobUuid>k4o8psqg-quid-gmvf-1001-a3bv4bts</pwg:JobUuid>
<scan:Age>7</scan:Age>
<pwg:ImagesCompleted>0</pwg:ImagesCompleted>
<pwg:ImagesToTransfer>0</pwg:ImagesToTransfer>
<scan:TransferRetryCount>0</scan:TransferRetryCount>
<pwg:JobState>Aborted</pwg:JobState>
<pwg:JobStateReasons>
<pwg:JobStateReason>JobCanceledAtDevice</pwg:JobStateReason>
</pwg:JobStateReasons>
</scan:JobInfo>
</scan:Jobs>
</scan:ScannerStatus>

Is the above last scanner status you sent me a long time after the scanner has completed scanning and have you actually manually canceled the job at the device (e.g. by using some button or touch input), as the JobStateReason says?

[roechr]

Nice to see progress even in other issues.
I try to watch them ... today I installed ScanBridge-nightly-20250201.apk from issue #47.
But no better. The 503 error still exists.

So I try to answer to your latest questions:

• http://[IPv4 address]:8080/eSCL/ScannerCapabilities works from my Firefox on Windows.
  It gives the same output as if no explicit port is written in the URL.

• Yes, the 503 error is thrown 1 second after the scanner starts moving.

• Yes, there is a sheet of printed paper in the flatbed. Using the ADF makes no difference.

Here is a screenshot of the web interface:

• Spent half of the afternoon to setup a openSUSE Live system to check other scan software.
  Result: Everything works fine!
  The combination of eSCL-sane and 3 different frontends always resulted in properly scanned files.

On my android smartphone I tested with the official app from Mopria:
https://mopria.org/scan-to-android
Here I have success, too!

So I don't think my printer is wrong or buggy.

BTW: Do you know the Mopria eSCL Specification? https://mopria.org/spec-download
Some days ago I was able to view the file, but today there is a problem with the captcha.

[Chrisimx]

http://[IPv4 address]:8080/eSCL/ScannerCapabilities works from my Firefox on Windows.
It gives the same output as if no explicit port is written in the URL

As the practical experience with #47 shows it does however actually make a difference in some cases whether you connect using 80 or 8080

Yes, the 503 error is thrown 1 second after the scanner starts moving.

That makes it the most likely hypothesis that your scanner requires a sort of polling. I read that some scanners do it like this in https://gist.github.com/markosjal/79d03cc4f1fd287016906e7ff6f07136

Spent half of the afternoon to setup a openSUSE Live system to check other scan software.
Result: Everything works fine!
The combination of eSCL-sane and 3 different frontends always resulted in properly scanned files.

Sorry that you had to put so much time in this. But this result is definitely better than if something would be wrong with your scanner. It would have been perfect if you had ran Wireshark while running your scans. Its basically a software that can record and visualize all network message. Then you can filter for the requests sent to your scanner and see what exactly eSCL-sane has done. It is not that difficult to use but as you are not a Linux user I don't expect you to spent time on that, especially because I'm pretty sure we've found the problem now. At least I hope so.

Do you know the Mopria eSCL Specification? https://mopria.org/spec-download
Some days ago I was able to view the file, but today there is a problem with the captcha.

Yeah, that is actually the document that eSCLKt, the supporting library for eSCL used by ScanBridge, is partially based on, in addition to the knowledge taken from testing with actual scanners. Not everything what is in the document is implemented by eSCLKt though, but some things also just aren't really used by scanners in the wild or are not necessary for a scanner app (e.g. push scans as described in the document). At the same time (as has shown multiple times with the issues here), several scanners also have extended the protocol in some way or another. The Mopria spec is definitely not complete. Even your scanner had properties in the scanner capabilities that are not specified in the Mopria spec. eSCLKt could ignore that in many cases, but then we would never know which additional properties and features scanners in the wild actually have and can't support them. So instead of ignoring, it errors out to get it reported here and these things get then added to a list of known additional features/expansions. Sometimes elements which I assume to be obligatory according to the spec also are missing (e.g. with #47 the scanner UUID is missing).
The Mopria document is honestly written very poorly (in many places it has unclarities and also see below, in such a small excerpt there are already things that I can only explain to me as errors). Especially if compared to more professional specs (e.g. the HTTP/2 spec) it is not a nice read, but at least its something.

From section "11.1
Description" of the spec:

After initiating the scan job, the client is responsible to pace the retrieval of the scan
pages. In this document, this transactional model is labeled: Pull Scan.
Once the job is created, the scanner SHOULD be in Pending or Processing state.
The client keeps checking the scanner status and the client uses the job URL to
upload (really? "upload"?, that can't be correct in this context, or maybe it can be?) next scan page after the JobState moves to Pending or Processing (didn't it say before that the scanner should already be in the Pending or Processing state right after job creation? For what should I wait in this case?). The
client keeps invoking this URL until the NextDocument request returns 404 Not
Found. When all pages are scanned, the JobState MUST indicate ‘Completed’
denoting there are no more pages to be uploaded. The JobState SHOULD indicate
‘Canceled’ or ‘Aborted’ if the job was either canceled by the user or due to internal
abort
If the NextDocument request times out, the client SHOULD check the JobState and
retry pulling the scan pages if the JobState is either Pending or Processing.
After the job is created, if the client fails to retrieve the scan pages for some reason
– could be even a network failure, the scanner MAY abort the scan job after a
certain time period determined by the scanner. The scanner SHOULD NOT cancel
the job if it can retain the scan pages for a longer period without resource
constraints.

ScanBridge does not do it exactly like this (even though this description is not very exact either). Actually, it does pretty exactly what is written as an example in Section "11.5
Usage Flow" . It was written with the spec in mind (as seen by orienting at the given example) but as long as it did work with my scanner it thought it was fine (my scanner does support doing it like 11.5 Usage Flow. Yours probably requires doing it more like what is meant with the quote above). It is difficult to say how hard it is to make ScanBridge work both mine and your scanner but it is definitely possible.

I will try to implement this usage flow and then talk back to you to see if it works

[Chrisimx]

I've now created 2 test APKs. These are not meant as actual fixes but will hopefully provide information on how a fix could look like.

The first one observes and logs the job status for 40 seconds. This will be interesting as we can then see if there are any important changes there. So try out that one. Leave it enough time as it will take at least 40 seconds. While doing that record a debug log and then send it to me.

The second one tries to access the new page endpoint until it actually returns something (until it is not 503 anymore). Record a debug log with this one. Give it at least a minute as well. If it then didn't retrieve the page you can close the app and send the debug log to me.

[Chrisimx]

That's very good! With both APKs your scanner sent an image. Can you now please test if the issue is fixed with this build? Check both ADF (with multiple pages) and Platen input source. If it does not work, please record another debug log

[roechr]

SUCCESS !!! 👍✨

Now it works, both flatbed and ADF ...

[Chrisimx]

Amazing! Thanks for hanging on so tightly and providing the necessary logs :)

Fixed with 9bc201f and #58