Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fields containing variable text (DA for radio buttons, field dictionary, widget annotation dictionary) #323

Open
a20dev opened this issue Aug 11, 2023 · 26 comments
Assignees
Labels
bug Something isn't correct question Further information is requested

Comments

@a20dev
Copy link

a20dev commented Aug 11, 2023

I couldn't find a definition of "fields containing variable text" in ISO 32000-2:2020. There are only a couple of examples in the NOTE of 12.7.4.3.

Adobe Acrobat, for instance, puts a DA entry into the field dictionary of radio button fields, see the attached document created with Adobe Acrobat. For check boxes, the appearance created by Adobe Acrobat uses a character from the ZaDB font for the check mark, so check boxes use text, but not variable text.

Moreover, Adobe Acrobat also puts a DA entry into one of the widget annotations for the radio button (see the attached document), but DA is defined for FreeText and Redact annotations only. Is DA not being defined for widget annotations a bug in ISO 32000 (standard not following existing practice) or is Adobe Acrobat's usage of DA in the widget annotation incorrect?

Adobe Acrobat actually uses the widget annotation's DA entry when recreating the appearance streams due to NeedAppearances.

rb-da.pdf

@a20dev a20dev added the bug Something isn't correct label Aug 11, 2023
@datalogics-pgallot
Copy link

The first paragraph of clause 12.7.4.3 "Variable Text" actually cites clause 12.5.6.19 "Widget annotations", which seems to be a carryover from PDF v1.4 (in section 8.6.2), if not earlier. Given that "Variable Text" is a subsection of the "Interactive Forms" section, it's probably an oversight that "DA" is not listed as an entry for "Table 191 — Additional entries specific to a widget annotation"

@petervwyatt petervwyatt added this to the Annotation-related milestone Aug 13, 2023
@petervwyatt petervwyatt added the question Further information is requested label Aug 13, 2023
@a20dev
Copy link
Author

a20dev commented Aug 14, 2023

it's probably an oversight that "DA" is not listed as an entry for "Table 191 — Additional entries specific to a widget annotation"

However, 12.7.3 claims "This presents no ambiguity, since the contents of the two kinds of dictionaries do not conflict" about merging field dictionaries and widget annotation dictionaries.

Moreover, inheritance of DA by a widget annotation from its parent field is not defined explicitely.

On the other hand, a lot of software uses DA in (unmerged) widget annotation dictionaries. Is there any software that doesn't?

@a20dev
Copy link
Author

a20dev commented Aug 14, 2023

Adobe Acrobat puts DA into the field dictionary for all types of fields (ie, all types of fields are "fields containing variable text") and puts DA into one of the widget annotation dictionaries (except for signature fields which cannot have multiple widget annotations).

@a20dev
Copy link
Author

a20dev commented Feb 13, 2024

bump

@mkl-public
Copy link

mkl-public commented Feb 18, 2024

Maybe one should allow DA generally for all fields and describe them as "if the PDF processor creates text-based appearances for the field type in question, it may use the DA value as described in the clause on Variable Text.""

@petervwyatt
Copy link
Member

Trying to converge on a solution...

So if DA was moved from "Table 228 — Additional entries common to all fields containing variable text" to "Table 226 — Entries common to all field dictionaries", its description would need to change to "Sometimes required" where the sometimes means if the field contains variable text. Consequently, various discussions about DA will also need an additional cross-reference added to Table 228 as well as 12.7.4.3.
PS. DA is already defined as inheritable and the interactive form dictionary (Table 224) may also contain a "document-wide default value".

I don't think DA needs to be added to "Table 191 — Additional entries specific to a widget annotation" since it can be considered as the (only) merged entry.

@lrosenthol
Copy link
Contributor

I am still unclear as to what the actual problem is with the text.

@petervwyatt
Copy link
Member

I think the problems stated here are:

  1. I couldn't find a definition of "fields containing variable text" in ISO 32000-2:2020

  2. Moreover, inheritance of DA by a widget annotation from its parent field is not defined explicitly.

  3. On the other hand, a lot of software uses DA in (unmerged) widget annotation dictionaries.

so changing it to allow all fields is a simple but imprecise solution for point 1.
I'm not 100% sure about points 2 and 3 since AFAICT having DA into a widget is effectively making a merged dict with a single key coming from fields (i.e. there are no other required keys that are needed from the field dict).

And could you confirm if a popular implementation is using or writing DA beyond what the spec says (i.e. does it write it for all fields)?

@mkl-public
Copy link

mkl-public commented Feb 19, 2024

I am still unclear as to what the actual problem is with the text.

I assume, mostly the first item on Peter's list is irritating, in particular in combination with the requirement character of section 12.7.4.3 "Variable text": As there is no definition of "fields containing variable text", many people assume the other way around that a field with a DA entry must be a "field containing variable text". And in that case section 12.7.4.3 "Variable text" appears to imply that this DA value must be used to create appearances for this field as described in that section.

What about e.g. signature fields with visible widgets? If they have a DA value, does this imply that the signing software has to create a text-only visualization along the lines of section 12.7.4.3 when filling (i.e. signing) the field?

The case of DA values on checkboxes fortunately has become irrelevant in PDF 2.0 as check boxes must bring pre-fabricated appearances along, so there is no need to create appearances for them anymore.


In real life this is handled in a relaxed way. Hardly any signing software is impressed by a DA on signature fields. PDF creators feel free to generate checkbox appearances using vector graphics in spite of a DA. And if the font set in the DA of a text field doesn't provide all the glyphs required for the user's input, sensible PDF processors use replacement fonts.

Nonetheless, the spec should be updated to prevent its requirements and the real use to deviate too far from each other.

@a20dev
Copy link
Author

a20dev commented Feb 19, 2024

Well, in real life, Adobe Acrobat both sets DA for signature fields and uses the value of DA for creating the appearance stream for signature fields.

@a20dev
Copy link
Author

a20dev commented Feb 19, 2024

IIRC, points 2 and 3 affect fields that have multiple widgets.

@mkl-public
Copy link

Well, in real life, Adobe Acrobat both sets DA for signature fields and uses the value of DA for creating the appearance stream for signature fields.

Maybe; so I might better have said "Hardly any signing software is impressed by a DA on signature fields." Software that creates such DA entries probably also uses them. Acrobat definitively doesn't do it as specified in 12.7.4.3 "Variable text", though. The last time I checked they still created the frm with the n0 and n2 layers which contradicts that section. Furthermore, you can configure text-less signature appearances in Acrobat.

@a20dev
Copy link
Author

a20dev commented Feb 19, 2024

Here's an example of Adobe Acrobat using DA when signing a signature field.
sig1-DA3-signed.pdf

@a20dev
Copy link
Author

a20dev commented Feb 19, 2024

I'm not 100% sure about points 2 and 3 since AFAICT having DA into a widget is effectively making a merged dict with a single key coming from fields (i.e. there are no other required keys that are needed from the field dict).

A merged dict is a field dictionary and hence has to have a T entry. But have a look at this file created with Adobe Acrobat. It has a text field Text2 that has a DA entry which specifies a redish color for nonstroking operations. It has two kids which are widget annotations, not merged field/annotation dictionaries as they don't have T entries. The first one looks as expected, the second one has a DA entry (which should not be there or should be ignored according to ISO 32000), which specifies blue for nonstroking operations. And Adobe Acrobat does pay attention to that DA entry.

Now, who's correct? Adobe Acrobat or the wording in ISO 32000?

I made that file with Adobe Acrobat by adding two text fields having the same name but different text colors.

text2-da.pdf

@mkl-public
Copy link

mkl-public commented Feb 19, 2024

Here's an example of Adobe Acrobat using DA when signing a signature field.

Yes, clearly the font from the signature field DA is used. But also clearly the internal structure of the signature appearance only loosely follow the section 12.7.4.3 "Variable text" ideas.

But have a look at this file created with Adobe Acrobat. It has a text field Text2 that has a DA entry which specifies a redish color for nonstroking operations. It has two kids which are widget annotations, not merged field/annotation dictionaries as they don't have T entries.

The file claims a PDF version 1.6. Before ISO 32000-2:2017 the distinction field vs. widget was not really clear cut. Acrobat is known to not act fully ISO 32000-2 compatible.

Also see #28 - T is optional in fields again in ISO 32000-2:2020 + errata. (Which I find a pity. But that's life.)

@petervwyatt
Copy link
Member

@lrosenthol @sgaither - care to comment?

@lrosenthol
Copy link
Contributor

As with all implementors, Adobe has chosen to only implement those parts of ISO 32000-2 that are important to our customer base. At this time, we have not made changes to form fields as we believe the Forms.Next work is a better place to invest our resources in this area.

So any implementation considerations would need to be taken in light of 32000-1.

@mkl-public
Copy link

As with all implementors, Adobe has chosen to only implement those parts of ISO 32000-2 that are important to our customer base. At this time, we have not made changes to form fields as we believe the Forms.Next work is a better place to invest our resources in this area.

Makes sense. And also illustrates why asking in case of conflict whether Acrobat is right or the spec is right, is a too simplified approach.

Just to be sure, though: Does that also imply that Acrobat partially handles PDFs that claim version 2.0 according to ISO 32000-1 in a possibly not ISO 32000-2 conforming way? Or is there some filter that would prevent such non-conformance by disallowing certain operations on 2.0 files?

@a20dev
Copy link
Author

a20dev commented Feb 20, 2024

I think the discrepancy between Adobe Acrobat's behavior and the PDF specification dates back to PDF 1.2.

@a20dev
Copy link
Author

a20dev commented Feb 20, 2024

Also see #28 - T is optional in fields again in ISO 32000-2:2020 + errata.

This still seems to prevent different representations of the same field from having different DA entries, that is, all representations have to have the same representation: A kid without T entry is to be treated as widget annotation and therefore DA is to be ignored. Perhaps I don't understand the resolution of #28.

@a20dev
Copy link
Author

a20dev commented Feb 20, 2024

Does #28 mean that widget annotation dictionaries not merged with a field dictionary no longer exist as all widget annotation dictionaries are now considered to be merged with the corresponding field dictionary?

@mkl-public
Copy link

I think the discrepancy between Adobe Acrobat's behavior and the PDF specification dates back to PDF 1.2.

Well, before PDF 1.7 in ISO form, i.e. before 2008, we only had the PDF References, and as a PDF expert in that year explained to me, the PDF References aren't "normative" in nature. Thus, digging in those old references can be interesting if one is interested in the development of the intentions and ideas people had in the context of PDF, but what one finds there shouldn't be taken too seriously.

Does #28 mean that widget annotation dictionaries not merged with a field dictionary no longer exist as all widget annotation dictionaries are now considered to be merged with the corresponding field dictionary?

Well, in combination with the quote from section 12.7.4.2 at the start of #28 - A field dictionary that does not have a partial field name (T entry) of its own shall not be considered a field but simply a Widget annotation - the rollback is really weird. An IMO good solution would have been to keep T required and to change that quoted sentence.

So now there is the question whether a field dictionary that shall not be considered a field may contain entries specific to field dictionaries. Or whether a PDF processor shall make use of such entries. Like your DA.

@a20dev
Copy link
Author

a20dev commented Feb 20, 2024

Are you implying that there are field dictionaries that are not field dictionaries? (Viz, those without T entry.) It gets weirder and weirder...

@a20dev
Copy link
Author

a20dev commented Feb 20, 2024

I mixed up "field" and "field dictionary":
A field is represented by a field dictionary.
A field dictionary represents either a field, a widget annotation, or both.

What happens if an interior field tree node (ie, one with kids) lacks a T entry? Probably the Kids entry is to be ignored.

@mkl-public
Copy link

I'm afraid we digress a bit. (But indeed, the clarification of the implications of the "A field dictionary that does not have a partial field name..." paragraph could be the topic of an issue by itself here.)

Essentially the change in #28 appears to allow what you observed as "Adobe Acrobat also puts a DA entry into one of the widget annotations".

The remaining problems are

  • What are fields containing variable text?
  • Does a PDF processor when filling a field containing variable text need to do so as described in 12.7.4.3 "Variable text"? In particular in cases that are not classic text fields, this would be very restrictive.

Peter's approach of moving DA from table 228 to table 226 coupled with a clarification that the 12.7.4.3 approach is a (maybe recommended) option but not a requirement would weaken the need for concrete answers.

@petervwyatt
Copy link
Member

@lrosenthol - could you please propose improved wording/clarifications for this and related errata?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't correct question Further information is requested
Projects
None yet
Development

No branches or pull requests

5 participants