Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update ProGuard default shrinking rules #2448

Merged
merged 11 commits into from
Aug 22, 2023
+129 −78
Merged

Update ProGuard default shrinking rules #2448

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
7fa9ef5
Update ProGuard default shrinking rules to correctly deal with classe…
sgjesse Jul 26, 2023
e384d4d
Merge branch 'google:main' into fix-proguard-rules
sgjesse Jul 27, 2023
f60e17d
Update test after changing default shrinking rules
sgjesse Jul 27, 2023
eea08d0
Adjust shrinker tests
Marcono1234 Aug 5, 2023
4c40a91
Merge pull request #1 from Marcono1234/marcono1234/fix-proguard-rules…
sgjesse Aug 7, 2023
c34126b
Update rules
sgjesse Aug 10, 2023
af98d32
Merge branch 'main' into fix-proguard-rules
sgjesse Aug 10, 2023
ac6e590
Addressed review comments
sgjesse Aug 15, 2023
8d14f1a
Addressed more review comments
sgjesse Aug 21, 2023
743d529
Addressed more review comments
sgjesse Aug 21, 2023
3468700
Merge branch 'main' into fix-proguard-rules
sgjesse Aug 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 13 additions & 15 deletions gson/src/main/resources/META-INF/proguard/gson.pro
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,25 +15,20 @@
# Note: Cannot perform finer selection here to only cover Gson annotations, see also https://stackoverflow.com/q/47515093
-keepattributes RuntimeVisibleAnnotations,AnnotationDefault


### The following rules are needed for R8 in "full mode" which only adheres to `-keepattribtues` if
### the corresponding class or field is matches by a `-keep` rule as well, see
### https://r8.googlesource.com/r8/+/refs/heads/main/compatibility-faq.md#r8-full-mode

# Keep class TypeToken (respectively its generic signature)
-keep class com.google.gson.reflect.TypeToken { *; }
# Keep class TypeToken (respectively its generic signature) if present
-if class com.google.gson.reflect.TypeToken
-keep,allowobfuscation class com.google.gson.reflect.TypeToken

# Keep any (anonymous) classes extending TypeToken
-keep,allowobfuscation class * extends com.google.gson.reflect.TypeToken

# Keep classes with @JsonAdapter annotation
-keep,allowobfuscation,allowoptimization @com.google.gson.annotations.JsonAdapter class *

# Keep fields with @SerializedName annotation, but allow obfuscation of their names
-keepclassmembers,allowobfuscation class * {
@com.google.gson.annotations.SerializedName <fields>;
}
Comment on lines -32 to -35
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity, why did you remove this here and add it below guarded by a -if class *? Does that make a difference for the end result?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, in R8 full mode we have a distinction between referenced and instantiated classes. For classes which are not seen by R8 as instantiated (e.g. if only referenced statically using .class as one will do in fromJson then -keepclassmembers will only keep static members and not instance members. For ProGuard and R8 in compatability mode the two will do the same.

Copy link
Collaborator

@Marcono1234 Marcono1234 Aug 15, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm thanks, ok that sounds like the previous rules would not have worked properly then because -keepclassmembers would not have kepts the non-static @SerializedName fields. Though I think the shrinker-test showed that it did work as expected?
I am bit confused, or am I misunderstanding your comment or are the shrinker-tests flawed and not close enough to real code?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One way to see the difference is to have an instance field annotated with @SerializedName which is not used in the code at all, then the field will be removed with the previous rule in R8 full mode (which is default from AGP 8.0.0). Then in a pass through from fromJson to toJson the field value will be lost. Of course it is a matter of taste if unused fields should be kept this way.


# Keep fields with any other Gson annotation
# Also allow obfuscation, assuming that users will additionally use @SerializedName or
# other means to preserve the field names
Expand All @@ -59,13 +54,16 @@
<init>();
}

# If a class is used in some way by the application and has fields annotated with @SerializedName,
# keep those fields and the constructors of the class
# For convenience this also matches classes without no-args constructor, in which case Gson uses JDK Unsafe
# Based on https://issuetracker.google.com/issues/150189783#comment11
# See also https://github.com/google/gson/pull/2420#discussion_r1241813541 for a more detailed explanation
# Keep fields annotated with @SerializedName for classes which are present.
# If classes with fields annotated with @SerializedName have a no-args
# constructor keep that as well.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you remove the link to the pull request discussion?

At least in my opinion this -if class * ... is pretty cryptic and its intended behavior is not directly obvious, so it would be good to have a reference explaining what this actually does.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, added those references back.

-if class *
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this if class *, -classeswithmembers... class <1> doesn't do anything relative to -classeswithmembers... class * , after reading some of the follow-up threads on that change.

Copy link
Collaborator

@Marcono1234 Marcono1234 Jul 29, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At least for R8 it does seem to make a difference: -keepclasseswithmembers ... class * always keeps the class, even if it isn't used. Whereas -if class * ... -keepclasseswithmembers ... class <1> only keeps the class if it is actually used.

Unfortunately this is not covered by the tests yet; have created #2455 to add a test for this.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks also for the other points you raised, but unless sgjesse wants to address them here as well could you please create a separate GitHub issue or pull request? Otherwise the discussion on this pull request here might drift too far away from the original topic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed the rules to be two -if rules

  1. Keep all @SerializedName fields for classes which are present after shrinking
  2. If a class with a @SerializedName field is present keep its no-args constructor

-keepclasseswithmembers,allowobfuscation,allowoptimization class <1> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you remove allowoptimization?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having allowoptimization can allow R8 to propagate field values, so if you have an explicit construction of a serialized class always setting a field to a specific value, then that single value can become the default value for the created instances instead of 0/null.

<init>(...);
Copy link
Collaborator

@Marcono1234 Marcono1234 Aug 13, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it intentional that you removed <init>(...) again and used <init>() below? My personal opinion in #2448 (review) was not blocking the merge of this pull request (sorry if I did not make that clear enough).

This change to use <init>() again might render some or all of the shrinker test changes of this pull request obsolete.
Edit: Maybe not, see comment below.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or at least I am a bit surprised that no exception occurs for ClassWithoutDefaultConstructor; I assume it might be related to your newly added -keepclasseswithmembers.

So, do I understand it correctly that your new approach of using -keepclasseswithmembers achieves the same thing as your initial <init>(...) approach (i.e. Gson can instantiate classes without no-args constructor using Unsafe; instead of failing because R8 made the class abstract). But this new approach is better than the <init>(...) rule because that would have unnecessarily kept the constructors with args around, even though they are not actually used by Gson?

-keepclasseswithmembers,allowobfuscation class <1> {
@com.google.gson.annotations.SerializedName <fields>;
}
-if class * {
@com.google.gson.annotations.SerializedName <fields>;
}
-keepclassmembers,allowobfuscation class <1> {
<init>();
}
3 changes: 3 additions & 0 deletions shrinker-test/proguard.pro
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,6 @@
-keepclassmembernames class com.example.NoSerializedNameMain$TestClassWithoutDefaultConstructor {
<fields>;
}
#-keep class com.example.ClassWithSerializedName {
# <init>(...);
#}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, no removed the rules in comments. Also removed the rule above, as that was also not needed. I probably added it while testing different rules.