-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Compile-time source generation for System.Text.Json #45448
Comments
|
partial class JsonSerializerContext
{
public static JsonTypeInfo<T> CreateTypeInfoViaReflection<T>(JsonSerializerOptions options);
public static JsonTypeInfo CreateTypeInfoViaReflection(Type type, JsonSerializerOptions options);
}
partial class JsonSerializerContext
{
// This can use lazy initialization, or other techniques, to make AddContext work without creating an options.
public JsonSerializerOptions Options { get; }
protected JsonSerializerContext(JsonSerializerOptions? options = null) { }
}
|
Can we add better justification for this than just "ASP.NET asked for this"? What are the examples where this overload is expected to be used, does it really need to be generic virtual method? Generic virtual methods are well known source of bad combinatorics expansion in AOT scenarios. We should be using them only when they are really needed. |
Just for my education, when are they typically really needed vs nice to have? |
When there are no alternatives or when the alternatives are much worse (e.g. require everybody write a ton of boiler plate code for common scenario). In this particular case, I would think that the non-generic |
@davidfowl okay to leave this method out for now till I get some usage samples/scenarios from you? Target would be preview 5/6. |
Adding a proposal to update the APIs to configure and use JSON source generation. This covers functionality added since the last review. Most of this functionality is prototyped/checked into .NET 6 and would only need minor clean-up, depending on how the review goes. cc @steveharter @eiriktsarpalis @eerhardt @stephentoub @ericstj APIs for source-generated code to usenamespace System.Text.Json.Serialization.Metadata
{
// Provides serialization info about a type
public partial class JsonTypeInfo
{
internal JsonTypeInfo() { }
}
// Provides serialization info about a type
public abstract partial class JsonTypeInfo<T> : JsonTypeInfo
{
internal JsonTypeInfo() { }
public Action<Utf8JsonWriter, T>? Serialize { get { throw null; } }
}
// Provide serialization info about a property or field of a POCO
public abstract partial class JsonPropertyInfo
{
internal JsonPropertyInfo() { }
}
+ The list of parameters in `CreatePropertyInfo<T>` below is too long, so we move the data to a struct. This should help with forward-compat since we can add members representing new serializer features.
+ public readonly struct JsonPropertyInfoValues<T>
+ {
+ public bool IsProperty { get; init; }
+ public bool IsPublic { get; init; }
+ public bool IsVirtual { get; init; }
+ public Type DeclaringType { get; init; }
+ public JsonTypeInfo PropertyTypeInfo { get; init; }
+ public JsonConverter<T>? Converter { get; init; }
+ public Func<object, T>? Getter { get; init; }
+ public Action<object, T>? Setter { get; init; }
+ public JsonIgnoreCondition? IgnoreCondition { get; init; }
+ public bool IsExtensionDataProperty { get; init; }
+ public JsonNumberHandling? NumberHandling { get; init; }
+ // The property's statically-declared name.
+ public string PropertyName { get; init; }
+ // The name to use when processing the property, specified by [JsonPropertyName(string)].
+ public string? JsonPropertyName { get; init; }
+ }
+ // For the same reason as `JsonPropertyInfoValues<T>` above, move object info to a struct.
+ // Configures information about an object with members that is deserialized using a parameterless ctor.
+ public readonly struct JsonObjectInfoValues<T>
+ {
+ // A method to create an instance of the type, using a parameterless ctor
+ public Func<T>? CreateObjectFunc { get; init; }
+ // A method to create an instance of the type, using a parameterized ctor
+ public Func<object[], T>? CreateObjectFunc { get; init; }
+ // Provides information about the type's properties and fields.
+ public Func<JsonSerializerContext, JsonPropertyInfo[]>? PropInitFunc { get; init; }
+ // Provides information about the type's ctor params.
+ public Func<JsonParameterInfo[]>? CtorParamInitFunc { get; init; }
+ // The number-handling setting for the type's properties and fields.
+ public JsonNumberHandling NumberHandling { get; init; }
+ // An optimized method to serialize instances of the type, given pre-defined serialization settings.
+ public Action<Utf8JsonWriter, T>? SerializeFunc { get; init; }
+ }
+ // Configures information about a collection
+ public readonly struct JsonCollectionInfo<T>
+ {
+ // A method to create an instance of the collection
+ public Func<T>? CreateObjectFunc { get; init; }
+ // Serialization metadata about the collection key type, if a dictionary.
+ public JsonTypeInfo? KeyInfo { get; init; }
+ // Serialization metadata about the collection element/value type.
+ public JsonTypeInfo ElementInfo { get; init; }
+ // The number-handling setting for the collection's elements.
+ public JsonNumberHandling NumberHandling { get; init; }
+ // An optimized method to serialize instances of the collection, given pre-defined serialization settings.
+ public Action<Utf8JsonWriter, T>? SerializeFunc { get; init; }
+ }
+ // Provides serialization info about a constructor parameter
+ public readonly struct JsonParameterInfo
+ {
+ public object? DefaultValue { readonly get { throw null; } init { } }
+ public bool HasDefaultValue { readonly get { throw null; } init { } }
+ public string Name { readonly get { throw null; } init { } }
+ public System.Type ParameterType { readonly get { throw null; } init { } }
+ public int Position { readonly get { throw null; } init { } }
+ }
// Object type and property info creators
public static partial class JsonMetadataServices
{
// Creator for an object type
- public static JsonTypeInfo<T> CreateObjectInfo<T>(JsonSerializerOptions options, Func<T>? createObjectFunc, Func<JsonSerializerContext, JsonPropertyInfo[]>? propInitFunc, JsonNumberHandling numberHandling, Action<Utf8JsonWriter, T>? serializeFunc) where T : notnull { throw null; }
+ public static JsonTypeInfo<T> CreateObjectInfo<T>(JsonSerializerOptions options, JsonObjectInfoValues<T> objectInfo) where T : notnull { throw null; }
// Creator for an object property
- public static JsonPropertyInfo CreatePropertyInfo<T>(JsonSerializerOptions options, bool isProperty, bool isPublic, bool isVirtual, Type declaringType, JsonTypeInfo propertyTypeInfo, JsonConverter<T>? converter, Func<object, T>? getter, Action<object, T>? setter, JsonIgnoreCondition? ignoreCondition, bool hasJsonInclude, JsonNumberHandling? numberHandling, string propertyName, string? jsonPropertyName) { throw null; }
+ public static JsonPropertyInfo CreatePropertyInfo<T>(JsonSerializerOptions options, JsonPropertyInfoValues<T> propertyInfo) { throw null; }
}
// Collection type info creators
public static partial class JsonMetadataServices
{
- public static JsonTypeInfo<TElement[]> CreateArrayInfo<TElement>(JsonSerializerOptions options, JsonTypeInfo elementInfo, JsonNumberHandling numberHandling, Action<Utf8JsonWriter, TElement[]>? serializeFunc) { throw null; }
+ public static JsonTypeInfo<TElement[]> CreateArrayInfo<TElement>(JsonSerializerOptions options, JsonCollectionInfo<TElement[]> info) { throw null; }
- public static JsonTypeInfo<TCollection> CreateDictionaryInfo<TCollection, TKey, TValue>(JsonSerializerOptions options, Func<TCollection> createObjectFunc, JsonTypeInfo keyInfo, JsonTypeInfo valueInfo, JsonNumberHandling numberHandling, Action<Utf8JsonWriter, TCollection>? serializeFunc) where TCollection : Dictionary<TKey, TValue> where TKey : notnull { throw null; }
+ public static JsonTypeInfo<TCollection> CreateDictionaryInfo<TCollection, TKey, TValue>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : Dictionary<TKey, TValue> where TKey : notnull { throw null; }
- public static JsonTypeInfo<TCollection> CreateListInfo<TCollection, TElement>(JsonSerializerOptions options, Func<TCollection>? createObjectFunc, JsonTypeInfo elementInfo, JsonNumberHandling numberHandling, Action<Utf8JsonWriter, TCollection>? serializeFunc) where TCollection : List<TElement> { throw null; }
+ public static JsonTypeInfo<TCollection> CreateListInfo<TCollection, TElement>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : List<TElement> { throw null; }
+ public static JsonTypeInfo<TCollection> CreateConcurrentQueueInfo<TCollection, TElement>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : ConcurrentQueue<TElement> { throw null; }
+ public static JsonTypeInfo<TCollection> CreateConcurrentStackInfo<TCollection, TElement>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : ConcurrentStack<TElement> { throw null; }
+ public static JsonTypeInfo<TCollection> CreateICollectionInfo<TCollection, TElement>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : ICollection<TElement> { throw null; }
+ public static JsonTypeInfo<TCollection> CreateIDictionaryInfo<TCollection>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : IDictionary { throw null; }
+ public static JsonTypeInfo<TCollection> CreateIDictionaryInfo<TCollection, TKey, TValue>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : IDictionary<TKey, TValue> where TKey : notnull { throw null; }
+ public static JsonTypeInfo<TCollection> CreateIEnumerableInfo<TCollection>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : IEnumerable { throw null; }
+ public static JsonTypeInfo<TCollection> CreateIEnumerableInfo<TCollection, TElement>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : IEnumerable<TElement> { throw null; }
+ public static JsonTypeInfo<TCollection> CreateIListInfo<TCollection>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : IList { throw null; }
+ public static JsonTypeInfo<TCollection> CreateIListInfo<TCollection, TElement>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : IList<TElement> { throw null; }
+ public static JsonTypeInfo<TCollection> CreateImmutableDictionaryInfo<TCollection, TKey, TValue>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info, Func<IEnumerable<KeyValuePair<TKey, TValue>>, TCollection> createRangeFunc) where TCollection : IReadOnlyDictionary<TKey, TValue> where TKey : notnull { throw null; }
+ public static JsonTypeInfo<TCollection> CreateImmutableEnumerableInfo<TCollection, TElement>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info, Func<IEnumerable<TElement>, TCollection> createRangeFunc) where TCollection : IEnumerable<TElement> { throw null; }
+ public static JsonTypeInfo<TCollection> CreateIReadOnlyDictionaryInfo<TCollection, TKey, TValue>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : IReadOnlyDictionary<TKey, TValue> where TKey : notnull { throw null; }
+ public static JsonTypeInfo<TCollection> CreateISetInfo<TCollection, TElement>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : ISet<TElement> { throw null; }
+ public static JsonTypeInfo<TCollection> CreateQueueInfo<TCollection, TElement>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : Queue<TElement> { throw null; }
+ public static JsonTypeInfo<TCollection> CreateStackInfo<TCollection, TElement>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : Stack<TElement> { throw null; }
+ public static JsonTypeInfo<TCollection> CreateStackOrQueueInfo<TCollection>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info, Action<TCollection, object?> addFunc) where TCollection : IEnumerable { throw null; }
}
} APIs to configure source generatornamespace System.Text.Json.Serialization
{
public enum JsonKnownNamingPolicy
{
Unspecified = 0,
CamelCase = 1,
}
public abstract partial class JsonSerializerContext
{
- protected JsonSerializerContext(JsonSerializerOptions? instanceOptions, JsonSerializerOptions? defaultOptions) { }
+ protected JsonSerializerContext(JsonSerializerOptions? options) { }
public JsonSerializerOptions Options { get { throw null; } }
public abstract JsonTypeInfo? GetTypeInfo(Type type);
+ // The set of options that are compatible with generated serialization and (future) deserialization logic for types in the context.
+ protected abstract JsonSerializerOptions? DesignTimeOptions { get; }
}
[AttributeUsage(AttributeTargets.Class, AllowMultiple=false)]
public partial sealed class JsonSourceGenerationOptionsAttribute : JsonAttribute
{
public JsonSourceGenerationOptionsAttribute() { }
public JsonIgnoreCondition DefaultIgnoreCondition { get { throw null; } set { } }
public bool IgnoreReadOnlyFields { get { throw null; } set { } }
public bool IgnoreReadOnlyProperties { get { throw null; } set { } }
- // Whether the generated source code should ignore converters added at runtime.
- public bool IgnoreRuntimeCustomConverters { get { throw null; } set { } }
public bool IncludeFields { get { throw null; } set { } }
public JsonKnownNamingPolicy PropertyNamingPolicy { get { throw null; } set { } }
public bool WriteIndented { get { throw null; } set { } }
public JsonSourceGenerationMode GenerationMode { get { throw null; } set { } }
}
[Flags]
public enum JsonSourceGenerationMode
{
Default = 0,
Metadata = 1,
Serialization = 2,
// Future
// Deserialization = 4
}
} |
namespace System.Text.Json.Serialization.Metadata
{
// Provides serialization info about a type
public partial class JsonTypeInfo
{
internal JsonTypeInfo();
}
// Provides serialization info about a type
public abstract partial class JsonTypeInfo<T> : JsonTypeInfo
{
internal JsonTypeInfo();
public Action<Utf8JsonWriter, T>? Serialize { get; }
}
// Provide serialization info about a property or field of a POCO
public abstract partial class JsonPropertyInfo
{
internal JsonPropertyInfo();
}
+ // The list of parameters in `CreatePropertyInfo<T>` below is too long, so we move the data to a struct. This should help with forward-compat since we can add members representing new serializer features.
+ public readonly struct JsonPropertyInfoValues<T>
+ {
+ public bool IsProperty { get; init; }
+ public bool IsPublic { get; init; }
+ public bool IsVirtual { get; init; }
+ public Type DeclaringType { get; init; }
+ public JsonTypeInfo PropertyTypeInfo { get; init; }
+ public JsonConverter<T>? Converter { get; init; }
+ public Func<object, T>? Getter { get; init; }
+ public Action<object, T>? Setter { get; init; }
+ public JsonIgnoreCondition? IgnoreCondition { get; init; }
+ public bool IsExtensionDataProperty { get; init; }
+ public JsonNumberHandling? NumberHandling { get; init; }
+ // The property's statically-declared name.
+ public string PropertyName { get; init; }
+ // The name to use when processing the property, specified by [JsonPropertyName(string)].
+ public string? JsonPropertyName { get; init; }
+ }
+ // For the same reason as `JsonPropertyInfoValues<T>` above, move object info to a struct.
+ // Configures information about an object with members that is deserialized using a parameterless ctor.
+ public readonly struct JsonObjectInfoValues<T>
+ {
+ // A method to create an instance of the type, using a parameterless ctor
+ public Func<T>? CreateObjectFunc { get; init; }
+ // A method to create an instance of the type, using a parameterized ctor
+ public Func<object[], T>? CreateObjectFunc { get; init; }
+ // Provides information about the type's properties and fields.
+ public Func<JsonSerializerContext, JsonPropertyInfo[]>? PropInitFunc { get; init; }
+ // Provides information about the type's ctor params.
+ public Func<JsonParameterInfo[]>? CtorParamInitFunc { get; init; }
+ // The number-handling setting for the type's properties and fields.
+ public JsonNumberHandling NumberHandling { get; init; }
+ // An optimized method to serialize instances of the type, given pre-defined serialization settings.
+ public Action<Utf8JsonWriter, T>? SerializeFunc { get; init; }
+ }
+ // Configures information about a collection
+ public readonly struct JsonCollectionInfo<T>
+ {
+ // A method to create an instance of the collection
+ public Func<T>? CreateObjectFunc { get; init; }
+ // Serialization metadata about the collection key type, if a dictionary.
+ public JsonTypeInfo? KeyInfo { get; init; }
+ // Serialization metadata about the collection element/value type.
+ public JsonTypeInfo ElementInfo { get; init; }
+ // The number-handling setting for the collection's elements.
+ public JsonNumberHandling NumberHandling { get; init; }
+ // An optimized method to serialize instances of the collection, given pre-defined serialization settings.
+ public Action<Utf8JsonWriter, T>? SerializeFunc { get; init; }
+ }
+ // Provides serialization info about a constructor parameter
+ public readonly struct JsonParameterInfo
+ {
+ public object? DefaultValue { readonly get; init; }
+ public bool HasDefaultValue { readonly get; init; }
+ public string Name { readonly get; init; }
+ public Type ParameterType { readonly get; init; }
+ public int Position { readonly get; init; }
+ }
// Object type and property info creators
public static partial class JsonMetadataServices
{
// Creator for an object type
- public static JsonTypeInfo<T> CreateObjectInfo<T>(JsonSerializerOptions options, Func<T>? createObjectFunc, Func<JsonSerializerContext, JsonPropertyInfo[]>? propInitFunc, JsonNumberHandling numberHandling, Action<Utf8JsonWriter, T>? serializeFunc) where T : notnull;
+ public static JsonTypeInfo<T> CreateObjectInfo<T>(JsonSerializerOptions options, JsonObjectInfoValues<T> objectInfo) where T : notnull;
// Creator for an object property
- public static JsonPropertyInfo CreatePropertyInfo<T>(JsonSerializerOptions options, bool isProperty, bool isPublic, bool isVirtual, Type declaringType, JsonTypeInfo propertyTypeInfo, JsonConverter<T>? converter, Func<object, T>? getter, Action<object, T>? setter, JsonIgnoreCondition? ignoreCondition, bool hasJsonInclude, JsonNumberHandling? numberHandling, string propertyName, string? jsonPropertyName);
+ public static JsonPropertyInfo CreatePropertyInfo<T>(JsonSerializerOptions options, JsonPropertyInfoValues<T> propertyInfo);
}
// Collection type info creators
public static partial class JsonMetadataServices
{
- public static JsonTypeInfo<TElement[]> CreateArrayInfo<TElement>(JsonSerializerOptions options, JsonTypeInfo elementInfo, JsonNumberHandling numberHandling, Action<Utf8JsonWriter, TElement[]>? serializeFunc);
+ public static JsonTypeInfo<TElement[]> CreateArrayInfo<TElement>(JsonSerializerOptions options, JsonCollectionInfo<TElement[]> info);
- public static JsonTypeInfo<TCollection> CreateDictionaryInfo<TCollection, TKey, TValue>(JsonSerializerOptions options, Func<TCollection> createObjectFunc, JsonTypeInfo keyInfo, JsonTypeInfo valueInfo, JsonNumberHandling numberHandling, Action<Utf8JsonWriter, TCollection>? serializeFunc) where TCollection : Dictionary<TKey, TValue> where TKey : notnull;
+ public static JsonTypeInfo<TCollection> CreateDictionaryInfo<TCollection, TKey, TValue>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : Dictionary<TKey, TValue> where TKey : notnull;
- public static JsonTypeInfo<TCollection> CreateListInfo<TCollection, TElement>(JsonSerializerOptions options, Func<TCollection>? createObjectFunc, JsonTypeInfo elementInfo, JsonNumberHandling numberHandling, Action<Utf8JsonWriter, TCollection>? serializeFunc) where TCollection : List<TElement>;
+ public static JsonTypeInfo<TCollection> CreateListInfo<TCollection, TElement>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : List<TElement>;
+ public static JsonTypeInfo<TCollection> CreateConcurrentQueueInfo<TCollection, TElement>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : ConcurrentQueue<TElement>;
+ public static JsonTypeInfo<TCollection> CreateConcurrentStackInfo<TCollection, TElement>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : ConcurrentStack<TElement>;
+ public static JsonTypeInfo<TCollection> CreateICollectionInfo<TCollection, TElement>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : ICollection<TElement>;
+ public static JsonTypeInfo<TCollection> CreateIDictionaryInfo<TCollection>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : IDictionary;
+ public static JsonTypeInfo<TCollection> CreateIDictionaryInfo<TCollection, TKey, TValue>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : IDictionary<TKey, TValue> where TKey : notnull;
+ public static JsonTypeInfo<TCollection> CreateIEnumerableInfo<TCollection>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : IEnumerable;
+ public static JsonTypeInfo<TCollection> CreateIEnumerableInfo<TCollection, TElement>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : IEnumerable<TElement>;
+ public static JsonTypeInfo<TCollection> CreateIListInfo<TCollection>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : IList;
+ public static JsonTypeInfo<TCollection> CreateIListInfo<TCollection, TElement>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : IList<TElement>;
+ public static JsonTypeInfo<TCollection> CreateImmutableDictionaryInfo<TCollection, TKey, TValue>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info, Func<IEnumerable<KeyValuePair<TKey, TValue>>, TCollection> createRangeFunc) where TCollection : IReadOnlyDictionary<TKey, TValue> where TKey : notnull;
+ public static JsonTypeInfo<TCollection> CreateImmutableEnumerableInfo<TCollection, TElement>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info, Func<IEnumerable<TElement>, TCollection> createRangeFunc) where TCollection : IEnumerable<TElement>;
+ public static JsonTypeInfo<TCollection> CreateIReadOnlyDictionaryInfo<TCollection, TKey, TValue>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : IReadOnlyDictionary<TKey, TValue> where TKey : notnull;
+ public static JsonTypeInfo<TCollection> CreateISetInfo<TCollection, TElement>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : ISet<TElement>;
+ public static JsonTypeInfo<TCollection> CreateQueueInfo<TCollection, TElement>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : Queue<TElement>;
+ public static JsonTypeInfo<TCollection> CreateStackInfo<TCollection, TElement>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info) where TCollection : Stack<TElement>;
+ public static JsonTypeInfo<TCollection> CreateStackOrQueueInfo<TCollection>(JsonSerializerOptions options, JsonCollectionInfo<TCollection> info, Action<TCollection, object?> addFunc) where TCollection : IEnumerable;
}
}
namespace System.Text.Json.Serialization
{
public enum JsonKnownNamingPolicy
{
Unspecified = 0,
CamelCase = 1,
}
public abstract partial class JsonSerializerContext
{
- protected JsonSerializerContext(JsonSerializerOptions? instanceOptions, JsonSerializerOptions? defaultOptions);
+ protected JsonSerializerContext(JsonSerializerOptions? options);
public JsonSerializerOptions Options { get; }
public abstract JsonTypeInfo? GetTypeInfo(Type type);
+ // The set of options that are compatible with generated serialization and (future) deserialization logic for types in the context.
+ protected abstract JsonSerializerOptions? DesignTimeOptions { get; }
}
[AttributeUsage(AttributeTargets.Class, AllowMultiple=false)]
public partial sealed class JsonSourceGenerationOptionsAttribute : JsonAttribute
{
public JsonSourceGenerationOptionsAttribute();
public JsonIgnoreCondition DefaultIgnoreCondition { get; set; }
public bool IgnoreReadOnlyFields { get; set; }
public bool IgnoreReadOnlyProperties { get; set; }
- // Whether the generated source code should ignore converters added at runtime.
- public bool IgnoreRuntimeCustomConverters { get; set; }
public bool IncludeFields { get; set; }
public JsonKnownNamingPolicy PropertyNamingPolicy { get; set; }
public bool WriteIndented { get; set; }
public JsonSourceGenerationMode GenerationMode { get; set; }
}
[Flags]
public enum JsonSourceGenerationMode
{
Default = 0,
Metadata = 1,
Serialization = 2,
// Future
// Deserialization = 4
}
} |
We've implemented the required functionality of this feature for 6.0, barring bug fixes and goodness. There'll likely be one more review of the APIs before we ship but I'll move this issue to 7.0 to make 6.0 tracking cleaner. |
We need a final pass at the source generator APIs: namespace System.Text.Json.Serialization.Metadata
{
// Provides serialization info about a type
public abstract partial class JsonTypeInfo<T> : JsonTypeInfo
{
internal JsonTypeInfo();
- public Action<Utf8JsonWriter, T>? Serialize { get; }
+ // Rename to SerializeHandler
+ public Action<Utf8JsonWriter, T>? SerializeHandler { get; }
}
public static partial class JsonMetadataServices
{
+ // Converter that handles JsonArray
+ public static System.Text.Json.Serialization.JsonConverter<JsonArray> JsonArrayConverter { get { throw null; } }
+ // Converter that handles JsonNode
+ public static System.Text.Json.Serialization.JsonConverter<JsonNode> JsonNodeConverter { get { throw null; } }
+ // Converter that handles JsonObject
+ public static System.Text.Json.Serialization.JsonConverter<JsonObject> JsonObjectConverter { get { throw null; } }
+ // Converter that handles JsonValue
+ public static System.Text.Json.Serialization.JsonConverter<JsonValue> JsonValueConverter { get { throw null; } }
+ // Converter that handles unsupported types.
+ public static System.Text.Json.Serialization.JsonConverter<T> GetUnsupportedTypeConverter<T>() { throw null; }
}
} |
@am11 do you have scenarios where you have a typed value i.e. some |
It just felt that one of the using System;
using System.Text.Json;
using System.Text.Json.Serialization;
// instead of:
string json = JsonSerializer.Serialize(new B(), typeof(B), new MySerializerContext());
// i was expecting this to work:
// string json = JsonSerializer.Serialize(new B(), new MySerializerContext());
//
// but it doesn't, since there is no generic overload accepting TValue and context arg, i.e.:
// Serialize<TValue>(TValue, JsonSerializerContext)
Console.WriteLine(json);
[JsonSerializable(typeof(B))]
public partial class MySerializerContext : JsonSerializerContext { }
public class B
{
public int Foo { get; } = 42;
} |
Yes we can consider this by design until a scenario necessitates new overloads. |
namespace System.Text.Json.Serialization.Metadata
{
// Provides serialization info about a type
public abstract partial class JsonTypeInfo<T> : JsonTypeInfo
{
internal JsonTypeInfo();
- public Action<Utf8JsonWriter, T>? Serialize { get; }
+ // Rename to SerializeHandler
+ public Action<Utf8JsonWriter, T>? SerializeHandler { get; }
}
public static partial class JsonMetadataServices
{
+ // Converter that handles JsonArray
+ public static JsonConverter<JsonArray> JsonArrayConverter { get; }
+ // Converter that handles JsonNode
+ public static JsonConverter<JsonNode> JsonNodeConverter { get; }
+ // Converter that handles JsonObject
+ public static JsonConverter<JsonObject> JsonObjectConverter { get; }
+ // Converter that handles JsonValue
+ public static JsonConverter<JsonValue> JsonValueConverter { get; }
+ // Converter that handles unsupported types.
+ public static JsonConverter<T> GetUnsupportedTypeConverter<T>();
}
} |
@layomia should we close this and open a fresh issue for 7.0.0? |
i do not see this being used for web api output. it will run the code for incomming objects, but outgoing objects show the context as null. |
@AnQueth I'm not sure I understand what you're saying. I would recommend creating a new issue with relevant details (and reproduction) so that we can take a closer look. |
Closing, since this issue defines work scoped to .NET 6. |
TODO (.NET 6):
Backlog (.NET 7/future) (click to view)
object
valuesJsonSerializerOptions
instance is compatible with fast-path logic[JsonSerializable]
instances on the same context: Use System.Text.Json's source generated code to deserialize WebEvents aspnetcore#32374 (comment).The “JSON Serializer recommendations for 5.0” document goes over benefits of generating additional source specific to serializable .NET types at compile time. These include faster startup performance, reduced memory usage, improved runtime throughput, and smaller size-on-disk of applications that use
System.Text.Json
. This document will discuss design considerations and technical details for a C# source generator that helps provide these benefits.Scenarios
System.Text.Json
to reduce the size of their appsGoals
System.Text.Json
(post ILLinker trimming)API Usage
Developers using
System.Text.Json
directlyNew pattern (includes size benefits where we trim out unused converters/reflection code)
"Code was generated for all serializable types and I know what type I'm processing"
"Code was generated for all serializable types but I don't know what type I'm processing"
Existing pattern (does not provide size benefits)
"Code was generated for all serializable types and I know what type I'm processing"
"Code was generated for all serializable types but I don't know what type I'm processing"
Developers using
System.Text.Json
indirectlyASP.NET Core
Blazor
The pre-generated metadata can be forwarded to the serializer directly via a new overload on the
System.Http.Net.Json.HttpClient.GetFromJsonAsync
method.MVC, WebAPIs, SignalR, Houdini
These services can retrieve user-generated context with already-existing APIs to configure serialization options. These do not come with the size benefits.
In the future, these services can expose APIs to directly take
JsonSerializerContext
instances. It is difficult today since the underlying calls to the different serializer overloads are based on runtime checks. The code is not separated from the root configuration of options, so the ILLinker cannot understand that it can trim out existing linker-unfriendly overloads of the serializer. A couple strategies to evntually achieve linker friendliness are:New APIs can then be added to accept context instances.
Other libraries performing serialization on behalf of others
Similar to the ASP.NET Core scenarios above, developers can provide new APIs that accept options or context instances to forward to the serializer on behalf of the user. If neither of this is viable, then the library author has a couple of options
Utf8JsonReader
andUtf8JsonWriter
.What about a global static to cache unbound
JsonSerializerContext
s?A global mechansim to cache
JsonSerializeContext
s may help with the viral nature of this design by allowing developers doing serialization on behalf of others to retreive the metadata from a static rather than needing new APIs to accept it. This mechanism is likely to be unreliable because it is difficult to establish a precedent for retrieving metadata in the event of collisions. For example, if mutliple contexts contain metadata for the same type, whose should win? First-one-wins? Last one? This question does matter because metadata for the same type may differ across assemblies (e.g. non-public members, specificallyprivate
). These are the same issues that make it difficult to expose the default options and make them mutable.Nonetheless, the proposed design is forward compatible with solving for this scenario if it is deemed important in the future.
API Proposal
New Metadata APIs for source generation
New
JsonSerializer
method overloadsNew
System.Net.Http.Json
method overloadsWhat's next?
JsonSerializerOptions
usagesDesign doc (Click to view)
Overview
A source generator is a .NET Standard 2.0 assembly that is loaded by the compiler. Source generators allow developers to generate C# source files that can be added to an assembly during the course of compilation. The
System.Text.Json
source generator (System.Text.Json.SourceGeneration.dll
) generates serialization metadata for JSON-serializable types in a project. The metadata generated for a type contains structured information in a format that can be optimally utilized by the serializer to serialize and deserialize instances of that type to and from JSON representations. Serializable types are indicated to the source generator via a new[JsonSerializable]
attribute. The generator then generates metadata for each type in the object graphs of each indicated type.In previous versions of
System.Text.Json
, serialization metadata was computed at runtime, during the first serialization or deserialization routine of every type in any object graph passed to the serializer. At a high level, this metadata includes delegates to constructors, properter setters and getters, along with user options indicated at both runtime and design (e.g. whether to ignore a property value when serializing and it isnull
). After this metadata is generated, the serializer performs the actual serialization and deserialization. The generation phase is based on reflection, and is computationally expensive both in terms of throughput and allocations. We can refer to this phase as the serializer's "warm-up" phase. With this design, we are concerned not only with the cost of the intial work done within the serializer, but with the cost of all the work when an application is first started because it usesSystem.Text.Json
. We can refer to this work collectively as the start time of the application.The fundamental approach of the design this document goes over is to shift this runtime metadata generation phase to compile-time, substantially reducing the cost of the first serialization or deserialization procedures. This metadata is generated to the compiling assembly, where it can be initialized and passed directly to
JsonSerializer
so that it doesn't have to generate it at runtime. This helps reduce the costs of the first serialization or deserialization of each type.The serializer supports a wide range of scenarios and has multiple layers of abstraction and indirection to navigate through when serializing and deserializing. In order to improve throughput for a specific scenario, one may consider generating specific and optimized serialization and deserialization logic. For now, there is no change to the actual serialization and deserialization logic for each type. The existing code-paths of the serializer are still used. This is to support complicated serializer usages like complex object graphs, and also complex serialization options that can be indicated at runtime (using
JsonSerializerOptions
) as well as design-time (using serialization attributes like[JsonIgnore]
). Attempting the generate code that adapts to each scenario can lead to large, complex, and unreliable code. A lightweight mode to generate low-level serialization logic for simple scenarios like POCOs and TechEmpower benchmarks is not yet implemented in this design, but is in scope for .NET 6.0. In the future, we can add optional knobs to improve throughput for other predetermined scenarios.This project introduces new patterns for using
JsonSerializer
, with the aim of improving performance in consuming applications. Let us see an example of what compile-time generated metadata looks like and how to interact with it in a project. Given an object:With the existing
JsonSerializer
functionality, this POCO may be serialized and deserialized as follows:With source generation, the serializable type may be indicated to the generator via
JsonSerializableAttribute
:The generator will then generate structured type metadata to the compilation assembly (click to view).
JsonSerializableAttribute.g.cs
JsonContext.g.cs
JsonContext.GetJsonClassInfo.g.cs
MyClass.g.cs
Int32.g.cs
StringArray.g.cs
String.g.cs
The generated type metadata can then be passed to new (de)serialization overloads as follows:
Using the source generator
The source generator can be consumed in any .NET C# project, including console application, class libraries, and Blazor applications. If the application's TFM is
net6.0
or upwards (inbox scenarios), then the generator will be part of the SDK(TODO: is this technically correct). For out-of-box scenarios such as .NET framework applications and .NET Standard-compatible libaries, the generator can be consumed via aSystem.Text.Json
NuGet package reference. See "Inbox Source Generators" for details on how we can ship inbox source generators.Design
Type discovery
Type discovery refers to how the source generator is made aware of types that will be passed to
JsonSerializer
. An explicit model where users manually indicate each type using a new assembly-level attribute ([assembly: JsonSerializable(Type)]
) is employed. This model is safe and ensures that we do not skip any types, or include unwanted types. In the future we can introduce an implicit model where the generator scan source files forT
s andType
instances passed to the variousJsonSerializer
methods, but it is expected that the explicit model remains relevant to cover cases where serializable types cannot easily be detected by inspecting source code.Here are some sample usages of the attribute for type discovery:
The attribute instances instruct the source generator to generate serialization metadata for the
MyClass
,object[]
,string
, andint
types. TheCanBeDynamic
property tells the generator to structure its output such that the metadata for thestring
,int
can be retrieved by an internal dictionary-based look up with the types as keys, since those values will be serialized polymorphically. TheMyClass
andobject[]
types will not be serialized polymorphically, so there is no need to specifyCanBeDynamic
.Based on the serializer usages in the example, we could expect that a source generator can inspect this code and automatically figure out what type needs to be serialized. This expectation is valid, and such functionality can be provided in the future. However, the explicit model will still be needed to handle scenarios such as the following:
Generated metadata
There are three major classes of types, corresponding to three major generated-metadata representations: primitives, objects (types that map to JSON object representations), and collections.
TODO: deep dive into different types of generated metadata.
How generating metadata helps meet goals
Faster startup & reduced private memory
Moving the generation of type metadata from runtime to compile time means that there is less work for the serializer to do on start-up, which leads to a reduction in the amount of time it takes to perform the first serialization or deserialization of each type.
The serializer uses Reflection.Emit where possible to generate fast member accessors to constructors, properties, and fields. Generating these IL methods takes a non-trivial time, but also consumes private memory. With source generators, we are able to generate delegates that statically invoke these accessors. These delegates are the used by the serializer alongside other metadata. This eliminates time and allocation cost due to Reflection emit.
All serialization and deserialization of JSON data is ultimately peformed within
System.Text.Json
converters. Today the serializer statically initializes several built-in converter instances to provide default functionality. User applications pay the cost of these allocations, even when only a few of these converters are needed given the input object graphs. With source generation, we can initialize only the converters that are needed by the types indicated to the generator.Given a very simple POCO like the one used for the TechEmpower benchmark, we can observe startup improvements when serializing and deserializing instances of the type:
Serialize
Old
Private bytes (KB): 2786.0
Elapsed time (ms): 41.0
New
Private bytes (KB): 2532.0
Elapsed time (ms): 30.0
Writer
Private bytes (KB): 337.0
Elapsed time (ms): 8.25
Deserialize
Old
Private bytes (KB): 1450
Elapsed time (ms): 30.25
New
Private bytes (KB): 916.0
Elapsed time (ms): 13.0
Reader*
Private bytes (KB): 457.0
Elapsed time (ms): 5.0
* not fair as only one prop with no complex lookup
Given the following object graph designed to similate a microservice that returns the weather forecase for the next 5 days, we can also notice startup improvements:
Serialize
Old
Private bytes (KB): 3209.0
Elapsed time (ms): 48.25
New
Private bytes (KB): 2693.0
Elapsed time (ms): 36.0
Writer
Private bytes (KB): 815
Elapsed time (ms): 15.5
Deserialize
Old
Private bytes (KB): 1698.0
Elapsed time (ms): 36.5
New
Private bytes (KB): 1093
Elapsed time (ms): 19.5
It is natural to wonder if we could yield more performance here given the simple scenarios and the performance of the low level reader and writer. It is indeed possible to do so in limited scenarios like the ones described above. It is in scope for 6.0 to provide a mode that is closer in performance to the reader and writer. It is also planned to provide more knobs in the future to provide better throughput for more complex scenarios. See the "Throughput" section below for more notes on this.
Also see this gist to see how the startup performance scales as we add more types, properties, and serialization attributes. TODO: get updated numbers.
Reduced app size
By generating metadata at compile-time instead of at runtime, two major things are done to reduce the size of the consuming application. First, we can detect which custom or built-in
System.Text.Json
converter types are needed in the application at runtime, and reference them statically in the generated metadata. This allows the ILLinker to trim out JSON converter types which will not be needed in the application at runtime. Similarly, reflecting over input types at compile-time eradicates the need to do so at compile-time. This eradicates the need for lots ofSystem.Reflection
APIs at runtime, so the ILLinker can trim code internal code inSystem.Text.Json
which interacts with those APIs. Unused source code further in the dependency graph is also trimmed out.Size reductions are based on how
JsonSerializerOptions
instances are created. If you use the existing patterns, then all the converters and reflection-based support will be rooted in the application for backwards compat:If you use a new pattern, then we can shed these types when unused:
When copying options, the new instance inherits the pattern of the old instance:
TODO:
NotSupportedException
is thrown if options is created for size, and metadata for a type hasn't been provided.TODO: do we need to expose a
JsonSerializerOptions.IsOptimizedForSize
so that users can know whether they can use an options instance with existing overloads.Given the weather forecast scenario from above, we can observe size reductions in both a Console app and the default Blazor app when processing JSON data. In a Blazor app, we get **~ 121 KB** of compressed dll size savings. In a console app we get about 400 KB in size reductions, post cross-gen and linker trimming. TODO: get updated numbers for console app.
TODO: if using POCO with 5 props as example, how many of those would we need in app before we start making the app bigger?
ILLinker friendliness
By avoiding runtime reflection, we avoid the primary coding pattern that is unfriendly to ILLinker analysis. Given that this code is trimmed out, applications that use the
System.Text.Json
go from having several ILLinker analysis warnings when trimming to having absolutely none. This means that applciations that use theSystem.Text.Json
source generator can be safely trimmed, provided there are no other warnings due to the user app itself, or other parts of the BCL.Throughput
Why not generate serialization code directly, using the
Utf8JsonReader
andUtf8JsonWriter
?The metadata-based design does not currently provide throughput improvements, but is forward-compatible with doing so in the future. Improving throughput means generating specific serialization and deserialization logic for each type in the object graph. Various serializer features would need to be baked into the generated code including:
List<Dictionary<string, Poco>>
orSalesOrder.Customer.Address.City
).This can lead to a large, complex, and unserviceable amount of code to generate. The large amount of code that can potentially be generated also conflicts with the goal of app size reductions. With this in mind, we decided to start with an approach that works well for all usages of the serializer.
In the future, we plan to build on the design and provide knobs to employ different source generation modes that also improve throughput for different scenarios. For instance, an application can indicate to the generator ahead of time that it will not use a naming policy or custom converters at runtime. With this information, we could generate a reasonable amount of very fast serialization and deserialization logic using the writer and reader directly. Another mode could be to generate optimal logic for a happy-path scenario, but also generate metadata as a fallback in case runtime-specified options need more complex logic.
A mode to provide throughput improvements for simple scenarios like the TechEmpower JSON benchmark is in scope for .NET 6.0, and this design will be updated to include it.
API Proposal
(click to view)
System.Text.Json.SourceGeneration.dll
The following API will be generated into the compiling assembly, to mark serializable types.
System.Text.Json.dll
System.Net.Http.Json.dll
Implementation considerations
Compatibility with existing
JsonSerializer
functionalityAll functionality that exists in
JsonSerializer
today will continue to do so after this new feature is implemented, assuming theJsonSerializerOptions
are not created for size optimizations. This includes rooting all converters and reflection-based logic for a runtime warm up of the serializer if the existing methods are used. If theJsonSerializerOptions.CreateForSizeOpts
method is used to create the options, then aNotSupportedException
will be thrown if serialization is attempted.What is the model for implementing and maintaining new features moving forward
For the first release, every feature that the serializer supports will be supported when the source generator is used. This involves implementing logic in
System.Text.Json.SourceGeneration.dll
to recognize static options like serializationa attributes. Each new feature needs to be implemented in the serializer (System.Text.Json.dll
), but now the source generator must also learn how to check whether it is used at design time, so that the appropriate metadata can be generated.Interaction between options instances and context classes
With the current design, a single
JsonSerializerOptions
instance can be used in multipleJsonSerializerContext
classes. Multiple context instances could populate the options instance with metadata like converters and customization options. This presents a challenge, as APIs likeJsonSerializerOptions.GetConverter(Type)
must now depend on first-one-wins semantics among the context classes to fetch a converter from. This can be worked around with validation to enforce a 1:1 mapping from options instance to context instance. An alternative option might be to make theJsonSerializerContext
type derive fromJsonSerializerOptions
. This enforces a 1:1 pairing, and might also be a more natural representation of the relationship given that options instances cache references to type metadata, as opposed to being just a lightweight POCO for holding runtime-specified serializer configuration.TODO: Discuss context class deriving from
JsonSerializerOptions
What does the metadata pattern mean for applications, APIs, services that perform JsonSerialization on behalf of others?
Services that perform JSON processing on behalf of others, such as
Blazor
client APIs,ASP.NET MVC
, and APIs in theSystem.Net.Http.Json
library must provide new APIs to acceptJsonSerializerContext
instances that were generated in the user's assembly.Versioning
TODO. How to ensure we don't invoke stale/buggy metadata implementations?
Integer property on generated JsonContext class(es) which we can check at runtime.
The text was updated successfully, but these errors were encountered: