-
Notifications
You must be signed in to change notification settings - Fork 2.1k
JsonInputFormatter, ModelState and error response message. #4607
Comments
If this is the case this is a very serious bug that would break everyone's app. Have a repro for this?
This isn't exactly accurate, model validation (data-annotations attributes) is a separate pass that takes place after formatters/model-binding run. There is some input validation that takes place during formatters/model-binding - this includes things like parsing JSON and converting strings to various types.
Can you provide a repro and example of the kinds of problems you're seeing? It's definitely the intent that errors from a formatter would work the same as errors from old-school model binding. |
public class MyController : Controller
{
public class MyModel
{
[JsonProperty(Required = Required.Always)]
public String Foo { get; set; }
}
[HttpPut]
[Route("")]
public async Task<IActionResult> GetWee([FromBody] MyModel myModel)
{
if (!ModelState.IsValid)
return HttpBadRequest(ModelState);
return Ok("Wee!");
}
public class MyOtherModel
{
[BindRequired]
public String Foo { get; set; }
}
[HttpPut]
[Route("other")]
public async Task<IActionResult> GetOther([FromBody] MyOtherModel myModel)
{
if (!ModelState.IsValid)
return HttpBadRequest(ModelState);
return Ok("Wee!");
}
} For both requests, submit with a payload of The first action (
The second action (
|
@Zoltu have you debugged this scenario? From your description, it sounds like everything is working as expected. In particular, the JSON response for the first case looks like the expected The I suspect the |
When I drop into a debugger and look at the exception thrown by the JSON deserializer it has a much better message. The message on the exception says something along the lines of, "Expected String Foo" and gives a line/column number of where it expected a Foo and didn't find one. In this case, since the object is so simple that is line 1 character 2 I think (I don't have the code in front of me). @dougbu Are you suggesting that I can put |
@blowdart anything to add?
Yup. |
I think JSON.NET deserialization exceptions are always safe to show the the client, but I can appreciate that such things shouldn't be assumed. It's there some other way to expose better error messages to the user? The crux of this issue is that it is currently difficult to write a quality public API with asp.net core because it is hard to get quality error messages back to the user. |
Nothing to add. I'd be ok with someone being able to configure detailed exceptions if they wanted to shoot themselves in the foot, but it can't be the default. In binding shouldn't these things show up as model state errors, or does the exception happen before then? |
@blowdart these W.r.t. configuring detailed exceptions in a response, someone could write their own |
@blowdart The exception occurs before the action so there is no opportunity for user-code to see/handle the error. Looking at the code, it appears that they are added to the ModelState as an error, but unless I'm missing something that doesn't do any good if the exception prevents the action from executing.
For the same reason as I just mentioned, I don't believe this is currently possible unless there is some way to stop the |
@Zoltu your action is executing. It calls |
Hmm, I thought I did that and didn't see the breakpoint hit. I'll try again this evening though, it could be that I am mis-remembering. Thanks! |
Putting errors on the 'root' object is a bug in RC1 that's been fixed. This can have an effect on validation when you need to validate multiple parameters, and might describe some of the badness you're seeing. There's a somewhat related discussion here: #3743 (comment) If you're seeing issues where model validation isn't triggered for other parameters, you might want to use the |
While not ideal, I was able to take the suggestions here and get a better error message out with the following: if (!ModelState.IsValid)
{
var error = ModelState.SelectMany(x => x.Value.Errors).First();
if (error.ErrorMessage != null && error.ErrorMessage != String.Empty)
return HttpBadRequest(error.ErrorMessage);
else if (error.Exception?.Message != null)
return HttpBadRequest(error.Exception.Message);
else
return HttpBadRequest(ModelState);
} I can probably figure out a relatively clean way to throw that into an extension method and then JSON validation failure responses will be improved across my application as long as I use it everywhere instead of I also tried switching over to using I still think there would be value in making the At this point I suppose this may have turned into a feature rather than a bug and I'm not sure if this is the right place to track such things so feel free to close if it isn't. |
I think this is quality feedback in an area we know that we want to improve (error reporting for JSON/API clients). I appreciate your patience and thoughts on this. |
I think there should be a way to customize the resulting JSON message generated by calling A good way would be to wrap them in an |
@markgarcia - there's nothing magic about I'd suggest overriding the requisite method on |
Should the
Should be:
Request and Response payloads works with camel casing. |
is there a simple way to convert your response object when using context.Result = new BadRequestObjectResult(context.ModelState); on a filter to lower camel case? I'm getting pascal case and I don't know how to change this |
@LRPalacios Good question, there are way? |
@LRPalacios , if you mean dictionary keys, they're serialized pascal case by default. You can change this as described here. services.AddMvc().AddJsonOptions(x =>
{
x.SerializerSettings.ContractResolver = new DefaultContractResolver
{
NamingStrategy = new CamelCaseNamingStrategy
{
ProcessDictionaryKeys = true
}
};
}); |
@alexb5dh thanks, I'm going to try it |
It sounds like the important takeaway from this is that we don't have a good default error message when the JSON is invalid. This is definitely something we should improve for 2.1.0 |
@SteveSandersonMS assigned to you as a companion to #4862 When you get an input error in JSON it lands here: https://github.com/aspnet/Mvc/blob/dev/src/Microsoft.AspNetCore.Mvc.Formatters.Json/JsonInputFormatter.cs#L155 Assume the usage of A few issues:
|
OK, I've been investigating what happens in different JSON
ProposalA. Make it simple to opt into exposing the raw exception message to clients Example: This exposes maximum info to clients, including details from Json.NET about why properties were invalid or what aspect of JSON parsing failed. This would work by changing B. By default, add more info to the generic error message Instead of just defaulting to This would work by expanding @rynowak's questions
Sometimes yes, sometimes no. Examples above.
This data structure does match the reality of what we are doing. Plus it just so happens to be precisely what some third-party components such as React-Formsy are looking for (so they can make validation messages appear at the right place in the UI). I don't have any reason to propose changing this. At some point we might want to support the HTTP Problem Details spec. This is a different-looking response structure, but it's possible to produce valid problem details JSON objects using only the information we normally have in a
We already do use the JSON keys for all the errors that come from Json.NET. We only use .NET property names for model validation errors, by which time we don't know about JSON keys.
Mainly I think we just need to expose all the information that's safe to expose by default ( Plus also give an easy and reasonably-discoverable solution to the people asking for the raw message info from Json.NET as per proposal [A]. |
I think as long as you don't reflect back the actual input, and it's just positional information it'd be fine. |
It would be positional information ( |
I guess if someone is silly enough to output the error message without encoding they're doing work to shoot themselves in the foot, so ... sure, fine by me. |
@SteveSandersonMS is anything in your description different after @kichalla's work in 83c3ac6 (adding |
Yes, this is a totally different workitem. Kiran's bug dealt with formatters swallowing totally unrelated exceptions such as This work item is about improving the user experience for bona-fide input validation errors. The hope is that returning the model state errors to a user will make it possible to troubleshoot your problems as client of an API |
Great notes @SteveSandersonMS ! Some thoughts:
I'm not crazy about this idea. If the developer has to make a code change to turn this on, then it's not on by default, then clients have to contact the developer to ask for it. At that point it might as well be logging. Unless we're willing to make something like this on by default for new code ( We also have a really long history of conflict, and building bad features that try to turn all exceptions into client errors. I think that if something like this is needed - then it means your second proposal is a failure.
IMO we should find a way to make this part of
1,000,000 times Yes!
I'm curious to know if you have any thoughts about how this will work with layering. We have a new
OK great, that's a good data point. What I was wondering was, do the 'keys' matter? It sounds like they do.
We do. The E2E for this feature is to use |
Re: opting into exposing exception messages
I'm fine with just not doing this, assuming it's possible for developers to expose the exception message on their own by writing a reasonably simple result filter (or similar). Or maybe, as per below, we might be able to just expose this info by default.
Strictly speaking this information already is in the Are you thinking we should turn I don't know what use cases there are for these extra properties on
Yes, we could use It looks like all the existing default usages of As for whether we can add all JSON deserialization errors into this privileged group: AFAIK this would be fine in practice today. Theoretically, a future version of Json.NET could start putting sensitive info into its exception messages, but theoretically so could If we decide we can't do this for Json.NET exceptions by default, we could instead put special-case logic into I guess this would count as a breaking change since people might have existing logic that looks in
And this happens by default because of @pranavkm's work on serializing |
It's an idea I had 💡 -
We should bring this up with him again and try to get a real sign off. We've had some hallway discussions about it, but I want to make sure it doesn't take anyone by surprise.
I'm still believe it's the kind of change we'd be willing to make in a minor release. We already put a generic error in the model state when we can't process the body, in this case we'd be adding a specific error.
Yes, |
Fodder for
Json.NET includes details of the request content in a few throw JsonReaderException.Create(this, "Could not convert string to integer: {0}.".FormatWith(CultureInfo.InvariantCulture, s)); The above is usually just a single character or value. But, I haven't attempted to find everything. Json.NET also includes potentially-sensitive server configuration details e.g. here: throw JsonReaderException.Create(this, "The reader's MaxDepth of {0} has been exceeded.".FormatWith(CultureInfo.InvariantCulture, _maxDepth)); |
@dougbu Thanks for that info! This does need to get fed into the decision about whether Json.NET exception messages are exposed by default or not. @rynowak It sounds like a plan is settling. To summarise:
I don't see any value right now in adding So @rynowak - do you agree with this plan? If so I'll get on with it. Also I will get in touch with Barry to make sure we have a final position on exposing the Json.NET messages or not. |
Yes, I think this is a great plan. 👍 |
Implemented in #7015 |
@SteveSandersonMS does #7015 implementation only reates to I'm on dotnet 2.1.301 |
I have been digging through the aspnet/mvc issues backlog as well as StackOverflow/Google and I have learned that when a client calls a controller/action that uses
[FromBody]
, theJsonInputFormatter
is used rather than the MVC Model binding stuff. This results inModelState.IsValid
returning true even when the model state isn't valid, because the actual model validation occurs as part of the input formatter processing, after theModelState
is already populated. This, of course, is problematic because it doesn't align with other properties likeFromRoute
where deserialization/parsing errors are included in theModelState
.I can accept that this is how things work though and move on. The problem I am now running into is that I can't figure out any way to surface the JSON deserializer error in the
BadRequest
response. Looking at https://github.com/aspnet/Mvc/blob/master/src/Microsoft.AspNet.Mvc.Formatters.Json/JsonInputFormatter.cs#L105, I can see that theModelState
is actually updated (Yay!), unfortunately because the Input Formatter proceeds with surfacing the error through its failure mechanisms as well (https://github.com/aspnet/Mvc/blob/master/src/Microsoft.AspNet.Mvc.Formatters.Json/JsonInputFormatter.cs#L133) there is no opportunity for user code to actually look at theModelState
and report the error it has in the usual ways (HttpBadRequest(ModelState)
).What is the ASP.NET Core 1.0 release plan for this? Unless I am missing something, it seems that I can't really leverage input formatters unless I am okay with never giving a useful response to my users. If I want to give them information to help troubleshoot what is wrong with their request I have to take in a
[FromBody] String
and deserialize by hand inside each of my actions, or build my own InputFormatter that doesn't actually return an error on failure and instead just populates theModelState
(asJsonInputFormatter
does), then claim success.Also, are there any workarounds short of writing my own input formatter that doesn't behave correctly (always returns success with Model errors)? I also tried adding an exception filter but it doesn't appear to catch InputFormatter problems, which isn't surprising as the
ExceptionFilter
is much later in the pipeline.The text was updated successfully, but these errors were encountered: