使用skywalking中遇上一个异常

[shuxinqin]

我们应用中使用skywalking中遇上一个异常，这个异常不定时出现。出现时的场景大概是这样的，asp.net core应用，在一个mvc的action中使用HttpClient/HttpWebRequest去发起一个http请求，发起的http请求就会报错，具体异常信息如下。
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.ArgumentException: An item with the same key has already been added. Key: status_code
at System.ThrowHelper.ThrowAddingDuplicateWithKeyArgumentException(Object key)
at System.Collections.Generic.Dictionary2.TryInsert(TKey key, TValue value, InsertionBehavior behavior) at SkyWalking.Context.Trace.AbstractTracingSpan.Tag(String key, String value) at SkyWalking.Context.Trace.ExitSpan.Tag(String key, String value) at SkyWalking.Context.Tag.StringTag.Set(ISpan span, String tagValue) at SkyWalking.Diagnostics.HttpClient.HttpClientDiagnosticProcessor.HttpResponse(HttpResponseMessage response) --- End of inner exception stack trace --- at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor) at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments) at SkyWalking.Diagnostics.TracingDiagnosticMethod.Invoke(String diagnosticName, Object value) at SkyWalking.Diagnostics.TracingDiagnosticObserver.OnNext(KeyValuePair2 value)
at System.Diagnostics.DiagnosticListener.Write(String name, Object value)
at System.Net.Http.DiagnosticsHandler.d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
at System.Net.Http.HttpClient.d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at publish_helper.HttpUtils.d__0.MoveNext()

[liuhaoyang]

Could you provide some demo for us?

[shuxinqin]

伪代码，下面 Test 是一个Action
[HttpPost]
public string Test(Message message)
{
//先异步调用
PostAsync("url", "{}", null);

        //再次异步调用，获取结果
        string ret = PostAsync("url", "{a:123}", null).Result; //然后在这时候调用，就会报异常了。这个异常不定时出现。同样的Test请求（参数一致），有时会出异常，有时没事
        return ret;
    }
    static async Task<string> PostAsync(string url, string data, string authorization, int timeout = 100, string contentType = "application/json")
    {
        string result = string.Empty;
        using (var httpClient = new HttpClient())
        {
            // 设置超时时间
            httpClient.Timeout = TimeSpan.FromSeconds(timeout);
            var request = new HttpRequestMessage
            {
                RequestUri = new Uri(url),
                Method = HttpMethod.Post
            };

            request.Content = new StringContent(data, Encoding.UTF8, contentType);

            if (!string.IsNullOrEmpty(authorization))
            {
                string[] arr = authorization.Split(' ');
                string scheme = "";
                string parameter;
                if (arr.Length > 1)
                {
                    scheme = arr[0];
                    parameter = arr[1];
                }
                else
                {
                    parameter = arr[0];
                }
                request.Headers.Authorization = new AuthenticationHeaderValue(scheme, parameter);
            }

            var response = await httpClient.SendAsync(request);

            response.EnsureSuccessStatusCode();

            result = await response.Content.ReadAsStringAsync();

            return result;
        }
    }

[shuxinqin]

使用的版本是 0.2.0

[liuhaoyang]

You should wait for the previous HttpClient request to end and then initiate a new HttpClient request.

public async Task<string> Test(Message message)
{
        await PostAsync("url", "{}", null);

        string ret = await PostAsync("url", "{a:123}", null);
        return ret;
    }

[shuxinqin]

目前我们线上的情况是：有时候用户填写完表单点击提交按钮ajax调用Test方法，第一次提交报错，然后他们再次点击就好了。同样的的请求，同样的参数，偶尔会出现异常，不是总出现。
如果是PostAsync("url", "{}", null);没加await导致，那么为什么会时好时坏呢？
ps：第一个PostAsync之所以不await是因为我们的业务逻辑不需要等待调用结果第一次调用结果。

[shuxinqin]

据同事反馈，他也遇上SkyWalking异常。他的场景是这样的，在asp.net core应用中，有一个后台运行的任务，需要频繁调用http请求，然后偶尔也会报错，异常就是由SkyWalking内部引发的。
因此，初步断定，原因是：如果在同一个请求中的action中，如果这个action内部有并行发起http请求的情况，这时候SkyWalking就会有可能出现异常。

[liuhaoyang]

目前使用AsyncLocal存储追踪上下文，并行会造成AsyncLocal的值被覆写

[shuxinqin]

System.ArgumentException: An item with the same key has already been added. Key: status_code
at System.ThrowHelper.ThrowAddingDuplicateWithKeyArgumentException(Object key)
at System.Collections.Generic.Dictionary2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
Dictionary是非线程安全的，根据异常提示分析，很可能是由于同一请求中并行执行了多个http请求导致。

[liuhaoyang]

Repeat with #102, this bug will be fixed in V0.8.

[caozhiyuan]

this bug at there : https://github.com/caozhiyuan/SkyAPM-dotnet/blob/0.7.0/src/SkyWalking.Core/Context/TracingContext.cs#L193 @liuhaoyang Stack problem.
and wrong span ref.