-
Notifications
You must be signed in to change notification settings - Fork 446
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mock for http exporters #1185
Mock for http exporters #1185
Conversation
Codecov Report
@@ Coverage Diff @@
## main #1185 +/- ##
=======================================
Coverage 92.96% 92.96%
=======================================
Files 196 196
Lines 7040 7040
=======================================
Hits 6544 6544
Misses 496 496
|
…lemetry-cpp into mock-http-exporters
@@ -22,7 +22,7 @@ $VCPKG_DIR="$SRC_DIR\vcpkg" | |||
|
|||
switch ($action) { | |||
"bazel.build" { | |||
bazel build $BAZEL_OPTIONS -- //... | |||
bazel build --copt=-DENABLE_TEST $BAZEL_OPTIONS -- //... |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit - --copt=-DENABLE_TEST
is also part of BAZEL_OPTIONS
, so can be removed here.
|
||
/** | ||
* Export | ||
* @param message message to export, it should be ExportTraceServiceRequest, | ||
* ExportMetricsServiceRequest or ExportLogsServiceRequest | ||
*/ | ||
sdk::common::ExportResult Export(const google::protobuf::Message &message) noexcept; | ||
|
||
VIRTUAL_TEST sdk::common::ExportResult Export(const google::protobuf::Message &message) noexcept; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit - googletest
suggests to mock the non-virtual function using template - https://chromium.googlesource.com/external/github.com/google/googletest/+/refs/tags/release-1.8.0/googlemock/docs/CookBook.md#mocking-nonvirtual-methods. Sharing if this is feasible, though I don't see much issue with the existing approach.
@@ -98,14 +104,16 @@ class OtlpHttpClient | |||
* Create an OtlpHttpClient using the given options. | |||
*/ | |||
explicit OtlpHttpClient(OtlpHttpClientOptions &&options); | |||
#ifdef ENABLE_TEST | |||
VIRTUAL_TEST ~OtlpHttpClient() {} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am thinking if mocking ext::http::client::HttpClient
would be a better approach (the way we do it for Zipkin) as this will allow us to test most of the existing code in OtlpHttpClient::Export() method. We may have to add ext::http::client::HttpClient object as dependency injection to OtlpHttpClient to make this feasible.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't use template because I thought it will be too much of noise in code.
Thanks for the idea, I will use dependency injection.
return http_request_; | ||
} | ||
|
||
MOCK_METHOD(void, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need to mock the SendRequest(), or can we create a empty (200 OK) response, and call the callback from same thread. Something like this
void SendRequest(EventHandler &callback) noexcept override
{
auto response = std::unique_ptr<Response>(new Response());
.. populate response object..
callback.OnResponse(response);
}
This will allow us to remove dependency of gtest/gmock from this target.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have used Mock for testing the request that is going to be sent. The other option would be to make the client test itself via:
std::function<void(http_client::nosend::Session *)> CreateJsonTester(
const std::string report_trace_id)
{
return [report_trace_id](http_client::nosend::Session *mock_session) {
auto check_json = nlohmann::json::parse(mock_session->GetRequest()->body_, nullptr, false);
// extract received_trace_id from request
EXPECT_EQ(received_trace_id, report_trace_id);
};
}
Then store this function into session, so whenever there is a request going to be sent the session tests itself:
void SendRequest(opentelemetry::ext::http::client::EventHandler &callback) noexcept override
{
if (test_input_)
{
test_input_(this);
}
// let the otlp_http_client to continue
Response response;
callback.OnResponse(response);
}
Shall I keep gMock or let the session test itself?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank. Let's keep the gMock for now, we can fix it later.
I am just thinking another solution to change signature for EventHandler::OnEvent to something like:
virtual void OnEvent(SessionState, void* data = nullptr, size_t size = 0) noexcept = 0;
The method in its current form is not useful. Sending back client implementation-specific data
would be useful in real-world scenarios like server certificate validation during SSL Handshake. And nosend client can then use this to return the Request object for validation, something like
void SendRequest(EventHandler &callback) noexcept override
{
callback.OnEvent(SessionState::Connected, (void*)http_request_.get(), size);
auto response = std::unique_ptr<Response>(new Response());
.. populate response object..
callback.OnResponse(response);
}
But don't want to delay this PR more, let's have this fix separately. We can have a github ticket to add this support / or remove mock from nosend client.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
good idea, thanks.
yes I agree let's merge this PR and add the support in another issue.
Please assign the new issue to me.
client factory should be fixed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR. Changes look good, apart from minor issues to address
- Remove mock dependency for nosend HTTP interface.
- Small change in HTTP factory implementation (discussing offline)
OtlpHttpClient http_client_; | ||
std::unique_ptr<OtlpHttpClient> http_client_; | ||
// For testing | ||
friend class OtlpHttpExporterTestPeer; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Duplicated with line 95?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks, cleaned.
OtlpHttpClient http_client_; | ||
std::unique_ptr<OtlpHttpClient> http_client_; | ||
// For testing | ||
friend class OtlpHttpLogExporterTestPeer; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Duplicated with line 94.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks, cleaned
Signed-off-by: owent <[email protected]>
Export all datas before notify `ForceFlush` to continue Signed-off-by: owent <[email protected]> Fix timeout for `wait_for` Signed-off-by: owent <[email protected]> Fix wakeup notify Signed-off-by: owent <[email protected]> 1. Reduce busy waiting of `BatchSpanProcessor::ForceFlush` and `BatchSpanProcessor::ForceFlush` when another thread is exporting data. 2. Fix the deadlock when `Shutdown` is called and background thread is finished after first checking of `is_shutdown_` and before setting `is_shutdown_` in `ForceFlush` of BatchSpanProcessor and `BatchSpanProcessor`. Signed-off-by: owent <[email protected]> Fix initialization of `is_shutdown_` in internal constructor of `OtlpHttpClient` Signed-off-by: owent <[email protected]> Add `ForceFlush` for `LoggerProvider` Signed-off-by: owent <[email protected]> Merge open-telemetry#1185 and add `OnEvent` for `nosend::HttpClient` Signed-off-by: owent <[email protected]> Add changelog Signed-off-by: owentou <[email protected]> Fix asan of overflow of chrono Signed-off-by: owentou <[email protected]> Fix shutdown timeout Signed-off-by: owentou <[email protected]> Fix thread-safety problem when destroying `OtlpHttpClient` Signed-off-by: owentou <[email protected]> Fix some compiler do not generate the correct move construct/assignment operator implementation Signed-off-by: owent <[email protected]> Fix compatibility Signed-off-by: owent <[email protected]> Allow concurrency otlp http session for otlp http client Signed-off-by: owent <[email protected]>
Export all datas before notify `ForceFlush` to continue Signed-off-by: owent <[email protected]> Fix timeout for `wait_for` Signed-off-by: owent <[email protected]> Fix wakeup notify Signed-off-by: owent <[email protected]> 1. Reduce busy waiting of `BatchSpanProcessor::ForceFlush` and `BatchSpanProcessor::ForceFlush` when another thread is exporting data. 2. Fix the deadlock when `Shutdown` is called and background thread is finished after first checking of `is_shutdown_` and before setting `is_shutdown_` in `ForceFlush` of BatchSpanProcessor and `BatchSpanProcessor`. Signed-off-by: owent <[email protected]> Fix initialization of `is_shutdown_` in internal constructor of `OtlpHttpClient` Signed-off-by: owent <[email protected]> Add `ForceFlush` for `LoggerProvider` Signed-off-by: owent <[email protected]> Merge open-telemetry#1185 and add `OnEvent` for `nosend::HttpClient` Signed-off-by: owent <[email protected]> Add changelog Signed-off-by: owentou <[email protected]> Fix asan of overflow of chrono Signed-off-by: owentou <[email protected]> Fix shutdown timeout Signed-off-by: owentou <[email protected]> Fix thread-safety problem when destroying `OtlpHttpClient` Signed-off-by: owentou <[email protected]> Fix some compiler do not generate the correct move construct/assignment operator implementation Signed-off-by: owent <[email protected]> Fix compatibility Signed-off-by: owent <[email protected]> Allow concurrency otlp http session for otlp http client Signed-off-by: owent <[email protected]>
Export all datas before notify `ForceFlush` to continue Signed-off-by: owent <[email protected]> Fix timeout for `wait_for` Signed-off-by: owent <[email protected]> Fix wakeup notify Signed-off-by: owent <[email protected]> 1. Reduce busy waiting of `BatchSpanProcessor::ForceFlush` and `BatchSpanProcessor::ForceFlush` when another thread is exporting data. 2. Fix the deadlock when `Shutdown` is called and background thread is finished after first checking of `is_shutdown_` and before setting `is_shutdown_` in `ForceFlush` of BatchSpanProcessor and `BatchSpanProcessor`. Signed-off-by: owent <[email protected]> Fix initialization of `is_shutdown_` in internal constructor of `OtlpHttpClient` Signed-off-by: owent <[email protected]> Add `ForceFlush` for `LoggerProvider` Signed-off-by: owent <[email protected]> Merge open-telemetry#1185 and add `OnEvent` for `nosend::HttpClient` Signed-off-by: owent <[email protected]> Add changelog Signed-off-by: owentou <[email protected]> Fix asan of overflow of chrono Signed-off-by: owentou <[email protected]> Fix shutdown timeout Signed-off-by: owentou <[email protected]> Fix thread-safety problem when destroying `OtlpHttpClient` Signed-off-by: owentou <[email protected]> Fix some compiler do not generate the correct move construct/assignment operator implementation Signed-off-by: owent <[email protected]> Fix compatibility Signed-off-by: owent <[email protected]> Allow concurrency otlp http session for otlp http client Signed-off-by: owent <[email protected]>
Export all datas before notify `ForceFlush` to continue Signed-off-by: owent <[email protected]> Fix timeout for `wait_for` Signed-off-by: owent <[email protected]> Fix wakeup notify Signed-off-by: owent <[email protected]> 1. Reduce busy waiting of `BatchSpanProcessor::ForceFlush` and `BatchSpanProcessor::ForceFlush` when another thread is exporting data. 2. Fix the deadlock when `Shutdown` is called and background thread is finished after first checking of `is_shutdown_` and before setting `is_shutdown_` in `ForceFlush` of BatchSpanProcessor and `BatchSpanProcessor`. Signed-off-by: owent <[email protected]> Fix initialization of `is_shutdown_` in internal constructor of `OtlpHttpClient` Signed-off-by: owent <[email protected]> Add `ForceFlush` for `LoggerProvider` Signed-off-by: owent <[email protected]> Merge open-telemetry#1185 and add `OnEvent` for `nosend::HttpClient` Signed-off-by: owent <[email protected]> Add changelog Signed-off-by: owentou <[email protected]> Fix asan of overflow of chrono Signed-off-by: owentou <[email protected]> Fix shutdown timeout Signed-off-by: owentou <[email protected]> Fix thread-safety problem when destroying `OtlpHttpClient` Signed-off-by: owentou <[email protected]> Fix some compiler do not generate the correct move construct/assignment operator implementation Signed-off-by: owent <[email protected]> Fix compatibility Signed-off-by: owent <[email protected]> Allow concurrency otlp http session for otlp http client Signed-off-by: owent <[email protected]>
Export all datas before notify `ForceFlush` to continue Signed-off-by: owent <[email protected]> Fix timeout for `wait_for` Signed-off-by: owent <[email protected]> Fix wakeup notify Signed-off-by: owent <[email protected]> 1. Reduce busy waiting of `BatchSpanProcessor::ForceFlush` and `BatchSpanProcessor::ForceFlush` when another thread is exporting data. 2. Fix the deadlock when `Shutdown` is called and background thread is finished after first checking of `is_shutdown_` and before setting `is_shutdown_` in `ForceFlush` of BatchSpanProcessor and `BatchSpanProcessor`. Signed-off-by: owent <[email protected]> Fix initialization of `is_shutdown_` in internal constructor of `OtlpHttpClient` Signed-off-by: owent <[email protected]> Add `ForceFlush` for `LoggerProvider` Signed-off-by: owent <[email protected]> Merge open-telemetry#1185 and add `OnEvent` for `nosend::HttpClient` Signed-off-by: owent <[email protected]> Add changelog Signed-off-by: owentou <[email protected]> Fix asan of overflow of chrono Signed-off-by: owentou <[email protected]> Fix shutdown timeout Signed-off-by: owentou <[email protected]> Fix thread-safety problem when destroying `OtlpHttpClient` Signed-off-by: owentou <[email protected]> Fix some compiler do not generate the correct move construct/assignment operator implementation Signed-off-by: owent <[email protected]> Fix compatibility Signed-off-by: owent <[email protected]> Allow concurrency otlp http session for otlp http client Signed-off-by: owent <[email protected]>
Fixes #1049 (issue)
Changes
removes
http
server from exporter tests, and uses mock for the clients.For significant contributions please make sure you have completed the following items:
CHANGELOG.md
updated for non-trivial changes