Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CT-1886] Include adapter_response in RunResultMsg #6703

Closed
jtcohen6 opened this issue Jan 24, 2023 · 1 comment · Fixed by #6709
Closed

[CT-1886] Include adapter_response in RunResultMsg #6703

jtcohen6 opened this issue Jan 24, 2023 · 1 comment · Fixed by #6709
Labels
bug Something isn't working logging

Comments

@jtcohen6
Copy link
Contributor

jtcohen6 commented Jan 24, 2023

adapter_response is being skipped during message serialization for execution results:

$ dbt --debug --log-format json run | jq .data.run_result.adapter_response
...
{}
...

def to_msg(self):
# TODO: add more fields
msg = RunResultMsg()
msg.status = str(self.status)
msg.message = cast_to_str(self.message)
msg.thread = self.thread_id
msg.execution_time = self.execution_time
msg.num_failures = cast_to_int(self.failures)
msg.timing_info = [ti.to_msg() for ti in self.timing]
# adapter_response
return msg

Reproduction case

I drop a breakpoint in here:

fire_event(
NodeFinished(
node_info=runner.node.node_info,
run_result=result.to_msg(),
)
)

ipdb> result.adapter_response
{'_message': 'CREATE VIEW (0 processed)', 'code': 'CREATE VIEW', 'bytes_processed': 0, 'location': 'US', 'project_id': 'dbt-test-env', 'job_id': 'b70d9a54-38a6-4e21-ad07-d08e75ace428', 'slot_ms': 0}
ipdb> result.to_msg()
RunResultMsg(status='success', message='CREATE VIEW (0 processed)', timing_info=[TimingInfoMsg(name='compile', started_at=datetime.datetime(2023, 1, 24, 10, 38, 9, 423443), completed_at=datetime.datetime(2023, 1, 24, 10, 38, 9, 426940)), TimingInfoMsg(name='execute', started_at=datetime.datetime(2023, 1, 24, 10, 38, 9, 427426), completed_at=datetime.datetime(2023, 1, 24, 10, 38, 10, 391500))], thread='Thread-1 (worker)', execution_time=0.9696011543273926, adapter_response={}, num_failures=0)
ipdb> result.to_msg().adapter_response
{}

Previous behavior

This does represent a functional regression in v1.4, relative to v1.3, when we used different serialization (mashumaro) for the result:

$ dbt --debug --log-format json run | jq .data.run_result.adapter_response
...
{
  "_message": "CREATE VIEW (0 processed)",
  "bytes_processed": 0,
  "code": "CREATE VIEW",
  "job_id": "9180b859-944f-4f91-a559-d772e0697c6d",
  "location": "US",
  "project_id": "dbt-test-env",
  "slot_ms": 0
}
...

Same breakpoint as above:

ipdb> result.to_dict()['adapter_response']
{'_message': 'CREATE VIEW', 'code': 'CREATE VIEW', 'rows_affected': -1}
@jtcohen6 jtcohen6 added bug Something isn't working logging labels Jan 24, 2023
@github-actions github-actions bot changed the title Include adapter_response in RunResultMsg [CT-1886] Include adapter_response in RunResultMsg Jan 24, 2023
@jtcohen6
Copy link
Contributor Author

I'm guessing this is because our proto representation of adapter_response can only be a dictionary with string keys + string values (given limited proto type system):

https://github.com/dbt-labs/dbt-core/blob/main/core/dbt/events/types.proto#L48

adapter_response: Dict[str, str] = betterproto.map_field(
6, betterproto.TYPE_STRING, betterproto.TYPE_STRING
)

Still, this would be better than not including the data at all:

     msg.timing_info = [ti.to_msg() for ti in self.timing] 
     msg.adapter_response = {k: str(v) for k, v in self.adapter_response.items()}
     return msg 
$ dbt --debug --log-format json run | jq .data.run_result.adapter_response
...
{
  "_message": "CREATE VIEW (0 processed)",
  "bytes_processed": "0",
  "code": "CREATE VIEW",
  "job_id": "10588bfb-7857-432a-950e-926e10c00a9e",
  "location": "US",
  "project_id": "dbt-test-env",
  "slot_ms": "0"
}
...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working logging
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant