Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(data-warehouse): Ensure dates are actual datetime formats #27777

Merged
merged 1 commit into from
Jan 22, 2025

Conversation

Gilbert09
Copy link
Member

Problem

  • Some mysql tables can return datetimes in string format and out of range at 0000-00-00T00:00:00 which is invalid in python
  • Fixes this ticket

Changes

  • Format string dates to be actual datetime objects when loading into pyarrows

Does this work well for both Cloud and self-hosted?

Yes

How did you test this code?

Tested with MySQL data

@Gilbert09 Gilbert09 requested a review from a team January 22, 2025 13:30
@Gilbert09 Gilbert09 merged commit b412151 into master Jan 22, 2025
93 checks passed
@Gilbert09 Gilbert09 deleted the tom/fix-invalid-dates branch January 22, 2025 15:53
fuziontech added a commit that referenced this pull request Jan 22, 2025
* master: (103 commits)
  feat(postgres-estimated-rows): pg Estimated Rows on Data Warehouse Sync (#27634)
  fix: revert darkmode class toggle, updated content on fills (#27783)
  chore: upgrade posthog-js (#27790)
  chore(editor-3001): add back join actions (#27740)
  feat: Add person distinct ID overrides squash job (as dagster job) (#27710)
  fix(created-by-sources): Adding `created_by` to sources (#27751)
  Revert "feat(data-warehouse): V2 pipeline release " (#27791)
  fix: typo for feature flags (#27786)
  fix(defer-unmounting): Defer unmounting of react elements (#27742)
  feat(data-warehouse): V2 pipeline release (#27732)
  fix(data-warehouse): Ensure dates are actual datetime formats (#27777)
  fix: enable hot reload for the products dir (#27746)
  fix: assignee selector when null (#27737)
  chore: clarify rrweb imports (#27776)
  chore(deps): Update posthog-js to 1.207.3 (#27779)
  feat(retention): filters on start/return event (#27770)
  fix(experiments): only show supported math functions (#27589)
  feat(web-analytics): Set unique conversions graph when adding conversions goal (#27774)
  chore: color design system part 1: banner and accents (#27756)
  chore(experiments): Add tests for funnel attribution options (#27752)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants