Skip to content

Commit

Permalink
Speed up glean_usage generation by caching the table getter (#4644)
Browse files Browse the repository at this point in the history
`get_tables` is deterministic under the assumption that the tables don't
change in between invocations. Which I hope holds here.
We therefore can just cache that value so that subsequent runs quickly
return without needing a roundtrip to BigQuery again.
  • Loading branch information
badboy authored Dec 4, 2023
1 parent 97c8a42 commit ca21c91
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion sql_generators/glean_usage/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
"""GLEAN Usage."""
from functools import partial
from functools import cache, partial
from pathlib import Path

import click
Expand Down Expand Up @@ -99,6 +99,7 @@ def generate(
elif exclude:
table_filter = partial(table_matches_patterns, exclude, True)

@cache
def get_tables(table_name="baseline_v1"):
baseline_tables = list_tables(
project_id=target_project,
Expand Down

0 comments on commit ca21c91

Please sign in to comment.