Skip to content

Commit

Permalink
Additional file monitoring instruction
Browse files Browse the repository at this point in the history
  • Loading branch information
Gordon Shotwell committed Dec 6, 2023
1 parent 1f376a0 commit bf5c115
Show file tree
Hide file tree
Showing 5 changed files with 36 additions and 4 deletions.
6 changes: 4 additions & 2 deletions file-monitoring/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Problem description

Data apps often sit at the end of a larger data processing pipeline, and so it is helpful to have your app update whenever the underlying data files change.
Data apps often sit at the end of a larger data processing pipeline, and so it is helpful to have your app update whenever the underlying data files change. An example data generating script can be found in `populate-logs.py` which irregularly appends a row to a data frame.

# Requirements
- App should read in and render a csv file, whenever that csv file changes the application should update without any user action.
- App should read in and render a csv file as a data frame, whenever that csv file changes the data frame output should rerender.
- The application shouldn't rerender if the csv file is unchanged.
- Othe components should be usable as the data frame rerenders.
4 changes: 4 additions & 0 deletions file-monitoring/logs.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
date,status
2023-12-05 18:48:35.076744,status5
2023-12-05 18:48:36.081387,status14
2023-12-05 18:48:37.088922,status19
17 changes: 17 additions & 0 deletions file-monitoring/populate-logs.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import pandas as pd
from datetime import datetime
from pathlib import Path

import random
import time

log_path = Path(__file__).parent / "logs.csv"

while True:
status = "status" + str(random.randint(0, 20))
# Create a new DataFrame with the current time and the random status
df = pd.DataFrame({"date": [datetime.now()], "status": [status]})

df.to_csv(log_path, mode="a", header=False, index=False)
# Wait for a second before the next append operation
time.sleep(random.randint(1, 5))
11 changes: 10 additions & 1 deletion file-monitoring/shiny/app.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
from shiny import Inputs, Outputs, Session, App, render, ui, reactive
from pathlib import Path
import pandas as pd
from datetime import datetime


@reactive.file_reader(Path(__file__).parent / "logs.csv")
Expand All @@ -9,7 +10,11 @@ def logs():


app_ui = ui.page_fillable(
ui.card(ui.output_data_frame("df")),
ui.layout_column_wrap(
ui.card(ui.output_data_frame("df")),
ui.card(ui.input_text("txt_in", "Enter text"), ui.output_text("text_out")),
width=1 / 2,
)
)


Expand All @@ -19,5 +24,9 @@ def df():
out = logs().sort_values("date", ascending=False)
return render.DataTable(out, width="500px", height="95%")

@render.text
def text_out():
return f"You wrote {input.txt_in()} at {datetime.now().time()}"


app = App(app_ui, server)
2 changes: 1 addition & 1 deletion file-monitoring/shiny/populate-logs.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,4 @@

df.to_csv(log_path, mode="a", header=False, index=False)
# Wait for a second before the next append operation
time.sleep(1)
time.sleep(random.randint(1, 5))

0 comments on commit bf5c115

Please sign in to comment.