Make --setup/--autoreload/--warm work with --num-procs

[hoxbro]

Resolves #6891
Resolves #6816

The following generate_session(app, initialize=True) gave file errors on Python versions below 3.12, so I have set initialize to False for those versions. I'm not entirely sure why it works fine on Python 3.12, but it could be related to sub-interpreters.

Postpone the setup script execution after the server starts. This means the setup script runs for each process; for example, it will run twice for the following example.

# app.py
import panel as pn

pn.pane.Markdown("Hello World").servable()

# background_job.py
from datetime import datetime
import os
import panel as pn

current_time = datetime.now()


def task() -> None:
    """Refresh the data in the database."""
    print(f"Ran task at {datetime.now()} (PID {os.getpid()})")


pn.state.schedule_task(name="load_data", callback=task, at=current_time, period="5s")

Command: panel serve app.py --setup background_job.py --num-procs 2

screenrecord-2024-06-11_16.39.30.mp4

Also tested:

panel serve app.py --num-procs 2 --warm
panel serve app.py --num-procs 2 --autoreload

[codecov]

Codecov Report

Attention: Patch coverage is 82.69231% with 9 lines in your changes missing coverage. Please review.

Project coverage is 82.23%. Comparing base (4849daf) to head (4a1592f).
Report is 20 commits behind head on main.

Files with missing lines	Patch %	Lines
panel/io/server.py	30.00%	7 Missing ⚠️
panel/command/serve.py	66.66%	2 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff            @@
##             main    #6913    +/-   ##
========================================
  Coverage   82.22%   82.23%            
========================================
  Files         328      330     +2     
  Lines       49269    49379   +110     
========================================
+ Hits        40513    40607    +94     
- Misses       8756     8772    +16     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[philippjfr]

This makes a lot sense to me. What's missing here, apart from tests?

[hoxbro]

What's missing here, apart from tests?

Time 🙃 This does not fix the problem observed on older versions of Python. It could, therefore, be that the loading of the file should also be postponed.

[philippjfr]

This does not fix the problem observed on older versions of Python.

How old are we talking?

[hoxbro]

Only working with Python 3.12, IIRC.

[MarcSkovMadsen]

Would this fix the --warm issue too?

[hoxbro]

Would this fix the --warm issue too?

Yes, it should be the case after the changes I made yesterday.

[instance01]

@MarcSkovMadsen @philippjfr - we relied on the setup script to be run before the server starts. This reduces some of the usefuleness of the setup script.

It is also still mentioned in the wiki "--setup SETUP Path to a setup script to run before server starts.". Might want to update the documentation to reflect that this is not supported any longer. Link to the documentation: https://panel.holoviz.org/how_to/server/commandline.html

I would consider this a minor breaking change and would appreciate mentioning it in the changelogs.