Performance: Add static TypeAdapters and reuse them.

[aaazzam]

Describe the current behavior

Today Prefect liberally uses Pydantic's TypeAdapter to validate models of type Generic[BaseModel], e.g.

async def read_variables(self, limit: Optional[int] = None) -> List[Variable]:
    """Reads all variables."""
    response = await self._client.post("/variables/filter", json={"limit": limit})
    return pydantic.TypeAdapter(List[Variable]).validate_python(response.json())

Morally, TypeAdapter under the hood is a lower-level create_model which has a non-trivial cost to create.

Pydantic
When creating an instance of TypeAdapter, the provided type must be analyzed and converted into a pydantic-core schema. This comes with some non-trivial overhead, so it is recommended to create a TypeAdapter for a given type just once and reuse it in loops or other performance-critical code.

This means today Prefect is paying that overhead in almost every client call that fetches a list.

Describe the proposed behavior

For each relevant Prefect BaseModel, e.g. Variable create a

_VariableListAdapter = TypeAdapter(List[Variable])

alongside that BaseModel for reuse. Note with our switch to defer_build, we need to explicitly turn on defer_build for each TypeAdapter so that we don't force the creation of a serializer accidentally.

https://docs.pydantic.dev/latest/api/config/#pydantic.config.ConfigDict.experimental_defer_build_mode

Example Use

No response

Additional context

No response