Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow user to constrain supported lock environments #6210

Merged
merged 2 commits into from
Aug 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion STYLE.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ The documentation is divided into:
1. When using `console` syntax, use `$` to indicate commands — everything else is output.
1. Never use the `bash` syntax when displaying command output.
1. Prefer `console` with `$` prefixed commands over `bash`.
1. Command output should rarely be included — it's hard to keep up to date.
1. Command output should rarely be included — it's hard to keep up-to-date.
1. Use `title` for example files, e.g., `pyproject.toml`, `Dockerfile`, or `example.py`.

## CLI
Expand Down
2 changes: 1 addition & 1 deletion crates/uv-client/src/httpcache/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ called a re-validation request.

A re-validation request includes with it some metadata (usually an "entity tag"
or `etag` for short) that was on the cached response (which is now stale).
When we send this request, the server can compare it with its most up to date
When we send this request, the server can compare it with its most up-to-date
version of the resource. If its entity tag matches the one we gave it (among
other possible criteria), then the server can skip returning the body and
instead just return a small HTTP 304 NOT MODIFIED response. When we get this
Expand Down
4 changes: 2 additions & 2 deletions crates/uv-git/src/git.rs
Original file line number Diff line number Diff line change
Expand Up @@ -626,9 +626,9 @@ enum FastPathRev {
/// date with what this rev resolves to on GitHub's server.
UpToDate,
/// The following SHA must be fetched in order for the local rev to become
/// up to date.
/// up-to-date.
NeedsFetch(GitOid),
/// Don't know whether local rev is up to date. We'll fetch _all_ branches
/// Don't know whether local rev is up-to-date. We'll fetch _all_ branches
/// and tags from the server and see what happens.
Indeterminate,
}
Expand Down
32 changes: 32 additions & 0 deletions crates/uv-resolver/src/lock.rs
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ pub struct Lock {
/// If this lockfile was built from a forking resolution with non-identical forks, store the
/// forks in the lockfile so we can recreate them in subsequent resolutions.
fork_markers: Vec<MarkerTree>,
/// The list of supported environments specified by the user.
supported_environments: Vec<MarkerTree>,
/// The range of supported Python versions.
requires_python: Option<RequiresPython>,
/// We discard the lockfile if these options don't match.
Expand Down Expand Up @@ -161,6 +163,7 @@ impl Lock {
requires_python,
options,
ResolverManifest::default(),
vec![],
graph.fork_markers.clone(),
)?;
Ok(lock)
Expand All @@ -173,6 +176,7 @@ impl Lock {
requires_python: Option<RequiresPython>,
options: ResolverOptions,
manifest: ResolverManifest,
supported_environments: Vec<MarkerTree>,
fork_markers: Vec<MarkerTree>,
) -> Result<Self, LockError> {
// Put all dependencies for each package in a canonical order and
Expand Down Expand Up @@ -329,6 +333,7 @@ impl Lock {
Ok(Self {
version,
fork_markers,
supported_environments,
requires_python,
options,
packages,
Expand All @@ -344,6 +349,13 @@ impl Lock {
self
}

/// Record the supported environments that were used to generate this lock.
#[must_use]
pub fn with_supported_environments(mut self, supported_environments: Vec<MarkerTree>) -> Self {
self.supported_environments = supported_environments;
self
}

/// Returns the number of packages in the lockfile.
pub fn len(&self) -> usize {
self.packages.len()
Expand Down Expand Up @@ -384,6 +396,11 @@ impl Lock {
self.options.exclude_newer
}

/// Returns the supported environments that were used to generate this lock.
pub fn supported_environments(&self) -> &[MarkerTree] {
&self.supported_environments
}

/// If this lockfile was built from a forking resolution with non-identical forks, return the
/// markers of those forks, otherwise `None`.
pub fn fork_markers(&self) -> &[MarkerTree] {
Expand Down Expand Up @@ -486,6 +503,7 @@ impl Lock {
if let Some(ref requires_python) = self.requires_python {
doc.insert("requires-python", value(requires_python.to_string()));
}

if !self.fork_markers.is_empty() {
let fork_markers = each_element_on_its_line_array(
self.fork_markers
Expand All @@ -496,6 +514,16 @@ impl Lock {
doc.insert("resolution-markers", value(fork_markers));
}

if !self.supported_environments.is_empty() {
let supported_environments = each_element_on_its_line_array(
self.supported_environments
.iter()
.filter_map(MarkerTree::contents)
.map(|marker| marker.to_string()),
);
doc.insert("supported-markers", value(supported_environments));
}

// Write the settings that were used to generate the resolution.
// This enables us to invalidate the lockfile if the user changes
// their settings.
Expand Down Expand Up @@ -951,6 +979,8 @@ struct LockWire {
/// forks in the lockfile so we can recreate them in subsequent resolutions.
#[serde(rename = "resolution-markers", default)]
fork_markers: Vec<MarkerTree>,
#[serde(rename = "supported-markers", default)]
supported_environments: Vec<MarkerTree>,
Copy link
Member Author

@charliermarsh charliermarsh Aug 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am tempted to rename these to: resolution-markers and supported-markers (i.e., rename environment-markers).

/// We discard the lockfile if these options match.
#[serde(default)]
options: ResolverOptions,
Expand All @@ -966,6 +996,7 @@ impl From<Lock> for LockWire {
version: lock.version,
requires_python: lock.requires_python,
fork_markers: lock.fork_markers,
supported_environments: lock.supported_environments,
options: lock.options,
manifest: lock.manifest,
packages: lock.packages.into_iter().map(PackageWire::from).collect(),
Expand Down Expand Up @@ -1005,6 +1036,7 @@ impl TryFrom<LockWire> for Lock {
wire.requires_python,
wire.options,
wire.manifest,
wire.supported_environments,
wire.fork_markers,
)
}
Expand Down
13 changes: 12 additions & 1 deletion crates/uv-resolver/src/resolution/graph.rs
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,18 @@ impl ResolutionGraph {
vec![]
}
ResolverMarkers::Fork(_) => {
panic!("A single fork must be universal");
resolutions
.iter()
.map(|resolution| {
resolution
.markers
.fork_markers()
.expect("A non-forking resolution exists in forking mode")
.clone()
})
// Any unsatisfiable forks were skipped.
.filter(|fork| !fork.is_false())
.collect()
}
}
} else {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Ok(
Lock {
version: 1,
fork_markers: [],
supported_environments: [],
requires_python: None,
options: ResolverOptions {
resolution_mode: Highest,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Ok(
Lock {
version: 1,
fork_markers: [],
supported_environments: [],
requires_python: None,
options: ResolverOptions {
resolution_mode: Highest,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Ok(
Lock {
version: 1,
fork_markers: [],
supported_environments: [],
requires_python: None,
options: ResolverOptions {
resolution_mode: Highest,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Ok(
Lock {
version: 1,
fork_markers: [],
supported_environments: [],
requires_python: None,
options: ResolverOptions {
resolution_mode: Highest,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Ok(
Lock {
version: 1,
fork_markers: [],
supported_environments: [],
requires_python: None,
options: ResolverOptions {
resolution_mode: Highest,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Ok(
Lock {
version: 1,
fork_markers: [],
supported_environments: [],
requires_python: None,
options: ResolverOptions {
resolution_mode: Highest,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Ok(
Lock {
version: 1,
fork_markers: [],
supported_environments: [],
requires_python: None,
options: ResolverOptions {
resolution_mode: Highest,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Ok(
Lock {
version: 1,
fork_markers: [],
supported_environments: [],
requires_python: None,
options: ResolverOptions {
resolution_mode: Highest,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Ok(
Lock {
version: 1,
fork_markers: [],
supported_environments: [],
requires_python: None,
options: ResolverOptions {
resolution_mode: Highest,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Ok(
Lock {
version: 1,
fork_markers: [],
supported_environments: [],
requires_python: None,
options: ResolverOptions {
resolution_mode: Highest,
Expand Down
4 changes: 4 additions & 0 deletions crates/uv-settings/src/settings.rs
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,10 @@ pub struct Options {
#[cfg_attr(feature = "schemars", schemars(skip))]
dev_dependencies: serde::de::IgnoredAny,

#[serde(default, skip_serializing)]
#[cfg_attr(feature = "schemars", schemars(skip))]
environments: serde::de::IgnoredAny,

#[serde(default, skip_serializing)]
#[cfg_attr(feature = "schemars", schemars(skip))]
managed: serde::de::IgnoredAny,
Expand Down
78 changes: 78 additions & 0 deletions crates/uv-workspace/src/environments.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
use std::str::FromStr;

use serde::ser::SerializeSeq;

use pep508_rs::MarkerTree;

#[derive(Debug, Default, Clone, Eq, PartialEq)]
pub struct SupportedEnvironments(Vec<MarkerTree>);

impl SupportedEnvironments {
/// Return the list of marker trees.
pub fn as_markers(&self) -> &[MarkerTree] {
&self.0
}

/// Convert the [`SupportedEnvironments`] struct into a list of marker trees.
pub fn into_markers(self) -> Vec<MarkerTree> {
self.0
}
}

/// Serialize a [`SupportedEnvironments`] struct into a list of marker strings.
impl serde::Serialize for SupportedEnvironments {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the difference from the auto-derive here that we drop empty markers that the user may have written?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We support providing a string or a list of strings -- either is fine.

fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: serde::Serializer,
{
let mut seq = serializer.serialize_seq(Some(self.0.len()))?;
for element in &self.0 {
if let Some(contents) = element.contents() {
seq.serialize_element(&contents)?;
}
}
seq.end()
}
}

/// Deserialize a marker string or list of marker strings into a [`SupportedEnvironments`] struct.
impl<'de> serde::Deserialize<'de> for SupportedEnvironments {
fn deserialize<D>(deserializer: D) -> Result<SupportedEnvironments, D::Error>
where
D: serde::Deserializer<'de>,
{
struct StringOrVecVisitor;

impl<'de> serde::de::Visitor<'de> for StringOrVecVisitor {
type Value = SupportedEnvironments;

fn expecting(&self, formatter: &mut std::fmt::Formatter) -> std::fmt::Result {
formatter.write_str("a string or a list of strings")
}

fn visit_str<E>(self, value: &str) -> Result<Self::Value, E>
where
E: serde::de::Error,
{
let marker = MarkerTree::from_str(value).map_err(serde::de::Error::custom)?;
Ok(SupportedEnvironments(vec![marker]))
}

fn visit_seq<A>(self, mut seq: A) -> Result<Self::Value, A::Error>
where
A: serde::de::SeqAccess<'de>,
{
let mut markers = Vec::new();

while let Some(elem) = seq.next_element::<String>()? {
let marker = MarkerTree::from_str(&elem).map_err(serde::de::Error::custom)?;
markers.push(marker);
}

Ok(SupportedEnvironments(markers))
}
}

deserializer.deserialize_any(StringOrVecVisitor)
}
}
2 changes: 2 additions & 0 deletions crates/uv-workspace/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
pub use environments::SupportedEnvironments;
pub use workspace::{
check_nested_workspaces, DiscoveryOptions, ProjectWorkspace, VirtualProject, Workspace,
WorkspaceError, WorkspaceMember,
};

mod environments;
pub mod pyproject;
pub mod pyproject_mut;
mod workspace;
33 changes: 32 additions & 1 deletion crates/uv-workspace/src/pyproject.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ use serde::{Deserialize, Serialize};
use thiserror::Error;
use url::Url;

use crate::environments::SupportedEnvironments;
use pep440_rs::VersionSpecifiers;
use pypi_types::{RequirementSource, VerbatimParsedUrl};
use uv_git::GitReference;
Expand Down Expand Up @@ -98,19 +99,49 @@ pub struct ToolUv {
"#
)]
pub managed: Option<bool>,
/// The project's development dependencies. Development dependencies will be installed by
/// default in `uv run` and `uv sync`, but will not appear in the project's published metadata.
#[cfg_attr(
feature = "schemars",
schemars(
with = "Option<Vec<String>>",
description = "PEP 508-style requirements, e.g., `ruff==0.5.0`, or `ruff @ https://...`."
)
)]
#[option(
default = r#"[]"#,
value_type = "list[str]",
example = r#"
dev_dependencies = ["ruff==0.5.0"]
"#
)]
pub dev_dependencies: Option<Vec<pep508_rs::Requirement<VerbatimParsedUrl>>>,
/// A list of supported environments against which to resolve dependencies.
///
/// By default, uv will resolve for all possible environments during a `uv lock` operation.
/// However, you can restrict the set of supported environments to improve performance and avoid
/// unsatisfiable branches in the solution space.
#[cfg_attr(
feature = "schemars",
schemars(
with = "Option<Vec<String>>",
description = "A list of environment markers, e.g. `python_version >= '3.6'`."
)
)]
#[option(
default = r#"[]"#,
value_type = "str | list[str]",
example = r#"
# Resolve for macOS, but not for Linux or Windows.
environments = ["sys_platform == 'darwin'"]
"#
)]
pub environments: Option<SupportedEnvironments>,
#[cfg_attr(
feature = "schemars",
schemars(
with = "Option<Vec<String>>",
description = "PEP 508 style requirements, e.g. `ruff==0.5.0`, or `ruff @ https://...`."
description = "PEP 508-style requirements, e.g. `ruff==0.5.0`, or `ruff @ https://...`."
)
)]
pub override_dependencies: Option<Vec<pep508_rs::Requirement<VerbatimParsedUrl>>>,
Expand Down
Loading
Loading