Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(weave): Add util to strip EXIF Metadata (and save 30-50% filesize per img) #2863

Merged
merged 6 commits into from
Nov 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,9 @@ repos:
files: ^examples/.*\.ipynb$
language: system
entry: jupyter nbconvert --ClearOutputPreprocessor.enabled=True --inplace
- id: strip-exif
name: Strip EXIF data
entry: python scripts/strip_exif.py
language: python
types: [image]
additional_dependencies: ["pillow"]
Binary file modified docs/docs/guides/core-types/imgs/audio-trace.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/docs/guides/core-types/imgs/cat-pumpkin-trace.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/docs/guides/integrations/imgs/anthropic_model.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/docs/guides/integrations/imgs/anthropic_ops.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/docs/guides/integrations/imgs/anthropic_tool.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/docs/guides/integrations/imgs/anthropic_trace.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/docs/guides/integrations/imgs/cerebras_calls.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/docs/guides/integrations/imgs/cerebras_model.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/docs/guides/integrations/imgs/cohere_model.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/docs/guides/integrations/imgs/cohere_ops.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/docs/guides/integrations/imgs/cohere_trace.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/docs/guides/integrations/imgs/dspy/dspy_trace.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/docs/guides/integrations/imgs/langchain_eval.png
Binary file modified docs/docs/guides/integrations/imgs/langchain_model.png
Binary file modified docs/docs/guides/integrations/imgs/langchain_trace.png
Binary file modified docs/docs/guides/integrations/imgs/litellm_gif.gif
Binary file modified docs/docs/guides/integrations/imgs/llamaindex_model.png
Binary file modified docs/docs/guides/integrations/imgs/llamaindex_rag.png
Binary file modified docs/docs/guides/integrations/imgs/mistral_model.png
Binary file modified docs/docs/guides/integrations/imgs/mistral_ops.png
Binary file modified docs/docs/guides/integrations/imgs/mistral_trace.png
Binary file modified docs/docs/guides/integrations/imgs/notdiamond/api-keys.png
Binary file modified docs/docs/guides/integrations/imgs/openai-model.png
Binary file modified docs/docs/guides/integrations/imgs/openai-pokedex.png
Binary file modified docs/docs/guides/integrations/imgs/openai.png
Binary file modified docs/docs/guides/integrations/imgs/simple_llamaindex.png
Binary file modified docs/docs/guides/tracking/imgs/feedback_call_header.png
Binary file modified docs/docs/guides/tracking/imgs/feedback_calls.png
Binary file modified docs/docs/guides/tracking/imgs/feedback_tab.png
Binary file modified docs/docs/guides/tracking/imgs/feedback_use.png
Binary file modified docs/docs/media/codegen/codegen_trace.png
Binary file modified docs/docs/media/codegen/eval_dash.png
Binary file modified docs/docs/media/codegen/eval_trace.png
Binary file modified docs/docs/media/dspy_optimization/1.png
Binary file modified docs/docs/media/dspy_optimization/2.png
Binary file modified docs/docs/media/dspy_optimization/3.png
Binary file modified docs/docs/media/dspy_optimization/4.png
Binary file modified docs/docs/media/dspy_optimization/5.png
Binary file modified docs/docs/media/intro/1.png
Binary file modified docs/docs/media/intro/10.png
Binary file modified docs/docs/media/intro/2.png
Binary file modified docs/docs/media/intro/3.png
Binary file modified docs/docs/media/intro/4.png
Binary file modified docs/docs/media/intro/5.png
Binary file modified docs/docs/media/intro/6.png
Binary file modified docs/docs/media/intro/7.png
Binary file modified docs/docs/media/intro/8.png
Binary file modified docs/docs/media/intro/9.png
Binary file modified docs/docs/media/multi-agent-structured-output/0.png
Binary file modified docs/docs/media/multi-agent-structured-output/1.png
Binary file modified docs/docs/media/multi-agent-structured-output/2.png
Binary file modified docs/docs/media/multi-agent-structured-output/3.png
Binary file modified docs/docs/media/pii/encrypt.png
Binary file modified docs/docs/media/pii/redact.png
Binary file modified docs/docs/media/pii/replace.png
Binary file modified docs/docs/media/summarization/dataset.png
Binary file modified docs/docs/media/summarization/eval_dash.png
Binary file modified docs/docs/media/summarization/model.png
Binary file modified docs/docs/media/summarization/summarization_trace.png
Binary file modified docs/static/img/call_edit_screenshot.png
Binary file modified docs/static/img/docusaurus-social-card.jpg
Binary file modified docs/static/img/docusaurus.png
Binary file modified docs/static/img/evals-hero.png
Binary file modified docs/static/img/export_modal.png
Binary file modified docs/static/img/logo-large-padded.png
Binary file modified docs/static/img/logo-large.png
Binary file modified docs/static/img/screenshots/basic_call.png
Binary file modified docs/static/img/screenshots/calls_filter.png
Binary file modified docs/static/img/screenshots/calls_macro.png
Binary file modified docs/static/img/tutorial-model_invoke3.png
Binary file modified docs/static/img/tutorial-model_re-use.png
Binary file modified docs/static/img/tutorial_trace_1.png
Binary file modified docs/static/img/tutorial_tracing_2_call_uri.png
Binary file modified docs/static/img/tutorial_tracing_2_nested_dinos.png
Binary file modified docs/static/img/weave-hero.png
Binary file modified docs/static/weave-ui-example.jpg
75 changes: 75 additions & 0 deletions scripts/strip_exif.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
#!/usr/bin/env python3
import sys
from pathlib import Path

from PIL import Image

IMAGE_EXTENSIONS = {".jpg", ".jpeg", ".png", ".tiff", ".tif", ".webp"}


def strip_exif(image_path: Path) -> bool:
"""Strip EXIF data from an image file if present.

Args:
image_path: Path to the image file

Returns:
bool: True if successful, False if failed
"""
try:
with Image.open(image_path) as img:
if not img.getexif():
return True

# Convert RGBA to RGB if needed (some formats don't support RGBA)
if img.mode == "RGBA":
img = img.convert("RGB")

# Create a new image without EXIF
data = list(img.getdata())
img_without_exif = Image.new(img.mode, img.size)
img_without_exif.putdata(data)
img_without_exif.save(image_path, format=img.format)
print(f"Stripped EXIF from: {image_path}")
return True

except Exception as e:
print(f"Error processing {image_path}: {e}", file=sys.stderr)
return False


def process_path(path: Path) -> None:
"""Process a single path, which may be a file or directory."""
if path.is_file():
if path.suffix.lower() in IMAGE_EXTENSIONS:
strip_exif(path)
elif path.is_dir():
found_images = False
for img_path in path.rglob("*"):
if img_path.suffix.lower() in IMAGE_EXTENSIONS:
found_images = True
strip_exif(img_path)
if not found_images:
print(f"No images found in directory: {path}")


def main() -> None:
"""Main function to handle CLI usage."""
if len(sys.argv) < 2:
print(
"Usage: strip_exif.py <file_or_directory> [file_or_directory ...]",
file=sys.stderr,
)
sys.exit(1)

# Handle multiple paths
for path_str in sys.argv[1:]:
path = Path(path_str)
if not path.exists():
print(f"Path does not exist: {path}", file=sys.stderr)
continue
process_path(path)


if __name__ == "__main__":
main()
Binary file modified sdks/node/logs.png
Binary file modified weave_query/testdata/spring-lamb.jpg
Loading