Code

Add image text to text pipeline#34170

Code

Merged

yonigozlan merged 45 commits intohuggingface:mainhuggingface/transformers:mainfrom yonigozlan:add-image-text-to-text-pipelineyonigozlan/transformers:add-image-text-to-text-pipelineCopy head branch name to clipboardOct 31, 2024

+988-33

Commits

Commits on Oct 31, 2024

• Standardize image-text-to-text-models-output

  Show description for de37f38
  yonigozlan

  committed
  de37f38

  Copy full SHA for de37f38

• nit var name post_process_image_text_to_text udop

  yonigozlan

  committed
  a7e56fa

  Copy full SHA for a7e56fa

• nit fix deprecation warnings

  yonigozlan

  committed
  9161a52

  Copy full SHA for 9161a52

• Add image-text-to-text pipeline

  yonigozlan

  committed
  04d4f77

  Copy full SHA for 04d4f77

• add support for image url in chat template for pipeline

  yonigozlan

  committed
  0aabcb2

  Copy full SHA for 0aabcb2

• Reformat to be fully compatible with chat templates

  yonigozlan

  committed
  5189f16

  Copy full SHA for 5189f16

• Add tests chat template

  yonigozlan

  committed
  a0c9075

  Copy full SHA for a0c9075

• Fix imports and tests

  yonigozlan

  committed
  726933f

  Copy full SHA for 726933f

• Add pipeline tag

  yonigozlan

  committed
  b1d7a34

  Copy full SHA for b1d7a34

• change logic handling of single prompt ans multiple images

  yonigozlan

  committed
  f92628c

  Copy full SHA for f92628c

• add pipeline mapping to models

  yonigozlan

  committed
  fa25411

  Copy full SHA for fa25411

• fix batched inference

  yonigozlan

  committed
  9fe26c7

  Copy full SHA for 9fe26c7

• fix tests

  yonigozlan

  committed
  316cf7d

  Copy full SHA for 316cf7d

• Add manual batching for preprocessing

  yonigozlan

  committed
  c633b4b

  Copy full SHA for c633b4b

• Fix outputs with nested images

  yonigozlan

  committed
  d6598da

  Copy full SHA for d6598da

• Add support for all common processing kwargs

  yonigozlan

  committed
  c8e5802

  Copy full SHA for c8e5802

• Add default padding when multiple text inputs (batch size>1)

  yonigozlan

  committed
  20cdd5a

  Copy full SHA for 20cdd5a

• nit change version deprecation warning

  yonigozlan

  committed
  5bc43be

  Copy full SHA for 5bc43be

• Add support for text only inference

  yonigozlan

  committed
  6cccf5f

  Copy full SHA for 6cccf5f

• add chat_template warnings

  yonigozlan

  committed
  ba8f85f

  Copy full SHA for ba8f85f

• Add pipeline tests and add copied from post process function

  yonigozlan

  committed
  00174e8

  Copy full SHA for 00174e8

• Fix batched pipeline tests

  yonigozlan

  committed
  8a65ea4

  Copy full SHA for 8a65ea4

• nit

  yonigozlan

  committed
  da05987

  Copy full SHA for da05987

• Fix pipeline tests blip2

  yonigozlan

  committed
  1f2dafb

  Copy full SHA for 1f2dafb

• remove unnecessary max_new_tokens

  yonigozlan

  committed
  d66e523

  Copy full SHA for d66e523

• revert processing kosmos2 and remove unnecessary max_new_tokens

  yonigozlan

  committed
  5056aa5

  Copy full SHA for 5056aa5

• fix pipeline tests idefics

  yonigozlan

  committed
  fe7e75d

  Copy full SHA for fe7e75d

• Force try loading processor if pipeline supports it

  yonigozlan

  committed
  b866c27

  Copy full SHA for b866c27

• revert load_processor change

  yonigozlan

  committed
  3118dac

  Copy full SHA for 3118dac

• hardcode loading only processor

  yonigozlan

  committed
  065542a

  Copy full SHA for 065542a

• remove unnecessary try except

  yonigozlan

  committed
  7f583df

  Copy full SHA for 7f583df

• skip imagetexttotext tests for kosmos2 as tiny model causes problems

  yonigozlan

  committed
  aad9ad4

  Copy full SHA for aad9ad4

• Make code clearer

  yonigozlan

  committed
  e227b83

  Copy full SHA for e227b83

• Address review comments

  yonigozlan

  committed
  8f370f4

  Copy full SHA for 8f370f4

• remove preprocessing logic from pipeline

  yonigozlan

  committed
  c82fe29

  Copy full SHA for c82fe29

• fix fuyu

  yonigozlan

  committed
  7e1fb07

  Copy full SHA for 7e1fb07

• add BC resize fuyu

  yonigozlan

  committed
  f581eaa

  Copy full SHA for f581eaa

• Move post_process_image_text_to_text to ProcessorMixin

  yonigozlan

  committed
  4eda963

  Copy full SHA for 4eda963

• add guard in post_process

  yonigozlan

  committed
  0263221

  Copy full SHA for 0263221

• fix zero shot object detection pipeline

  yonigozlan

  committed
  45c1706

  Copy full SHA for 45c1706

• add support for generator input in pipeline

  yonigozlan

  committed
  2e69b97

  Copy full SHA for 2e69b97

• nit

  yonigozlan

  committed
  58a6fb8

  Copy full SHA for 58a6fb8

• change default image-text-to-text model to llava onevision

  yonigozlan

  committed
  66c017c

  Copy full SHA for 66c017c

• fix owlv2 size dict

  yonigozlan

  committed
  5772312

  Copy full SHA for 5772312

• Change legacy deprecation warning to only show when True

  yonigozlan

  committed
  61cc576

  Copy full SHA for 61cc576