Skip to content

README.md

Latest commit

 

History

History
128 lines (113 loc) · 3.51 KB

README.md

File metadata and controls

128 lines (113 loc) · 3.51 KB
Raw

Instructions for GLIGEN

Installation

Move to the GLIGEN folder:

cd GLIGEN

Then run:

uv sync --index-strategy unsafe-best-match

Training

For setting up the training data, please follow the instructions in the official GLIGEN repository.

Text grounding (Per instance bounding box and caption)

Default $\epsilon$-supervision

uv run main_with_periodic_eval.py \
  --name=train_text_grounding_with_eval \
  --yaml_file=configs/GoldG+SBU+CC3M+O365_box_text_with_eval.yaml \
  --batch_size=16 \
  --workers=8 \
  --total_iters=200000 \
  --save_every_iters=1000 \
  --seed=123

Default $x_0$-supervision

uv run main_x0_with_periodic_eval.py \
  --name=train_text_grounding_x0_with_eval \
  --yaml_file=configs/GoldG+SBU+CC3M+O365_box_text_with_eval.yaml \
  --batch_size=16 \
  --workers=8 \
  --total_iters=200000 \
  --save_every_iters=1000 \
  --seed=123

Default $v$-supervision

uv run main_v_with_periodic_eval.py \
  --name=train_text_grounding_v_with_eval \
  --yaml_file=configs/GoldG+SBU+CC3M+O365_box_text_with_eval.yaml \
  --batch_size=16 \
  --workers=8 \
  --total_iters=200000 \
  --save_every_iters=1000 \
  --seed=123

Text and image grounding (Per instance bounding box, appearance image, caption)

Default $\epsilon$-supervision

uv run main_with_periodic_eval.py \
  --name=train_text_image_grounding_with_eval \
  --yaml_file=configs/GoldG+SBU+CC3M+O365_box_text_image_with_eval.yaml \
  --batch_size=16 \
  --workers=8 \
  --total_iters=200000 \
  --save_every_iters=1000 \
  --seed=123

Default $x_0$-supervision

uv run main_x0_with_periodic_eval.py \
  --name=train_text_image_grounding_x0_with_eval \
  --yaml_file=configs/GoldG+SBU+CC3M+O365_box_text_image_with_eval.yaml \
  --batch_size=16 \
  --workers=8 \
  --total_iters=200000 \
  --save_every_iters=1000 \
  --seed=123

Default $v$-supervision

uv run main_v_with_periodic_eval.py \
  --name=train_text_image_grounding_v_with_eval \
  --yaml_file=configs/GoldG+SBU+CC3M+O365_box_text_image_with_eval.yaml \
  --batch_size=16 \
  --workers=8 \
  --total_iters=200000 \
  --save_every_iters=1000 \
  --seed=123

Evaluation

First download these json files:

To generate and evaluate the control models use the followings:

Text grounding (Per instance bounding box and caption)

uv run generate_box_eval_samples.py \
  --ckpt_path OUTPUT/<ckpt_path> \
  --data_path <path_to_coco_2017_validation_set> \
  --metadata_path cocoval2017_position_eval.json \
  --image_size 512 \
  --batch_size 8 \
  --num_workers 8 \
  --ddim_steps 50 \
  --cfg_scale 7.5 \
  --save_path <save_path>

Text and image grounding (Per instance bounding box, appearance image, caption)

uv run generate_box_image_eval_samples.py \
  --ckpt_path OUTPUT/<ckpt_path> \
  --data_path <path_to_coco_2017_validation_set> \
  --metadata_path cocoval2017_text_image_grounding_position_eval.json \
  --coco_subet_json instances_val2017.json \
  --image_size 512 \
  --batch_size 8 \
  --num_workers 8 \
  --ddim_steps 50 \
  --cfg_scale 7.5 \
  --save_path <save_path>