transformers/tests
Joao Gante b0f0c61899
Add SynthID (watermerking by Google DeepMind) (#34350)
* Add SynthIDTextWatermarkLogitsProcessor

* esolving comments.

* Resolving comments.

* esolving commits,

* Improving SynthIDWatermark tests.

* switch to PT version

* detector as pretrained model + style

* update training + style

* rebase

* Update logits_process.py

* Improving SynthIDWatermark tests.

* Shift detector training to wikitext negatives and stabilize with lower learning rate.

* Clean up.

* in for 7B

* cleanup

* upport python 3.8.

* README and final cleanup.

* HF Hub upload and initiaze.

* Update requirements for synthid_text.

* Adding SynthIDTextWatermarkDetector.

* Detector testing.

* Documentation changes.

* Copyrights fix.

* Fix detector api.

* ironing out errors

* ironing out errors

* training checks

* make fixup and make fix-copies

* docstrings and add to docs

* copyright

* BC

* test docstrings

* move import

* protect type hints

* top level imports

* watermarking example

* direct imports

* tpr fpr meaning

* process_kwargs

* SynthIDTextWatermarkingConfig docstring

* assert -> exception

* example updates

* no immutable dict (cant be serialized)

* pack fn

* einsum equivalent

* import order

* fix test on gpu

* add detector example

---------

Co-authored-by: Sumedh Ghaisas <sumedhg@google.com>
Co-authored-by: Marc Sun <marc@huggingface.co>
Co-authored-by: sumedhghaisas2 <138781311+sumedhghaisas2@users.noreply.github.com>
Co-authored-by: raushan <raushan@huggingface.co>
2024-10-23 21:18:52 +01:00
..
agents Decorator for easier tool building (#33439) 2024-09-18 11:07:51 +02:00
benchmark [Test refactor 1/5] Per-folder tests reorganization (#15725) 2022-02-23 15:46:28 -05:00
bettertransformer Fixed malapropism error (#26660) 2023-10-09 11:04:57 +02:00
deepspeed Trainer - deprecate tokenizer for processing_class (#32385) 2024-10-02 14:08:46 +01:00
extended [tests] skip tests for xpu (#33553) 2024-09-19 19:28:04 +01:00
fixtures Implementation of SuperPoint and AutoModelForKeypointDetection (#28966) 2024-03-19 14:43:02 +00:00
fsdp 🚨🚨🚨 Update min version of accelerate to 0.26.0 (#32627) 2024-08-20 11:42:36 +02:00
generation Add SynthID (watermerking by Google DeepMind) (#34350) 2024-10-23 21:18:52 +01:00
models skip test_pipeline_depth_estimation temporarily (#34316) 2024-10-23 17:27:51 +02:00
optimization fix: Fixed the 1st argument name in classmethods (#31907) 2024-07-11 12:11:50 +01:00
peft_integration [PEFT] Support low_cpu_mem_usage option for PEFT loading adapters (#33725) 2024-10-03 16:15:36 +02:00
pipelines Add option for running ffmpeg_microphone_live as a background process (#32838) 2024-10-22 15:56:41 +02:00
quantization Add GGUF for starcoder2 (#34094) 2024-10-14 10:22:49 +02:00
repo_utils Refactor CI: more explicit (#30674) 2024-08-30 18:17:25 +02:00
sagemaker Trainer - deprecate tokenizer for processing_class (#32385) 2024-10-02 14:08:46 +01:00
tokenization Fix for slow the bug tokenizer adding spaces to single id decodes (#32564) 2024-09-18 12:32:02 +02:00
trainer Enable users to use their own loss functions + deal with prefetching for grad accum (#34198) 2024-10-17 17:01:56 -04:00
utils Attn implementation for composite models (#32238) 2024-10-22 06:54:44 +02:00
__init__.py GPU text generation: mMoved the encoded_prompt to correct device 2020-01-06 15:11:12 +01:00
test_backbone_common.py Align backbone stage selection with out_indices & out_features (#27606) 2023-12-20 18:33:17 +00:00
test_configuration_common.py Refactor: Removed un-necessary object base class (#32230) 2024-07-26 10:33:02 +02:00
test_feature_extraction_common.py Split common test from core tests (#24284) 2023-06-15 07:30:24 -04:00
test_image_processing_common.py Add DetrImageProcessorFast (#34063) 2024-10-21 09:05:05 -04:00
test_image_transforms.py fix: center_crop occasionally outputs off-by-one dimension matrix (#30934) 2024-05-21 13:56:52 +01:00
test_modeling_common.py T5 compile compatibilty (#34089) 2024-10-22 08:23:53 +02:00
test_modeling_flax_common.py add sdpa to ViT [follow up of #29325] (#30555) 2024-05-16 10:56:11 +01:00
test_modeling_tf_common.py [TF] Fix Tensorflow XLA Generation on limited seq_len models (#33903) 2024-10-05 16:20:50 +02:00
test_pipeline_mixin.py Sync video classification pipeline with huggingface_hub spec (#34288) 2024-10-22 13:33:49 +01:00
test_processing_common.py Uniformize model processors (#31368) 2024-10-02 10:41:08 +02:00
test_sequence_feature_extraction_common.py Fix typo (#25966) 2023-09-05 10:12:25 +02:00
test_tokenization_common.py Trainer - deprecate tokenizer for processing_class (#32385) 2024-10-02 14:08:46 +01:00