transformers/tests
Arthur 9cccb3a838
[Persimmon] Add support for persimmon (#26042)
* intiial commit

* updates

* nits

* update conversion script

* update conversion script

* use path to load

* add tips etc

* some modeling logic

* modeling update

* more nits

* nits

* normal layer norm

* update config and doc

* nits

* update doc remove unused

* update

* fix inits and stuff

* fixup

* revert wrong changes

* updates

* more nits

* add default config values to the configuration file

* fixup happy

* update

* 2 tests left

* update readmes

* more nits

* slow test and more documentation

* update readme

* fix licences

* styling

* use fast if possible when saving tokenizer

* remove todo

* remove tokenization tests

* small last nits

* Apply suggestions from code review

Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>

* nits to skip the timout doctest

* fix integration test

* fix test

* update eos token

* update to allow fast tokenization

* styling

* fix codeLlama as well for the update post processor

* Apply suggestions from code review

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

* add more copied from statements

* update

* doc passes doctest

* remove `# final layer norm?`

* change docstring prompot

* update

* Update README.md

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* don't doctest the conversion script as it requires more packages

* don't init a model in the config

* oups

* fix doctest

---------

Co-authored-by: Matt <Rocketknight1@users.noreply.github.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-09-12 11:33:27 +02:00
..
benchmark
bettertransformer Add methods to PreTrainedModel to use PyTorch's BetterTransformer (#21259) 2023-04-27 11:03:42 +02:00
deepspeed deepspeed resume from ckpt fixes and adding support for deepspeed optimizer and HF scheduler (#25863) 2023-09-05 22:31:20 +05:30
extended [tests] switch to torchrun (#22712) 2023-04-12 08:25:45 -07:00
fixtures
generation Generate: general test for decoder-only generation from inputs_embeds (#25687) 2023-08-23 19:17:01 +01:00
models [Persimmon] Add support for persimmon (#26042) 2023-09-12 11:33:27 +02:00
optimization
peft_integration [PEFT] Fix PeftConfig save pretrained when calling add_adapter (#25738) 2023-08-25 08:19:11 +02:00
pipelines [CI] Fix red CI and ERROR failed should show (#25995) 2023-09-05 20:16:00 +02:00
quantization modify context length for GPTQ + version bump (#25899) 2023-09-06 11:45:47 -04:00
repo_utils Document check copies (#25291) 2023-08-04 14:56:29 +02:00
sagemaker Avoid invalid escape sequences, use raw strings (#22936) 2023-04-25 09:17:56 -04:00
tokenization [ PreTrainedTokenizerFast] Keep properties from fast tokenizer (#25053) 2023-07-25 18:45:01 +02:00
tools Add support for for loops in python interpreter (#24429) 2023-06-26 09:58:14 -04:00
trainer Patch with accelerate xpu (#25714) 2023-09-05 15:41:42 +01:00
utils Fix test_load_img_url_timeout (#25976) 2023-09-05 11:34:28 +02:00
__init__.py
test_backbone_common.py Add ViTDet (#25524) 2023-08-29 10:03:52 +01:00
test_configuration_common.py Deal with nested configs better in base class (#25237) 2023-08-04 14:56:09 +02:00
test_configuration_utils.py Deal with nested configs better in base class (#25237) 2023-08-04 14:56:09 +02:00
test_feature_extraction_common.py Split common test from core tests (#24284) 2023-06-15 07:30:24 -04:00
test_feature_extraction_utils.py Split common test from core tests (#24284) 2023-06-15 07:30:24 -04:00
test_image_processing_common.py Input data format (#25464) 2023-08-16 17:45:02 +01:00
test_image_processing_utils.py Run hub tests (#24807) 2023-07-13 15:25:45 -04:00
test_image_transforms.py Add input_data_format argument, image transforms (#25462) 2023-08-11 15:09:31 +01:00
test_modeling_common.py [resize_embedding] Introduce pad_to_multiple_of and guidance (#25088) 2023-08-17 17:00:32 +02:00
test_modeling_flax_common.py Split common test from core tests (#24284) 2023-06-15 07:30:24 -04:00
test_modeling_flax_utils.py Split common test from core tests (#24284) 2023-06-15 07:30:24 -04:00
test_modeling_tf_common.py Skip test_onnx_runtime_optimize for now (#25560) 2023-08-17 11:23:16 +02:00
test_modeling_tf_utils.py Split common test from core tests (#24284) 2023-06-15 07:30:24 -04:00
test_modeling_utils.py Skip warning if tracing with dynamo (#25581) 2023-09-08 21:13:33 +02:00
test_pipeline_mixin.py Add Text-To-Speech pipeline (#24952) 2023-08-17 17:34:47 +01:00
test_sequence_feature_extraction_common.py Fix typo (#25966) 2023-09-05 10:12:25 +02:00
test_tokenization_common.py [split_special_tokens] Add support for split_special_tokens argument to encode (#25081) 2023-08-18 13:26:27 +02:00
test_tokenization_utils.py Split common test from core tests (#24284) 2023-06-15 07:30:24 -04:00