Joao Gante
0d33381fad
Tag tests as slow ⌛ ( #21537 )
...
begone slow tests
2023-02-09 14:46:15 +00:00
Victor Sonck
3a726777ca
Fix ClearML Integration to run in ClearML pipelines and external Tasks. ( #21531 )
...
* Added clearml pipeline fix for when task is already initialized
* Correctly initialize
2023-02-09 09:28:55 -05:00
Motoki Wu
17109ecfb8
Fix missing unfinished_sequences ( #21529 )
...
fix missing unfinished_sequences
2023-02-09 09:06:22 -05:00
Joao Gante
2edf9a857b
Generate: TF .generate() can now be exported with dynamic length ( #21474 )
2023-02-09 12:52:30 +00:00
Joao Gante
e69f9715eb
Generate: make TF .generate() signature == PT .generate() signature ( #21525 )
2023-02-09 11:10:13 +00:00
Yih-Dar
c35bb6de54
Add __len__ method to _LazyAutoMapping ( #21522 )
...
Add `__len__` method to `_LazyAutoMapping`
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-02-08 20:35:14 +01:00
Motoki Wu
9960506cbe
Fix multiple eos_token_ids in model.generate(...) ( #21461 )
...
* add tests with multiple eos_token_ids
* make math.prod instead of sum
* make fixup
* fix long and also use np.prod since math.prod does not exist <python 3.8
* make fixup
* add prod util
* use prod util instead of np.prod
* make fixup
* previous .long location
* use tensor ops
* remove prod
* remove prod
* update device
* make fixup
* fix none
2023-02-08 13:48:46 -05:00
Nicolas Patry
06d940efc3
Fixing backward compatiblity image_processor in pipeline. ( #21513 )
2023-02-08 19:36:20 +01:00
Stas Bekman
8ea994d3c5
[tests] add missing report_to none ( #21505 )
...
[tests] report_to none
2023-02-08 09:32:40 -08:00
Thomas Wang
98d5b72727
Update OPT conversion script to work for OPT-IML ( #21519 )
2023-02-08 18:31:10 +01:00
Matthijs Hollemans
fe616f35c8
no more dummies for speech processors ( #21517 )
2023-02-08 11:41:54 -05:00
Joao Gante
1d9c26a4b8
Generate: TF compute_transition_scores ( #21341 )
2023-02-08 16:36:43 +00:00
Stefan Schweter
d3046dad80
[Doc] Minor URL fixes in PyTorch Text Classification Readme ( #21511 )
...
docs: fix some references in PyTorch text classification readme
2023-02-08 09:39:52 -05:00
dependabot[bot]
e024cd715e
Bump cryptography from 36.0.2 to 39.0.1 in /examples/research_projects/decision_transformer ( #21507 )
...
Bump cryptography in /examples/research_projects/decision_transformer
Bumps [cryptography](https://github.com/pyca/cryptography ) from 36.0.2 to 39.0.1.
- [Release notes](https://github.com/pyca/cryptography/releases )
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst )
- [Commits](https://github.com/pyca/cryptography/compare/36.0.2...39.0.1 )
---
updated-dependencies:
- dependency-name: cryptography
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-08 09:25:06 -05:00
Guillaume Klein
ca905ba28e
Exclude the madeup words from M2M100Tokenizer.vocab_size ( #20976 )
2023-02-08 09:19:06 -05:00
Katie Le
cc1d0685b3
Wrap RemBert integration test forward passes with torch.no_grad() ( #21503 )
...
added with torch.no_grad() to the integration tests and applied make style
Co-authored-by: Bibi <Bibi@katies-mac.local>
2023-02-08 14:00:52 +01:00
Sylvain Gugger
5b67ab9924
Fix import in Accelerate for find_exec_bs ( #21501 )
2023-02-07 16:45:59 -05:00
Prajwal Kailas
eb1771ef1f
Check for mapping/dict in distributed_concat function ( #21500 )
...
check for mapping/dict in distributed_concat function
Co-authored-by: prajwal967 <user.email>
2023-02-07 16:45:37 -05:00
Stefan Schweter
7e51a441e4
Add XLM-V to Model Doc ( #21498 )
...
* doc: introduce new section for XLM-V model
* doc: mention more details for XLM-V integration
* docs: paper abstract in italics, model identifier for base model added
* doc: mention new XLM-V support
* auto: add XLM-V mapping
* doc: run make fix-copies ;)
2023-02-07 16:43:19 -05:00
Adrian Sager La Ganga
a3034c7004
Add inverse sqrt learning rate scheduler ( #21495 )
...
* added inverse sqrt lr scheduler
* Updated get_scheduler in src/transformers/optimization.py
* Updated src/transformers/__init__.py
* Added inverse sqrt lr scheduler test
* Updated docs/source/en/main_classes/optimizer_schedules.mdx
* Ran style and quality scripts
* Fix get_inverse_sqrt_schedule docstring
* Comment implementation URL
2023-02-07 15:00:50 -05:00
Stas Bekman
b9af152efb
[tokenizer] sanitize saved config ( #21483 )
...
* [tokenizer] sanitize saved config
* rm config["name_or_path"] test
2023-02-07 10:51:45 -08:00
Sylvain Gugger
67d074874d
Cleanup quality ( #21493 )
...
* Remove mentions of flake8/isort
* Clean up inits
* Deall with all other inits
* Last special rule for dummy files
2023-02-07 12:27:31 -05:00
raghavanone
571fa585b6
Add limit_all_gathers option to fsdp_config and fix forward_prefetch bug ( #21489 )
...
* Add limit_all_gathers option to fsdp_config and fix forward_prefetch bug
* Fix black issue
* Fix ruff failure
* Incorporate PR feedbacks
* Incorporate PR feedbacks
* Incorporate PR feedbacks
2023-02-07 12:27:06 -05:00
Yih-Dar
479322bfaa
A new test to check config attributes being used ( #21453 )
...
* Add a new test to check config attributes being used
* Add a new test to check config attributes being used
* Add a new test to check config attributes being used
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Apply suggestions
* Update allowed cases - part 1
* Update allowed cases - part 2
* final
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-02-07 17:49:30 +01:00
Arthur
9e7f84a556
[OPT] Adds GPT2TokenizerFast to the list of tokenizer to use for OPT. ( #20823 )
...
* Add ("opt", ("GPT2Tokenizer", "GPT2TokenizerFast" if is_tokenizers_available() else None)),
* skip failing test
* Add ("opt", ("GPT2Tokenizer", "GPT2TokenizerFast" if is_tokenizers_available() else None)),
* skip failing test
2023-02-07 17:35:28 +01:00
raghavanone
8a303f527f
Sanity check the type of id2label and label2id arguments of from_pretrained for TokenClassification models ( #21490 )
...
* Sanity check the type of id2label and label2id arguments of from_pretrained for TokenClassification models
* Incorporate PR feedbacks
* Incorporate PR feedbacks
2023-02-07 10:44:43 -05:00
Matt
28ec07d8ad
Typos/fixes to link syntax ( #21450 )
...
* Typos/fixes to link syntax
* Trying section headers
* Add header formatting for Rule #3
2023-02-07 15:19:19 +00:00
Jeroen Van Der Donckt
bbe98ea9c3
🖊️ fix typo in pytorch semantic segmentation readme ( #21492 )
2023-02-07 09:39:24 -05:00
Iulian Taiatu
8581fbaa6d
changed "ot" to "to" ( #21488 )
2023-02-07 09:31:32 -05:00
Younes Belkada
fa0ae17958
[Doc] Fix int8 docs ( #21487 )
...
fix int8 docs
2023-02-07 15:09:27 +01:00
Joao Gante
1e4cf8bb44
Generate: TF can now generate from embeddings in encoder-decoder models ( #21475 )
2023-02-07 11:18:23 +00:00
Arthur
12eb528b5a
[CI ] Remove past in favor of pat_key_values ( #21443 )
...
* fix past renamed to past_key_value
* update more `past`that were ski^êd
* fixup
* remove changes made to rag
* refactor `_reorder_cache` to use `past_key_values`
* fix git `prepare_inputs_for_generation` to pass tests when false is needed in use_cache
2023-02-07 09:51:35 +01:00
Sylvain Gugger
5b49376202
Deprecate parallelize API ( #21448 )
...
* Deprecate parallelize API
* Add documentation
* Fix copies
2023-02-06 19:39:13 -05:00
Sylvain Gugger
cc8407522a
Fix epoch number when resuming training ( #21478 )
2023-02-06 19:34:34 -05:00
dependabot[bot]
35f93f299f
Bump oauthlib from 3.2.1 to 3.2.2 in /examples/research_projects/decision_transformer ( #21481 )
...
Bump oauthlib in /examples/research_projects/decision_transformer
Bumps [oauthlib](https://github.com/oauthlib/oauthlib ) from 3.2.1 to 3.2.2.
- [Release notes](https://github.com/oauthlib/oauthlib/releases )
- [Changelog](https://github.com/oauthlib/oauthlib/blob/master/CHANGELOG.rst )
- [Commits](https://github.com/oauthlib/oauthlib/compare/v3.2.1...v3.2.2 )
---
updated-dependencies:
- dependency-name: oauthlib
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-06 18:27:14 -05:00
Sylvain Gugger
6f79d26442
Update quality tooling for formatting ( #21480 )
...
* Result of black 23.1
* Update target to Python 3.7
* Switch flake8 to ruff
* Configure isort
* Configure isort
* Apply isort with line limit
* Put the right black version
* adapt black in check copies
* Fix copies
2023-02-06 18:10:56 -05:00
lewtun
b7bb2b59f7
Add tips for generation with Int8 models ( #21424 )
...
* Add tips for generation with Int8 models
* Empty commit to trigger CI
* Apply suggestions from code review
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
* Update docs/source/en/perf_infer_gpu_one.mdx
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
---------
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-02-06 20:25:40 +01:00
Joao Gante
10056d898e
OPT: BLIP2-ready prepare_inputs_for_generation ( #21477 )
2023-02-06 18:19:17 +00:00
Nolwenn Bernard
baf4bacb1f
[i18n-fr] Translate index page to French ( #21458 )
...
* Translate index page to French
* Fix indent
* Fix toctree
* Replace missing file by in_translation
* Add index
* Update docs/source/fr/index.mdx
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
---------
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-02-06 12:25:49 -05:00
Stas Bekman
3b9a1dc132
[examples] improve block_size warning message ( #21463 )
2023-02-06 08:36:12 -08:00
Nicolas Patry
4435c7f52c
Removing more_itertools dependency. ( #21473 )
...
* Removing `more_itertools` dependency.
* Update examples/research_projects/vqgan-clip/requirements.txt
2023-02-06 17:33:20 +01:00
Joao Gante
4943331015
Generate: TF can now accept custom logits processors ( #21454 )
2023-02-06 15:44:47 +00:00
Matthijs Hollemans
e215e6ded2
make SpeechT5 doc examples deterministic ( #21470 )
...
* make doc examples deterministic
* add IGNORE_RESULT
2023-02-06 15:43:55 +01:00
Kaustubh Dhole
182afb7dc6
Fixed RAG script which was failing on dummy example ( #21416 )
...
* do not use prefix="val" for test
The dummy example fails when test_epoch_end is called. The prefix="test" should be dynamic in the log metrics too.
* Create test.source
* Create test.target
2023-02-06 09:27:34 -05:00
Irene López
7dbee87e09
Fix PushToHubCallback import in Share a model docs ( #21457 )
...
docs: update PushToHubCallback import in docs
2023-02-06 09:26:22 -05:00
Jinen Setpal
5ac1c7ea85
Added documentation for DagsHubCallback ( #21452 )
...
updated documentation
2023-02-06 09:24:18 -05:00
jianan-gu
ae31831879
Add perf numbers for perf_train_cpu ( #20974 )
...
* Update perf_train_cpu.mdx
* Update perf_train_cpu.mdx
* Update perf_train_cpu.mdx
* Update docs/source/en/perf_train_cpu.mdx
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update perf_train_cpu.mdx
* Update perf_train_cpu.mdx
* Update perf_train_cpu.mdx
* Update perf_train_cpu.mdx
---------
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2023-02-06 09:20:43 -05:00
Yih-Dar
0db5d911fc
Fix SpeechT5ForSpeechToSpeechIntegrationTests device issue ( #21460 )
...
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-02-06 10:43:07 +01:00
Yih-Dar
59d5edef34
Avoid flaky generation sampling tests ( #21445 )
...
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2023-02-03 22:01:25 +01:00
agossard
31c351c4d3
For IterableDataset, return DataLoader using self._train_batch_size. … ( #21447 )
...
For IterableDataset, return DataLoader using self._train_batch_size. This is consistent with how we generate a regular DataLoader, and leads to the correct args.per_device_train_batch_size eventually ending up on each GPU.
2023-02-03 15:32:48 -05:00
