* [t5 doc] typos
a few run away backticks
@sgugger
* style
* [trainer] put fp16 args together
this PR proposes a purely cosmetic change that puts all the fp16 args together - so they are easier to manager/read
@sgugger
* style
* create model
* add integration
* save current state
* make integration tests pass
* add one more test
* add explanation to tests
* remove from bart
* add padding
* remove unnecessary test
* make all tests pass
* re-add cookie cutter tests
* finish PyTorch
* fix attention test
* Update tests/test_modeling_common.py
* revert change
* remove unused file
* add string to doc
* save intermediate
* make tf integration tests pass
* finish tf
* fix doc
* fix docs again
* add led to doctree
* add to auto tokenizer
* added tips for led
* make style
* apply jplus statements
* correct tf longformer
* apply lysandres suggestions
* apply sylvains suggestions
* Apply suggestions from code review
* Use extlinks to point hyperlink with the version of code
* Point to version on release and master until then
* Apply style
* Correct links
* Add missing backtick
* Simple missing backtick after all.
Co-authored-by: Raghavendra Sugeeth P S <raghav-5305@raghav-5305.csez.zohocorpin.com>
Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
* --model_parallel hasn't been implemented for most models
* make the help clear as well
* implement is_parallelizable; use it
* oops
* remove property
This PR proposes to:
* auto-flush `transformers` logging
When using logging for tracing signals from different parts of the code and which could be mixed with print debug this aids to get all the logging events synchronized.
I don't think this change will introduce any performance impacts.
If it helps someone here is the code I used to sync `transformers` logging with various other debug prints.
I was porting bart to MP and I needed to trace that the device switching happens correctly and I added a bunch of logger.info calls inside `modeling_bart.py` and also had some other helpers `print` debug messages which weren't logger based:
```
# auto flush std streams
from sys import stdout, stderr
def stdout_write_flush(args, w=stderr.write): w(args); stderr.flush()
def stderr_write_flush(args, w=stderr.write): w(args); stderr.flush()
stdout.write = stdout_write_flush
stderr.write = stderr_write_flush
from transformers import BartTokenizer, BartForConditionalGeneration, BartConfig
import logging
import transformers.utils.logging
import transformers.models.bart.modeling_bart
# I wanted a shorter simpler format
handlers = transformers.utils.logging._get_library_root_logger().handlers
for handler in handlers:
formatter = logging.Formatter("[%(funcName)s] %(message)s")
handler.setFormatter(formatter)
transformers.models.bart.modeling_bart.logger.setLevel(transformers.logging.INFO)
```
@LysandreJik, @sgugger, @patrickvonplaten
This PR:
* fixes trainer to have the logger agree with the actual default `output_dir`, but setting it one place and passing it as an argument to both places
@sgugger
* fix a bug in eval_batch_retrieval
* should return parser as well as other staticmethod
* remove duplicate argument
* these kwargs are no longer accepted (cause TypeError in self.generator.generate of modeling_rag.py)
* fixed file paths in README
* moved an arg to add_ray_specific_args
```
python -c "from apex.normalization import FusedProphetNetLayerNorm"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: cannot import name 'FusedProphetNetLayerNorm' from 'apex.normalization' (/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/apex/normalization/__init__.py)
```
It looks like this code has never been tested, so it silently fails inside try/except.
Discovered this by accident in https://github.com/huggingface/transformers/issues/9338#issuecomment-752217708
