mirror of https://github.com/huggingface/transformers.git synced 2025-07-31 02:02:21 +06:00

History

Patrick von Platen 0d1f67e651 [Flax] Add wav2vec2 (#12271 ) * fix_torch_device_generate_test * remove @ * start flax wav2vec2 * save intermediate * forward pass has correct shape * add weight norm * add files * finish ctc * make style * finish gumbel quantizer * correct docstrings * correct some more files * fix vit * finish quality * correct tests * correct docstring * correct tests * start wav2vec2 pretraining script * save intermediate * start pretraining script * finalize pretraining script * finish * finish * small typo * finish * correct * Apply suggestions from code review Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> Co-authored-by: Suraj Patil <surajp815@gmail.com> * make style * push Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> Co-authored-by: Suraj Patil <surajp815@gmail.com>		2021-06-30 18:44:23 +01:00
..
flax	[Flax] Example scripts - correct weight decay (#12409 )	2021-06-29 12:01:08 +01:00
legacy	remove extra white space from log format (#12360 )	2021-06-25 13:20:14 -07:00
pytorch	[Examples] Added context manager to datasets map (#12367 )	2021-06-28 09:14:00 -07:00
research_projects	[Flax] Add wav2vec2 (#12271 )	2021-06-30 18:44:23 +01:00
tensorflow	Tensorflow LM examples (#12358 )	2021-06-28 19:31:44 +01:00
README.md	Examples reorg (#11350 )	2021-04-21 11:11:20 -04:00

README.md

Examples

This folder contains actively maintained examples of use of 🤗 Transformers organized along NLP tasks. If you are looking for an example that used to be in this folder, it may have moved to the corresponding framework subfolder (pytorch, tensorflow or flax), our research projects subfolder (which contains frozen snapshots of research projects) or to the legacy subfolder.

While we strive to present as many use cases as possible, the scripts in this folder are just examples. It is expected that they won't work out-of-the box on your specific problem and that you will be required to change a few lines of code to adapt them to your needs. To help you with that, most of the examples fully expose the preprocessing of the data. This way, you can easily tweak them.

This is similar if you want the scripts to report another metric than the one they currently use: look at the compute_metrics function inside the script. It takes the full arrays of predictions and labels and has to return a dictionary of string keys and float values. Just change it to add (or replace) your own metric to the ones already reported.

Please discuss on the forum or in an issue a feature you would like to implement in an example before submitting a PR: we welcome bug fixes but since we want to keep the examples as simple as possible, it's unlikely we will merge a pull request adding more functionality at the cost of readability.

Important note

Important

To make sure you can successfully run the latest versions of the example scripts, you have to install the library from source and install some example-specific requirements. To do this, execute the following steps in a new virtual environment:

git clone https://github.com/huggingface/transformers
cd transformers
pip install .

Then cd in the example folder of your choice and run

pip install -r requirements.txt

To browse the examples corresponding to released versions of 🤗 Transformers, click on the line below and then on your desired version of the library:

Examples for older versions of 🤗 Transformers

Alternatively, you can find switch your cloned 🤗 Transformers to a specific version (for instance with v3.5.1) with

git checkout tags/v3.5.1

and run the example command as usual afterward.