riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-03 12:50:06 +06:00
Code Issues Actions 28 Packages Projects Releases Wiki Activity
19,295 Commits 859 Branches 224 Tags 2 GiB
b3b7789cbc
Commit Graph

269 Commits

Author SHA1 Message Date
Julien Chaumond
e36bd94345
[ci] Run all tests on (self-hosted) GPU (#3020) 
* Create self-hosted.yml

* Update self-hosted.yml

* Update self-hosted.yml

* Update self-hosted.yml

* Update self-hosted.yml

* Update self-hosted.yml

* do not run slow tests, for now

* [ci] For comparison with circleci, let's also run CPU-tests

* [ci] reorganize

* clearer filenames

* [ci] Final tweaks before merging

* rm slow tests on circle ci

* Trigger CI

* On GPU this concurrency was way too high
 2020-02-28 21:11:08 -05:00
Julien Chaumond
e693cd1e87 [ci] Run slow tests every day 2020-02-24 19:54:47 -05:00
Julien Chaumond
4fc63151af [ci] Attempt to fix #2844 2020-02-24 19:51:34 -05:00
Sam Shleifer
0ed630f139
Attempt to increase timeout for circleci slow tests (#2844) 2020-02-13 09:11:03 -05:00
Morgan Funtowicz
6aa7973aec Fix circleci cuInit error on Tensorflow >= 2.1.0. 
Tensorflow 2.1.0 introduce a new dependency model where pip install tensorflow would install tf with GPU support.
Before it would just install with CPU support, thus CircleCI is looking for NVidia driver version at initialization of the
tensorflow related tests but fails as their is no NVidia Driver running.

Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>
 2020-02-10 13:24:37 +01:00
alberduris
81d6841b4b GPU text generation: mMoved the encoded_prompt to correct device 2020-01-06 15:11:12 +01:00
alberduris
dd4df80f0b Moved the encoded_prompts to correct device 2020-01-06 15:11:12 +01:00
Aymeric Augustin
0ffc8eaf53 Enforce target version for black. 
This should stabilize formatting.
 2020-01-05 12:52:14 -05:00
Aymeric Augustin
10724a8123 Run the slow tests every Monday morning. 2019-12-24 09:09:43 +01:00
Aymeric Augustin
8a6881822a Run some tests on Python 3.7. 
This will improve version coverage.
 2019-12-23 21:06:23 +01:00
Aymeric Augustin
76a1417f2a Include all optional dependencies in extras. 
Take advantage of this to simplify the Circle CI configuration.

Don't bother with tensorboardX: it's a fallback for PyTorch < 1.1.0.
 2019-12-23 19:14:31 +01:00
Aymeric Augustin
23dad8447c Install deps from setup.py for building docs. 
requirements.txt isn't up to date.
 2019-12-23 17:06:32 +01:00
Aymeric Augustin
0dddc1494d Remove py3 marker. 2019-12-22 18:38:56 +01:00
Aymeric Augustin
6be7cdda66 Move source code inside a src subdirectory. 
This prevents transformers from being importable simply because the CWD
is the root of the git repository, while not being importable from other
directories. That led to inconsistent behavior, especially in examples.

Once you fetch this commit, in your dev environment, you must run:

    $ pip uninstall transformers
    $ pip install -e .
 2019-12-22 14:15:13 +01:00
Aymeric Augustin
ced0a94204 Switch test files to the standard test_*.py scheme. 2019-12-22 14:15:13 +01:00
Aymeric Augustin
067395d5c5 Move tests outside of library. 2019-12-22 13:47:17 +01:00
Aymeric Augustin
c11b3e2926 Sort imports for optional third-party libraries. 
These libraries aren't always installed in the virtual environment where
isort is running. Declaring them properly avoids mixing these
third-party imports with local imports.
 2019-12-22 11:19:13 +01:00
Aymeric Augustin
577a03664d Enforce flake8 in CI. 2019-12-22 11:00:04 +01:00
Aymeric Augustin
9e80fc7b2f Enforce isort in CI. 
We need https://github.com/timothycrosley/isort/pull/1000 but there's no
release with this fix yet, so we'll install from GitHub.
 2019-12-22 10:59:00 +01:00
Aymeric Augustin
6e5291a915 Enforce black in CI. 2019-12-21 17:53:18 +01:00
Aymeric Augustin
343c094f21 Run examples separately from tests. 
This optimizes the total run time of the Circle CI test suite.
 2019-12-21 08:43:19 +01:00
Aymeric Augustin
80caf79d07 Prevent excessive parallelism in PyTorch. 
We're already using as many processes in parallel as we have CPU cores.
Furthermore, the number of core may be incorrectly calculated as 36
(we've seen this in pytest-xdist) which make compound the problem.

PyTorch performance craters without this.
 2019-12-21 08:43:19 +01:00
Aymeric Augustin
bb3bfa2d29 Distribute tests from the same file to the same worker. 
This should prevent two issues:

- hitting API rate limits for tests that hit the HF API
- multiplying the cost of expensive test setups
 2019-12-21 08:43:19 +01:00
Aymeric Augustin
29cbab98f0 Parallelize tests on Circle CI. 
Set the number of CPUs manually based on the Circle CI resource class,
or else we're getting 36 CPUs, which is far too much (perhaps that's
the underlying hardware and not what Circle CI allocates to us).

Don't parallelize the custom tokenizers tests because they take less
than one second to run and parallelization actually makes them slower.
 2019-12-21 08:43:19 +01:00
thomwolf
15dda5ea32 remove python 2 tests for circle-ci cc @aaugustin @julien-c @LysandreJik 2019-12-20 13:20:41 +01:00
thomwolf
e4baa68ddb tick-tock cc @julien-c 2019-12-19 20:37:26 +01:00
Thomas Wolf
d5712f7cac
Merge branch 'master' into check-link-validity 2019-12-12 08:00:51 +01:00
Julien Chaumond
371c5ddfad Py2 tests for Lysandre 2019-12-11 18:32:27 -05:00
Julien Chaumond
5505cf7014 Run tests on Py2 too, for Lysandre 2019-12-11 18:32:27 -05:00
Julien Chaumond
9cb97c0c0f Actually run the tests 2019-12-11 18:32:27 -05:00
Julien Chaumond
95854c4a2f Actually run the tests 2019-12-11 18:32:27 -05:00
Julien Chaumond
d2100428d3 Update to new test infra and only run conditionally 2019-12-11 18:32:27 -05:00
Masatoshi Suzuki
597ba7feb3 Support testing Japanese BERT tokenizers 2019-12-11 18:32:27 -05:00
Rémi Louf
f230d91b43 check the validity of links 
We add a script and a CI workflow to check that all download links
present in the source code are valid.
 2019-12-06 09:41:28 +01:00
Lysandre
fbcc5ff9fb Change branch to master 2019-10-22 18:01:10 -04:00
Lysandre
69eba0ab19 Edit script path 2019-10-22 17:53:52 -04:00
Lysandre
bc3e57d551 Multi version doc deployment 2019-10-22 17:51:30 -04:00
thomwolf
2a4fef837a move Circle-CI from TF2-rc0 to official TF2 2019-10-10 15:57:35 +02:00
Julien Chaumond
22d2fded2c [docs] Fix doc auto-deploy 
Co-Authored-By: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
 2019-09-26 18:22:45 -04:00
thomwolf
31c23bd5ee [BIG] pytorch-transformers => transformers 2019-09-26 10:15:53 +02:00
thomwolf
0f091062d4 Merge branch 'glue-example' into tf2 2019-09-25 10:21:52 +02:00
thomwolf
2167e366ba update circleCi 2019-09-24 13:27:45 +02:00
thomwolf
e9a103c17a bidirectional conversion TF <=> PT - extended tests 2019-09-24 13:25:50 +02:00
LysandreJik
11ac4b9555 [CI] Symbolic link for documentation 2019-09-11 10:13:44 +02:00
thomwolf
e30579f764 no pytest version checking 2019-09-08 15:02:06 +03:00
thomwolf
518307dfcd test suite independent of framework 2019-09-08 15:02:06 +03:00
thomwolf
9d0a11a68c update dependencies and circle-ci 2019-09-08 15:02:06 +03:00
thomwolf
ad0ab9afe9 fix test when tf is not here 2019-09-08 15:02:06 +03:00
LysandreJik
3fbf301bba [CI] Updated resource size for python 3 tests 2019-09-02 12:35:14 -04:00
Julien Chaumond
0fd0b674e6 [ci] legible output [skip ci] 2019-08-30 20:36:26 -04:00
Julien Chaumond
b65a994f59 [ci] decrease parallelism to increase success prob 2019-08-30 20:33:16 -04:00
LysandreJik
e7fba4bef5 Documentation auto-deploy 2019-08-29 12:14:29 -04:00
thomwolf
7322c314a6 remove python2 testing for examples 2019-07-12 14:24:08 +02:00
thomwolf
273617b86d update config - fix gpt/gpt-2 from pretrained 2019-07-11 22:45:03 +02:00
thomwolf
6b13f4cb3a update circle-ci 2019-07-11 22:36:35 +02:00
thomwolf
2b644785f0 add tests on examples and large circle ci config 2019-07-11 22:31:50 +02:00
LysandreJik
3f56ad5aff Updated CircleCI's config.yml to use a large resource class. 2019-07-09 18:50:59 -04:00
thomwolf
a4f980547f remove circle ci parallelism 2019-07-05 12:31:34 +02:00
thomwolf
0bab55d5d5 [BIG] name change 2019-07-05 11:55:36 +02:00
thomwolf
cf86d23eff parallelism in circlci 2019-07-04 17:02:21 +02:00
thomwolf
99ae5ab883 update config tests and circle-ci 2019-07-02 12:40:39 +02:00
thomwolf
411981a080 remove slow circle-ci 2019-06-20 08:54:18 +02:00
thomwolf
cd110835a0 coverage in circle-ci 2019-04-30 11:35:40 +02:00
thomwolf
1f5fc95b68 add code coverage 2019-04-30 11:05:26 +02:00
thomwolf
31d387604c adding s3 model tests with --runslow 2019-04-17 11:58:27 +02:00
thomwolf
8197eb9f10 update Circle CI config 2019-02-11 10:22:10 +01:00
thomwolf
525eba68ab update Circle CI 2019-02-11 10:19:25 +01:00
thomwolf
34bdb7f9cb update circle-ci for python 2.7 and 3.5 2019-02-06 00:25:12 +01:00
Julien Chaumond
8da280ebbe Setup CI 2018-12-20 16:33:39 -05:00