riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-04 21:30:07 +06:00
Code Issues Actions 28 Packages Projects Releases Wiki Activity
15,053 Commits 860 Branches 225 Tags 2 GiB
3f9f749325
Commit Graph

456 Commits

Author SHA1 Message Date
Aymeric Augustin
d73eb552e8 Remove requirements.txt. 
It's redundant with setup.py and, also, incomplete (e.g. numpy).
 2019-12-23 19:15:08 +01:00
Aymeric Augustin
76a1417f2a Include all optional dependencies in extras. 
Take advantage of this to simplify the Circle CI configuration.

Don't bother with tensorboardX: it's a fallback for PyTorch < 1.1.0.
 2019-12-23 19:14:31 +01:00
Aymeric Augustin
f2522869ea Review and update setup.py. 2019-12-23 18:45:42 +01:00
Aymeric Augustin
1c62e87b34 Use built-in open(). 
On Python 3, `open is io.open`.
 2019-12-22 18:38:56 +01:00
Aymeric Augustin
d6eaf4e6d2 Update comments mentioning Python 2. 2019-12-22 18:38:56 +01:00
Aymeric Augustin
6be7cdda66 Move source code inside a src subdirectory. 
This prevents transformers from being importable simply because the CWD
is the root of the git repository, while not being importable from other
directories. That led to inconsistent behavior, especially in examples.

Once you fetch this commit, in your dev environment, you must run:

    $ pip uninstall transformers
    $ pip install -e .
 2019-12-22 14:15:13 +01:00
Aymeric Augustin
158e82e061 Sort imports with isort. 
This is the result of:

    $ isort --recursive examples templates transformers utils hubconf.py setup.py
 2019-12-22 10:57:46 +01:00
Aymeric Augustin
fa84ae26d6 Reformat source code with black. 
This is the result of:

    $ black --line-length 119 examples templates transformers utils hubconf.py setup.py

There's a lot of fairly long lines in the project. As a consequence, I'm
picking the longest widely accepted line length, 119 characters.

This is also Thomas' preference, because it allows for explicit variable
names, to make the code easier to understand.
 2019-12-21 17:52:29 +01:00
Aymeric Augustin
a4c9338b83 Prevent parallel downloads of the same file with a lock. 
Since the file is written to the filesystem, a filesystem lock is the
way to go here. Add a dependency on the third-party filelock library to
get cross-platform functionality.
 2019-12-21 08:43:19 +01:00
Lysandre
a436574bfd Release: v2.3.0 2019-12-20 16:22:20 -05:00
thomwolf
73fcebf7ec update serving command 2019-12-20 13:47:35 +01:00
thomwolf
407093b3fa Merge branch 'cli' of https://github.com/huggingface/transformers into cli 2019-12-19 20:26:51 +01:00
thomwolf
c7be096c39 Merge branch 'master' into cli 2019-12-19 20:26:08 +01:00
Morgan Funtowicz
a305067f2d Removed __main__ 2019-12-19 19:41:48 +01:00
Lysandre
5e289f69bc regex 2019.12.17 install fails with Python 2 2019-12-17 15:54:05 -05:00
Morgan Funtowicz
d7c62661a3 Provide serving dependencies for tensorflow and pytorch (serving-tf, serving-torch) 2019-12-17 11:23:39 +01:00
Lysandre
7bd11dda6f Release: v2.2.2 2019-12-13 16:45:30 -05:00
thomwolf
72c36b9ea2 [WIP] - CLI 2019-12-10 11:33:14 +01:00
Aymeric Augustin
35401fe50f Remove dependency on pytest for running tests (#2055) 
* Switch to plain unittest for skipping slow tests.

Add a RUN_SLOW environment variable for running them.

* Switch to plain unittest for PyTorch dependency.

* Switch to plain unittest for TensorFlow dependency.

* Avoid leaking open files in the test suite.

This prevents spurious warnings when running tests.

* Fix unicode warning on Python 2 when running tests.

The warning was:

    UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal

* Support running PyTorch tests on a GPU.

Reverts 27e015bd.

* Tests no longer require pytest.

* Make tests pass on cuda
 2019-12-06 13:57:38 -05:00
Julien Chaumond
e4fbf3e2cc CLI for authenticated file sharing 2019-12-04 00:52:23 -05:00
LysandreJik
8101924a68 Patch: v2.2.1 2019-12-03 11:20:26 -05:00
Lysandre
ae98d45991 Release: v2.2.0 2019-11-26 14:12:44 -05:00
Lysandre
3ddce1d74c Release: 2.1.1 2019-10-11 06:37:49 -04:00
LysandreJik
9c2e0a4acf Release: 2.1.0 2019-10-09 12:14:03 -04:00
thomwolf
e4e35296fb update setup.py metadata 2019-09-26 13:52:24 +02:00
thomwolf
9676d1a2a8 update readme and setup.py 2019-09-26 13:47:58 +02:00
thomwolf
31c23bd5ee [BIG] pytorch-transformers => transformers 2019-09-26 10:15:53 +02:00
thomwolf
9d0a11a68c update dependencies and circle-ci 2019-09-08 15:02:06 +03:00
thomwolf
89fd3450a6 Release: 1.2.0 2019-09-04 13:32:18 +02:00
Shijie Wu
ca4baf8ca1 Match order of casing in OSS XLM; Improve document; Clean up dependency 2019-08-27 20:03:18 -04:00
Shijie Wu
e85123d398 Add custom tokenizer for zh and ja 2019-08-23 20:27:52 -04:00
Shijie Wu
436ce07218 Tokenization behave the same as original XLM proprocessing for most languages except zh, ja and th; Change API to allow specifying language in tokenize 2019-08-23 14:40:17 -04:00
LysandreJik
fe02e45e48 Release: 1.1.0 2019-08-15 11:15:08 -04:00
thomwolf
58830807d1 inidicate we only support pytorch 1.0.0+ now 2019-08-05 14:38:59 +02:00
thomwolf
6b70760204 typos 2019-07-16 21:21:03 +02:00
thomwolf
ed7549bb1a release version 1.0 2019-07-16 16:10:58 +02:00
thomwolf
eb91f6437e update readme and setup 2019-07-05 12:30:15 +02:00
thomwolf
0bab55d5d5 [BIG] name change 2019-07-05 11:55:36 +02:00
thomwolf
32da75486b add tokenizer and tests 2019-06-21 11:09:51 +02:00
thomwolf
b832d5bb8a Release: 0.6.2 2019-04-25 21:37:47 +02:00
thomwolf
e0855e8929 forgot to add regex to requirements :( 2019-02-18 11:54:51 +01:00
thomwolf
009ee86a19 fix tests - bump up version 2019-02-17 23:57:23 +01:00
thomwolf
321d70a7a9 bump up to 0.5.1 2019-02-13 10:11:20 +01:00
thomwolf
448937c00d python 2 compatibility 2019-02-06 00:07:46 +01:00
thomwolf
eed51c5bdf add OpenAI GPT 2019-01-08 12:26:58 +01:00
Patrick Sodré
87c1244c7d Convert scripts into entry_points 
The recommended approach to create launch scripts is to use entry_points
and console_scripts.

xref: https://packaging.python.org/guides/distributing-packages-using-setuptools/#scripts
 2018-12-19 02:26:08 +00:00
thomwolf
ae88eb88a4 set encoding to 'utf-8' in calls to open 2018-12-14 13:48:58 +01:00
thomwolf
1cbb32a542 include version number + comment in setup.py 2018-12-13 12:50:44 +01:00
thomwolf
ce52177638 added version in __init__.py 2018-12-13 12:50:44 +01:00
thomwolf
258eb50086 bump up version 2018-11-30 22:55:33 +01:00
thomwolf
ce37b8e481 bump version in setup.py 2018-11-26 10:45:48 +01:00
thomwolf
c8cba67742 clean up readme and examples 2018-11-17 12:19:16 +01:00
thomwolf
a99b971738 bump up version minor 2018-11-17 10:43:39 +01:00
thomwolf
4e46affc34 updating examples 2018-11-17 10:30:54 +01:00
thomwolf
f920eff8c3 update readme 2018-11-17 08:42:45 +01:00
thomwolf
1de35b624b preparing for first release 2018-11-15 20:56:10 +01:00