riaz.somc/transformers
1
0
Fork 0
mirror of https://github.com/huggingface/transformers.git synced 2025-07-18 20:18:24 +06:00
Code Issues Actions 23 Packages Projects Releases Wiki Activity
15,053 Commits 875 Branches 227 Tags 2 GiB
3f9f749325
Commit Graph

15053 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Aymeric Augustin
71f94a8a1c Remove unused variables in src. 2019-12-23 22:38:09 +01:00
Aymeric Augustin
81422c4e6d Remove unused variables in examples. 2019-12-23 22:29:02 +01:00
Aymeric Augustin
072750f4dc
Merge pull request #2288 from aaugustin/better-handle-optional-imports 
Improve handling of optional imports
 2019-12-23 22:28:47 +01:00
Aymeric Augustin
4621ad6f9d Use the same pattern as everywhere else. 
This is really just for consistency.
 2019-12-23 21:30:04 +01:00
Aymeric Augustin
a31d4a2971 Reraise ImportError when sentencepiece isn't installed. 
Else, the next line fails with a confusion exception because the spm
variable isn't defined.
 2019-12-23 21:27:42 +01:00
Aymeric Augustin
c8b0c1e551 Improve exception type. 
ImportError isn't really appropriate when there's no import involved.
 2019-12-23 21:27:38 +01:00
Aymeric Augustin
4c09a96096 Simplify re-raising exceptions. 
Most module use the simpler `raise` version. Normalize those that don't.
 2019-12-23 21:20:54 +01:00
Aymeric Augustin
5565dcdd35 Remove warning when scikit-learn isn't available. 
Most users don't need it.
 2019-12-23 21:16:26 +01:00
Aymeric Augustin
8a6881822a Run some tests on Python 3.7. 
This will improve version coverage.
 2019-12-23 21:06:23 +01:00
Aymeric Augustin
7a865821d9 Remove stray egg-info directory automatically. 
If a user or contributor ran `pip install -e .` on transformers < 3.0,
pip created a transformers.egg-info directory next to the transformers
directory at the root of the repository.

In transformers 3.0, the source is in a `src` subdirectory.
`pip install -e .` creates a transformers.egg-info directory there.
However, pip will still pick transformers.egg-info from the previous
location. This is a bug: https://github.com/pypa/pip/issues/5466

Users and contributors are likely to hit this problem because the
documentation for transformers 3.0 relies heavily on extra_requires
which didn't exist in earlier versions, so aren't defined in a stale
transformers.egg-info directory.

If such a directory exists, remove it. It's autogenerated, gitignored
and not supposed to contain anything of value.
 2019-12-23 21:06:23 +01:00
Aymeric Augustin
70373a5f7c Update contribution instructions. 
Also provide shortcuts in a Makefile.
 2019-12-23 21:05:30 +01:00
Aymeric Augustin
c3783399db Remove redundant requirements with transformers. 2019-12-23 19:17:27 +01:00
Aymeric Augustin
d79e9c9a9a Remove docs/requirements.txt. 
It's superseded by the "docs" extras.
 2019-12-23 19:17:07 +01:00
Aymeric Augustin
d73eb552e8 Remove requirements.txt. 
It's redundant with setup.py and, also, incomplete (e.g. numpy).
 2019-12-23 19:15:08 +01:00
Aymeric Augustin
9fcc532df6 Remove requirements-dev.txt. 
It was generated once, likely in a non-reproducible way (pip freeze
in a contributor's local environment), and never updated.
 2019-12-23 19:14:36 +01:00
Aymeric Augustin
76a1417f2a Include all optional dependencies in extras. 
Take advantage of this to simplify the Circle CI configuration.

Don't bother with tensorboardX: it's a fallback for PyTorch < 1.1.0.
 2019-12-23 19:14:31 +01:00
Aymeric Augustin
9fc8dcb2a0 Standardize import. 
Every other file uses this pattern.
 2019-12-23 18:45:42 +01:00
Aymeric Augustin
f2522869ea Review and update setup.py. 2019-12-23 18:45:42 +01:00
Alan deLevie
7cef764ec0
Typo in tokenization_utils.py 
avoir -> avoid
 2019-12-23 12:14:50 -05:00
Aymeric Augustin
23dad8447c Install deps from setup.py for building docs. 
requirements.txt isn't up to date.
 2019-12-23 17:06:32 +01:00
Aymeric Augustin
d8e33dbd67 Fix path to source code in docs config. 
This should fix API docs, which went AWOL with yesterday's changes.
 2019-12-23 16:49:35 +01:00
thomwolf
59b123bc50 fix tqdm logging level 2019-12-23 16:47:24 +01:00
Thomas Wolf
ba2378ced5
Merge pull request #2264 from upura/fix-doclink 
Fix doc link in README
 2019-12-23 12:31:00 +01:00
Thomas Wolf
e4e2a666c9
Merge pull request #2276 from ShnitzelKiller/scatterfix 
fix error due to wrong argument name to Tensor.scatter()
 2019-12-23 12:19:48 +01:00
James Noeckel
398bb03f98 fix out-of-place call to scatter, whose named argument name is source, not src 2019-12-22 23:30:52 -08:00
Aymeric Augustin
ce50305e5b
Merge pull request #2270 from aaugustin/remove-python-2 
Remove support for Python 2
 2019-12-22 23:04:37 +01:00
Aymeric Augustin
1a948d7020 Switch from comments to annotations for types. 2019-12-22 18:56:01 +01:00
Aymeric Augustin
1c62e87b34 Use built-in open(). 
On Python 3, `open is io.open`.
 2019-12-22 18:38:56 +01:00
Aymeric Augustin
d6eaf4e6d2 Update comments mentioning Python 2. 2019-12-22 18:38:56 +01:00
Aymeric Augustin
45841eaf7b Remove references to Python 2 in documentation. 2019-12-22 18:38:56 +01:00
Aymeric Augustin
0dddc1494d Remove py3 marker. 2019-12-22 18:38:56 +01:00
Aymeric Augustin
75a23d24af Remove import fallbacks. 2019-12-22 18:38:56 +01:00
Aymeric Augustin
798b3b3899 Remove sys.version_info[0] == 2 or 3. 2019-12-22 18:38:42 +01:00
Aymeric Augustin
8af25b1664 Remove six. 2019-12-22 17:56:09 +01:00
Aymeric Augustin
6b2200fc88 Remove u-prefixes. 2019-12-22 17:47:54 +01:00
Aymeric Augustin
c824d15aa1 Remove __future__ imports. 2019-12-22 17:47:54 +01:00
Aymeric Augustin
b6ea0f43ae Remove duplicate -v flag. 2019-12-22 17:47:27 +01:00
Thomas Wolf
5daca95ddd
Merge pull request #2268 from aaugustin/improve-repository-structure 
Improve repository structure
 2019-12-22 16:41:53 +01:00
Thomas Wolf
54abc67aec
Merge pull request #2255 from aaugustin/implement-best-practices 
Implement some Python best practices
 2019-12-22 16:31:11 +01:00
Aymeric Augustin
00204f2b4c Replace CommonTestCases for tokenizers with a mixin. 
This is the same change as for (TF)CommonTestCases for modeling.
 2019-12-22 15:35:25 +01:00
Aymeric Augustin
a3c5883f2c Rename file for consistency. 2019-12-22 15:35:25 +01:00
Aymeric Augustin
daf8bebcdd Remove unused GPTModelTester. 
It isn't imported anywhere.
 2019-12-22 15:35:25 +01:00
Aymeric Augustin
345c23a60f Replace (TF)CommonTestCases for modeling with a mixin. 
I suspect the wrapper classes were created in order to prevent the
abstract base class (TF)CommonModelTester from being included in test
discovery and running, because that would fail.

I solved this by replacing the abstract base class with a mixin.

Code changes are just de-indenting and automatic reformattings
performed by black to use the extra line space.
 2019-12-22 15:35:18 +01:00
Aymeric Augustin
7e98e211f0 Remove unittest.main() in test modules. 
This construct isn't used anymore these days.

Running python tests/test_foo.py puts the tests/ directory on
PYTHONPATH, which isn't representative of how we run tests.

Use python -m unittest tests/test_foo.py instead.
 2019-12-22 14:42:03 +01:00
Aymeric Augustin
6be7cdda66 Move source code inside a src subdirectory. 
This prevents transformers from being importable simply because the CWD
is the root of the git repository, while not being importable from other
directories. That led to inconsistent behavior, especially in examples.

Once you fetch this commit, in your dev environment, you must run:

    $ pip uninstall transformers
    $ pip install -e .
 2019-12-22 14:15:13 +01:00
Aymeric Augustin
ced0a94204 Switch test files to the standard test_*.py scheme. 2019-12-22 14:15:13 +01:00
Aymeric Augustin
067395d5c5 Move tests outside of library. 2019-12-22 13:47:17 +01:00
Aymeric Augustin
698f9e3d7a Remove trailing whitespace in README. 2019-12-22 13:29:58 +01:00
Aymeric Augustin
c11b3e2926 Sort imports for optional third-party libraries. 
These libraries aren't always installed in the virtual environment where
isort is running. Declaring them properly avoids mixing these
third-party imports with local imports.
 2019-12-22 11:19:13 +01:00
Aymeric Augustin
2a34d5b71b Stabilize import order for packaging. 
I don't want to consider it a dependency of transformers, but it's
usually there in local development and usually not there in CI.
 2019-12-22 11:07:31 +01:00