# Sharing custom models The 🤗 Transformers library is designed to be easily extensible. Every model is fully coded in a given subfolder of the repository with no abstraction, so you can easily copy a modeling file and tweak it to your needs. Once you are happy with those tweaks and trained a model you want to share with the community, there are simple steps to push on the Model Hub not only the weights of your model, but also the code it relies on, so that anyone in the community can use it, even if it's not present in the 🤗 Transformers library. This also applies to configurations and tokenizers (support for feature extractors and processors is coming soon). ## Sending the code to the Hub First, make sure your model is fully defined in a `.py` file. It can rely on relative imports to some other files as long as all the files are in the same directory (we don't support submodules for this feature yet). For instance, let's say you have a `modeling.py` file and a `configuration.py` file in a folder of the current working directory named `awesome_model`, and that the modeling file defines an `AwesomeModel`, the configuration file a `AwesomeConfig`. ``` . └── awesome_model ├── __init__.py ├── configuration.py └── modeling.py ``` The `__init__.py` can be empty, it's just there so that Python detects `awesome_model` can be use as a module. Here is an example of what the configuration file could look like: ```py from transformers import PretrainedConfig class AwesomeConfig(PretrainedConfig): model_type = "awesome" def __init__(self, attribute=1, hidden_size=42, **kwargs): self.attribute = attribute self.hidden_size = hidden_size super().__init__(**kwargs) ``` and the modeling file could have content like this: ```py import torch from transformers import PreTrainedModel from .configuration import AwesomeConfig class AwesomeModel(PreTrainedModel): config_class = AwesomeConfig base_model_prefix = "base" def __init__(self, config): super().__init__(config) self.linear = torch.nn.Linear(config.hidden_size, config.hidden_size) def forward(self, x): return self.linear(x) ``` `AwesomeModel` should subclass [`PreTrainedModel`] and `AwesomeConfig` should subclass [`PretrainedConfig`]. The easiest way to achieve this is to copy the modeling and configuration files of the model closest to the one you're coding, and then tweaking them. If copying a modeling files from the library, you will need to replace all the relative imports at the top of the file to import from the `transformers` package. Note that you can re-use (or subclass) an existing configuration/model. To share your model with the community, follow those steps: first import the custom objects. ```py from awesome_model.configuration import AwesomeConfig from awesome_model.modeling import AwesomeModel ``` Then you have to tell the library you want to copy the code files of those objects when using the `save_pretrained` method and properly register them with a given Auto class (especially for models), just run: ```py AwesomeConfig.register_for_auto_class() AwesomeModel.register_for_auto_class("AutoModel") ``` Note that there is no need to specify an auto class for the configuration (there is only one auto class for them, [`AutoConfig`]) but it's different for models. Your custom model could be suitable for sequence classification (in which case you should do `AwesomeModel.register_for_auto_class("AutoModelForSequenceClassification")`) or any other task, so you have to specify which one of the auto classes is the correct one for your model. Next, just create the config and models as you would any other Transformer models: ```py config = AwesomeConfig() model = AwesomeModel(config) ``` then train your model. Alternatively, you could load a pretrained checkpoint you have already trained in your model. Once everything is ready, you just have to do: ```py model.save_pretrained("save_dir") ``` which will not only save the model weights and the configuration in json format, but also copy the modeling and configuration `.py` files in this folder, so you can directly upload the result to the Hub. If you have already logged in to Hugging face with ```bash huggingface-cli login ``` or in a notebook with ```py from huggingface_hub import notebook_login notebook_login() ``` you can push your model and its code to the Hub with the following: ```py model.push_to_hub("model-identifier") ``` See the [sharing tutorial](model_sharing) for more information on the push to Hub method. ## Using a model with custom code You can use any configuration, model or tokenizer with custom code files in its repository with the auto-classes and the `from_pretrained` method. The only thing is that you have to add an extra argument to make sure you have read the online code and trust the author of that model, to avoid executing malicious code on your machine: ```py from transformers import AutoModel model = AutoModel.from_pretrained("model-checkpoint", trust_remote_code=True) ``` It is also strongly encouraged to pass a commit hash as a `revision` to make sure the author of the models did not update the code with some malicious new lines (unless you fully trust the authors of the models). ```py commit_hash = "b731e5fae6d80a4a775461251c4388886fb7a249" model = AutoModel.from_pretrained("model-checkpoint", trust_remote_code=True, revision=commit_hash) ``` Note that when browsing the commit history of the model repo on the Hub, there is a button to easily copy the commit hash of any commit.