mirror of
https://github.com/huggingface/transformers.git
synced 2025-07-08 07:10:06 +06:00
a52478253b
Some checks are pending
Self-hosted runner (benchmark) / Benchmark (aws-g5-4xlarge-cache) (push) Waiting to run
Build documentation / build (push) Waiting to run
Slow tests on important models (on Push - A10) / Get all modified files (push) Waiting to run
Slow tests on important models (on Push - A10) / Slow & FA2 tests (push) Blocked by required conditions
Secret Leaks / trufflehog (push) Waiting to run
Update Transformers metadata / build_and_package (push) Waiting to run
* updates * feedback * badges * fix? * fix? * fix? * fix?
59 lines
2.6 KiB
Markdown
59 lines
2.6 KiB
Markdown
# Cohere
|
|
|
|
<div class="flex flex-wrap space-x-1">
|
|
<img alt="PyTorch" src="https://img.shields.io/badge/PyTorch-DE3412?style=flat&logo=pytorch&logoColor=white">
|
|
<img alt="FlashAttention" src="https://img.shields.io/badge/%E2%9A%A1%EF%B8%8E%20FlashAttention-eae0c8?style=flat">
|
|
<img alt="SDPA" src="https://img.shields.io/badge/SDPA-DE3412?style=flat&logo=pytorch&logoColor=white">
|
|
<img alt="Tensor parallelism" src="https://img.shields.io/badge/Tensor%20parallelism-06b6d4?style=flat&logoColor=white">
|
|
</div>
|
|
|
|
## Overview
|
|
[C4AI Command R7B](https://cohere.com/blog/command-r7b) is an open weights research release of a 7B billion parameter model developed by Cohere and Cohere For AI. It has advanced capabilities optimized for various use cases, including reasoning, summarization, question answering, and code. The model is trained to perform sophisticated tasks including Retrieval Augmented Generation (RAG) and tool use. The model also has powerful agentic capabilities that can use and combine multiple tools over multiple steps to accomplish more difficult tasks. It obtains top performance on enterprise-relevant code use cases. C4AI Command R7B is a multilingual model trained on 23 languages.
|
|
|
|
The model features three layers with sliding window attention (window size 4096) and ROPE for efficient local context modeling and relative positional encoding. A fourth layer uses global attention without positional embeddings, enabling unrestricted token interactions across the entire sequence.
|
|
|
|
The model has been trained on 23 languages: English, French, Spanish, Italian, German, Portuguese, Japanese, Korean, Arabic, Chinese, Russian, Polish, Turkish, Vietnamese, Dutch, Czech, Indonesian, Ukrainian, Romanian, Greek, Hindi, Hebrew, and Persian.
|
|
|
|
## Usage tips
|
|
The model and tokenizer can be loaded via:
|
|
|
|
```python
|
|
# pip install transformers
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM
|
|
|
|
model_id = "CohereForAI/c4ai-command-r7b-12-2024"
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
|
model = AutoModelForCausalLM.from_pretrained(model_id)
|
|
|
|
# Format message with the command-r chat template
|
|
messages = [{"role": "user", "content": "Hello, how are you?"}]
|
|
input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
|
|
|
|
gen_tokens = model.generate(
|
|
input_ids,
|
|
max_new_tokens=100,
|
|
do_sample=True,
|
|
temperature=0.3,
|
|
)
|
|
|
|
gen_text = tokenizer.decode(gen_tokens[0])
|
|
print(gen_text)
|
|
```
|
|
|
|
## Cohere2Config
|
|
|
|
[[autodoc]] Cohere2Config
|
|
|
|
## Cohere2Model
|
|
|
|
[[autodoc]] Cohere2Model
|
|
- forward
|
|
|
|
|
|
## Cohere2ForCausalLM
|
|
|
|
[[autodoc]] Cohere2ForCausalLM
|
|
- forward
|
|
|
|
|