pglorio 33cb1f7b61 Add Zamba2 (#34517) ... * First commit * Finish model implementation * First commit * Finish model implementation * Register zamba2 * generated modeling and configuration * generated modeling and configuration * added hybrid cache * fix attention_mask in mamba * dropped unused loras * fix flash2 * config docstrings * fix config and fwd pass * make fixup fixes * text_modeling_zamba2 * small fixes * make fixup fixes * Fix modular model converter * added inheritances in modular, renamed zamba cache * modular rebase * new modular conversion * fix generated modeling file * fixed import for Zamba2RMSNormGated * modular file cleanup * make fixup and model tests * dropped inheritance for Zamba2PreTrainedModel * make fixup and unit tests * Add inheritance of rope from GemmaRotaryEmbedding * moved rope to model init * drop del self.self_attn and del self.feed_forward * fix tests * renamed lora -> adapter * rewrote adapter implementation * fixed tests * Fix torch_forward in mamba2 layer * Fix torch_forward in mamba2 layer * Fix torch_forward in mamba2 layer * Dropped adapter in-place sum * removed rope from attention init * updated rope * created get_layers method * make fixup fix * make fixup fixes * make fixup fixes * update to new attention standard * update to new attention standard * make fixup fixes * minor fixes * cache_position * removed cache_position postion_ids use_cache * remove config from modular * removed config from modular (2) * import apply_rotary_pos_emb from llama * fixed rope_kwargs * Instantiate cache in Zamba2Model * fix cache * fix @slow decorator * small fix in modular file * Update docs/source/en/model_doc/zamba2.md Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> * several minor fixes * inherit mamba2decoder fwd and drop position_ids in mamba * removed docstrings from modular * reinstate zamba2 attention decoder fwd * use regex for tied keys * Revert "use regex for tied keys" This reverts commit 9007a522b1. * use regex for tied keys * add cpu to slow forward tests * dropped config.use_shared_mlp_adapter * Update docs/source/en/model_doc/zamba2.md Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> * re-convert from modular --------- Co-authored-by: root <root@node-2.us-southcentral1-a.compute.internal> Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>		2025-01-27 10:51:23 +01:00
..
__init__.py	[Test refactor 1/5] Per-folder tests reorganization (#15725)	2022-02-23 15:46:28 -05:00
test_beam_constraints.py	Generate: move generation_*.py src files into generation/*.py (#20096)	2022-11-09 15:34:08 +00:00
test_beam_search.py	Time to Say Goodbye, torch 1.7 and 1.8 (#22291)	2023-03-21 19:22:01 +01:00
test_candidate_generator.py	Refactoring AssistedCandidateGenerator for Improved Modularity and Reusability (#35009)	2024-12-12 15:47:05 +01:00
test_configuration_utils.py	[generate] can instantiate GenerationConfig(cache_implementation="static") (#35679)	2025-01-16 17:04:54 +00:00
test_flax_logits_process.py	Adding FlaxNoRepeatNGramLogitsProcessor (#29677)	2024-04-02 11:39:33 +02:00
test_flax_utils.py	Fix CI (#34458)	2024-10-29 08:26:04 +01:00
test_framework_agnostic.py	Generation: fix handling of special tokens (#31254)	2024-06-06 15:21:32 +05:00
test_fsdp.py	Default synced_gpus to True when using FullyShardedDataParallel (#33483)	2024-10-10 14:09:04 -04:00
test_logits_process.py	use torch.testing.assertclose instead to get more details about error in cis (#35659)	2025-01-24 16:55:28 +01:00
test_stopping_criteria.py	Dynamic number of speculative tokens in order to accelerate speculative decoding (#33258)	2024-09-11 14:22:28 +02:00
test_streamers.py	Implement AsyncTextIteratorStreamer for asynchronous streaming (#34931)	2024-12-20 12:08:12 +01:00
test_tf_logits_process.py	fix: multilingual midel convert to tflite get wrong token (#32079)	2024-08-27 11:44:09 +02:00
test_tf_utils.py	Revert workaround for TF safetensors loading (#30128)	2024-04-09 11:04:18 +01:00
test_utils.py	Add Zamba2 (#34517)	2025-01-27 10:51:23 +01:00