mirror of
https://github.com/huggingface/transformers.git
synced 2025-07-13 09:40:06 +06:00
a2ef3cf537
* Iterative generation using input embeds * Add Janus model * discard changes * Janus imports * Refactor config and processor * Added Vision tower of Janus * Import Janus Image processor * Vision tower fixes * Refactor code * Added VQ Model * Complete model integration * temp conversion script * processor refactor * Adding files to facilitate pulling * Fixes after debugging * Skip test for these models * Add Janus Model * discard changes * Janus imports * Refactor config and processor * Added Vision tower of Janus * Import Janus Image processor * Vision tower fixes * Refactor code * Added VQ Model * Complete model integration * temp conversion script * processor refactor * Adding files to facilitate pulling * Fixes after debugging * Refactor to Text config * ✨ Added generate function * Saving intermediate convert file. Still need to read configs from the hub and convert them to our format. * Adding version that reads from the JSON files. Still have to tweak some parameters manually. * relative imports * Initial tests * Refactor image processor * Seemingly working version of the conversion script, will need to test further. * Adding command message * Fixing conflicting JanusTextConfig class * Incorporating some of the discussed changes. * Small fix to create dir. * Removing system from JINJA template * Adding draft processor tests * style fixes * Minor fixes and enhancement * added generation config * Initial tests * Small modifications, tests are now passing. * Small changes I noticed while reading code. * more fixes * Added JanusModel class * Small merge adaptations * Small merge adaptations * Image processing tests passing * More tests and fixes * Convert script updated and refactored * Tests and cleanup * make style * Postprocessing for image generation * generate refactor * fixes * - Passing tests that write a part of the model to cpu (e.g. test_cpu_offload) - Passing tests of dispatching SDPA - Only gradient checkpointing tests are left. * Removing temporary code * Changes * Writing change to modular * Added JanusVisionModel. SDPA dispatch tests pass more robustly. Gradient checkpoint tests are next * Gradient checkpoint tests passing * Removing debug code * Major generate refactor 😮💨 * Temp changes for testing * Green quality CI * 2 out of 4 integration tests passing * breadcrumbs * Usage Examples * Regenerate modeling after merge * dirty code * JanusIntegrationTest are passing * breadcrumbs * happy CI * fixes * Changing template * nits * Text generation logits matching original codebase at 100% precision * Remove ./tmp from git tracking * Remove ./tmp from git tracking * Checkpointing changes after reviewing * Fixing code in docstrings * CHanging comments and small bug in convert file * Fixing bug in image_token_id for 7B version * Removing line that was added by both of us * Pushing changes after discussion. Only one left is to change the key mapping for convert file. * Updating module file * New convert file using dict. Tested that it is equivalent to the old one by: - comparing keys in a script - comparing checksums of the output files between version generated with the current convert script and those generated with the old script. This is a more reliable test. * revert changes * mistake * consistency change for CI * make style * doc fixes * more fixes * experimenting with masking out pad token * checkpoint * Batched generation with multi-images working for 1B models. Will test 7B next. * Device fix. * Writing changes to modular, previous ones were written to modeling just for quick testing. * Using passed processor attention mask (only in modeling for now) * Matching performance done in the non-standard way * Working version of batched generation. Will change how some args are passed to make it more similar to language case * More compliant version of the code * Removed duplicated `_prepare_4d_causal_attention_mask_with_cache_position` * Updating modular file, making masked filling with paddings more efficient * Slightly more efficient version * Modifying JanusVisionModel to be a wrapper * Fixing test to comply with new names * Modular overhaul * More refactoring * - Changing JanusVisionModel back - Changing forward pass - Adding boi token to the comparison * - Removing whole context model_ids - Using inherited implementation of prepare_inputs_for_generation * Moving the way boi token is passed to the model * Fixing sdpa test * Minor changes * testing changes * Minor fix * - Adding postprocessing test - checking values of generated image on integration test * changes * Removing pooled attention vision module, fixing convert script as a consequence * More changes * Fixes * Draft after merge * Bug fixes * More bug fix * Fixing docs * Nits * Refactor return dict * Moving image post processing test to main processor post process * Passing guidance_scale as kwarg * make style * 🔥 refactor * make style * Update and green CI * Nits and tests update * up * Added MID block * fix * Dead code * update testcase * update * model_id change * init_weight changes --------- Co-authored-by: hsilva664 <metallic-silver@hotmail.com> |
||
|---|---|---|
| .. | ||
| internal | ||
| main_classes | ||
| model_doc | ||
| quantization | ||
| tasks | ||
| _config.py | ||
| _redirects.yml | ||
| _toctree.yml | ||
| accelerate.md | ||
| add_new_model.md | ||
| add_new_pipeline.md | ||
| agents.md | ||
| attention_interface.md | ||
| attention.md | ||
| backbones.md | ||
| cache_explanation.md | ||
| chat_extras.md | ||
| chat_templating_multimodal.md | ||
| chat_templating_writing.md | ||
| chat_templating.md | ||
| community.md | ||
| contributing.md | ||
| conversations.md | ||
| custom_models.md | ||
| debugging.md | ||
| deepspeed.md | ||
| executorch.md | ||
| fast_tokenizers.md | ||
| feature_extractors.md | ||
| fsdp.md | ||
| generation_features.md | ||
| generation_strategies.md | ||
| gguf.md | ||
| glossary.md | ||
| gpu_selection.md | ||
| how_to_hack_models.md | ||
| hpo_train.md | ||
| image_processors.md | ||
| index.md | ||
| installation.md | ||
| kv_cache.md | ||
| llm_optims.md | ||
| llm_tutorial_optimization.md | ||
| llm_tutorial.md | ||
| model_memory_anatomy.md | ||
| model_sharing.md | ||
| model_summary.md | ||
| models.md | ||
| modular_transformers.md | ||
| notebooks.md | ||
| optimizers.md | ||
| pad_truncation.md | ||
| peft.md | ||
| perf_hardware.md | ||
| perf_infer_cpu.md | ||
| perf_infer_gpu_multi.md | ||
| perf_infer_gpu_one.md | ||
| perf_torch_compile.md | ||
| perf_train_cpu_many.md | ||
| perf_train_cpu.md | ||
| perf_train_gpu_many.md | ||
| perf_train_gpu_one.md | ||
| perf_train_special.md | ||
| perf_train_tpu_tf.md | ||
| perplexity.md | ||
| philosophy.md | ||
| pipeline_gradio.md | ||
| pipeline_tutorial.md | ||
| pipeline_webserver.md | ||
| pr_checks.md | ||
| processors.md | ||
| quicktour.md | ||
| run_scripts.md | ||
| serialization.md | ||
| serving.md | ||
| task_summary.md | ||
| tasks_explained.md | ||
| testing.md | ||
| tf_xla.md | ||
| tflite.md | ||
| tokenizer_summary.md | ||
| tools.md | ||
| torchscript.md | ||
| trainer.md | ||
| training.md | ||
| troubleshooting.md | ||