🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support
This report presents the forensic synthetic code analysis of huggingface/accelerate, a Python project with 9,777 GitHub stars. SynthScan v2.0 examined 77,993 lines of code across 337 source files, recording 1283 pattern matches distributed across 15 syntactic categories. The overall adjusted score of 16.9 places this repository in the Moderate AI signal band.
The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).
Longitudinal tracking requires multiple scan runs. Once this repository is re-scanned after new commits land, this chart will visualise how the synthetic code signal evolves over time — enabling you to detect whether AI authorship is growing, stabilising, or being actively corrected by human engineers.
Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.
This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.
The scanner identified 1283 distinct pattern matches across 15 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_big_modeling.py | 191 | def test_init_empty_weights_very_large_model(self): | CODE |
| LOW | tests/test_big_modeling.py | 222 | def test_cpu_offload_with_unused_submodules(self): | CODE |
| LOW | tests/test_big_modeling.py | 276 | def test_disk_offload_with_unused_submodules(self): | CODE |
| LOW | tests/test_big_modeling.py | 317 | def test_dispatch_model_and_remove_hook(self): | CODE |
| LOW | tests/test_big_modeling.py | 356 | def test_dispatch_model_with_non_persistent_buffers(self): | CODE |
| LOW | tests/test_big_modeling.py | 368 | def test_dispatch_model_tied_weights(self): | CODE |
| LOW | tests/test_big_modeling.py | 377 | def test_dispatch_model_tied_weights_memory(self): | CODE |
| LOW | tests/test_big_modeling.py | 442 | def test_dispatch_model_tied_weights_memory_with_nested_offload_cpu(self): | CODE |
| LOW | tests/test_big_modeling.py | 543 | def test_dispatch_model_tied_weights_memory_with_nested_offload_disk(self): | CODE |
| LOW | tests/test_big_modeling.py | 649 | def test_dispatch_model_multi_devices(self): | CODE |
| LOW | tests/test_big_modeling.py | 682 | def test_dispatch_model_move_offloaded_model(self): | CODE |
| LOW | tests/test_big_modeling.py | 692 | def test_dispatch_model_move_model_warning(self): | CODE |
| LOW | tests/test_big_modeling.py | 749 | def test_dispatch_model_with_unused_submodules(self): | CODE |
| LOW | tests/test_big_modeling.py | 765 | def test_dispatch_model_with_unused_submodules_multi_device(self): | CODE |
| LOW | tests/test_big_modeling.py | 780 | def test_dispatch_model_disk_only(self): | CODE |
| LOW | tests/test_big_modeling.py | 795 | def test_dispatch_model_force_hooks(self): | CODE |
| LOW | tests/test_big_modeling.py | 807 | def test_load_checkpoint_and_dispatch(self): | CODE |
| LOW | tests/test_big_modeling.py | 828 | def test_load_checkpoint_and_dispatch_device_map_none(self): | CODE |
| LOW | tests/test_big_modeling.py | 847 | def test_load_checkpoint_and_dispatch_multi_device(self): | CODE |
| LOW | tests/test_big_modeling.py | 872 | def test_load_checkpoint_and_dispatch_with_unused_submodules(self): | CODE |
| LOW | tests/test_big_modeling.py | 899 | def test_load_checkpoint_and_dispatch_multi_device_with_unused_submodules(self): | CODE |
| LOW | tests/test_big_modeling.py | 926 | def test_cpu_offload_with_hook(self): | CODE |
| LOW⚡ | tests/test_optimizer.py | 26 | def test_accelerated_optimizer_pickling(self): | CODE |
| LOW⚡ | tests/test_optimizer.py | 40 | def test_accelerated_optimizer_step_was_skipped(self): | CODE |
| LOW⚡ | tests/test_data_loader.py | 445 | def test_split_batches_validates_dynamic_batch_size(self): | CODE |
| LOW⚡ | tests/test_data_loader.py | 455 | def test_split_batches_validates_non_divisible_batch_size(self): | CODE |
| LOW⚡ | tests/test_data_loader.py | 463 | def check_iterable_dataset_shards( | CODE |
| LOW | tests/test_data_loader.py | 47 | def parameterized_custom_name_func(func, param_num, param): | CODE |
| LOW | tests/test_data_loader.py | 100 | def check_batch_sampler_shards(self, batch_sampler, expected, split_batches=False, even_batches=True): | CODE |
| LOW | tests/test_data_loader.py | 110 | def test_batch_sampler_shards_with_no_splits(self): | CODE |
| LOW | tests/test_data_loader.py | 179 | def test_batch_sampler_shards_with_splits(self): | CODE |
| LOW | tests/test_data_loader.py | 231 | def test_batch_sampler_shards_with_no_splits_no_even(self): | CODE |
| LOW | tests/test_data_loader.py | 300 | def test_batch_sampler_shards_with_splits_no_even(self): | CODE |
| LOW | tests/test_data_loader.py | 352 | def test_batch_sampler_with_varying_batch_size(self): | CODE |
| LOW | tests/test_data_loader.py | 362 | def test_batch_sampler_varying_batch_size_even_batches(self): | CODE |
| LOW | tests/test_data_loader.py | 424 | def test_batch_sampler_varying_batch_size_many_processes(self): | CODE |
| LOW | tests/test_data_loader.py | 503 | def test_iterable_dataset_shard(self): | CODE |
| LOW | tests/test_data_loader.py | 520 | def test_iterable_dataset_using_none_batch_size(self): | CODE |
| LOW | tests/test_data_loader.py | 527 | def test_iterable_dataset_with_non_tensor_samples(self): | CODE |
| LOW | tests/test_data_loader.py | 578 | def test_dataloader_inheritance(self): | CODE |
| LOW | tests/test_data_loader.py | 626 | def test_end_of_dataloader_dispatcher(self): | CODE |
| LOW | tests/test_data_loader.py | 635 | def test_set_epoch_in_batch_sampler(self): | CODE |
| LOW | tests/test_data_loader.py | 649 | def test_skip_first_batches_preserves_iteration(self): | CODE |
| LOW | tests/test_data_loader.py | 668 | def test_skip_first_batches_does_not_reset_sampler_epoch(self): | CODE |
| LOW | tests/test_data_loader.py | 687 | def test_iterable_dataset_native_sharding_when_n_shards_equals_num_processes(self): | CODE |
| LOW | tests/test_data_loader.py | 700 | def test_ensure_dataloader_gets_cleaned_up(self): | CODE |
| LOW | tests/test_data_loader.py | 750 | def test_end_of_dataloader_dispatcher(self): | CODE |
| LOW | tests/test_data_loader.py | 762 | def test_dataloader_state_dict(self, num_workers): | CODE |
| LOW | tests/test_data_loader.py | 789 | def test_dataloader_dispatcher_state_dict(self, num_workers): | CODE |
| LOW | tests/test_data_loader.py | 816 | def test_dataloader_inheritance(self): | CODE |
| LOW | tests/test_data_loader.py | 844 | def test_stateful_dataloader_adapter_equivalent_to_torchdata_stateful_dataloader(self, num_workers): | CODE |
| LOW | tests/test_data_loader.py | 954 | def test_decoupled_stateful_dataloader_adapter_equivalent_to_torchdata_stateful_dataloader(self, num_workers): | CODE |
| LOW | tests/test_utils.py | 143 | def test_patch_environment_key_exists(self): | CODE |
| LOW | tests/test_utils.py | 162 | def test_patch_environment_restores_on_error(self): | CODE |
| LOW | tests/test_utils.py | 181 | def test_can_undo_convert_outputs(self): | CODE |
| LOW | tests/test_utils.py | 242 | def test_dynamo_extract_model_keep_torch_compile(self): | CODE |
| LOW | tests/test_utils.py | 253 | def test_dynamo_extract_model_remove_torch_compile(self): | CODE |
| LOW⚡ | tests/test_utils.py | 269 | def test_check_os_kernel_no_warning_when_release_gt_min(self): | CODE |
| LOW⚡ | tests/test_utils.py | 276 | def test_check_os_kernel_no_warning_when_not_linux(self): | CODE |
| LOW⚡ | tests/test_utils.py | 283 | def test_check_os_kernel_warning_when_release_lt_min(self): | CODE |
| 314 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | manim_animations/dataloaders/stage_6.py | 15 | CODE | |
| LOW | manim_animations/dataloaders/stage_2.py | 15 | CODE | |
| LOW | manim_animations/dataloaders/stage_3.py | 15 | CODE | |
| LOW | manim_animations/dataloaders/stage_7.py | 15 | CODE | |
| LOW | manim_animations/dataloaders/stage_0.py | 15 | CODE | |
| LOW | manim_animations/dataloaders/stage_4.py | 15 | CODE | |
| LOW | manim_animations/dataloaders/stage_5.py | 15 | CODE | |
| LOW | manim_animations/dataloaders/stage_5.py | 168 | CODE | |
| LOW | manim_animations/dataloaders/stage_1.py | 15 | CODE | |
| LOW | manim_animations/big_model_inference/stage_2.py | 15 | CODE | |
| LOW | manim_animations/big_model_inference/stage_3.py | 15 | CODE | |
| LOW | manim_animations/big_model_inference/stage_4.py | 15 | CODE | |
| LOW | manim_animations/big_model_inference/stage_5.py | 15 | CODE | |
| LOW | manim_animations/big_model_inference/stage_1.py | 15 | CODE | |
| LOW | src/accelerate/logging.py | 14 | CODE | |
| LOW | src/accelerate/parallelism_config.py | 30 | CODE | |
| LOW | src/accelerate/__init__.py | 16 | CODE | |
| LOW | src/accelerate/__init__.py | 17 | CODE | |
| LOW | src/accelerate/__init__.py | 17 | CODE | |
| LOW | src/accelerate/__init__.py | 17 | CODE | |
| LOW | src/accelerate/__init__.py | 17 | CODE | |
| LOW | src/accelerate/__init__.py | 17 | CODE | |
| LOW | src/accelerate/__init__.py | 17 | CODE | |
| LOW | src/accelerate/__init__.py | 17 | CODE | |
| LOW | src/accelerate/__init__.py | 26 | CODE | |
| LOW | src/accelerate/__init__.py | 27 | CODE | |
| LOW | src/accelerate/__init__.py | 28 | CODE | |
| LOW | src/accelerate/__init__.py | 28 | CODE | |
| LOW | src/accelerate/__init__.py | 29 | CODE | |
| LOW | src/accelerate/__init__.py | 30 | CODE | |
| LOW | src/accelerate/__init__.py | 31 | CODE | |
| LOW | src/accelerate/__init__.py | 31 | CODE | |
| LOW | src/accelerate/__init__.py | 31 | CODE | |
| LOW | src/accelerate/__init__.py | 31 | CODE | |
| LOW | src/accelerate/__init__.py | 31 | CODE | |
| LOW | src/accelerate/__init__.py | 31 | CODE | |
| LOW | src/accelerate/__init__.py | 31 | CODE | |
| LOW | src/accelerate/__init__.py | 31 | CODE | |
| LOW | src/accelerate/__init__.py | 31 | CODE | |
| LOW | src/accelerate/__init__.py | 31 | CODE | |
| LOW | src/accelerate/__init__.py | 31 | CODE | |
| LOW | src/accelerate/__init__.py | 31 | CODE | |
| LOW | src/accelerate/__init__.py | 31 | CODE | |
| LOW | src/accelerate/__init__.py | 31 | CODE | |
| LOW | src/accelerate/__init__.py | 51 | CODE | |
| LOW | src/accelerate/accelerator.py | 15 | CODE | |
| LOW | src/accelerate/accelerator.py | 167 | CODE | |
| LOW | src/accelerate/state.py | 15 | CODE | |
| LOW | src/accelerate/state.py | 63 | CODE | |
| LOW | src/accelerate/state.py | 66 | CODE | |
| LOW | src/accelerate/state.py | 69 | CODE | |
| LOW | src/accelerate/state.py | 72 | CODE | |
| LOW | src/accelerate/state.py | 761 | CODE | |
| LOW | src/accelerate/test_utils/__init__.py | 14 | CODE | |
| LOW | src/accelerate/test_utils/__init__.py | 14 | CODE | |
| LOW | src/accelerate/test_utils/__init__.py | 14 | CODE | |
| LOW | src/accelerate/test_utils/__init__.py | 14 | CODE | |
| LOW | src/accelerate/test_utils/__init__.py | 14 | CODE | |
| LOW | src/accelerate/test_utils/__init__.py | 14 | CODE | |
| LOW | src/accelerate/test_utils/__init__.py | 14 | CODE | |
| 329 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | setup.py | 1 | # Copyright 2021 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | setup.py | 101 | COMMENT | |
| LOW | setup.py | 121 | # 9. Add release notes to the tag in github once everything is looking hunky-dory. | COMMENT |
| LOW | tests/test_big_modeling.py | 1 | # Copyright 2022 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_optimizer.py | 1 | # Copyright 2022 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_data_loader.py | 1 | # Copyright 2021 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_sagemaker.py | 1 | # Copyright 2021 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_utils.py | 1 | # Copyright 2021 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_cpu.py | 1 | # Copyright 2022 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_launch.py | 1 | # Copyright 2021 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | …ts/test_load_checkpoint_and_dispatch_with_broadcast.py | 1 | # Copyright 2024 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_logging.py | 1 | # Copyright 2024 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_memory_utils.py | 1 | # Copyright 2022 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_metrics.py | 1 | # Copyright 2021 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_imports.py | 1 | # Copyright 2024 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/__init__.py | 1 | # Copyright 2025 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_fp8.py | 1 | # Copyright 2025 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_kwargs_handlers.py | 1 | # Copyright 2021 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_scheduler.py | 1 | # Copyright 2021 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_modeling_utils.py | 1 | # Copyright 2022 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_compile.py | 1 | # Copyright 2025 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_dataclasses.py | 1 | # Copyright 2022 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_accelerator.py | 1 | # Copyright 2022 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_tracking.py | 1 | # Copyright 2022 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_multidevice.py | 1 | # Copyright 2021 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_quantization.py | 1 | # Copyright 2023 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_hooks.py | 1 | # Copyright 2022 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_grad_sync.py | 1 | # Copyright 2021 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/xla_spawn.py | 1 | # Copyright 2021 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_offload.py | 1 | # Copyright 2022 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_cli.py | 1 | # Copyright 2022 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_tpu.py | 1 | # Copyright 2021 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_examples.py | 1 | # Copyright 2022 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_state_checkpointing.py | 1 | # Copyright 2022 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/deepspeed/test_deepspeed_gradient_accumulation.py | 1 | # Copyright 2022 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/deepspeed/test_alst_ulysses_sp.py | 1 | # Copyright 2022 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/deepspeed/test_deepspeed.py | 1 | # Copyright 2022 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/deepspeed/test_deepspeed_multiple_model.py | 1 | # Copyright 2024 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/tp/test_tp.py | 1 | # Copyright 2022 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/tp/fsdp2_tp_preparation.py | 1 | # Copyright 2022 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/fsdp/test_fsdp.py | 1 | # Copyright 2022 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | utils/stale.py | 1 | # Copyright 2022 The HuggingFace Team, the AllenNLP library authors. All rights reserved. | COMMENT |
| LOW | utils/log_reports.py | 1 | # Copyright 2022 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | examples/nlp_example.py | 1 | # Copyright 2021 The HuggingFace Inc. team. All rights reserved. | COMMENT |
| LOW | examples/nlp_example.py | 21 | from transformers import AutoModelForSequenceClassification, AutoTokenizer, get_linear_schedule_with_warmup, set_seed | COMMENT |
| LOW | examples/multigpu_remote_launcher.py | 1 | # Copyright 2023 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | examples/complete_cv_example.py | 1 | # Copyright 2021 The HuggingFace Inc. team. All rights reserved. | COMMENT |
| LOW | examples/complete_cv_example.py | 21 | from timm import create_model | COMMENT |
| LOW | examples/finetune_lm_tpu.py | 1 | # Copyright 2025 The HuggingFace Inc. team. All rights reserved. | COMMENT |
| LOW | examples/complete_nlp_example.py | 1 | # Copyright 2021 The HuggingFace Inc. team. All rights reserved. | COMMENT |
| LOW | examples/complete_nlp_example.py | 21 | from torch.utils.data import DataLoader | COMMENT |
| LOW | examples/cv_example.py | 1 | # Copyright 2021 The HuggingFace Inc. team. All rights reserved. | COMMENT |
| LOW | examples/cv_example.py | 21 | from timm import create_model | COMMENT |
| LOW | examples/by_feature/gradient_accumulation.py | 1 | # Copyright 2021 The HuggingFace Inc. team. All rights reserved. | COMMENT |
| LOW | examples/by_feature/gradient_accumulation.py | 21 | from torch.utils.data import DataLoader | COMMENT |
| LOW | examples/by_feature/memory.py | 1 | # Copyright 2022 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | examples/by_feature/memory.py | 21 | from torch.optim import AdamW | COMMENT |
| LOW | examples/by_feature/local_sgd.py | 1 | # Copyright 2023 The HuggingFace Inc. team. All rights reserved. | COMMENT |
| LOW | examples/by_feature/local_sgd.py | 21 | from torch.utils.data import DataLoader | COMMENT |
| LOW | examples/by_feature/automatic_gradient_accumulation.py | 1 | # Copyright 2022 The HuggingFace Team. All rights reserved. | COMMENT |
| 169 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | examples/nlp_example.py | 0 | creates a set of `dataloader`s for the `salesforce/wikitext` dataset, using "huggingfacetb/smollm-360m" as the tokenizer | STRING |
| HIGH | examples/by_feature/gradient_accumulation.py | 0 | creates a set of `dataloader`s for the `salesforce/wikitext` dataset, using "huggingfacetb/smollm-360m" as the tokenizer | STRING |
| HIGH | examples/by_feature/memory.py | 0 | creates a set of `dataloader`s for the `salesforce/wikitext` dataset, using "huggingfacetb/smollm-360m" as the tokenizer | STRING |
| HIGH | examples/by_feature/local_sgd.py | 0 | creates a set of `dataloader`s for the `salesforce/wikitext` dataset, using "huggingfacetb/smollm-360m" as the tokenizer | STRING |
| HIGH | examples/by_feature/automatic_gradient_accumulation.py | 0 | creates a set of `dataloader`s for the `salesforce/wikitext` dataset, using "huggingfacetb/smollm-360m" as the tokenizer | STRING |
| HIGH | examples/by_feature/multi_process_metrics.py | 0 | creates a set of `dataloader`s for the `salesforce/wikitext` dataset, using "huggingfacetb/smollm-360m" as the tokenizer | STRING |
| HIGH | examples/by_feature/early_stopping.py | 0 | creates a set of `dataloader`s for the `salesforce/wikitext` dataset, using "huggingfacetb/smollm-360m" as the tokenizer | STRING |
| HIGH | examples/by_feature/profiler.py | 0 | creates a set of `dataloader`s for the `salesforce/wikitext` dataset, using "huggingfacetb/smollm-360m" as the tokenizer | STRING |
| HIGH | examples/by_feature/schedule_free.py | 0 | creates a set of `dataloader`s for the `salesforce/wikitext` dataset, using "huggingfacetb/smollm-360m" as the tokenizer | STRING |
| HIGH | examples/by_feature/tracking.py | 0 | creates a set of `dataloader`s for the `salesforce/wikitext` dataset, using "huggingfacetb/smollm-360m" as the tokenizer | STRING |
| HIGH | examples/by_feature/ddp_comm_hook.py | 0 | creates a set of `dataloader`s for the `salesforce/wikitext` dataset, using "huggingfacetb/smollm-360m" as the tokenizer | STRING |
| HIGH | examples/by_feature/checkpointing.py | 0 | creates a set of `dataloader`s for the `salesforce/wikitext` dataset, using "huggingfacetb/smollm-360m" as the tokenizer | STRING |
| HIGH | …ure/gradient_accumulation_for_autoregressive_models.py | 0 | creates a set of `dataloader`s for the `salesforce/wikitext` dataset, using "huggingfacetb/smollm-360m" as the tokenizer | STRING |
| HIGH | benchmarks/fp8/ms_amp/ddp.py | 0 | this script tests to ensure that `accelerate` performs at the same level as raw `torchao`. this particular script verifi | STRING |
| HIGH | benchmarks/fp8/ms_amp/non_distributed.py | 0 | this script tests to ensure that `accelerate` performs at the same level as raw `torchao`. this particular script verifi | STRING |
| HIGH | benchmarks/fp8/transformer_engine/distrib_deepspeed.py | 0 | this script tests to ensure that `accelerate` performs at the same level as raw `torchao`. this particular script verifi | STRING |
| HIGH | benchmarks/fp8/transformer_engine/ddp.py | 0 | this script tests to ensure that `accelerate` performs at the same level as raw `torchao`. this particular script verifi | STRING |
| HIGH | benchmarks/fp8/transformer_engine/non_distributed.py | 0 | this script tests to ensure that `accelerate` performs at the same level as raw `torchao`. this particular script verifi | STRING |
| HIGH | benchmarks/fp8/transformer_engine/fsdp.py | 0 | this script tests to ensure that `accelerate` performs at the same level as raw `torchao`. this particular script verifi | STRING |
| HIGH | benchmarks/fp8/torchao/distrib_deepspeed.py | 0 | this script tests to ensure that `accelerate` performs at the same level as raw `torchao`. this particular script verifi | STRING |
| HIGH | benchmarks/fp8/torchao/ddp.py | 0 | this script tests to ensure that `accelerate` performs at the same level as raw `torchao`. this particular script verifi | STRING |
| HIGH | benchmarks/fp8/torchao/non_distributed.py | 0 | this script tests to ensure that `accelerate` performs at the same level as raw `torchao`. this particular script verifi | STRING |
| HIGH | benchmarks/fp8/torchao/fsdp.py | 0 | this script tests to ensure that `accelerate` performs at the same level as raw `torchao`. this particular script verifi | STRING |
| HIGH | benchmarks/fp8/ms_amp/fp8_utils.py | 0 | returns a tuple of: - model - optimizer - train dataloader (prepared) - eval dataloader (prepared) - lr scheduler suitab | STRING |
| HIGH | benchmarks/fp8/transformer_engine/fp8_utils.py | 0 | returns a tuple of: - model - optimizer - train dataloader (prepared) - eval dataloader (prepared) - lr scheduler suitab | STRING |
| HIGH | benchmarks/fp8/torchao/fp8_utils.py | 0 | returns a tuple of: - model - optimizer - train dataloader (prepared) - eval dataloader (prepared) - lr scheduler suitab | STRING |
| HIGH | benchmarks/fp8/ms_amp/fp8_utils.py | 0 | same thing as `accelerator.get_named_parameters` returns a list of the named parameters of the model (extracted from par | STRING |
| HIGH | benchmarks/fp8/transformer_engine/fp8_utils.py | 0 | same thing as `accelerator.get_named_parameters` returns a list of the named parameters of the model (extracted from par | STRING |
| HIGH | benchmarks/fp8/torchao/fp8_utils.py | 0 | same thing as `accelerator.get_named_parameters` returns a list of the named parameters of the model (extracted from par | STRING |
| HIGH | …_utils/scripts/external_deps/test_ds_multiple_model.py | 0 | creates a set of `dataloader`s for the `glue` dataset. args: accelerator (`accelerator`): an `accelerator` object batch_ | STRING |
| HIGH | …e/test_utils/scripts/external_deps/test_performance.py | 0 | creates a set of `dataloader`s for the `glue` dataset. args: accelerator (`accelerator`): an `accelerator` object batch_ | STRING |
| HIGH | …test_utils/scripts/external_deps/test_checkpointing.py | 0 | creates a set of `dataloader`s for the `glue` dataset. args: accelerator (`accelerator`): an `accelerator` object batch_ | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_tracking.py | 117 | CODE | |
| LOW | tests/test_multidevice.py | 85 | CODE | |
| LOW | tests/test_hooks.py | 66 | CODE | |
| LOW | tests/test_examples.py | 93 | CODE | |
| LOW | tests/deepspeed/test_deepspeed_gradient_accumulation.py | 71 | CODE | |
| LOW | tests/deepspeed/test_deepspeed.py | 281 | CODE | |
| LOW | tests/deepspeed/test_deepspeed.py | 650 | CODE | |
| LOW | tests/deepspeed/test_deepspeed.py | 934 | CODE | |
| LOW | tests/deepspeed/test_deepspeed.py | 979 | CODE | |
| LOW | tests/deepspeed/test_deepspeed.py | 1034 | CODE | |
| LOW | tests/fsdp/test_fsdp.py | 139 | CODE | |
| LOW | tests/fsdp/test_fsdp.py | 214 | CODE | |
| LOW | tests/fsdp/test_fsdp.py | 291 | CODE | |
| LOW | tests/fsdp/test_fsdp.py | 332 | CODE | |
| LOW | tests/fsdp/test_fsdp.py | 429 | CODE | |
| LOW | tests/fsdp/test_fsdp.py | 763 | CODE | |
| LOW | tests/fsdp/test_fsdp.py | 865 | CODE | |
| LOW | examples/complete_cv_example.py | 74 | CODE | |
| LOW | examples/complete_nlp_example.py | 50 | CODE | |
| LOW | examples/by_feature/local_sgd.py | 121 | CODE | |
| LOW | examples/by_feature/deepspeed_with_config_support.py | 264 | CODE | |
| LOW | …ure/gradient_accumulation_for_autoregressive_models.py | 144 | CODE | |
| LOW | examples/by_feature/multi_process_metrics.py | 125 | CODE | |
| LOW | examples/by_feature/megatron_lm_gpt_pretraining.py | 247 | CODE | |
| LOW | examples/by_feature/fsdp_with_peak_mem_tracking.py | 140 | CODE | |
| LOW | examples/by_feature/profiler.py | 119 | CODE | |
| LOW | examples/by_feature/cross_validation.py | 139 | CODE | |
| LOW | examples/by_feature/checkpointing.py | 124 | CODE | |
| LOW | examples/inference/distributed/florence2.py | 44 | CODE | |
| LOW | examples/inference/distributed/florence2.py | 143 | CODE | |
| LOW | examples/inference/distributed/llava_next_video.py | 138 | CODE | |
| LOW | …inference/distributed/distributed_speech_generation.py | 152 | CODE | |
| LOW | benchmarks/fp8/transformer_engine/ddp.py | 39 | CODE | |
| LOW | benchmarks/fp8/transformer_engine/fsdp.py | 47 | CODE | |
| LOW | manim_animations/dataloaders/stage_6.py | 19 | CODE | |
| LOW | manim_animations/dataloaders/stage_7.py | 18 | CODE | |
| LOW | manim_animations/dataloaders/stage_5.py | 18 | CODE | |
| LOW | src/accelerate/logging.py | 49 | CODE | |
| LOW | src/accelerate/hooks.py | 298 | CODE | |
| LOW | src/accelerate/hooks.py | 402 | CODE | |
| LOW | src/accelerate/data_loader.py | 1016 | CODE | |
| LOW | src/accelerate/data_loader.py | 1395 | CODE | |
| LOW | src/accelerate/data_loader.py | 213 | CODE | |
| LOW | src/accelerate/data_loader.py | 806 | CODE | |
| LOW | src/accelerate/launchers.py | 43 | CODE | |
| LOW | src/accelerate/inference.py | 101 | CODE | |
| LOW | src/accelerate/scheduler.py | 54 | CODE | |
| LOW | src/accelerate/accelerator.py | 279 | CODE | |
| LOW | src/accelerate/accelerator.py | 1300 | CODE | |
| LOW | src/accelerate/accelerator.py | 1397 | CODE | |
| LOW | src/accelerate/accelerator.py | 1414 | CODE | |
| LOW | src/accelerate/accelerator.py | 1580 | CODE | |
| LOW | src/accelerate/accelerator.py | 1673 | CODE | |
| LOW | src/accelerate/accelerator.py | 1769 | CODE | |
| LOW | src/accelerate/accelerator.py | 2134 | CODE | |
| LOW | src/accelerate/accelerator.py | 2506 | CODE | |
| LOW | src/accelerate/accelerator.py | 2619 | CODE | |
| LOW | src/accelerate/accelerator.py | 2818 | CODE | |
| LOW | src/accelerate/accelerator.py | 2946 | CODE | |
| LOW | src/accelerate/accelerator.py | 3068 | CODE | |
| 94 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM⚡ | tests/test_modeling_utils.py | 775 | # Create a model where modules cannot be allocated without fallback_allocation | COMMENT |
| MEDIUM⚡ | tests/test_modeling_utils.py | 776 | # Define the inner module with its layers | COMMENT |
| MEDIUM | tests/test_modeling_utils.py | 809 | # Create a model where even the smallest submodules cannot fit | COMMENT |
| MEDIUM | tests/test_modeling_utils.py | 834 | # Create a model with deeper hierarchy | COMMENT |
| MEDIUM | tests/test_modeling_utils.py | 854 | # Create a model with tied weights | COMMENT |
| MEDIUM | tests/test_modeling_utils.py | 873 | # Create a model with buffers | COMMENT |
| MEDIUM | tests/test_modeling_utils.py | 961 | # Create a model with a ModuleList containing more than 10 elements | COMMENT |
| MEDIUM | tests/deepspeed/test_deepspeed_gradient_accumulation.py | 155 | # Create a simple dataloader for prepare to work | COMMENT |
| MEDIUM | examples/inference/pippy/gpt2.py | 29 | # Create an example model | COMMENT |
| MEDIUM | examples/inference/pippy/gpt2.py | 44 | # Create a pipeline stage from the model | COMMENT |
| MEDIUM | examples/inference/pippy/t5.py | 38 | # Create an example model | COMMENT |
| MEDIUM | examples/inference/pippy/t5.py | 55 | # Create a pipeline stage from the model | COMMENT |
| MEDIUM | examples/inference/pippy/llama.py | 34 | # Create a pipeline stage from the model | COMMENT |
| MEDIUM | examples/inference/pippy/bert.py | 29 | # Create an example model | COMMENT |
| MEDIUM | examples/inference/pippy/bert.py | 45 | # Create a pipeline stage from the model | COMMENT |
| MEDIUM | src/accelerate/big_modeling.py | 603 | >>> # Create a model and initialize it with empty weights | STRING |
| MEDIUM | src/accelerate/test_utils/scripts/test_ops.py | 58 | # Create a non-contiguous tensor (enforce non-contiguity after device memory allocation) | COMMENT |
| MEDIUM | src/accelerate/utils/environment.py | 345 | # The below code is based on https://github.com/NVIDIA/DeepLearningExamples/blob/master/TensorFlow2/Language | COMMENT |
| MEDIUM | src/accelerate/utils/dataclasses.py | 2000 | # Create a function that will be used to initialize the parameters of the model | COMMENT |
| MEDIUM | src/accelerate/utils/bnb.py | 389 | # Create a copy of the model | STRING |
| MEDIUM | src/accelerate/utils/modeling.py | 1348 | # Initialize the variables | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_big_modeling.py | 434 | except Exception as e: | CODE |
| LOW | tests/test_big_modeling.py | 519 | except Exception as e: | CODE |
| LOW | tests/test_big_modeling.py | 623 | except Exception as e: | CODE |
| LOW⚡ | tests/test_optimizer.py | 33 | except Exception as e: | CODE |
| LOW | examples/inference/distributed/llava_next_video.py | 131 | except Exception as e: | CODE |
| MEDIUM | examples/inference/distributed/llava_next_video.py | 132 | print(f"Error processing video {video_path}: {str(e)}") | CODE |
| LOW | src/accelerate/accelerator.py | 3137 | except Exception: | CODE |
| LOW | src/accelerate/checkpointing.py | 317 | except Exception: | CODE |
| LOW | src/accelerate/test_utils/testing.py | 875 | except Exception as e: | CODE |
| LOW | src/accelerate/test_utils/testing.py | 892 | except Exception as e: | CODE |
| MEDIUM | src/accelerate/test_utils/testing.py | 136 | def parse_flag_from_env(key, default=False): | CODE |
| LOW | src/accelerate/test_utils/scripts/test_merge_weights.py | 151 | except Exception: | CODE |
| LOW | src/accelerate/utils/memory.py | 181 | except Exception as e: | CODE |
| LOW | src/accelerate/utils/imports.py | 236 | except Exception as e: | CODE |
| LOW | src/accelerate/utils/imports.py | 386 | except Exception: | CODE |
| LOW | src/accelerate/utils/environment.py | 220 | except Exception: | CODE |
| LOW | src/accelerate/utils/environment.py | 241 | except Exception: | CODE |
| LOW⚡ | src/accelerate/utils/modeling.py | 771 | except Exception: | CODE |
| LOW⚡ | src/accelerate/utils/modeling.py | 779 | except Exception: | CODE |
| LOW⚡ | src/accelerate/utils/modeling.py | 787 | except Exception: | CODE |
| LOW⚡ | src/accelerate/utils/modeling.py | 795 | except Exception: | CODE |
| LOW⚡ | src/accelerate/utils/modeling.py | 803 | except Exception: | CODE |
| LOW⚡ | src/accelerate/utils/modeling.py | 811 | except Exception: | CODE |
| LOW⚡ | src/accelerate/utils/modeling.py | 819 | except Exception: | CODE |
| LOW | src/accelerate/utils/modeling.py | 213 | except Exception: | CODE |
| LOW | src/accelerate/commands/launch.py | 1024 | except Exception: | CODE |
| LOW | src/accelerate/commands/launch.py | 1077 | except Exception: | CODE |
| LOW | src/accelerate/commands/launch.py | 1167 | except Exception: | CODE |
| LOW | src/accelerate/commands/config/config_utils.py | 55 | except Exception: | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/fsdp/test_fsdp.py | 165 | # Check if torch enum works | COMMENT |
| LOW | tests/fsdp/test_fsdp.py | 174 | # Check if name works | COMMENT |
| LOW | examples/by_feature/early_stopping.py | 200 | # Check if we should stop the training on any processes | COMMENT |
| LOW | src/accelerate/accelerator.py | 1916 | # Check if the model is already a FSDP model due to `Manual Wrapping` and if so, | COMMENT |
| LOW | src/accelerate/accelerator.py | 2998 | # Set is_xla_gradients_synced to True to avoid all-reduce twice in the AcceleratedOptimizer step. | COMMENT |
| LOW | src/accelerate/accelerator.py | 3785 | # Check if folder exists | COMMENT |
| LOW | src/accelerate/big_modeling.py | 492 | # Check if we are using multi-gpus with RTX 4000 series | COMMENT |
| LOW | …erate/test_utils/scripts/test_distributed_data_loop.py | 272 | # Check if all elements are present in the sorted list of iterated samples | COMMENT |
| LOW | src/accelerate/utils/deepspeed.py | 66 | # Check if the optimizer is PyTorch's Adagrad. | COMMENT |
| LOW | src/accelerate/utils/other.py | 404 | # Check if it's a model and remove duplicates | COMMENT |
| LOW | src/accelerate/utils/environment.py | 214 | # Check if they have the right driver version | COMMENT |
| LOW | src/accelerate/utils/bnb.py | 332 | # Check if the current key is not in the `modules_to_not_convert` | COMMENT |
| LOW | src/accelerate/utils/bnb.py | 401 | # Check if it is a base model | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | examples/inference/distributed/llava_next_video.py | 138 | CODE | |
| LOW | …/inference/distributed/distributed_image_generation.py | 57 | CODE | |
| LOW | src/accelerate/hooks.py | 491 | CODE | |
| LOW | src/accelerate/hooks.py | 586 | CODE | |
| LOW | src/accelerate/hooks.py | 262 | CODE | |
| LOW | src/accelerate/data_loader.py | 1016 | CODE | |
| LOW | src/accelerate/data_loader.py | 545 | CODE | |
| LOW | src/accelerate/data_loader.py | 750 | CODE | |
| LOW | src/accelerate/launchers.py | 43 | CODE | |
| LOW | src/accelerate/accelerator.py | 279 | CODE | |
| LOW | src/accelerate/big_modeling.py | 315 | CODE | |
| LOW | src/accelerate/big_modeling.py | 520 | CODE | |
| LOW | src/accelerate/state.py | 899 | CODE | |
| LOW | src/accelerate/utils/environment.py | 91 | CODE | |
| LOW | src/accelerate/utils/modeling.py | 1295 | CODE | |
| LOW | src/accelerate/utils/modeling.py | 1805 | CODE | |
| LOW | src/accelerate/commands/config/config_args.py | 100 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_big_modeling.py | 53 | logger = logging.getLogger(__name__) | CODE |
| LOW | tests/test_tracking.py | 87 | logger = logging.getLogger(__name__) | CODE |
| LOW | tests/test_state_checkpointing.py | 42 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/accelerate/big_modeling.py | 58 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/accelerate/state.py | 75 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/accelerate/utils/environment.py | 31 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/accelerate/utils/environment.py | 364 | def set_numa_affinity(local_process_index: int, verbose: Optional[bool] = None) -> None: | CODE |
| LOW | src/accelerate/utils/dataclasses.py | 67 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/accelerate/utils/bnb.py | 41 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/accelerate/utils/modeling.py | 70 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/accelerate/commands/to_fsdp2.py | 68 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/accelerate/commands/launch.py | 70 | logger = logging.getLogger(__name__) | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| CRITICAL | tests/test_big_modeling.py | 634 | for pointer, pointer_dict in model.compute1.weight_submodule._hf_hook.tied_params_map.items(): | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | examples/config_yaml_templates/deepspeed.yaml | 3 | # With DeepSpeed, we utilize a deepspeed config file for the entire configuration | COMMENT |
| MEDIUM | examples/config_yaml_templates/multi_node.yaml | 2 | # Generally it's recommended to look at the SLURM config template for a more robust multi-node setup | COMMENT |
| LOW | src/accelerate/big_modeling.py | 425 | # Note: To handle the disk offloading case, we can not simply use weights_map[param_name].data_ptr() as | COMMENT |
| LOW | src/accelerate/utils/modeling.py | 1009 | # We can't just set the memory to model_size // num_devices as it will end being too small: each GPU will get | COMMENT |
| MEDIUM | src/accelerate/utils/fsdp_utils.py | 873 | """Removes parameter name modifiers in order to map them back to their original names. | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | examples/inference/distributed/phi2.py | 80 | # If you need to gather tensors, you can use gather from accelerate.utils | COMMENT |
| MEDIUM | src/accelerate/utils/imports.py | 336 | # With torch 1.12, you can use torch.backends.mps | COMMENT |
| MEDIUM | src/accelerate/utils/imports.py | 337 | # With torch 2.0.0, you can use torch.mps | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | examples/finetune_lm_tpu.py | 144 | # ============================================================================= | COMMENT |
| MEDIUM | examples/finetune_lm_tpu.py | 146 | # ============================================================================= | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_offload.py | 49 | # TODO: add tests on what is inside the index | COMMENT |
| LOW | tests/test_offload.py | 54 | # TODO: add tests on the fact weights are properly loaded | COMMENT |