Repository Analysis

deepspeedai/DeepSpeedExamples

Example models using DeepSpeed

26.1 Moderate AI signal View on GitHub

Analysis Overview

This report presents the forensic synthetic code analysis of deepspeedai/DeepSpeedExamples, a Python project with 6,834 GitHub stars. SynthScan v2.0 examined 103,715 lines of code across 736 source files, recording 1346 pattern matches distributed across 21 syntactic categories. The overall adjusted score of 26.1 places this repository in the Moderate AI signal band.

The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).

26.1
Adjusted Score
26.1
Raw Score
100%
Time Factor
2026-08-04
Last Push
6.8K
Stars
Python
Language
103.7K
Lines of Code
736
Files
1.3K
Pattern Hits
2026-08-09
Scan Date
0.42
HC Hit Rate

What These Metrics Mean

Adjusted Score
Primary synthetic code indicator. Raw score normalised per 1,000 lines of code and multiplied by the temporal discount factor. This is the definitive comparative metric — use it to rank repositories by AI authorship density.
Raw Score
The unmodified sum of all severity-weighted, context-multiplied pattern match scores before temporal discounting. Reflects the absolute signal strength independent of when the repository was last active.
Time Factor
The temporal discount multiplier (0–100%) applied to the raw score. Repositories last updated before ChatGPT's launch (Nov 2022) receive a 5% factor. Full signal is only assigned to repositories active in the post-adoption era (Jan 2024+).
Pattern Hits
Total count of individual pattern matches across all files and categories. A high hit count with a low score may indicate a very large codebase with isolated AI snippets; a low count with a high score indicates dense, concentrated AI signatures.
HC Hit Rate
High+Critical pattern hits per file, averaged across the repository. This orthogonal signal catches repositories where a few files are densely packed with high-severity AI tells — a strong indicator even when the normalised score appears moderate due to codebase size.
Lines of Code / Files
Total lines and files analysed. The scanner examines 94 file extensions. These denominators are used to normalise the score, enabling fair comparison between repositories of vastly different sizes.

Score History

Longitudinal tracking requires multiple scan runs. Once this repository is re-scanned after new commits land, this chart will visualise how the synthetic code signal evolves over time — enabling you to detect whether AI authorship is growing, stabilising, or being actively corrected by human engineers.

No multi-scan history yet — run the scanner again to build trend data.

Severity Breakdown

Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.

CRITICAL 0HIGH 311MEDIUM 47LOW 988

Directory Score Breakdown

This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.

Pattern Findings

The scanner identified 1346 distinct pattern matches across 21 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.

Cross-File Repetition302 hits · 1510 pts
SeverityFileLineSnippetContext
HIGH…aining/data_efficiency/vit_finetuning/main_imagenet.py0computes the accuracy over the k top predictions for the specified values of kSTRING
HIGHtraining/data_efficiency/vit_finetuning/utils/utils.py0computes the accuracy over the k top predictions for the specified values of kSTRING
HIGHtraining/imagenet/main.py0computes the accuracy over the k top predictions for the specified values of kSTRING
HIGHtraining/data_efficiency/gpt_finetuning/analyze_data.py0fine-tuning the library models for causal language modeling (gpt, gpt-2, ctrl, ...) on a text file or a dataset without STRING
HIGH…g/data_efficiency/gpt_finetuning/run_clm_no_trainer.py0fine-tuning the library models for causal language modeling (gpt, gpt-2, ctrl, ...) on a text file or a dataset without STRING
HIGHcompression/gpt2/run_clm_no_trainer.py0fine-tuning the library models for causal language modeling (gpt, gpt-2, ctrl, ...) on a text file or a dataset without STRING
HIGHtraining/bing_bert/deepspeed_train.py0utility function for checkpointing model + optimizer dictionaries the main purpose for this is to be able to resume traiSTRING
HIGHtraining/bing_bert/run_glue_classifier_bert_large.py0utility function for checkpointing model + optimizer dictionaries the main purpose for this is to be able to resume traiSTRING
HIGHtraining/BingBertGlue/run_glue_classifier_bert_large.py0utility function for checkpointing model + optimizer dictionaries the main purpose for this is to be able to resume traiSTRING
HIGHtraining/bing_bert/run_glue_classifier_bert_base.py0a single training/test example for simple sequence classification.STRING
HIGHtraining/bing_bert/run_glue_classifier_bert_large.py0a single training/test example for simple sequence classification.STRING
HIGHtraining/BingBertGlue/run_glue_classifier_bert_base.py0a single training/test example for simple sequence classification.STRING
HIGHtraining/BingBertGlue/run_glue_classifier_bert_large.py0a single training/test example for simple sequence classification.STRING
HIGHtraining/bing_bert/run_glue_classifier_bert_base.py0constructs a inputexample. args: guid: unique id for the example. text_a: string. the untokenized text of the first sequSTRING
HIGHtraining/bing_bert/run_glue_classifier_bert_large.py0constructs a inputexample. args: guid: unique id for the example. text_a: string. the untokenized text of the first sequSTRING
HIGHtraining/BingBertGlue/run_glue_classifier_bert_base.py0constructs a inputexample. args: guid: unique id for the example. text_a: string. the untokenized text of the first sequSTRING
HIGHtraining/BingBertGlue/run_glue_classifier_bert_large.py0constructs a inputexample. args: guid: unique id for the example. text_a: string. the untokenized text of the first sequSTRING
HIGHtraining/bing_bert/run_glue_classifier_bert_base.py0base class for data converters for sequence classification data sets.STRING
HIGHtraining/bing_bert/run_glue_classifier_bert_large.py0base class for data converters for sequence classification data sets.STRING
HIGHtraining/BingBertGlue/run_glue_classifier_bert_base.py0base class for data converters for sequence classification data sets.STRING
HIGHtraining/BingBertGlue/run_glue_classifier_bert_large.py0base class for data converters for sequence classification data sets.STRING
HIGHtraining/bing_bert/run_glue_classifier_bert_base.py0gets a collection of `inputexample`s for the train set.STRING
HIGHtraining/bing_bert/run_glue_classifier_bert_large.py0gets a collection of `inputexample`s for the train set.STRING
HIGHtraining/BingBertGlue/run_glue_classifier_bert_base.py0gets a collection of `inputexample`s for the train set.STRING
HIGHtraining/BingBertGlue/run_glue_classifier_bert_large.py0gets a collection of `inputexample`s for the train set.STRING
HIGHtraining/bing_bert/run_glue_classifier_bert_base.py0gets a collection of `inputexample`s for the dev set.STRING
HIGHtraining/bing_bert/run_glue_classifier_bert_large.py0gets a collection of `inputexample`s for the dev set.STRING
HIGHtraining/BingBertGlue/run_glue_classifier_bert_base.py0gets a collection of `inputexample`s for the dev set.STRING
HIGHtraining/BingBertGlue/run_glue_classifier_bert_large.py0gets a collection of `inputexample`s for the dev set.STRING
HIGHtraining/bing_bert/run_glue_classifier_bert_base.py0processor for the multinli mismatched data set (glue version).STRING
HIGHtraining/bing_bert/run_glue_classifier_bert_large.py0processor for the multinli mismatched data set (glue version).STRING
HIGHtraining/BingBertGlue/run_glue_classifier_bert_base.py0processor for the multinli mismatched data set (glue version).STRING
HIGHtraining/BingBertGlue/run_glue_classifier_bert_large.py0processor for the multinli mismatched data set (glue version).STRING
HIGHtraining/bing_bert/run_glue_classifier_bert_base.py0truncates a sequence pair in place to the maximum length.STRING
HIGHtraining/bing_bert/run_glue_classifier_bert_large.py0truncates a sequence pair in place to the maximum length.STRING
HIGHtraining/BingBertGlue/run_glue_classifier_bert_base.py0truncates a sequence pair in place to the maximum length.STRING
HIGHtraining/BingBertGlue/run_glue_classifier_bert_large.py0truncates a sequence pair in place to the maximum length.STRING
HIGH…ning/bing_bert/pytorch_pretrained_bert/optimization.py0implements bert version of adam algorithm with weight decay fix. params: lr: learning rate warmup: portion of t_total foSTRING
HIGH…g/BingBertGlue/pytorch_pretrained_bert/optimization.py0implements bert version of adam algorithm with weight decay fix. params: lr: learning rate warmup: portion of t_total foSTRING
HIGH…/BingBertSquad/pytorch_pretrained_bert/optimization.py0implements bert version of adam algorithm with weight decay fix. params: lr: learning rate warmup: portion of t_total foSTRING
HIGH…ning/bing_bert/pytorch_pretrained_bert/optimization.py0performs a single optimization step. arguments: closure (callable, optional): a closure that reevaluates the model and rSTRING
HIGH…g/BingBertGlue/pytorch_pretrained_bert/optimization.py0performs a single optimization step. arguments: closure (callable, optional): a closure that reevaluates the model and rSTRING
HIGH…/BingBertSquad/pytorch_pretrained_bert/optimization.py0performs a single optimization step. arguments: closure (callable, optional): a closure that reevaluates the model and rSTRING
HIGH…ning/bing_bert/pytorch_pretrained_bert/tokenization.py0runs basic whitespace cleaning and splitting on a peice of text.STRING
HIGH…g/BingBertGlue/pytorch_pretrained_bert/tokenization.py0runs basic whitespace cleaning and splitting on a peice of text.STRING
HIGH…/BingBertSquad/pytorch_pretrained_bert/tokenization.py0runs basic whitespace cleaning and splitting on a peice of text.STRING
HIGH…ning/bing_bert/pytorch_pretrained_bert/tokenization.py0runs end-to-end tokenization: punctuation splitting + wordpieceSTRING
HIGH…g/BingBertGlue/pytorch_pretrained_bert/tokenization.py0runs end-to-end tokenization: punctuation splitting + wordpieceSTRING
HIGH…/BingBertSquad/pytorch_pretrained_bert/tokenization.py0runs end-to-end tokenization: punctuation splitting + wordpieceSTRING
HIGH…ning/bing_bert/pytorch_pretrained_bert/tokenization.py0converts a sequence of tokens into ids using the vocab.STRING
HIGH…g/BingBertGlue/pytorch_pretrained_bert/tokenization.py0converts a sequence of tokens into ids using the vocab.STRING
HIGH…/BingBertSquad/pytorch_pretrained_bert/tokenization.py0converts a sequence of tokens into ids using the vocab.STRING
HIGH…ning/bing_bert/pytorch_pretrained_bert/tokenization.py0converts a sequence of ids in wordpiece tokens using the vocab.STRING
HIGH…g/BingBertGlue/pytorch_pretrained_bert/tokenization.py0converts a sequence of ids in wordpiece tokens using the vocab.STRING
HIGH…/BingBertSquad/pytorch_pretrained_bert/tokenization.py0converts a sequence of ids in wordpiece tokens using the vocab.STRING
HIGH…ning/bing_bert/pytorch_pretrained_bert/tokenization.py0instantiate a pretrainedbertmodel from a pre-trained model file. download and cache the pre-trained model file if neededSTRING
HIGH…g/BingBertGlue/pytorch_pretrained_bert/tokenization.py0instantiate a pretrainedbertmodel from a pre-trained model file. download and cache the pre-trained model file if neededSTRING
HIGH…/BingBertSquad/pytorch_pretrained_bert/tokenization.py0instantiate a pretrainedbertmodel from a pre-trained model file. download and cache the pre-trained model file if neededSTRING
HIGH…ning/bing_bert/pytorch_pretrained_bert/tokenization.py0runs basic tokenization (punctuation splitting, lower casing, etc.).STRING
HIGH…g/BingBertGlue/pytorch_pretrained_bert/tokenization.py0runs basic tokenization (punctuation splitting, lower casing, etc.).STRING
242 more matches not shown…
Unused Imports408 hits · 408 pts
SeverityFileLineSnippetContext
LOW…aining/data_efficiency/vit_finetuning/main_imagenet.py28CODE
LOW…aining/data_efficiency/vit_finetuning/main_imagenet.py30CODE
LOW…aining/data_efficiency/vit_finetuning/main_imagenet.py30CODE
LOWtraining/data_efficiency/vit_finetuning/main_cifar.py16CODE
LOWtraining/data_efficiency/vit_finetuning/main_cifar.py20CODE
LOWtraining/data_efficiency/vit_finetuning/main_cifar.py21CODE
LOWtraining/data_efficiency/vit_finetuning/main_cifar.py23CODE
LOWtraining/data_efficiency/vit_finetuning/main_cifar.py29CODE
LOWtraining/data_efficiency/vit_finetuning/main_cifar.py30CODE
LOWtraining/data_efficiency/vit_finetuning/main_cifar.py33CODE
LOW…ining/data_efficiency/vit_finetuning/utils/__init__.py15CODE
LOW…ining/data_efficiency/vit_finetuning/utils/__init__.py15CODE
LOW…ining/data_efficiency/vit_finetuning/utils/__init__.py15CODE
LOW…ining/data_efficiency/vit_finetuning/utils/__init__.py15CODE
LOW…ining/data_efficiency/vit_finetuning/utils/__init__.py15CODE
LOW…ining/data_efficiency/vit_finetuning/utils/__init__.py15CODE
LOW…ining/data_efficiency/vit_finetuning/utils/__init__.py15CODE
LOW…ining/data_efficiency/vit_finetuning/utils/__init__.py15CODE
LOW…ining/data_efficiency/vit_finetuning/utils/__init__.py16CODE
LOWtraining/data_efficiency/vit_finetuning/utils/utils.py15CODE
LOWtraining/data_efficiency/vit_finetuning/utils/utils.py17CODE
LOWtraining/data_efficiency/vit_finetuning/utils/utils.py21CODE
LOWtraining/data_efficiency/vit_finetuning/utils/utils.py23CODE
LOWtraining/data_efficiency/vit_finetuning/utils/utils.py24CODE
LOWtraining/data_efficiency/vit_finetuning/utils/utils.py30CODE
LOWtraining/data_efficiency/vit_finetuning/utils/utils.py32CODE
LOW…ning/data_efficiency/vit_finetuning/models/__init__.py15CODE
LOW…ning/data_efficiency/vit_finetuning/models/__init__.py16CODE
LOW…ning/data_efficiency/vit_finetuning/models/__init__.py17CODE
LOWtraining/data_efficiency/vit_finetuning/models/vit.py26CODE
LOWtraining/data_efficiency/vit_finetuning/models/vit.py33CODE
LOWtraining/data_efficiency/gpt_finetuning/analyze_data.py27CODE
LOWtraining/data_efficiency/gpt_finetuning/analyze_data.py29CODE
LOWtraining/data_efficiency/gpt_finetuning/analyze_data.py31CODE
LOWtraining/data_efficiency/gpt_finetuning/analyze_data.py32CODE
LOWtraining/data_efficiency/gpt_finetuning/analyze_data.py34CODE
LOWtraining/data_efficiency/gpt_finetuning/analyze_data.py37CODE
LOWtraining/data_efficiency/gpt_finetuning/analyze_data.py37CODE
LOWtraining/data_efficiency/gpt_finetuning/analyze_data.py37CODE
LOWtraining/data_efficiency/gpt_finetuning/analyze_data.py38CODE
LOWtraining/data_efficiency/gpt_finetuning/analyze_data.py39CODE
LOWtraining/data_efficiency/gpt_finetuning/analyze_data.py41CODE
LOWtraining/data_efficiency/gpt_finetuning/analyze_data.py41CODE
LOWtraining/data_efficiency/gpt_finetuning/analyze_data.py41CODE
LOWtraining/data_efficiency/gpt_finetuning/analyze_data.py41CODE
LOWtraining/data_efficiency/gpt_finetuning/analyze_data.py41CODE
LOWtraining/data_efficiency/gpt_finetuning/analyze_data.py55CODE
LOWtraining/data_efficiency/gpt_finetuning/analyze_data.py65CODE
LOW…g/data_efficiency/gpt_finetuning/run_clm_no_trainer.py31CODE
LOW…g/data_efficiency/gpt_finetuning/run_clm_no_trainer.py32CODE
LOW…g/data_efficiency/gpt_finetuning/run_clm_no_trainer.py34CODE
LOW…g/data_efficiency/gpt_finetuning/run_clm_no_trainer.py39CODE
LOWtraining/pipeline_parallelism/train.py12CODE
LOWtraining/bing_bert/deepspeed_train.py11CODE
LOWtraining/bing_bert/deepspeed_train.py18CODE
LOWtraining/bing_bert/deepspeed_train.py19CODE
LOWtraining/bing_bert/deepspeed_train.py19CODE
LOWtraining/bing_bert/deepspeed_train.py19CODE
LOWtraining/bing_bert/deepspeed_train.py21CODE
LOWtraining/bing_bert/deepspeed_train.py21CODE
348 more matches not shown…
Hyper-Verbose Identifiers160 hits · 168 pts
SeverityFileLineSnippetContext
LOWtraining/data_efficiency/vit_finetuning/models/vit.py662def _create_vision_transformer(variant, pretrained=False, **kwargs):CODE
LOW…g/tensor_parallel/hf_integration/train_bench_length.py67def smart_tokenizer_and_embedding_resize(CODE
LOW…g/tensor_parallel/hf_integration/train_bench_length.py193def make_supervised_data_module(tokenizer: transformers.PreTrainedTokenizer, data_args) -> Dict:CODE
LOWtraining/tensor_parallel/hf_integration/train.py65def smart_tokenizer_and_embedding_resize(CODE
LOWtraining/tensor_parallel/hf_integration/train.py188def make_supervised_data_module(tokenizer: transformers.PreTrainedTokenizer, data_args) -> Dict:CODE
LOW…aining/tensor_parallel/basic_example/autotp_example.py27 def get_data_parallel_world_size(self):CODE
LOW…aining/tensor_parallel/basic_example/autotp_example.py30 def get_model_parallel_world_size(self):CODE
LOW…tensor_parallel/basic_example/autotp_memory_compare.py28 def get_data_parallel_world_size(self):CODE
LOW…tensor_parallel/basic_example/autotp_memory_compare.py31 def get_model_parallel_world_size(self):CODE
LOW…sor_parallel/custom_patterns/autotp_custom_patterns.py32 def get_data_parallel_world_size(self):CODE
LOW…sor_parallel/custom_patterns/autotp_custom_patterns.py35 def get_model_parallel_world_size(self):CODE
LOWtraining/opsd/losses.py64def chunked_distillation_loss(CODE
LOWtraining/opsd/losses.py122def streamed_distillation_loss(CODE
LOWtraining/opsd/tests/test_losses.py28def test_positive_when_different(loss_type):CODE
LOWtraining/opsd/tests/test_losses.py38def test_chunking_equivalent_to_unchunked(loss_type):CODE
LOWtraining/opsd/tests/test_losses.py48def test_mask_excludes_tokens():CODE
LOWtraining/opsd/tests/test_losses.py58def test_gradient_flows_to_student():CODE
LOWtraining/opsd/tests/test_losses.py69def test_gradient_does_not_flow_to_teacher_when_detached():CODE
LOWtraining/opsd/tests/test_losses.py79def test_unknown_loss_type_raises():CODE
LOWtraining/opsd/tests/test_losses.py87def test_shape_mismatch_raises():CODE
LOWtraining/opsd/tests/test_losses.py95def test_mask_shape_mismatch_raises():CODE
LOWtraining/opsd/tests/test_losses.py104def test_temperature_changes_loss_but_stays_finite(temperature):CODE
LOWtraining/opsd/tests/test_losses.py123def test_all_zero_mask_returns_zero():CODE
LOWtraining/opsd/tests/test_losses.py132def test_build_response_mask_basic():CODE
LOWtraining/opsd/tests/test_losses.py140def test_build_response_mask_validates_shapes():CODE
LOWtraining/opsd/tests/test_teacher_caching.py17def test_round_trip_preserves_values_within_dtype():CODE
LOWtraining/opsd/tests/test_teacher_caching.py29def test_chunk_slicing_is_correct():CODE
LOWtraining/opsd/tests/test_teacher_caching.py39def test_invalid_chunk_bounds_raise():CODE
LOWtraining/opsd/tests/test_teacher_caching.py54def test_rejects_gpu_resident_logits():CODE
LOWtraining/opsd/tests/test_teacher_caching.py61def test_dtype_override_in_chunk_to_device():CODE
LOWtraining/opsd/tests/test_teacher_caching.py68def test_free_releases_buffer():CODE
LOWtraining/opsd/tests/test_teacher_caching.py82def test_streamed_chunked_loss_matches_full_loss():CODE
LOWtraining/opsd/tests/test_data.py36def test_collator_left_pads_real_tokens_to_right_edge():CODE
LOWtraining/opsd/tests/test_data.py55def test_collator_truncates_to_max_prompt_length():CODE
LOWtraining/opsd/tests/test_data.py64def test_collator_falls_back_to_eos_when_no_pad_id():CODE
LOWtraining/opsd/tests/test_data.py74def test_collator_raises_without_pad_and_eos():CODE
LOWtraining/opsd/tests/test_data.py80def test_prompt_dataset_reads_jsonl(tmp_path):CODE
LOWtraining/opsd/tests/test_data.py89def test_prompt_dataset_missing_field_raises(tmp_path):CODE
LOWtraining/bing_bert/deepspeed_train.py366def prepare_optimizer_parameters(args, model):CODE
LOWtraining/bing_bert/nvidia_bert_dataset_provider.py30def create_pretraining_dataset(input_file, max_predictions_per_seq,CODE
LOWtraining/bing_bert/run_glue_classifier_bert_base.py82class DataProcessor(object):CODE
LOWtraining/bing_bert/run_glue_classifier_bert_base.py411def convert_examples_to_features(examples, label_list, max_seq_length,CODE
LOWtraining/bing_bert/run_glue_classifier_bert_large.py109class DataProcessor(object):CODE
LOWtraining/bing_bert/run_glue_classifier_bert_large.py453def convert_examples_to_features(examples, label_list, max_seq_length,CODE
LOW…ch_pretrained_bert/convert_tf_checkpoint_to_pytorch.py31def convert_tf_checkpoint_to_pytorch(tf_checkpoint_path, bert_config_file,CODE
LOWtraining/bing_bert/nvidia/modelingpreln_layerdrop.py79def get_sparse_attention_config(args, num_heads):CODE
LOWtraining/bing_bert/nvidia/modelingpreln_layerdrop.py112def get_sparse_attention_utils(sparse_attention_config):CODE
LOWtraining/bing_bert/nvidia/modelingpreln.py78def get_sparse_attention_config(args, num_heads):CODE
LOWtraining/bing_bert/nvidia/modelingpreln.py110def get_sparse_attention_utils(sparse_attention_config):CODE
LOWtraining/bing_bert/turing/dataset.py344 def create_masked_lm_predictions(self, tokens):CODE
LOWtraining/BingBertGlue/nvidia_bert_dataset_provider.py30def create_pretraining_dataset(input_file, max_predictions_per_seq,CODE
LOWtraining/BingBertGlue/run_glue_classifier_bert_base.py81class DataProcessor(object):CODE
LOWtraining/BingBertGlue/run_glue_classifier_bert_base.py410def convert_examples_to_features(examples, label_list, max_seq_length,CODE
LOWtraining/BingBertGlue/run_glue_classifier_bert_large.py110class DataProcessor(object):CODE
LOWtraining/BingBertGlue/run_glue_classifier_bert_large.py454def convert_examples_to_features(examples, label_list, max_seq_length,CODE
LOW…ch_pretrained_bert/convert_tf_checkpoint_to_pytorch.py31def convert_tf_checkpoint_to_pytorch(tf_checkpoint_path, bert_config_file,CODE
LOWtraining/BingBertGlue/nvidia/modelingpreln_layerdrop.py79def get_sparse_attention_config(args, num_heads):CODE
LOWtraining/BingBertGlue/nvidia/modelingpreln_layerdrop.py112def get_sparse_attention_utils(sparse_attention_config):CODE
LOWtraining/BingBertGlue/nvidia/modelingpreln.py78def get_sparse_attention_config(args, num_heads):CODE
LOWtraining/BingBertGlue/nvidia/modelingpreln.py110def get_sparse_attention_utils(sparse_attention_config):CODE
100 more matches not shown…
Deep Nesting149 hits · 148 pts
SeverityFileLineSnippetContext
LOWtraining/gan/gan_baseline_train.py14CODE
LOWtraining/gan/gan_deepspeed_train.py16CODE
LOW…aining/data_efficiency/vit_finetuning/main_imagenet.py109CODE
LOW…aining/data_efficiency/vit_finetuning/main_imagenet.py513CODE
LOWtraining/data_efficiency/vit_finetuning/utils/utils.py63CODE
LOWtraining/data_efficiency/vit_finetuning/utils/utils.py75CODE
LOWtraining/data_efficiency/vit_finetuning/models/vit.py525CODE
LOWtraining/data_efficiency/vit_finetuning/models/vit.py631CODE
LOWtraining/data_efficiency/vit_finetuning/models/vit.py530CODE
LOW…g/data_efficiency/gpt_finetuning/run_clm_no_trainer.py241CODE
LOW…g/data_efficiency/gpt_finetuning/run_clm_no_trainer.py497CODE
LOWtraining/tensor_parallel/hf_integration/utils.py39CODE
LOWtraining/bing_bert/deepspeed_train.py132CODE
LOWtraining/bing_bert/deepspeed_train.py366CODE
LOWtraining/bing_bert/run_glue_classifier_bert_base.py546CODE
LOWtraining/bing_bert/run_glue_classifier_bert_base.py572CODE
LOWtraining/bing_bert/run_glue_classifier_bert_large.py587CODE
LOWtraining/bing_bert/run_glue_classifier_bert_large.py613CODE
LOW…ch_pretrained_bert/convert_tf_checkpoint_to_pytorch.py31CODE
LOW…ning/bing_bert/pytorch_pretrained_bert/tokenization.py297CODE
LOWtraining/bing_bert/nvidia/modelingpreln_layerdrop.py79CODE
LOWtraining/bing_bert/nvidia/modelingpreln_layerdrop.py119CODE
LOWtraining/bing_bert/nvidia/modelingpreln_layerdrop.py631CODE
LOWtraining/bing_bert/nvidia/modelingpreln.py78CODE
LOWtraining/bing_bert/nvidia/modelingpreln.py116CODE
LOWtraining/bing_bert/turing/sources.py167CODE
LOWtraining/bing_bert/turing/sources.py258CODE
LOWtraining/BingBertGlue/run_glue_classifier_bert_base.py545CODE
LOWtraining/BingBertGlue/run_glue_classifier_bert_base.py571CODE
LOWtraining/BingBertGlue/run_glue_classifier_bert_large.py588CODE
LOWtraining/BingBertGlue/run_glue_classifier_bert_large.py614CODE
LOW…ch_pretrained_bert/convert_tf_checkpoint_to_pytorch.py31CODE
LOW…g/BingBertGlue/pytorch_pretrained_bert/tokenization.py297CODE
LOWtraining/BingBertGlue/nvidia/modelingpreln_layerdrop.py79CODE
LOWtraining/BingBertGlue/nvidia/modelingpreln_layerdrop.py119CODE
LOWtraining/BingBertGlue/nvidia/modelingpreln_layerdrop.py631CODE
LOWtraining/BingBertGlue/nvidia/modeling.py70CODE
LOWtraining/BingBertGlue/nvidia/modelingpreln.py78CODE
LOWtraining/BingBertGlue/nvidia/modelingpreln.py116CODE
LOWtraining/BingBertGlue/turing/sources.py167CODE
LOWtraining/BingBertGlue/turing/sources.py258CODE
LOWtraining/DeepSpeed-Domino/domino/language_model.py276CODE
LOWtraining/DeepSpeed-Domino/domino/language_model.py544CODE
LOWtraining/DeepSpeed-Domino/domino/training.py476CODE
LOWtraining/DeepSpeed-Domino/domino/training.py739CODE
LOWtraining/imagenet/main.py141CODE
LOWtraining/imagenet/main.py453CODE
LOWtraining/deepspeed_finetune_demo/convert_ds_to_hf.py55CODE
LOWtraining/deepspeed_finetune_demo/finetune_llama.py254CODE
LOW…g/deepspeed_finetune_demo/evaluate/gsm8k/eval_gsm8k.py13CODE
LOW…ning/deepspeed_finetune_demo/evaluate/mmlu/gen_mmlu.py31CODE
LOW…ing/deepspeed_finetune_demo/evaluate/mmlu/eval_mmlu.py13CODE
LOWtraining/stable_diffusion/train_sd_distil_lora.py707CODE
LOW…ng/stable_diffusion/local_pipeline_stable_diffusion.py279CODE
LOW…ng/stable_diffusion/local_pipeline_stable_diffusion.py518CODE
LOW…aining/BingBertSquad/convert_bert_ckpt_to_deepspeed.py25CODE
LOW…aining/BingBertSquad/convert_bert_ckpt_to_deepspeed.py173CODE
LOW…aining/BingBertSquad/convert_bert_ckpt_to_deepspeed.py282CODE
LOWtraining/BingBertSquad/nvidia_run_squad_baseline.py112CODE
LOWtraining/BingBertSquad/nvidia_run_squad_baseline.py183CODE
89 more matches not shown…
Over-Commented Block112 hits · 112 pts
SeverityFileLineSnippetContext
LOW…atch_size_and_lr/variable_batch_size_and_lr_example.py141 "data_sampling": {COMMENT
LOW…g/data_efficiency/variable_batch_size_and_lr/README.md81 "dynamic_batching": {COMMENT
LOWtraining/data_efficiency/vit_finetuning/main_cifar.py1# Copyright 2021 Google LLCCOMMENT
LOW…ncy/vit_finetuning/bash_script/run_cifar_random_ltd.sh1#!/bin/bashCOMMENT
LOW…/vit_finetuning/bash_script/run_imagenet_random_ltd.sh21# --deepspeed \COMMENT
LOW…ining/data_efficiency/vit_finetuning/utils/__init__.py1# Copyright 2021 Google LLCCOMMENT
LOW…ining/data_efficiency/vit_finetuning/utils/get_data.py1# Copyright 2021 Google LLCCOMMENT
LOWtraining/data_efficiency/vit_finetuning/utils/utils.py1# Copyright 2021 Google LLCCOMMENT
LOW…ning/data_efficiency/vit_finetuning/models/__init__.py1# Copyright 2021 Google LLCCOMMENT
LOW…ining/data_efficiency/gpt_finetuning/learning_rates.py1# This code is adopted from https://github.com/deepspeedai/Megatron-DeepSpeed/blob/main/megatron/learning_rates.pyCOMMENT
LOWtraining/data_efficiency/gpt_finetuning/analyze_data.py1#!/usr/bin/env pythonCOMMENT
LOW…g/data_efficiency/gpt_finetuning/run_clm_no_trainer.py1#!/usr/bin/env pythonCOMMENT
LOW…ency/gpt_finetuning/bash_script/run_base_random_ltd.sh21# --random_ltd \COMMENT
LOW…g/tensor_parallel/hf_integration/train_bench_length.py1# Copyright 2023 Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen LiCOMMENT
LOWtraining/tensor_parallel/hf_integration/train.py1# Copyright 2023 Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen LiCOMMENT
LOWtraining/bing_bert/run_glue_classifier_bert_base.py1# coding=utf-8COMMENT
LOWtraining/bing_bert/run_glue_classifier_bert_base.py441 # tokens: [CLS] the dog is hairy . [SEP]COMMENT
LOWtraining/bing_bert/NOTICE.txt21Open Source License/Copyright Notice.COMMENT
LOWtraining/bing_bert/run_glue_classifier_bert_large.py1# coding=utf-8COMMENT
LOWtraining/bing_bert/run_glue_classifier_bert_large.py481 # type_ids: 0 0 0 0 0 0 0 0 1 1 1 1 1 1COMMENT
LOW…/mpi_train_bert_onebitlamb_bsz32k_seq512_infiniband.sh1#!/bin/bashCOMMENT
LOW…/mpi_train_bert_onebitlamb_bsz64k_seq128_infiniband.sh1#!/bin/bashCOMMENT
LOW…et/mpi_train_bert_onebitlamb_bsz32k_seq512_ethernet.sh1#!/bin/bashCOMMENT
LOW…et/mpi_train_bert_onebitlamb_bsz64k_seq128_ethernet.sh1#!/bin/bashCOMMENT
LOW…ch_pretrained_bert/convert_tf_checkpoint_to_pytorch.py1# coding=utf-8COMMENT
LOW…ning/bing_bert/pytorch_pretrained_bert/optimization.py1# coding=utf-8COMMENT
LOW…ning/bing_bert/pytorch_pretrained_bert/tokenization.py1# coding=utf-8COMMENT
LOWtraining/bing_bert/pytorch_pretrained_bert/modeling.py1# coding=utf-8COMMENT
LOWtraining/bing_bert/nvidia/modelingpreln_layerdrop.py1# DeepSpeed note, code taken from commit 3d59216cec89a363649b4fe3d15295ba936ced0fCOMMENT
LOWtraining/bing_bert/nvidia/modelingpreln_layerdrop.py681COMMENT
LOWtraining/bing_bert/nvidia/modelingpreln.py1# DeepSpeed note, code taken from commit 3d59216cec89a363649b4fe3d15295ba936ced0fCOMMENT
LOWtraining/bing_bert/nvidia/modelingpreln.py661# super(BertEncoder, self).__init__()COMMENT
LOW…d/mpi_train_bert_onebitadam_bsz4k_seq128_infiniband.sh1#!/bin/bashCOMMENT
LOW…net/mpi_train_bert_onebitadam_bsz4k_seq128_ethernet.sh1#!/bin/bashCOMMENT
LOWtraining/BingBertGlue/run_glue_classifier_bert_base.py1# coding=utf-8COMMENT
LOWtraining/BingBertGlue/run_glue_classifier_bert_base.py441 # type_ids: 0 0 0 0 0 0 0COMMENT
LOWtraining/BingBertGlue/run_glue_classifier_bert_large.py1# coding=utf-8COMMENT
LOWtraining/BingBertGlue/run_glue_classifier_bert_large.py481 # tokens: [CLS] is this jack ##son ##ville ? [SEP] no it is not . [SEP]COMMENT
LOW…ch_pretrained_bert/convert_tf_checkpoint_to_pytorch.py1# coding=utf-8COMMENT
LOW…g/BingBertGlue/pytorch_pretrained_bert/optimization.py1# coding=utf-8COMMENT
LOW…g/BingBertGlue/pytorch_pretrained_bert/tokenization.py1# coding=utf-8COMMENT
LOW…ining/BingBertGlue/pytorch_pretrained_bert/modeling.py1# coding=utf-8COMMENT
LOWtraining/BingBertGlue/nvidia/modelingpreln_layerdrop.py1# DeepSpeed note, code taken from commit 3d59216cec89a363649b4fe3d15295ba936ced0fCOMMENT
LOWtraining/BingBertGlue/nvidia/modelingpreln_layerdrop.py681COMMENT
LOWtraining/BingBertGlue/nvidia/modeling.py1# DeepSpeed note, code taken from commit 3d59216cec89a363649b4fe3d15295ba936ced0fCOMMENT
LOWtraining/BingBertGlue/nvidia/modeling.py581#class BertEncoder(nn.Module):COMMENT
LOWtraining/BingBertGlue/nvidia/modelingpreln.py1# DeepSpeed note, code taken from commit 3d59216cec89a363649b4fe3d15295ba936ced0fCOMMENT
LOWtraining/BingBertGlue/nvidia/modelingpreln.py661COMMENT
LOWtraining/BingBertGlue/nvidia/modelingpreln.py1021COMMENT
LOWtraining/imagenet/extract_ILSVRC.sh1#!/bin/bashCOMMENT
LOWtraining/imagenet/extract_ILSVRC.sh21# │ ├── ILSVRC2012_val_00002138.JPEGCOMMENT
LOWtraining/imagenet/extract_ILSVRC.sh41# 2. extract and copy contents of .tar file into directoryCOMMENT
LOWtraining/imagenet/extract_ILSVRC.sh61# get script from soumith and run; this script creates all class directories and moves images into corresponding directoCOMMENT
LOWtraining/deepspeed_finetune_demo/run_and_evaluate.sh1#!/bin/bashCOMMENT
LOWtraining/stable_diffusion/train_sd_distil_lora.py1#!/usr/bin/env pythonCOMMENT
LOW…ng/stable_diffusion/local_pipeline_stable_diffusion.py1# Copyright 2023 The HuggingFace Team. All rights reserved.COMMENT
LOWtraining/BingBertSquad/nvidia_run_squad_baseline.py1# coding=utf-8COMMENT
LOWtraining/BingBertSquad/nvidia_run_squad_baseline.py341def _improve_answer_span(doc_tokens, input_start, input_end, tokenizer,COMMENT
LOWtraining/BingBertSquad/nvidia_run_squad_baseline.py381 # Because of the sliding window approach taken to scoring documents, a singleCOMMENT
LOWtraining/BingBertSquad/nvidia_run_squad_baseline.py561 # (whitespace tokenized) tokens and our WordPiece tokenized tokens. SoCOMMENT
52 more matches not shown…
Self-Referential Comments30 hits · 88 pts
SeverityFileLineSnippetContext
MEDIUMtraining/HelloDeepSpeed/train_bert_ds.py149 # Create the labels firstCOMMENT
MEDIUMtraining/HelloDeepSpeed/train_bert.py119 # Create the labels firstCOMMENT
MEDIUMtraining/DeepSpeed-Domino/pretrain_gpt3_13b.sh1# This file is adapted from pretrain_gpt.sh in Megatron-LMCOMMENT
MEDIUMtraining/DeepSpeed-Domino/pretrain_gpt.py2# This file is adapted from pretrain_gpt.py in Megatron-LMCOMMENT
MEDIUMtraining/DeepSpeed-Domino/pretrain_gpt3_6.7b.sh1# This file is adapted from pretrain_gpt.sh in Megatron-LMCOMMENT
MEDIUMtraining/DeepSpeed-Domino/domino/language_model.py1# This file is adapted from language_model.py in Megatron-LMCOMMENT
MEDIUMtraining/DeepSpeed-Domino/domino/gpt_model.py1# This file is adapted from gpt_model.py in Megatron-LMCOMMENT
MEDIUMtraining/DeepSpeed-Domino/domino/training.py2# This file is adapted from training.py in Megatron-LMCOMMENT
MEDIUMtraining/stable_diffusion/train_sd_distil_lora.py1235 # Create the pipeline using using the trained modules and save it.STRING
MEDIUMtraining/cifar/cifar10_deepspeed.py237 # Define the test dataloader.COMMENT
MEDIUMtraining/cifar/cifar10_deepspeed.py356 # Define the Classification Cross-Entropy loss function.COMMENT
MEDIUM…chmarks/inference/deepspeedometer/tests/test_config.py22 # Create a config that would generate 6 benchmark settingsCOMMENT
MEDIUM…ce/deepspeedometer/src/deepspeedometer/sample_input.py95 # Define the dataset and data collatorCOMMENT
MEDIUM…ce/deepspeedometer/src/deepspeedometer/sample_input.py99 # Define the training argumentsCOMMENT
MEDIUMbenchmarks/inference/mii/src/sample_input.py95 # Define the dataset and data collatorCOMMENT
MEDIUMbenchmarks/inference/mii/src/sample_input.py99 # Define the training argumentsCOMMENT
MEDIUMbenchmarks/inference/mii/src/utils.py150 # Create the parser, inheriting from the server and/or client parsersCOMMENT
MEDIUMbenchmarks/autosp/distributed_attention.py65 # Create the attention mask from padding mask + causal maskCOMMENT
MEDIUMcompression/cifar/utils.py5# This file is part of PyHessian library.COMMENT
MEDIUM…lications/DeepSpeed-VisualChat/utils/model/vis_proj.py28# The following code is adopted from QWen-ClipCOMMENT
MEDIUM…ons/DeepSpeed-VisualChat/utils/data/ocr_vqa_dataset.py1# This file is adapted from https://github.com/open-mmlab/Multimodal-GPTCOMMENT
MEDIUM…eepSpeed-VisualChat/utils/data/cc_sbu_align_dataset.py1# This file is adapted from https://github.com/open-mmlab/Multimodal-GPTCOMMENT
MEDIUM…ations/DeepSpeed-VisualChat/utils/data/dial_dataset.py1# This file is adapted from https://github.com/open-mmlab/Multimodal-GPTCOMMENT
MEDIUM…plications/DeepSpeed-VisualChat/utils/data/__init__.py1# This file is adapted from https://github.com/open-mmlab/Multimodal-GPTCOMMENT
MEDIUMapplications/DeepSpeed-VisualChat/utils/data/builder.py1# This file is adapted from https://github.com/open-mmlab/Multimodal-GPTCOMMENT
MEDIUM…cations/DeepSpeed-VisualChat/utils/data/vqa_dataset.py1# This file is adapted from https://github.com/open-mmlab/Multimodal-GPTCOMMENT
MEDIUM…ions/DeepSpeed-VisualChat/utils/data/aokvqa_dataset.py1# This file is adapted from https://github.com/open-mmlab/Multimodal-GPTCOMMENT
MEDIUM…tions/DeepSpeed-VisualChat/utils/data/llava_dataset.py1# This file is adapted from https://github.com/open-mmlab/Multimodal-GPTCOMMENT
MEDIUM…eepSpeed-VisualChat/utils/data/coco_caption_dataset.py1# This file is adapted from https://github.com/open-mmlab/Multimodal-GPTCOMMENT
MEDIUMapplications/DeepSpeed-Chat/dschat/utils/utils.py152# This function is a modified version of code available in the from_pretrained API of HuggingFace TransformersCOMMENT
AI Structural Patterns72 hits · 72 pts
SeverityFileLineSnippetContext
LOWtraining/data_efficiency/vit_finetuning/models/vit.py229CODE
LOWtraining/data_efficiency/vit_finetuning/models/vit.py252CODE
LOWtraining/data_efficiency/vit_finetuning/models/vit.py282CODE
LOWtraining/data_efficiency/vit_finetuning/models/vit.py328CODE
LOWtraining/opsd/teacher.py90CODE
LOW…ning/bing_bert/pytorch_pretrained_bert/optimization.py108CODE
LOWtraining/bing_bert/pytorch_pretrained_bert/modeling.py80CODE
LOWtraining/bing_bert/pytorch_pretrained_bert/modeling.py162CODE
LOWtraining/bing_bert/nvidia/modelingpreln_layerdrop.py269CODE
LOWtraining/bing_bert/nvidia/modelingpreln_layerdrop.py351CODE
LOWtraining/bing_bert/nvidia/modelingpreln.py266CODE
LOWtraining/bing_bert/nvidia/modelingpreln.py348CODE
LOWtraining/HelloDeepSpeed/train_bert_ds.py595CODE
LOWtraining/HelloDeepSpeed/train_bert.py554CODE
LOW…g/BingBertGlue/pytorch_pretrained_bert/optimization.py108CODE
LOW…ining/BingBertGlue/pytorch_pretrained_bert/modeling.py80CODE
LOW…ining/BingBertGlue/pytorch_pretrained_bert/modeling.py162CODE
LOWtraining/BingBertGlue/nvidia/modelingpreln_layerdrop.py269CODE
LOWtraining/BingBertGlue/nvidia/modelingpreln_layerdrop.py351CODE
LOWtraining/BingBertGlue/nvidia/modeling.py217CODE
LOWtraining/BingBertGlue/nvidia/modeling.py298CODE
LOWtraining/BingBertGlue/nvidia/modelingpreln.py266CODE
LOWtraining/BingBertGlue/nvidia/modelingpreln.py348CODE
LOWtraining/DeepSpeed-Domino/domino/language_model.py433CODE
LOWtraining/stable_diffusion/train_sd_distil_lora.py545CODE
LOW…ng/stable_diffusion/local_pipeline_stable_diffusion.py518CODE
LOW…/BingBertSquad/pytorch_pretrained_bert/optimization.py80CODE
LOW…ning/BingBertSquad/pytorch_pretrained_bert/modeling.py78CODE
LOW…ning/BingBertSquad/pytorch_pretrained_bert/modeling.py158CODE
LOW…aining/BingBertSquad/turing/modelingpreln_layerdrop.py267CODE
LOW…aining/BingBertSquad/turing/modelingpreln_layerdrop.py349CODE
LOWtraining/BingBertSquad/turing/nvidia_modeling.py217CODE
LOWtraining/BingBertSquad/turing/nvidia_modeling.py299CODE
LOWtraining/BingBertSquad/turing/nvidia_modelingpreln.py220CODE
LOWtraining/BingBertSquad/turing/nvidia_modelingpreln.py304CODE
LOW…eedometer/src/deepspeedometer/clients/openai_client.py54CODE
LOW…speedometer/src/deepspeedometer/clients/vllm_client.py85CODE
LOWbenchmarks/inference/mii/src/client.py115CODE
LOWbenchmarks/inference/mii/src/client.py184CODE
LOWbenchmarks/autosp/distributed_attention.py93CODE
LOWbenchmarks/autosp/ring_attention.py391CODE
LOWbenchmarks/autosp/ring_attention.py417CODE
LOWbenchmarks/autosp/ring_attention.py444CODE
LOW…pression/bert/huggingface_transformer/modeling_bert.py538CODE
LOW…pression/bert/huggingface_transformer/modeling_bert.py895CODE
LOW…pression/bert/huggingface_transformer/modeling_bert.py1050CODE
LOW…pression/bert/huggingface_transformer/modeling_bert.py1161CODE
LOW…pression/bert/huggingface_transformer/modeling_bert.py1320CODE
LOW…pression/bert/huggingface_transformer/modeling_bert.py1410CODE
LOW…pression/bert/huggingface_transformer/modeling_bert.py1525CODE
LOW…pression/bert/huggingface_transformer/modeling_bert.py1625CODE
LOW…pression/bert/huggingface_transformer/modeling_bert.py1724CODE
LOW…pression/bert/huggingface_transformer/modeling_bert.py1814CODE
LOW…pression/reasoning_aware_compression/rac/sequential.py129CODE
LOW…ression/reasoning_aware_compression/rac/calibration.py182CODE
LOWcompression/cifar/resnet.py67CODE
LOWcompression/cifar/resnet.py125CODE
LOW…ions/DeepSpeed-VisualChat/utils/model/modeling_dsvl.py277CODE
LOW…t/utils/model/third_party_model/qwen_clip/qwen_clip.py243CODE
LOW…odel/third_party_model/hf_model/configuration_llama.py106CODE
12 more matches not shown…
Modern Structural Boilerplate45 hits · 45 pts
SeverityFileLineSnippetContext
LOW…ining/data_efficiency/vit_finetuning/utils/__init__.py18__all__ = [ "get_dataset", "ImageMemFolder", "AverageMeter", "ProgressMeter", "accuracy", "get_optimizer", "get_scheduleCODE
LOWtraining/data_efficiency/vit_finetuning/models/vit.py1097__all__ = [ 'vits16r224', 'vits16r384', 'lvits16r224','lvits16r384', 'Block']CODE
LOWtraining/data_efficiency/gpt_finetuning/analyze_data.py68logger = logging.getLogger(__name__)CODE
LOW…g/data_efficiency/gpt_finetuning/run_clm_no_trainer.py65logger = logging.getLogger(__name__)CODE
LOWtraining/bing_bert/run_glue_classifier_bert_base.py48logger = logging.getLogger(__name__)CODE
LOWtraining/bing_bert/run_glue_classifier_bert_large.py48logger = logging.getLogger(__name__)CODE
LOW…ning/bing_bert/pytorch_pretrained_bert/tokenization.py28logger = logging.getLogger(__name__)CODE
LOW…aining/bing_bert/pytorch_pretrained_bert/file_utils.py24logger = logging.getLogger(__name__) # pylint: disable=invalid-nameCODE
LOWtraining/bing_bert/pytorch_pretrained_bert/modeling.py37logger = logging.getLogger(__name__)CODE
LOWtraining/bing_bert/nvidia/modelingpreln_layerdrop.py48logger = logging.getLogger(__name__)CODE
LOWtraining/bing_bert/nvidia/modelingpreln.py47logger = logging.getLogger(__name__)CODE
LOWtraining/bing_bert/turing/logger.py8logger = logging.getLogger(__name__)CODE
LOWtraining/bing_bert/turing/file_utils.py39logger = logging.getLogger(__name__) # pylint: disable=invalid-nameCODE
LOWtraining/BingBertGlue/run_glue_classifier_bert_base.py47logger = logging.getLogger(__name__)CODE
LOWtraining/BingBertGlue/run_glue_classifier_bert_large.py49logger = logging.getLogger(__name__)CODE
LOW…g/BingBertGlue/pytorch_pretrained_bert/tokenization.py28logger = logging.getLogger(__name__)CODE
LOW…ing/BingBertGlue/pytorch_pretrained_bert/file_utils.py24logger = logging.getLogger(__name__) # pylint: disable=invalid-nameCODE
LOW…ining/BingBertGlue/pytorch_pretrained_bert/modeling.py37logger = logging.getLogger(__name__)CODE
LOWtraining/BingBertGlue/nvidia/modelingpreln_layerdrop.py48logger = logging.getLogger(__name__)CODE
LOWtraining/BingBertGlue/nvidia/modeling.py47logger = logging.getLogger(__name__)CODE
LOWtraining/BingBertGlue/nvidia/modelingpreln.py47logger = logging.getLogger(__name__)CODE
LOWtraining/BingBertGlue/turing/logger.py8logger = logging.getLogger(__name__)CODE
LOWtraining/BingBertGlue/turing/file_utils.py39logger = logging.getLogger(__name__) # pylint: disable=invalid-nameCODE
LOW…aining/BingBertSquad/convert_bert_ckpt_to_deepspeed.py15logger = logging.getLogger(__name__)CODE
LOWtraining/BingBertSquad/nvidia_run_squad_baseline.py51logger = logging.getLogger(__name__)CODE
LOWtraining/BingBertSquad/nvidia_run_squad_deepspeed.py54logger = logging.getLogger(__name__)CODE
LOW…/BingBertSquad/pytorch_pretrained_bert/tokenization.py28logger = logging.getLogger(__name__)CODE
LOW…ng/BingBertSquad/pytorch_pretrained_bert/file_utils.py24logger = logging.getLogger(__name__) # pylint: disable=invalid-nameCODE
LOW…ning/BingBertSquad/pytorch_pretrained_bert/modeling.py38logger = logging.getLogger(__name__)CODE
LOW…aining/BingBertSquad/turing/modelingpreln_layerdrop.py48logger = logging.getLogger(__name__)CODE
LOWtraining/BingBertSquad/turing/nvidia_modeling.py47logger = logging.getLogger(__name__)CODE
LOWtraining/BingBertSquad/turing/file_utils.py39logger = logging.getLogger(__name__) # pylint: disable=invalid-nameCODE
LOWtraining/BingBertSquad/turing/nvidia_modelingpreln.py47logger = logging.getLogger(__name__)CODE
LOWtraining/MoQ/run_glue.py68logger = logging.getLogger(__name__)CODE
LOW…me/model_checkpoint/torch/serialization_orig_v2.6.0.py41__all__ = [CODE
LOW…me/model_checkpoint/torch/serialization_fast_v2.6.0.py41__all__ = [CODE
LOWbenchmarks/autosp/ring_attention.py14__all__ = ["update_out_and_lse", "RingComm", "get_default_args"]CODE
LOWcompression/bert/util.py16logger = logging.getLogger(__name__)CODE
LOWcompression/bert/run_glue_lkd.py54logger = logging.getLogger(__name__)CODE
LOWcompression/bert/run_glue_no_trainer.py54logger = logging.getLogger(__name__)CODE
LOWcompression/reasoning_aware_compression/rac/__init__.py24__all__ = [CODE
LOWcompression/gpt2/run_clm_no_trainer.py60logger = logging.getLogger(__name__)CODE
LOWcompression/cifar/resnet.py13__all__ = ['resnet']CODE
LOW…at/training/step1_supervised_finetuning/prompt_eval.py16logger = logging.getLogger(__name__)CODE
LOW…eneration/run-generation-script/test-run-generation.py51logger = logging.getLogger(__name__)CODE
Decorative Section Separators9 hits · 30 pts
SeverityFileLineSnippetContext
MEDIUMtraining/opsd/trainer.py77 # ------------------------------------------------------------------COMMENT
MEDIUMtraining/opsd/trainer.py79 # ------------------------------------------------------------------COMMENT
MEDIUMtraining/opsd/trainer.py97 # ------------------------------------------------------------------COMMENT
MEDIUMtraining/opsd/trainer.py99 # ------------------------------------------------------------------COMMENT
MEDIUMtraining/opsd/trainer.py187 # ------------------------------------------------------------------COMMENT
MEDIUMtraining/opsd/trainer.py189 # ------------------------------------------------------------------COMMENT
MEDIUMtraining/cifar/cifar10_tutorial.py298# ----------------COMMENT
MEDIUMtraining/cifar/cifar10_tutorial.py342# -------------------------COMMENT
MEDIUMtraining/cifar/cifar10_tutorial.py347# -------------------COMMENT
Docstring Block Structure4 hits · 20 pts
SeverityFileLineSnippetContext
HIGH…ng/stable_diffusion/local_pipeline_stable_diffusion.py538 Function invoked when calling the pipeline for generation. Args: prompt (`str` or `List[stSTRING
HIGH…me/model_checkpoint/torch/serialization_orig_v2.6.0.py673 Restores `storage` using a deserializer function registered for the `location`. This function looks in the regSTRING
HIGH…me/model_checkpoint/torch/serialization_fast_v2.6.0.py673 Restores `storage` using a deserializer function registered for the `location`. This function looks in the regSTRING
HIGH…ce/stable-diffusion/local_pipeline_stable_diffusion.py540 Function invoked when calling the pipeline for generation. Args: prompt (`str` or `List[stSTRING
AI Slop Vocabulary9 hits · 20 pts
SeverityFileLineSnippetContext
LOW…ining/data_efficiency/gpt_finetuning/learning_rates.py66 # If the learning rate is constant, just return the initial value.COMMENT
LOWtraining/BingBertSquad/nvidia_run_squad_baseline.py365 # "Japanese", we just use "Japanese" as the annotation. This is fairly rareCOMMENT
LOWtraining/BingBertSquad/nvidia_run_squad_baseline.py523 # just create a nonce prediction in this case to avoid failure.COMMENT
LOWtraining/BingBertSquad/nvidia_run_squad_baseline.py583 # can fail in certain cases in which case we just return `orig_text`.COMMENT
LOWtraining/BingBertSquad/nvidia_run_squad_deepspeed.py369 # "Japanese", we just use "Japanese" as the annotation. This is fairly rareCOMMENT
LOWtraining/BingBertSquad/nvidia_run_squad_deepspeed.py527 # just create a nonce prediction in this case to avoid failure.COMMENT
LOWtraining/BingBertSquad/nvidia_run_squad_deepspeed.py587 # can fail in certain cases in which case we just return `orig_text`.COMMENT
MEDIUMtraining/cifar/cifar10_tutorial.py350# - `Train a state-of-the-art ResNet network on imagenet`_COMMENT
MEDIUMtraining/cifar/cifar10_tutorial.py358# .. _Train a state-of-the-art ResNet network on imagenet: https://github.com/pytorch/examples/tree/master/imagenetCOMMENT
Excessive Try-Catch Wrapping15 hits · 19 pts
SeverityFileLineSnippetContext
LOWtraining/DeepSpeed-SuperOffload/finetune_zero3.py220 except Exception as e:CODE
LOWtraining/DeepSpeed-SuperOffload/finetune_zero3.py370 except Exception as e:CODE
LOWtraining/DeepSpeed-SuperOffload/finetune_zero3.py377 except Exception as e:CODE
MEDIUMtraining/bf16_master_weight/gather_memory.py65 print(f"Error: Log directory '{log_dir}' does not exist")CODE
LOW…me/model_checkpoint/torch/serialization_orig_v2.6.0.py513 except Exception as e:STRING
LOW…me/model_checkpoint/torch/serialization_orig_v2.6.0.py1534 except Exception: # saving the source is optional, so we can ignore any errorsCODE
MEDIUM…me/model_checkpoint/torch/serialization_orig_v2.6.0.py1531def _check_container_source(container_type, source_file, original_source):CODE
LOW…me/model_checkpoint/torch/serialization_fast_v2.6.0.py513 except Exception as e:STRING
LOW…me/model_checkpoint/torch/serialization_fast_v2.6.0.py1538 except Exception: # saving the source is optional, so we can ignore any errorsCODE
MEDIUM…me/model_checkpoint/torch/serialization_fast_v2.6.0.py1535def _check_container_source(container_type, source_file, original_source):CODE
MEDIUMbenchmarks/inference/collect_results.py121 print(f"Error found in {file_path}, collecting error info...")CODE
LOW…dometer/src/deepspeedometer/clients/azure_ml_client.py72 except Exception as e:CODE
LOWbenchmarks/inference/mii/src/client.py254 except Exception as e:CODE
LOWbenchmarks/autosp/correctness/correctness_run.py32 except Exception:CODE
MEDIUMbenchmarks/autosp/correctness/correctness_run.py26def get_host_ip():CODE
Cross-Language Confusion4 hits · 14 pts
SeverityFileLineSnippetContext
HIGHtraining/opsd/losses.py12 * ``forward_kl`` = D_KL(teacher || student) — mode-covering for studentSTRING
HIGHtraining/opsd/losses.py13 * ``reverse_kl`` = D_KL(student || teacher) — mode-seeking for studentSTRING
HIGHtraining/opsd/losses.py14 * ``jsd`` = 0.5 * D_KL(P || M) + 0.5 * D_KL(Q || M), M = (P+Q)/2STRING
HIGHtraining/opsd/tests/test_losses.py8 cd examples/opsd && python -m pytest tests/test_losses.py -vSTRING
Structural Annotation Overuse7 hits · 14 pts
SeverityFileLineSnippetContext
LOWcompression/cifar/run_compress.sh5## Step 1: obtain a checkpoint (to be compressed)COMMENT
LOWcompression/cifar/run_compress.sh11### Step 2: compress: channel pruningCOMMENT
LOWcompression/cifar/run_compress.sh25### Step 1: obtain a checkpoint (to be compressed)COMMENT
LOWcompression/cifar/run_compress.sh32#### Step 2: compress: channel pruningCOMMENT
LOWapplications/DeepSpeed-Chat/training/README.md7### Step 1: Supervised FinetuningCOMMENT
LOWapplications/DeepSpeed-Chat/training/README.md16### Step 2: Reward Model FinetuningCOMMENT
LOWapplications/DeepSpeed-Chat/training/README.md34### Step 3: RLHF finetuningCOMMENT
Verbosity Indicators5 hits · 10 pts
SeverityFileLineSnippetContext
LOWtraining/cifar/cifar10_tutorial.py213# But we need to check if the network has learnt anything at all.COMMENT
LOWcompression/cifar/run_compress.sh5## Step 1: obtain a checkpoint (to be compressed)COMMENT
LOWcompression/cifar/run_compress.sh11### Step 2: compress: channel pruningCOMMENT
LOWcompression/cifar/run_compress.sh25### Step 1: obtain a checkpoint (to be compressed)COMMENT
LOWcompression/cifar/run_compress.sh32#### Step 2: compress: channel pruningCOMMENT
Redundant / Tautological Comments7 hits · 10 pts
SeverityFileLineSnippetContext
LOW…g/tensor_parallel/hf_integration/train_bench_length.py243 # Print message except when distributed but not rank 0COMMENT
LOWtraining/tensor_parallel/hf_integration/train.py238 # Print message except when distributed but not rank 0COMMENT
LOWtraining/DeepSpeed-Domino/domino/training.py660 # Set grad to zero.COMMENT
LOW…mpression/bash_script/collect_traces_zero_inference.sh25# Set NVME_OFFLOAD_DIR to spill parameters to NVMe instead of CPU memory.COMMENT
LOW…ions/DeepSpeed-Chat/dschat/utils/model/reward_model.py99 # Check if there is any padding otherwise take length of sequenceCOMMENT
LOW…cations/DeepSpeed-Chat/dschat/utils/data/data_utils.py364 if sft_train_datasets: # Check if sft_train_datasets is not emptyCODE
LOW…cations/DeepSpeed-Chat/dschat/utils/data/data_utils.py370 if sft_eval_datasets: # Check if sft_eval_datasets is not emptyCODE
AI Response Leakage1 hit · 8 pts
SeverityFileLineSnippetContext
HIGH…/huggingface/stable-diffusion/test-stable-diffusion.py8# In this example the SD inference pipeline is optimized based on recommendations in the research paperCOMMENT
Example Usage Blocks2 hits · 3 pts
SeverityFileLineSnippetContext
LOWtraining/bf16_master_weight/run_comparison.sh4# Usage:COMMENT
LOWbenchmarks/autosp/correctness/correctness.sh10# Usage:COMMENT
Modern AI Meta-Vocabulary1 hit · 3 pts
SeverityFileLineSnippetContext
MEDIUM…re_compression/bash_script/run_calibration_ablation.sh11# rac those prompts plus the dense model's own chain-of-thoughtCOMMENT
Slop Phrases2 hits · 2 pts
SeverityFileLineSnippetContext
LOWtraining/stable_diffusion/train_sd_distil_lora.py850 # make sure to pop weight so that corresponding model is not saved againSTRING
LOWbenchmarks/inference/mii/src/utils.py200 # If we are not running the benchmark, we need to make sure to only have oneCOMMENT
Fake / Example Data2 hits · 2 pts
SeverityFileLineSnippetContext
LOW…pression/bert/huggingface_transformer/modeling_bert.py1386 dummy_token = torch.full(STRING
LOW…pression/bert/huggingface_transformer/modeling_bert.py1389 input_ids = torch.cat([input_ids, dummy_token], dim=1)STRING