Repository Analysis

deepspeedai/DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

15.1 Moderate AI signal View on GitHub

Analysis Overview

This report presents the forensic synthetic code analysis of deepspeedai/DeepSpeed, a Python project with 42,843 GitHub stars. SynthScan v2.0 examined 317,439 lines of code across 1589 source files, recording 3604 pattern matches distributed across 22 syntactic categories. The overall adjusted score of 15.1 places this repository in the Moderate AI signal band.

The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).

15.1
Adjusted Score
15.1
Raw Score
100%
Time Factor
2026-08-01
Last Push
42.8K
Stars
Python
Language
317.4K
Lines of Code
1.6K
Files
3.6K
Pattern Hits
2026-08-02
Scan Date
0.10
HC Hit Rate

What These Metrics Mean

Adjusted Score
Primary synthetic code indicator. Raw score normalised per 1,000 lines of code and multiplied by the temporal discount factor. This is the definitive comparative metric — use it to rank repositories by AI authorship density.
Raw Score
The unmodified sum of all severity-weighted, context-multiplied pattern match scores before temporal discounting. Reflects the absolute signal strength independent of when the repository was last active.
Time Factor
The temporal discount multiplier (0–100%) applied to the raw score. Repositories last updated before ChatGPT's launch (Nov 2022) receive a 5% factor. Full signal is only assigned to repositories active in the post-adoption era (Jan 2024+).
Pattern Hits
Total count of individual pattern matches across all files and categories. A high hit count with a low score may indicate a very large codebase with isolated AI snippets; a low count with a high score indicates dense, concentrated AI signatures.
HC Hit Rate
High+Critical pattern hits per file, averaged across the repository. This orthogonal signal catches repositories where a few files are densely packed with high-severity AI tells — a strong indicator even when the normalised score appears moderate due to codebase size.
Lines of Code / Files
Total lines and files analysed. The scanner examines 94 file extensions. These denominators are used to normalise the score, enabling fair comparison between repositories of vastly different sizes.

Score History

This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.

Severity Breakdown

Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.

CRITICAL 8HIGH 154MEDIUM 227LOW 3215

Directory Score Breakdown

This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.

Pattern Findings

The scanner identified 3604 distinct pattern matches across 22 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.

Hyper-Verbose Identifiers1720 hits · 1730 pts
SeverityFileLineSnippetContext
LOWcsrc/aio/py_test/io_engine.py119def io_engine_multiprocessing(args, read_op):CODE
LOWcsrc/aio/py_test/test_ds_aio_utils.py86def create_page_locked_tensor(num_elem, use_accelerator, aio_handle=None):CODE
LOWdeepspeed/compat.py23def get_annotations_from_namespace(namespace: Mapping[str, object]) -> Dict[str, Any]:CODE
LOWdeepspeed/elasticity/utils.py9def is_torch_elastic_compatible():CODE
LOWdeepspeed/elasticity/elasticity.py27def get_candidate_batch_sizes(base_list, max_acceptable_batch_size):CODE
LOWdeepspeed/elasticity/elasticity.py191def _compatible_ds_version_check(target_deepspeed_version: str):CODE
LOWdeepspeed/elasticity/elasticity.py208def ensure_immutable_elastic_config(runtime_elastic_config_dict: dict):CODE
LOWdeepspeed/launcher/runner.py275def _stable_remove_duplicates(data):STRING
LOWdeepspeed/launcher/runner.py391def parse_inclusion_exclusion(resource_pool, inclusion, exclusion):STRING
LOWdeepspeed/model_implementations/diffusers/vae.py37 def _create_cuda_graph_decoder(self, *inputs, **kwargs):CODE
LOWdeepspeed/model_implementations/diffusers/vae.py80 def _create_cuda_graph_encoder(self, *inputs, **kwargs):CODE
LOW…eed/model_implementations/transformers/clip_encoder.py27 def _build_causal_attention_mask(self, bsz, seq_len, dtype):CODE
LOWdeepspeed/module_inject/auto_ep_layer.py85def apply_scores_before_experts_if_enabled(CODE
LOWdeepspeed/module_inject/auto_ep_layer.py96def _split_plan_from_expert_counts(CODE
LOWdeepspeed/module_inject/auto_ep_layer.py167def compute_split_plan_from_expert_indices(CODE
LOWdeepspeed/module_inject/auto_ep_layer.py289def unpermute_by_local_expert(CODE
LOWdeepspeed/module_inject/auto_ep_layer.py541 def set_deepspeed_parallelism(CODE
LOWdeepspeed/module_inject/auto_tp.py344 def set_tensor_parallel_config(self, mp_size, mp_group):CODE
LOWdeepspeed/module_inject/auto_tp.py458 def _create_row_parallel_layer(self, module, spec: TPLayerSpec, name: str):CODE
LOWdeepspeed/module_inject/auto_tp.py476 def _create_column_parallel_layer(self, module, spec: TPLayerSpec, name: str):CODE
LOWdeepspeed/module_inject/auto_tp.py513 def _configure_gathered_column_tie_fallbacks(self):CODE
LOWdeepspeed/module_inject/auto_tp.py632 def _replace_autoep_shared_experts(self, autoep_layer, autoep_name):CODE
LOWdeepspeed/module_inject/auto_tp.py735 def _replace_last_linear_module(self, r_module):CODE
LOWdeepspeed/module_inject/fusedqkv_utils.py10def split_by_qkvlist_and_refuse(qkv_list, split_size, split_dim=0, cat_dim=0):CODE
LOWdeepspeed/module_inject/fusedqkv_utils.py162def shard_value_with_share_qk(CODE
LOWdeepspeed/module_inject/auto_ep_folding.py130def expected_folding_group_tables(spec: ParallelFoldingSpec) -> FoldingGroupTables:CODE
LOWdeepspeed/module_inject/auto_ep_folding.py265def mark_autoep_folding_router_parameter(param) -> None:CODE
LOWdeepspeed/module_inject/auto_ep_folding.py281def mark_autoep_folding_partial_router_parameter(param) -> None:CODE
LOWdeepspeed/module_inject/auto_ep_folding.py298def mark_autoep_folding_sp_sharded_layernorm_parameter(param) -> None:CODE
LOWdeepspeed/module_inject/auto_ep_folding.py318def _is_model_parallel_param_marker(param) -> bool:CODE
LOWdeepspeed/module_inject/auto_ep_folding.py322def _autoep_folding_param_family(param, *, param_name: str | None = None) -> str | None:CODE
LOWdeepspeed/module_inject/auto_ep_folding.py341def autoep_folding_gradient_reduction_strategy(CODE
LOWdeepspeed/module_inject/auto_ep_folding.py425def reduce_autoep_folding_gradient(CODE
LOWdeepspeed/module_inject/auto_ep_folding.py465def is_autoep_folding_gradient_corrected(param) -> bool:CODE
LOWdeepspeed/module_inject/auto_ep_folding.py469def clear_autoep_folding_gradient_corrected(param) -> None:CODE
LOWdeepspeed/module_inject/auto_ep_folding.py474def apply_folding_correction_to_grad_buffer(CODE
LOWdeepspeed/module_inject/auto_ep_folding.py496def assert_group_matches_spec(existing_rank_lists, spec: ParallelFoldingSpec, *, group_kind: str = "ep_edp") -> None:CODE
LOWdeepspeed/module_inject/auto_ep.py40def _remove_transformers_output_capture_hooks(model: nn.Module) -> int:CODE
LOWdeepspeed/module_inject/auto_ep.py69def _raise_if_duplicate_moe_specs(specs: list[MoELayerSpec]) -> None:CODE
LOWdeepspeed/module_inject/auto_ep.py110def _get_num_experts_from_config(model_config, preset: MoEModelPreset) -> int | None:CODE
LOWdeepspeed/module_inject/auto_ep.py165def _infer_hidden_and_ffn_size(CODE
LOWdeepspeed/module_inject/auto_ep.py489 def _replace_moe_layer_without_retarget(CODE
LOWdeepspeed/module_inject/auto_ep.py518 def _retarget_transformers_output_recorders(self, spec: MoELayerSpec, replacement: nn.Module) -> None:CODE
LOWdeepspeed/module_inject/auto_ep.py568 def _requires_selected_preset_detection(self) -> bool:CODE
LOWdeepspeed/module_inject/auto_ep.py579 def _raise_no_moe_layers_detected(self, presets_to_try: list[tuple[str, MoEModelPreset]]) -> None:CODE
LOWdeepspeed/module_inject/auto_ep_config.py265def validate_autoep_post_detection(CODE
LOWdeepspeed/module_inject/auto_ep_config.py330def fill_autoep_config_from_hf(config: AutoEPConfig, model_config) -> None:CODE
LOWdeepspeed/module_inject/replace_module.py189def replace_transformer_layer(orig_layer_impl, model, checkpoint_dict, config, model_config):CODE
LOWdeepspeed/module_inject/auto_tp_model_utils.py74def build_mpt_atten_bias_tensor(self,CODE
LOWdeepspeed/module_inject/load_checkpoint.py23def load_model_with_checkpoint(r_module,CODE
LOWdeepspeed/module_inject/layers.py38def _build_param_uc_conversion_meta(*,CODE
LOWdeepspeed/module_inject/layers.py60def _build_param_uc_restore_meta(*,CODE
LOWdeepspeed/module_inject/layers.py414 def _should_materialize_tp_partition(self):CODE
LOWdeepspeed/module_inject/layers.py474def configure_tensor_parallel_runtime(config):CODE
LOWdeepspeed/module_inject/layers.py481def _get_param_uc_conversion_meta(param: torch.Tensor) -> Optional[Dict[str, Any]]:CODE
LOWdeepspeed/module_inject/layers.py496def collect_autotp_universal_checkpoint_info(model: nn.Module) -> Dict[str, Any]:CODE
LOWdeepspeed/module_inject/layers.py595 def _is_replaced_module_weight(self, param: torch.Tensor) -> bool:CODE
LOWdeepspeed/module_inject/layers.py1098def _infer_subparam_logical_shapes(weight_shape, shape, partition_dim, name=None):CODE
LOWdeepspeed/module_inject/layers.py1200def _partition_logical_tensor(tensor, partition_dim, tp_world_size, tp_index, name=None, subparam_sizes=None):CODE
LOWdeepspeed/module_inject/module_quantize.py9def quantize_transformer_layer(orig_layer_impl, model, megatron=False, preln=False):CODE
1660 more matches not shown…
Cross-File Repetition145 hits · 725 pts
SeverityFileLineSnippetContext
HIGHcsrc/aio/py_test/aio_bench_generate_param.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGHcsrc/aio/py_test/ds_aio_args.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGHcsrc/aio/py_test/test_ds_aio_utils.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGHcsrc/aio/py_test/validate_async_io.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGHcsrc/aio/py_test/test_ds_aio.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGHcsrc/aio/py_test/aio_bench_perf_sweep.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGHcsrc/aio/py_test/ds_aio_basic.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGHcsrc/aio/py_test/parse_aio_stats.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGHcsrc/gds/py_test/validate_gds.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGHdeepspeed/nvme/ds_aio_args.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGHdeepspeed/nvme/test_ds_aio_utils.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGHdeepspeed/nvme/validate_async_io.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGHdeepspeed/nvme/test_ds_aio.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGHdeepspeed/nvme/perf_generate_param.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGHdeepspeed/nvme/ds_aio_basic.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGHdeepspeed/nvme/ds_aio_handle.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGHdeepspeed/nvme/parse_nvme_stats.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGHdeepspeed/nvme/perf_run_sweep.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGH…eed/runtime/swap_tensor/pipelined_optimizer_swapper.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGH…d/runtime/swap_tensor/partitioned_optimizer_swapper.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGHcsrc/aio/py_test/ds_aio_job.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGHdeepspeed/nvme/ds_aio_job.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGHdeepspeed/runtime/swap_tensor/optimizer_utils.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGH…speed/runtime/swap_tensor/partitioned_param_swapper.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGHdeepspeed/runtime/swap_tensor/utils.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGHdeepspeed/runtime/swap_tensor/async_swapper.py0functionality of swapping tensors to/from (nvme) storage devices.STRING
HIGHdeepspeed/module_inject/containers/gptj.py0necessary to implement for `hybridsplitqkvcontainer`STRING
HIGHdeepspeed/module_inject/containers/opt.py0necessary to implement for `hybridsplitqkvcontainer`STRING
HIGHdeepspeed/module_inject/containers/llama.py0necessary to implement for `hybridsplitqkvcontainer`STRING
HIGHdeepspeed/module_inject/containers/gptneo.py0necessary to implement for `hybridsplitqkvcontainer`STRING
HIGHdeepspeed/module_inject/containers/internlm.py0necessary to implement for `hybridsplitqkvcontainer`STRING
HIGHdeepspeed/module_inject/containers/llama2.py0necessary to implement for `hybridsplitqkvcontainer`STRING
HIGHdeepspeed/module_inject/containers/llama.py0necessary to implement for `hybridgatedmlpcontainer`STRING
HIGHdeepspeed/module_inject/containers/internlm.py0necessary to implement for `hybridgatedmlpcontainer`STRING
HIGHdeepspeed/module_inject/containers/llama2.py0necessary to implement for `hybridgatedmlpcontainer`STRING
HIGHdeepspeed/runtime/tensor_parallel/config.py0number of devices to split the model across using tensor parallelism.STRING
HIGHdeepspeed/inference/config.py0number of devices to split the model across using tensor parallelism.STRING
HIGHdeepspeed/inference/v2/config_v2.py0number of devices to split the model across using tensor parallelism.STRING
HIGHdeepspeed/runtime/tensor_parallel/config.py0configuration for tensor parallelism used to split the model across several gpus. expects a dictionary containing valuesSTRING
HIGHdeepspeed/inference/config.py0configuration for tensor parallelism used to split the model across several gpus. expects a dictionary containing valuesSTRING
HIGHdeepspeed/inference/v2/config_v2.py0configuration for tensor parallelism used to split the model across several gpus. expects a dictionary containing valuesSTRING
HIGHdeepspeed/runtime/zero/stage_1_and_2.py0returns a dict containing the current state of this :class:`fp16_optimizer` instance. this dict contains attributes of :STRING
HIGHdeepspeed/runtime/zero/stage3.py0returns a dict containing the current state of this :class:`fp16_optimizer` instance. this dict contains attributes of :STRING
HIGHdeepspeed/runtime/fp16/fused_optimizer.py0returns a dict containing the current state of this :class:`fp16_optimizer` instance. this dict contains attributes of :STRING
HIGHdeepspeed/runtime/fp16/unfused_optimizer.py0returns a dict containing the current state of this :class:`fp16_optimizer` instance. this dict contains attributes of :STRING
HIGHdeepspeed/runtime/zero/stage_1_and_2.py0loads a state_dict created by an earlier call to state_dict(). if ``fp16_optimizer_instance`` was constructed from some STRING
HIGHdeepspeed/runtime/zero/stage3.py0loads a state_dict created by an earlier call to state_dict(). if ``fp16_optimizer_instance`` was constructed from some STRING
HIGHdeepspeed/runtime/fp16/fused_optimizer.py0loads a state_dict created by an earlier call to state_dict(). if ``fp16_optimizer_instance`` was constructed from some STRING
HIGHdeepspeed/runtime/fp16/unfused_optimizer.py0loads a state_dict created by an earlier call to state_dict(). if ``fp16_optimizer_instance`` was constructed from some STRING
HIGHdeepspeed/runtime/fp16/onebit/zoadam.py0performs a single optimization step. arguments: closure (callable, optional): a closure that reevaluates the model and rSTRING
HIGHdeepspeed/runtime/fp16/onebit/adam.py0performs a single optimization step. arguments: closure (callable, optional): a closure that reevaluates the model and rSTRING
HIGHdeepspeed/ops/lamb/fused_lamb.py0performs a single optimization step. arguments: closure (callable, optional): a closure that reevaluates the model and rSTRING
HIGHdeepspeed/runtime/fp16/onebit/zoadam.py0overrides load_state_dict() to add special handling when loading checkpointsSTRING
HIGHdeepspeed/runtime/fp16/onebit/lamb.py0overrides load_state_dict() to add special handling when loading checkpointsSTRING
HIGHdeepspeed/runtime/fp16/onebit/adam.py0overrides load_state_dict() to add special handling when loading checkpointsSTRING
HIGH…e/v2/kernels/ragged_ops/blocked_flash/blocked_flash.py0return preferred granulatity for blocked kv-cache implementation.STRING
HIGH…s/implementations/attention/dense_blocked_attention.py0return preferred granulatity for blocked kv-cache implementation.STRING
HIGH…peed/inference/v2/modules/interfaces/attention_base.py0return preferred granulatity for blocked kv-cache implementation.STRING
HIGH…gged_ops/linear_blocked_kv_rotary/blocked_kv_rotary.py0cuda kernel implementation that will perform rotary position embeddings on the queries and keys before copying into a blSTRING
HIGH…ops/linear_blocked_kv_rotary/linear_blocked_kv_copy.py0cuda kernel implementation that will perform rotary position embeddings on the queries and keys before copying into a blSTRING
85 more matches not shown…
Unused Imports714 hits · 695 pts
SeverityFileLineSnippetContext
LOWcsrc/aio/py_test/torch_io.py10CODE
LOWcsrc/aio/py_test/ds_aio_basic.py13CODE
LOWcsrc/aio/py_test/ds_aio_handle.py11CODE
LOWcsrc/aio/py_test/torch_fastio_engine.py11CODE
LOWdeepspeed/git_version_info.py8CODE
LOWdeepspeed/__init__.py17CODE
LOWdeepspeed/__init__.py18CODE
LOWdeepspeed/__init__.py36CODE
LOWdeepspeed/__init__.py37CODE
LOWdeepspeed/__init__.py42CODE
LOWdeepspeed/__init__.py42CODE
LOWdeepspeed/__init__.py49CODE
LOWdeepspeed/__init__.py50CODE
LOWdeepspeed/__init__.py51CODE
LOWdeepspeed/__init__.py52CODE
LOWdeepspeed/__init__.py52CODE
LOWdeepspeed/__init__.py53CODE
LOWdeepspeed/__init__.py53CODE
LOWdeepspeed/__init__.py55CODE
LOWdeepspeed/__init__.py58CODE
LOWdeepspeed/__init__.py59CODE
LOWdeepspeed/__init__.py29CODE
LOWdeepspeed/elasticity/config.py7CODE
LOWdeepspeed/elasticity/__init__.py6CODE
LOWdeepspeed/elasticity/__init__.py6CODE
LOWdeepspeed/elasticity/__init__.py6CODE
LOWdeepspeed/elasticity/__init__.py8CODE
LOWdeepspeed/elasticity/__init__.py8CODE
LOWdeepspeed/elasticity/__init__.py8CODE
LOWdeepspeed/elasticity/__init__.py10CODE
LOWdeepspeed/model_implementations/__init__.py6CODE
LOWdeepspeed/model_implementations/__init__.py7CODE
LOWdeepspeed/pipe/__init__.py6CODE
LOWdeepspeed/pipe/__init__.py6CODE
LOWdeepspeed/pipe/__init__.py6CODE
LOWdeepspeed/module_inject/auto_ep_layer.py15CODE
LOWdeepspeed/module_inject/auto_tp.py14CODE
LOWdeepspeed/module_inject/auto_ep_folding.py11CODE
LOWdeepspeed/module_inject/__init__.py6CODE
LOWdeepspeed/module_inject/__init__.py6CODE
LOWdeepspeed/module_inject/__init__.py6CODE
LOWdeepspeed/module_inject/__init__.py6CODE
LOWdeepspeed/module_inject/__init__.py6CODE
LOWdeepspeed/module_inject/__init__.py7CODE
LOWdeepspeed/module_inject/__init__.py8CODE
LOWdeepspeed/module_inject/__init__.py9CODE
LOWdeepspeed/module_inject/__init__.py9CODE
LOWdeepspeed/module_inject/__init__.py9CODE
LOWdeepspeed/module_inject/__init__.py9CODE
LOWdeepspeed/module_inject/__init__.py9CODE
LOWdeepspeed/module_inject/__init__.py9CODE
LOWdeepspeed/module_inject/__init__.py9CODE
LOWdeepspeed/module_inject/__init__.py10CODE
LOWdeepspeed/module_inject/__init__.py11CODE
LOWdeepspeed/module_inject/__init__.py11CODE
LOWdeepspeed/module_inject/__init__.py11CODE
LOWdeepspeed/module_inject/__init__.py11CODE
LOWdeepspeed/module_inject/__init__.py11CODE
LOWdeepspeed/module_inject/auto_ep.py11CODE
LOWdeepspeed/module_inject/auto_ep_config.py7CODE
654 more matches not shown…
Decorative Section Separators169 hits · 570 pts
SeverityFileLineSnippetContext
MEDIUMdeepspeed/module_inject/auto_ep_layer.py31# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/module_inject/auto_ep_layer.py33# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/module_inject/auto_ep_layer.py49# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/module_inject/auto_ep_layer.py51# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/module_inject/auto_ep_layer.py350# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/module_inject/auto_ep_layer.py352# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/module_inject/auto_ep_config.py36# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/module_inject/auto_ep_config.py38# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/module_inject/auto_ep_config.py90# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/module_inject/auto_ep_config.py92# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/runtime/rollout/hybrid_engine_rollout.py95 # ------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/runtime/rollout/hybrid_engine_rollout.py97 # ------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/compression/basic_layer.py746# -----------------COMMENT
MEDIUMdeepspeed/compression/basic_layer.py748# -----------------COMMENT
MEDIUM…epspeed/ops/transformer/inference/triton/matmul_ext.py19# -----------------------------------------------------------------------------COMMENT
MEDIUM…epspeed/ops/transformer/inference/triton/matmul_ext.py125# -----------------------------------------------------------------------------COMMENT
MEDIUM…epspeed/ops/transformer/inference/triton/matmul_ext.py470# -----------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/auto_sp.py82 # ------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/auto_sp.py84 # ------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/auto_sp.py96 # ------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/auto_sp.py98 # ------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/auto_sp.py115 # ------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/auto_sp.py117 # ------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/auto_sp.py129# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/auto_sp.py131# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/autosp_detector.py17# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/autosp_detector.py19# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/autosp_detector.py67# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/autosp_detector.py69# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/autosp_detector.py84# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/autosp_detector.py86# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/test_autosp.py274# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/test_autosp.py276# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/test_autosp.py28# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/test_autosp.py31# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/test_autosp.py87# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/test_autosp.py89# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/test_autosp.py109# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/test_autosp.py111# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/test_autosp.py161# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/test_autosp.py163# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/test_autosp.py215# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/test_autosp.py217# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/test_autosp.py393# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/test_autosp.py395# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/test_autosp.py503# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/test_autosp.py505# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/test_autosp.py612# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/test_autosp.py614# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/autosp_vit.py72 # ------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/autosp_vit.py74 # ------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/autosp_vit.py104 # -------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/autosp_vit.py106 # -------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/autosp_vit.py138 # -------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/autosp_vit.py140 # -------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/autosp_vit.py155 # -------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/sequence/autosp_vit.py159 # -------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/moe/ep_experts.py26# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/moe/ep_experts.py28# ---------------------------------------------------------------------------COMMENT
MEDIUMdeepspeed/moe/ep_experts.py83# ---------------------------------------------------------------------------COMMENT
109 more matches not shown…
Deep Nesting366 hits · 328 pts
SeverityFileLineSnippetContext
LOWcsrc/aio/py_test/aio_bench_perf_sweep.py158CODE
LOWcsrc/aio/py_test/parse_aio_stats.py36CODE
LOWcsrc/aio/py_test/parse_aio_stats.py88CODE
LOWdeepspeed/env_report.py83CODE
LOWdeepspeed/elasticity/elastic_agent.py127CODE
LOWdeepspeed/elasticity/elasticity.py41CODE
LOWdeepspeed/elasticity/elasticity.py233CODE
LOWdeepspeed/launcher/runner.py243CODE
LOWdeepspeed/launcher/runner.py310CODE
LOWdeepspeed/launcher/runner.py436CODE
LOWdeepspeed/launcher/launcher_helper.py55CODE
LOWdeepspeed/launcher/launch.py145CODE
LOWdeepspeed/module_inject/auto_tp.py156CODE
LOWdeepspeed/module_inject/auto_tp.py228CODE
LOWdeepspeed/module_inject/auto_tp.py291CODE
LOWdeepspeed/module_inject/auto_tp.py632CODE
LOWdeepspeed/module_inject/auto_tp.py653CODE
LOWdeepspeed/module_inject/fusedqkv_utils.py29CODE
LOWdeepspeed/module_inject/fusedqkv_utils.py124CODE
LOWdeepspeed/module_inject/policy.py131CODE
LOWdeepspeed/module_inject/auto_ep.py165CODE
LOWdeepspeed/module_inject/auto_ep.py219CODE
LOWdeepspeed/module_inject/auto_ep.py283CODE
LOWdeepspeed/module_inject/replace_module.py29CODE
LOWdeepspeed/module_inject/replace_module.py88CODE
LOWdeepspeed/module_inject/replace_module.py189CODE
LOWdeepspeed/module_inject/replace_module.py630CODE
LOWdeepspeed/module_inject/replace_module.py690CODE
LOWdeepspeed/module_inject/load_checkpoint.py23CODE
LOWdeepspeed/module_inject/load_checkpoint.py60CODE
LOWdeepspeed/module_inject/load_checkpoint.py222CODE
LOWdeepspeed/module_inject/load_checkpoint.py63CODE
LOWdeepspeed/module_inject/layers.py496CODE
LOWdeepspeed/module_inject/layers.py1098CODE
LOWdeepspeed/module_inject/autotp_config.py145CODE
LOWdeepspeed/monitor/csv_monitor.py22CODE
LOWdeepspeed/monitor/csv_monitor.py40CODE
LOWdeepspeed/monitor/tensorboard.py27CODE
LOWdeepspeed/nvme/parse_nvme_stats.py36CODE
LOWdeepspeed/nvme/parse_nvme_stats.py88CODE
LOWdeepspeed/nvme/perf_run_sweep.py167CODE
LOWdeepspeed/checkpoint/universal_checkpoint.py99CODE
LOWdeepspeed/checkpoint/autoep_universal.py207CODE
LOWdeepspeed/checkpoint/autoep_universal.py266CODE
LOWdeepspeed/checkpoint/utils.py41CODE
LOWdeepspeed/checkpoint/ds_to_universal.py161CODE
LOWdeepspeed/checkpoint/ds_to_universal.py249CODE
LOWdeepspeed/checkpoint/ds_to_universal.py550CODE
LOWdeepspeed/checkpoint/ds_to_universal.py759CODE
LOWdeepspeed/runtime/hybrid_engine.py168CODE
LOWdeepspeed/runtime/hybrid_engine.py274CODE
LOWdeepspeed/runtime/hybrid_engine.py305CODE
LOWdeepspeed/runtime/config.py205CODE
LOWdeepspeed/runtime/config.py250CODE
LOWdeepspeed/runtime/config.py678CODE
LOWdeepspeed/runtime/config.py924CODE
LOWdeepspeed/runtime/quantize.py51CODE
LOWdeepspeed/runtime/quantize.py129CODE
LOWdeepspeed/runtime/base_optimizer.py315CODE
LOWdeepspeed/runtime/bf16_optimizer.py232CODE
306 more matches not shown…
Over-Commented Block165 hits · 164 pts
SeverityFileLineSnippetContext
LOWcsrc/cpu/comm/shm_interface.cpp1// Copyright (c) Microsoft Corporation.COMMENT
LOWcsrc/cpu/comm/ccl.cpp1// Copyright (c) Microsoft Corporation.COMMENT
LOWcsrc/cpu/comm/shm.cpp1// Copyright (c) Microsoft Corporation.COMMENT
LOWcsrc/cpu/comm/riscv64/shm.h61 __attribute__((target("arch=+v,+zvfh")));COMMENT
LOWcsrc/cpu/comm/riscv64/shm.h81#define CVT_FP16_TO_FP32(X) cvt_fp16_to_fp32(X, vl)COMMENT
LOWcsrc/cpu/comm/arm64/shm.h1// Copyright (c) Microsoft Corporation.COMMENT
LOWcsrc/cpu/comm/arm64/shm.h81// Reduce functions down below use vectorized algorithm, the number of bytes processed eachCOMMENT
LOWcsrc/cpu/comm/arm64/shm.h101#define VADD_F32_2VL(A, B) vaddq_f32(A, B)COMMENT
LOWcsrc/cpu/comm/x86_64/shm.h61#define VLOAD_U16(X) _mm256_loadu_si256((__m256i*)(X))COMMENT
LOWcsrc/deepspeed4science/evoformer_attn/kernel_forward.h41#include <vector>COMMENT
LOWcsrc/deepspeed4science/evoformer_attn/kernel_forward.h61#include "cutlass/gemm/threadblock/default_mma_core_sm80.h"COMMENT
LOWcsrc/deepspeed4science/evoformer_attn/kernel_backward.h41#include <vector>COMMENT
LOWcsrc/deepspeed4science/evoformer_attn/kernel_backward.h61#include "cutlass/epilogue/warp/fragment_iterator_tensor_op.h"COMMENT
LOWcsrc/deepspeed4science/evoformer_attn/kernel_backward.h801 // part1 - after Q.K / dV / dO.VCOMMENT
LOW…ed4science/evoformer_attn/gemm/custom_mma_multistage.h41COMMENT
LOW…ed4science/evoformer_attn/gemm/custom_mma_multistage.h61/////////////////////////////////////////////////////////////////////////////////////////////////COMMENT
LOW…/deepspeed4science/evoformer_attn/gemm/mma_from_smem.h41COMMENT
LOW…/deepspeed4science/evoformer_attn/gemm/mma_from_smem.h61#include "../epilogue/epilogue_thread_apply_logsumexp.h"COMMENT
LOW…/deepspeed4science/evoformer_attn/gemm/mma_from_smem.h321 // BEGIN smemCOMMENT
LOW…/deepspeed4science/evoformer_attn/gemm/mma_from_smem.h661template <COMMENT
LOW…/deepspeed4science/evoformer_attn/gemm/mma_from_smem.h1301 /// Iterates over tiles of A operand in global memoryCOMMENT
LOW…/deepspeed4science/evoformer_attn/gemm/mma_from_smem.h1381COMMENT
LOW…eepspeed4science/evoformer_attn/gemm/custom_mma_base.h41COMMENT
LOW…eed4science/evoformer_attn/gemm/custom_mma_pipelined.h61/////////////////////////////////////////////////////////////////////////////////////////////////COMMENT
LOW…ence/evoformer_attn/epilogue/epilogue_rescale_output.h61#include "cutlass/layout/tensor.h"COMMENT
LOW…d4science/evoformer_attn/epilogue/epilogue_pipelined.h61#include "cutlass/functional.h"COMMENT
LOW…ttn/iterators/predicated_tile_iterator_residual_last.h61namespace threadblock {COMMENT
LOW…ttn/iterators/predicated_tile_iterator_residual_last.h81/// Adding a logical coordinate offset may be performed at the time the iteratorCOMMENT
LOW…ttn/iterators/predicated_tile_iterator_residual_last.h101///COMMENT
LOW…ttn/iterators/predicated_tile_iterator_residual_last.h121//COMMENT
LOW…ttn/iterators/predicated_tile_iterator_residual_last.h1861 }COMMENT
LOW…mer_attn/iterators/epilogue_predicated_tile_iterator.h41 Mostly copied from "cutlass/epilogue/threadblock/predicated_tile_iterator.h"COMMENT
LOW…rators/predicated_tile_access_iterator_residual_last.h221 int const* indices = nullptr)COMMENT
LOW…rators/predicated_tile_access_iterator_residual_last.h781COMMENT
LOW…rators/predicated_tile_access_iterator_residual_last.h821 /// Gets the maskCOMMENT
LOW…rators/predicated_tile_access_iterator_residual_last.h1701 AccessType* get() const { return reinterpret_cast<AccessType*>(iterator_.get()); }COMMENT
LOW…rators/predicated_tile_access_iterator_residual_last.h1741COMMENT
LOWcsrc/transformer/ds_transformer_cuda.cpp1// Copyright (c) Microsoft Corporation.COMMENT
LOWcsrc/transformer/inference/csrc/pt_binding.cpp1// Copyright (c) Microsoft Corporation.COMMENT
LOW…transformer/inference/includes/inference_cuda_layers.h1// Copyright (c) Microsoft Corporation.COMMENT
LOW…sformer/inference/includes/inference_cublas_wrappers.h1// Copyright (c) Microsoft Corporation.COMMENT
LOWcsrc/transformer/inference/includes/inference_context.h1// Copyright (c) Microsoft Corporation.COMMENT
LOWcsrc/xpu/includes/type_shim.h1// Copyright (c) Microsoft Corporation.COMMENT
LOWcsrc/xpu/includes/simd.h1// Copyright (c) Microsoft Corporation.COMMENT
LOWcsrc/xpu/includes/simd.h21#define SIMD_SET(x) _mm512_set1_ps(x)COMMENT
LOWcsrc/xpu/includes/simd.h41#define SIMD_STORE(a, d) _mm256_storeu_ps(a, d)COMMENT
LOWcsrc/xpu/includes/compat.h1// Copyright (c) Microsoft Corporation.COMMENT
LOWcsrc/xpu/includes/cpu_adam.h1// Copyright (c) Microsoft Corporation.COMMENT
LOWcsrc/xpu/adagrad/cpu_adagrad.cpp1// Copyright (c) Microsoft Corporation.COMMENT
LOWcsrc/xpu/adam/multi_tensor_apply.dp.hpp1// Copyright (c) Microsoft Corporation.COMMENT
LOWcsrc/includes/ds_transformer_cuda.h1// Copyright (c) Microsoft Corporation.COMMENT
LOWcsrc/includes/cpu_lion.h1// Copyright (c) Microsoft Corporation.COMMENT
LOWcsrc/includes/deepcompile.h1// Copyright (c) Microsoft Corporation.COMMENT
LOWcsrc/includes/context.h1// Copyright (c) Microsoft Corporation.COMMENT
LOWcsrc/includes/cublas_wrappers.h1// Copyright (c) Microsoft Corporation.COMMENT
LOWcsrc/includes/cpu_adagrad.h1// Copyright (c) Microsoft Corporation.COMMENT
LOWcsrc/includes/ds_kernel_utils.h1// Copyright (c) Microsoft Corporation.COMMENT
LOWcsrc/includes/ds_kernel_utils.h21#include <cuda_bf16.h>COMMENT
LOWcsrc/includes/type_shim.h1// Copyright (c) Microsoft Corporation.COMMENT
LOWcsrc/includes/custom_cuda_layers.h1// Copyright (c) Microsoft Corporation.COMMENT
105 more matches not shown…
Self-Referential Comments38 hits · 112 pts
SeverityFileLineSnippetContext
MEDIUMdeepspeed/launcher/runner.py276 # Create a new list in the same order as original but with duplicatesSTRING
MEDIUMdeepspeed/module_inject/layers.py425 # This function is designed forCOMMENT
MEDIUMdeepspeed/module_inject/containers/base_moe.py6# Create a container object to save model-specific tensors using the policy file above.COMMENT
MEDIUMdeepspeed/module_inject/containers/base.py6# Create a container object to save model-specific tensors using the policy file above.COMMENT
MEDIUMdeepspeed/runtime/zero/parameter_offload.py612 """This function is used to recursively obtain the granularity of each module."""STRING
MEDIUM…peed/runtime/activation_checkpointing/checkpointing.py489 """This function is adapted from torch.utils.checkpoint withSTRING
MEDIUM…peed/runtime/activation_checkpointing/checkpointing.py705 """This function is union of `torch.utils.checkpoint._checkpoint_without_reentrant` and `CheckpointFunction` in thisSTRING
MEDIUMdeepspeed/comm/comm.py857 # Create a torch backend object, initialize torch distributed, and assign to cdbCOMMENT
MEDIUMdeepspeed/utils/comms_logging.py186 # Create a snapshot to avoid concurrent modification issuesCOMMENT
MEDIUMdeepspeed/utils/comms_logging.py270 # Create a snapshot of the dictionary to avoid concurrent modification issuesCOMMENT
MEDIUMdeepspeed/compile/backend.py180 # Create a set of parameter indices for quick lookupCOMMENT
MEDIUMdeepspeed/compile/backend.py321 # Create an InputStorage instance for this specific graphCOMMENT
MEDIUMdeepspeed/compile/input_storage.py112 # Create a tensor with the stored metadata (original behavior for non-int tensors)COMMENT
MEDIUMdeepspeed/compile/profilers/comm_profile.py152 # Create an interpolation functionCOMMENT
MEDIUMdeepspeed/compression/basic_layer.py213 # This function is used for row/col pruningCOMMENT
MEDIUM…nference/v2/model_implementations/mixtral/container.py6# Create a container object to save model-specific tensors using the policy file above.COMMENT
MEDIUM…ence/v2/model_implementations/qwen_v2_moe/container.py6# Create a container object to save model-specific tensors using the policy file above.COMMENT
MEDIUM…/inference/v2/model_implementations/phi3/containers.py6# Create a container object to save model-specific tensors using the policy file above.COMMENT
MEDIUM…d/inference/v2/model_implementations/qwen/container.py6# Create a container object to save model-specific tensors using the policy file above.COMMENT
MEDIUM…nference/v2/model_implementations/qwen_v2/container.py6# Create a container object to save model-specific tensors using the policy file above.COMMENT
MEDIUM…ference/v2/model_implementations/llama_v2/container.py6# Create a container object to save model-specific tensors using the policy file above.COMMENT
MEDIUM…ed/inference/v2/model_implementations/opt/container.py6# Create a container object to save model-specific tensors using the policy file above.COMMENT
MEDIUM…inference/v2/model_implementations/falcon/container.py6# Create a container object to save model-specific tensors using the policy file above.COMMENT
MEDIUM…d/inference/v2/model_implementations/phi/containers.py6# Create a container object to save model-specific tensors using the policy file above.COMMENT
MEDIUM…nference/v2/model_implementations/mistral/container.py6# Create a container object to save model-specific tensors using the policy file above.COMMENT
MEDIUM…epspeed/ops/sparse_attention/sparse_attention_utils.py15 """This class provides some utility functions that are use integrating sparse attention into transformer models.STRING
MEDIUMdeepspeed/moe/sharded_moe.py152# The following functions are extracted and scriptedCOMMENT
MEDIUMdeepspeed/moe/sharded_moe.py201 # Create a mask for 1st's expert per tokenCOMMENT
MEDIUMdeepspeed/moe/sharded_moe.py302 # Create a mask for 1st's expert per tokenCOMMENT
MEDIUMdeepspeed/moe/sharded_moe.py308 # Create a mask for 2nd's expert per token using Gumbel-max trickCOMMENT
MEDIUMdeepspeed/moe/utils.py99 # Create the param MoE groups, leave param assign to next stepCOMMENT
MEDIUMop_builder/builder.py333 # Create a compiler object.COMMENT
MEDIUMop_builder/builder.py339 # Create a temporary directory to hold test files.COMMENT
MEDIUMop_builder/builder.py342 # Define a simple C program that calls the function in questionCOMMENT
MEDIUMtests/unit/ulysses_alst/test_ulysses_sp_hf.py207 # Create a mock PEFT model object that has config but doesn't inherit from PreTrainedModelCOMMENT
MEDIUM…ivation_checkpointing/test_activation_checkpointing.py272 # Create a simple topology for testingCOMMENT
MEDIUM…ivation_checkpointing/test_activation_checkpointing.py288 # Create a mock GPT2 model with different layer typesCOMMENT
MEDIUMtests/unit/runtime/half_precision/onebit/test_onebit.py1257 # Create a model with normal and empty parametersCOMMENT
Hallucination Indicators8 hits · 95 pts
SeverityFileLineSnippetContext
CRITICAL…epspeed/ops/sparse_attention/sparse_attention_utils.py33 original_max_position = model.bert.embeddings.position_embeddings.weight.size(0)CODE
CRITICAL…epspeed/ops/sparse_attention/sparse_attention_utils.py36 model.bert.embeddings.position_embeddings.weight.data = model.bert.embeddings.position_embeddings.weight.repCODE
CRITICAL…epspeed/ops/sparse_attention/sparse_attention_utils.py45 extended_position_embedding = model.roberta.embeddings.position_embeddings.weight.new_empty(CODE
CRITICALtests/unit/modeling.py456 self.attention.output.LayerNorm.weight.register_hook(lambda x, self=self: grads.append([x, "N2_W"]))CODE
CRITICALtests/unit/modeling.py457 self.attention.output.LayerNorm.bias.register_hook(lambda x, self=self: grads.append([x, "N2_B"]))CODE
CRITICALtests/unit/v1/zero/test_zero.py1357 assert model.l2.linear.weight.ds_tensor.numel() == ds_tensor_numelCODE
CRITICALtests/unit/v1/zero/test_zero.py1360 assert model.l4.linear.weight.ds_tensor.numel() == ds_tensor_numelCODE
CRITICALtests/unit/inference/test_inference.py360 if pipe.tokenizer.model_max_length < deepspeed.ops.transformer.inference.config.DeepSpeedInferenceConfig(CODE
AI Structural Patterns95 hits · 92 pts
SeverityFileLineSnippetContext
LOWdeepspeed/__init__.py93CODE
LOW…d/model_implementations/transformers/ds_transformer.py111CODE
LOWdeepspeed/module_inject/policy.py48CODE
LOWdeepspeed/module_inject/auto_ep_folding.py184CODE
LOWdeepspeed/module_inject/layers.py60CODE
LOWdeepspeed/module_inject/layers.py383CODE
LOWdeepspeed/module_inject/containers/gptj.py49CODE
LOWdeepspeed/module_inject/containers/opt.py62CODE
LOWdeepspeed/module_inject/containers/llama.py62CODE
LOWdeepspeed/module_inject/containers/gptneo.py63CODE
LOWdeepspeed/module_inject/containers/internlm.py54CODE
LOWdeepspeed/module_inject/containers/llama2.py59CODE
LOWdeepspeed/module_inject/containers/gptneox.py43CODE
LOWdeepspeed/module_inject/containers/bloom.py57CODE
LOWdeepspeed/checkpoint/autoep_universal.py68CODE
LOWdeepspeed/linear/optimized_linear.py204CODE
LOWdeepspeed/runtime/lr_schedules.py447CODE
LOWdeepspeed/runtime/quantize.py16CODE
LOWdeepspeed/runtime/quantize.py100CODE
LOWdeepspeed/runtime/quantize.py113CODE
LOWdeepspeed/runtime/quantize.py121CODE
LOWdeepspeed/runtime/bf16_optimizer.py39CODE
LOWdeepspeed/runtime/engine.py238CODE
LOW…a_pipeline/data_sampling/variable_batch_size_and_lr.py23CODE
LOW…a_pipeline/data_sampling/variable_batch_size_and_lr.py165CODE
LOW…a_pipeline/data_sampling/variable_batch_size_and_lr.py432CODE
LOW…d/runtime/data_pipeline/data_sampling/data_analyzer.py25CODE
LOW…d/runtime/data_pipeline/data_sampling/data_analyzer.py459CODE
LOWdeepspeed/runtime/domino/transformer.py247CODE
LOWdeepspeed/runtime/pipe/module.py125CODE
LOWdeepspeed/runtime/sequence_parallel/ulysses_sp.py227CODE
LOWdeepspeed/runtime/sequence_parallel/ulysses_sp.py198CODE
LOWdeepspeed/runtime/zero/parameter_offload.py119CODE
LOWdeepspeed/runtime/zero/stage_1_and_2.py146CODE
LOWdeepspeed/runtime/zero/partition_parameters.py930CODE
LOWdeepspeed/runtime/zero/mics.py65CODE
LOWdeepspeed/runtime/zero/stage3.py160CODE
LOWdeepspeed/runtime/fp16/fused_optimizer.py40CODE
LOWdeepspeed/runtime/fp16/unfused_optimizer.py31CODE
LOWdeepspeed/runtime/fp16/onebit/zoadam.py58CODE
LOWdeepspeed/runtime/fp16/onebit/lamb.py62CODE
LOWdeepspeed/runtime/fp16/onebit/adam.py46CODE
LOWdeepspeed/comm/comm.py792CODE
LOWdeepspeed/profiling/flops_profiler/profiler.py1204CODE
LOWdeepspeed/compression/basic_layer.py672CODE
LOWdeepspeed/compression/basic_layer.py695CODE
LOWdeepspeed/compression/basic_layer.py118CODE
LOWdeepspeed/compression/utils.py96CODE
LOWdeepspeed/compression/utils.py140CODE
LOWdeepspeed/compression/utils.py181CODE
LOWdeepspeed/compression/utils.py217CODE
LOWdeepspeed/inference/quantization/utils.py93CODE
LOWdeepspeed/inference/quantization/utils.py149CODE
LOWdeepspeed/ops/lamb/fused_lamb.py41CODE
LOWdeepspeed/ops/sparse_attention/sparsity_config.py250CODE
LOWdeepspeed/ops/transformer/transformer.py89CODE
LOWdeepspeed/ops/transformer/transformer.py399CODE
LOW…speed/ops/transformer/inference/diffusers_attention.py91CODE
LOW…speed/ops/transformer/inference/diffusers_attention.py192CODE
LOW…speed/ops/transformer/inference/diffusers_attention.py87CODE
35 more matches not shown…
Excessive Try-Catch Wrapping67 hits · 78 pts
SeverityFileLineSnippetContext
LOWcsrc/aio/py_test/parse_aio_stats.py53 except Exception:CODE
LOWdeepspeed/launcher/launch.py232 except Exception as e:CODE
LOWdeepspeed/module_inject/auto_ep_presets/base.py200 except Exception as exc:CODE
LOWdeepspeed/module_inject/containers/bloom.py116 except Exception as e:CODE
MEDIUMdeepspeed/checkpoint/deepspeed_checkpoint.py274 print(f'Error: {file} is not existent')CODE
MEDIUMdeepspeed/checkpoint/reshape_utils.py31 print(f'Error: {file} is not existent')CODE
MEDIUMdeepspeed/runtime/engine.py817def __del__(self):CODE
LOWdeepspeed/runtime/engine.py820 except Exception as exc:CODE
LOWdeepspeed/runtime/torch_autocast.py51 except Exception as e:CODE
LOWdeepspeed/runtime/config_utils.py77 except Exception as e:CODE
LOWdeepspeed/runtime/config_utils.py95 except Exception as e:CODE
LOW…d/runtime/data_pipeline/data_sampling/data_analyzer.py560 except Exception as e:CODE
LOWdeepspeed/runtime/superoffload/superoffload_utils.py151 except Exception as e:CODE
LOWdeepspeed/runtime/superoffload/superoffload_utils.py217 except Exception as e:CODE
LOW…ntime/checkpoint_engine/decoupled_checkpoint_engine.py65 except Exception as e:CODE
LOW…ntime/checkpoint_engine/decoupled_checkpoint_engine.py116 except Exception:CODE
LOW…ntime/checkpoint_engine/decoupled_checkpoint_engine.py216 except Exception as e:CODE
LOW…ntime/checkpoint_engine/decoupled_checkpoint_engine.py222 except Exception:CODE
MEDIUM…ntime/checkpoint_engine/decoupled_checkpoint_engine.py42def init_decoupled_checkpoint(config_params, dp_writer_config, save_event, save_CODE
MEDIUM…ntime/checkpoint_engine/decoupled_checkpoint_engine.py113def __del__(self):CODE
LOW…time/checkpoint_engine/datastates_checkpoint_engine.py25 except Exception as e:CODE
LOWdeepspeed/comm/mori.py168 except Exception as e:CODE
LOWdeepspeed/comm/mori.py224 except Exception as e:CODE
LOWdeepspeed/comm/torch.py138 except Exception:CODE
LOWdeepspeed/utils/torch.py58 except Exception:CODE
LOWdeepspeed/utils/torch.py67 except Exception:CODE
LOWdeepspeed/compile/input_storage.py108 except Exception as e:CODE
LOWdeepspeed/compile/input_storage.py139 except Exception as e:CODE
LOWdeepspeed/compile/profilers/graph_profile.py44 except Exception as e:CODE
LOWdeepspeed/compile/profilers/graph_profile.py126 except Exception:CODE
LOWdeepspeed/compile/profilers/graph_profile.py168 except Exception as e:CODE
LOWdeepspeed/compile/profilers/graph_profile.py333 except Exception as e:CODE
LOWdeepspeed/autotuning/autotuner.py33except Exception as e:CODE
LOWdeepspeed/autotuning/tuner/base_tuner.py70 except Exception:CODE
LOWdeepspeed/moe/sharded_moe.py50except Exception:CODE
LOWdeepspeed/moe/ep_tp_dispatch.py195 except Exception:CODE
LOWci/tests_fetcher.py728 except Exception: # noqa: BLE001 -- explain is a diagnostic, never fail CI on itCODE
LOWci/tests_fetcher.py736 except Exception: # noqa: BLE001CODE
LOWci/test_tests_fetcher.py375 except Exception as e: # noqa: BLE001CODE
LOWci/test_torch_latest.py592 except Exception as exc: # noqa: BLE001CODE
LOWop_builder/builder.py211 except Exception:CODE
LOWop_builder/builder.py384 except Exception:CODE
LOWop_builder/builder.py414 except Exception as e:CODE
LOWop_builder/builder.py476 except Exception as e:CODE
MEDIUMop_builder/builder.py404def cpu_arch(self):CODE
MEDIUMop_builder/builder.py428def get_cuda_compile_flag(self):CODE
LOWop_builder/npu/builder.py33 except Exception:CODE
LOWop_builder/supa/fused_adam.py14except Exception:CODE
LOWop_builder/supa/fused_lion.py8except Exception:CODE
LOWop_builder/supa/fused_lamb.py10except Exception:CODE
LOWtests/unit/v1/nvme/test_aio_gil.py123 except Exception as error:CODE
LOWtests/unit/v1/nvme/test_aio_gil.py156 except Exception as error:CODE
LOWtests/unit/v1/nvme/test_aio_gil.py165 except Exception as error:CODE
MEDIUMtests/unit/v1/nvme/test_aio_gil.py119def run_read(index):CODE
MEDIUMtests/unit/v1/nvme/test_aio_gil.py151def run_reads():CODE
MEDIUMtests/unit/v1/nvme/test_aio_gil.py159def manage_locked_tensors():CODE
LOWtests/unit/runtime/zero/test_zero_grad_clip.py79 except Exception as e:CODE
LOWtests/unit/hybrid_engine/test_he_lora.py35 except Exception:CODE
LOWtests/unit/hybrid_engine/test_he_lora.py69 except Exception:CODE
LOWtests/model/Megatron_GPT2/run_checkpoint_test.py458 except Exception:CODE
7 more matches not shown…
Redundant / Tautological Comments27 hits · 38 pts
SeverityFileLineSnippetContext
LOWsetup.py260# Set cuda_version to 0.0 if cpu-only.COMMENT
LOWsetup.py263# Set hip_version to 0.0 if cpu-only.COMMENT
LOWdeepspeed/module_inject/auto_tp.py685 # Check if embedding matches any patternCOMMENT
LOWdeepspeed/module_inject/auto_ep.py252 # Check if MoE block has tuple return contract (legacy transformers)COMMENT
LOWdeepspeed/module_inject/load_checkpoint.py107 # Check if the weight tensor is for the QKV parameterCOMMENT
LOWdeepspeed/module_inject/load_checkpoint.py149 # Check if the weight tensor is for the QKV parameterCOMMENT
LOWdeepspeed/module_inject/layers.py590 # Check if the parameters belong to a replaced layer (indicated by a specific attribute)COMMENT
LOWdeepspeed/monitor/csv_monitor.py61 # Open file and record event. Insert header if this is the first time writingCOMMENT
LOWdeepspeed/runtime/engine.py2924 # Check if loss scaling was required but not appliedSTRING
LOWdeepspeed/runtime/engine.py3136 # Set flag to prevent hooks from firing (we'll manually call prologue/epilogue)STRING
LOWdeepspeed/runtime/utils.py66 # Check if first element is a dict (parameter groups) or a ParameterCOMMENT
LOWdeepspeed/runtime/utils.py824 # Print message except when distributed but not rank 0COMMENT
LOWdeepspeed/runtime/config_utils.py72 # Check if there is a new param and if it should be set with a valueCOMMENT
LOWdeepspeed/runtime/pipe/engine.py861 # Set _running_engine_backward to avoid RuntimeError in post-backward hookCOMMENT
LOWdeepspeed/runtime/zenflow/zenflow_stage_1_and_2.py483 # Set numel to next partition's offsetCOMMENT
LOW…speed/runtime/swap_tensor/partitioned_param_swapper.py135 #Check if partitioned param or numel in a tensor is swappable or notCOMMENT
LOWdeepspeed/runtime/zero/stage_1_and_2.py1346 # Set numel to next partition's offsetCOMMENT
LOW…peed/runtime/activation_checkpointing/checkpointing.py175 # Check if we have added the stateCOMMENT
LOW…ntime/checkpoint_engine/decoupled_checkpoint_engine.py139 # Check if process is still aliveCOMMENT
LOWdeepspeed/inference/engine.py182 # Check if local CUDA graphs can be created in replacement modulesCOMMENT
LOWdeepspeed/ops/adam/zenflow_torch_adam.py12# Check if we have PyTorch >= 2.0 for ZenFlow featuresCOMMENT
LOWdeepspeed/ops/adam/cpu_adam.py222 # Check if we can rollback (step count must be > 0)COMMENT
LOWop_builder/xpu/builder.py147 # Set CXX to icpx (Intel oneAPI DPC++ compiler) so that .cpp/.dp.cppCOMMENT
LOWtests/unit/runtime/pipe/test_pipe.py198 # Check if models have same weights after trainingCOMMENT
LOWtests/unit/runtime/pipe/test_pipe.py288 # Check if all layers have seen different shapesCOMMENT
LOWtests/unit/moe/test_moe.py128 # Set numel to next partition's offsetCOMMENT
LOWtests/small_model_debugging/test.py14 # Print message except when distributed but not rank 0COMMENT
AI Slop Vocabulary18 hits · 32 pts
SeverityFileLineSnippetContext
MEDIUMdeepspeed/compat.py20# Backwards compatible, applies best practices (use annotationlib) from python 3.14 onwards.COMMENT
LOWdeepspeed/nebula/constants.py35# Customer can just set NEBULA_ENABLE_NEBULA_LOAD to False. Then useSTRING
MEDIUMdeepspeed/runtime/compiler.py108# robust version of @torch.compileCOMMENT
MEDIUMdeepspeed/runtime/engine.py3634 # utilize dp_world_size for allreduce averageSTRING
LOWdeepspeed/runtime/pipe/engine.py778 # Some models just return loss from forward()COMMENT
LOWdeepspeed/runtime/pipe/module.py277 # that case and just use it in forward()COMMENT
MEDIUMdeepspeed/runtime/zero/partition_parameters.py483 # since skip_init won't involve any computations or weight adjustments, we can directly utilize postCOMMENT
LOWdeepspeed/utils/comms_logging.py107 # If this comm_op has already been logged with this message size, just add to existing recordCOMMENT
MEDIUMdeepspeed/compile/util.py107 # to utilize that information. This is left as future work when real needsCOMMENT
LOWdeepspeed/inference/v2/modules/heuristics.py51 # Currently, we only have one implementation, so we just return it.COMMENT
LOWdeepspeed/inference/v2/modules/heuristics.py70 # Currently, we only have one implementation, so we just return it.COMMENT
LOWdeepspeed/inference/v2/modules/heuristics.py132 # Currently, we only have one implementation, so we just return it.COMMENT
LOWdeepspeed/inference/v2/modules/heuristics.py153 # Currently, we only have one implementation, so we just return it.COMMENT
LOWdeepspeed/inference/v2/modules/heuristics.py193 # Currently, we only have one implementation, so we just return it.COMMENT
MEDIUMtests/unit/util.py226 # environment, so to make the test robust a combined entry of both streams is availableSTRING
LOWtests/unit/v1/zero/test_zero.py887 # TODO. add testing for this - for now we just call it to make sure itCOMMENT
LOWtests/unit/v1/zero/test_zero.py1276 # TODO. add testing for this - for now we just call it to make sure itCOMMENT
MEDIUMtests/unit/checkpoint/test_zero_optimizer.py670 # This test calls deepspeed.initialize(), so use the harness' file-storeCOMMENT
Docstring Block Structure6 hits · 30 pts
SeverityFileLineSnippetContext
HIGHdeepspeed/__init__.py108Initialize the DeepSpeed Engine. Arguments: args: an object containing local_rank and deepspeed_config fielSTRING
HIGHdeepspeed/elasticity/elasticity.py234Core deepspeed elasticity API. Given an elastic config (similar to the example below) DeepSpeed will compute a totalSTRING
HIGHdeepspeed/runtime/compiler.py43 Decorator factory to enable compiling of a function if the minimum PyTorch version requirement is met. Args: STRING
HIGHdeepspeed/runtime/utils.py1445 Count the number of parameters that participate in the currently running backward graph. This helper is designSTRING
HIGHdeepspeed/runtime/zero/linear.py153Applies a linear transformation to the incoming data: :math:`y = xA^T + b`. The weights are pre-transposed and storeSTRING
HIGHdeepspeed/autotuning/autotuner.py189Initialize and return a resource manager Args: active_resources ([dict]): A dictionary of hostname STRING
Modern Structural Boilerplate20 hits · 22 pts
SeverityFileLineSnippetContext
LOWdeepspeed/__init__.py84def set_optimizer_flags(config_class: DeepSpeedConfig, model: torch.nn.Module) -> None:CODE
LOWdeepspeed/module_inject/tp_plan_converter.py10logger = logging.getLogger(__name__)CODE
LOWdeepspeed/module_inject/auto_ep_config.py24__all__ = [CODE
LOWdeepspeed/module_inject/layers.py23__all__ = [CODE
LOWdeepspeed/module_inject/auto_ep_preset_adapters.py16__all__ = [CODE
LOWdeepspeed/runtime/engine.py5576 def _set_deepcompile_active(self, active: bool) -> None:CODE
LOWdeepspeed/runtime/rollout/__init__.py22__all__ = [CODE
LOWdeepspeed/runtime/zero/partitioned_param_coordinator.py37def update_recompute_parameters(sub_module: Module, param: Parameter) -> None:CODE
LOWdeepspeed/runtime/superoffload/superoffload_utils.py189 def _set_cpu_affinity(self, cpuadam_cores_perc: float) -> None:CODE
LOWdeepspeed/compile/custom_ops/__init__.py9__all__ = ["all_to_all", "sp_dp_registry", "sp_compat"]CODE
LOW…ence/v2/model_implementations/inference_policy_base.py43 def set_transformer_params(self, prefixes: Union[str, Iterable[str]], containers: List[LayerContainer]) -> None:CODE
LOW…ence/v2/model_implementations/inference_policy_base.py52 def set_non_transformer_params(self, container: LayerContainer) -> None:CODE
LOW…ence/v2/model_implementations/inference_policy_base.py55 def set_unmapped_params(self, prefixes: Union[str, Iterable[str]]) -> None:CODE
LOW…rence/v2/model_implementations/layer_container_base.py272 def set_dependency(self, dep_name: str, dep_value: torch.Tensor) -> None:CODE
LOWdeepspeed/sequence/auto_sp.py39logger = logging.getLogger(__name__)CODE
LOWdeepspeed/sequence/auto_sp.py134def _set_module_by_name(model: nn.Module, dotted_name: str, new_module: nn.Module) -> None:CODE
LOWdeepspeed/moe/ep_kernels.py30logger = logging.getLogger(__name__)CODE
LOWdeepspeed/moe/layer.py86 def set_deepspeed_parallelism(self, use_data_before_expert_parallel_: bool = False) -> None:CODE
LOWtests/unit/modeling.py47logger = logging.getLogger(__name__)CODE
LOWtests/unit/modelingpreln.py43logger = logging.getLogger(__name__)CODE
Fake / Example Data21 hits · 21 pts
SeverityFileLineSnippetContext
LOWdeepspeed/compile/util.py136 return [n for n in graph.nodes if n.op == "placeholder"]CODE
LOWdeepspeed/compile/list_schedule.py221 if node.op == 'placeholder':CODE
LOWdeepspeed/compile/inductor.py106 if n.op == "placeholder" and n.name in pm.param_names:CODE
LOWdeepspeed/compile/fx.py49 primals = [n for n in graph.nodes if n.op == "placeholder"]CODE
LOWdeepspeed/compile/fx.py50 non_primals = [n for n in graph.nodes if n.op != "placeholder"]CODE
LOWdeepspeed/compile/fx.py121 activation_nodes_set = set([n for n in graph.nodes if n.op == "placeholder" and n.name in activation_node_names])CODE
LOWdeepspeed/compile/passes/sp_compile.py192 if node.op == "placeholder" and "val" in node.meta:CODE
LOWdeepspeed/compile/passes/prefetch.py96 if node.op != "placeholder":CODE
LOWdeepspeed/compile/passes/prefetch.py151 if (node.op != "placeholder"CODE
LOWdeepspeed/compile/passes/prefetch.py152 and node.target != torch.ops.dc.reload_parameter) and order_rev[i + 1].op == "placeholder":CODE
LOWdeepspeed/compile/passes/offload_adam_states.py353 or node.op == 'placeholder' \CODE
LOWdeepspeed/compile/passes/offload_adam_states.py384 if node.op != 'placeholder':CODE
LOWdeepspeed/compile/passes/offload_adam_states.py407 if node.op != 'placeholder' and not inserted_sync:CODE
LOWdeepspeed/compile/passes/offload_adam_states.py418 or node.op == 'placeholder' \CODE
LOWdeepspeed/compile/passes/offload_adam_states.py474 if node.op == "output" or node.op == "placeholder":CODE
LOWdeepspeed/compile/passes/offload_adam_states.py509 if node.op != 'placeholder' and not inserted_offload and is_first_graph:CODE
LOWdeepspeed/compile/profilers/graph_profile.py185 if n.op in {"placeholder", "output"}:CODE
LOWdeepspeed/compile/profilers/graph_profile.py348 if n.op in {"placeholder", "output"}:CODE
LOWtests/unit/v1/compile/test_compile_autosp.py118 assert node.op == "placeholder"CODE
LOWtests/unit/v1/compile/test_compile_autosp.py133 assert node.op == "placeholder"CODE
LOWtests/unit/v1/compile/test_compile_autosp.py149 assert node.op == "placeholder"CODE
Cross-Language Confusion3 hits · 20 pts
SeverityFileLineSnippetContext
HIGHci/test_torch_latest.py568 assert "pull_request.head.sha || github.sha" in deployCODE
HIGHci/test_torch_latest.py569 assert "pull_request.head.repo.full_name || github.repository" in deployCODE
HIGHci/test_torch_latest.py542 trusted_ref = "ref: ${{ github.event.pull_request.base.sha || github.sha }}"CODE
Structural Annotation Overuse11 hits · 16 pts
SeverityFileLineSnippetContext
LOWCONTRIBUTING.md129### Step 1: proposal and discussionCOMMENT
LOWCONTRIBUTING.md142### Step 2: implementation and verificationCOMMENT
LOWCONTRIBUTING.md151### Step 3: release and maintenanceCOMMENT
LOWdocs/_tutorials/universal-checkpointing.md26### Step 1: Create ZeRO CheckpointCOMMENT
LOWdocs/_tutorials/universal-checkpointing.md41### Step 2: Convert ZeRO Checkpoint to Universal FormatCOMMENT
LOWdocs/_tutorials/universal-checkpointing.md61### Step 3: Resume Training with Universal CheckpointCOMMENT
LOWdocs/_tutorials/monitor.md87# Step 1: Import monitor (and DeepSpeed config, if needed)COMMENT
LOWdocs/_tutorials/monitor.md91# Step 2: Initialized monitor with DeepSpeed config (get DeepSpeed config object, if needed)COMMENT
LOWdocs/_tutorials/monitor.md110 # Step 3: Create list of 3-tuple records (single entry in this case)COMMENT
LOWdocs/_tutorials/monitor.md112 # Step 4: Call monitor.write_events on the list from step 3COMMENT
LOWdocs/_tutorials/comms-logging.md91 # Step 3: Call `deepspeed.comm.log_summary()`COMMENT
Verbosity Indicators5 hits · 8 pts
SeverityFileLineSnippetContext
LOWdeepspeed/runtime/zenflow/zenflow_stage_1_and_2.py779 # Step 1:- Calculate gradient norm using bit-16 gradsCOMMENT
LOWdeepspeed/runtime/zero/stage_1_and_2.py2236 # Step 1:- Calculate gradient norm using bit-16 gradsCOMMENT
LOWdeepspeed/runtime/zero/stage_1_and_2.py2242 # Step 2:- run optimizer and upscaling simultaneouslyCOMMENT
LOWdeepspeed/runtime/zero/stage_1_and_2.py2301 # Step 3:- run the optimizer if no offloadingCOMMENT
LOWdeepspeed/runtime/zero/stage_1_and_2.py2304 # Step 4:- get rid of the fp32 gradients. Not needed anymoreCOMMENT
TODO Padding3 hits · 4 pts
SeverityFileLineSnippetContext
LOWcsrc/cpu/comm/ccl.cpp213// TODO: implement torch's async_op behavior, document it.COMMENT
LOW…ointing/test_activation_checkpointing_non_reentrant.py6# TODO: add tests with model parallelism for activation partitioning and other features.COMMENT
LOW…ivation_checkpointing/test_activation_checkpointing.py6# TODO: add tests with model parallelism for activation partitioning and other features.COMMENT
Modern AI Meta-Vocabulary1 hit · 3 pts
SeverityFileLineSnippetContext
MEDIUMci/tests_fetcher.py152 # Core orchestration on the deepspeed.initialize() critical path. TestsCOMMENT
Slop Phrases1 hit · 2 pts
SeverityFileLineSnippetContext
LOWdeepspeed/runtime/zero/stage3.py888 # We need to keep the reference to this buffer to make sure you can free it in `offload_states`COMMENT
Overly Generic Function Names1 hit · 0 pts
SeverityFileLineSnippetContext
LOWdeepspeed/runtime/compiler.py55 def my_function():STRING