Repository Analysis

hpcaitech/ColossalAI

Making large AI models cheaper, faster and more accessible

22.8 Moderate AI signal View on GitHub

Analysis Overview

This report presents the forensic synthetic code analysis of hpcaitech/ColossalAI, a Python project with 41,412 GitHub stars. SynthScan v2.0 examined 335,260 lines of code across 2148 source files, recording 4661 pattern matches distributed across 23 syntactic categories. The overall adjusted score of 22.8 places this repository in the Moderate AI signal band.

The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).

22.8
Adjusted Score
22.8
Raw Score
100%
Time Factor
2026-07-13
Last Push
41.4K
Stars
Python
Language
335.3K
Lines of Code
2.1K
Files
4.7K
Pattern Hits
2026-07-14
Scan Date
0.18
HC Hit Rate

What These Metrics Mean

Adjusted Score
Primary synthetic code indicator. Raw score normalised per 1,000 lines of code and multiplied by the temporal discount factor. This is the definitive comparative metric — use it to rank repositories by AI authorship density.
Raw Score
The unmodified sum of all severity-weighted, context-multiplied pattern match scores before temporal discounting. Reflects the absolute signal strength independent of when the repository was last active.
Time Factor
The temporal discount multiplier (0–100%) applied to the raw score. Repositories last updated before ChatGPT's launch (Nov 2022) receive a 5% factor. Full signal is only assigned to repositories active in the post-adoption era (Jan 2024+).
Pattern Hits
Total count of individual pattern matches across all files and categories. A high hit count with a low score may indicate a very large codebase with isolated AI snippets; a low count with a high score indicates dense, concentrated AI signatures.
HC Hit Rate
High+Critical pattern hits per file, averaged across the repository. This orthogonal signal catches repositories where a few files are densely packed with high-severity AI tells — a strong indicator even when the normalised score appears moderate due to codebase size.
Lines of Code / Files
Total lines and files analysed. The scanner examines 94 file extensions. These denominators are used to normalise the score, enabling fair comparison between repositories of vastly different sizes.

Score History

This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.

Severity Breakdown

Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.

CRITICAL 1HIGH 381MEDIUM 642LOW 3637

Directory Score Breakdown

This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.

Pattern Findings

The scanner identified 4661 distinct pattern matches across 23 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.

Decorative Section Separators531 hits · 1862 pts
SeverityFileLineSnippetContext
MEDIUM…sts/test_shardformer/test_model/test_shard_deepseek.py130 # ===================================================================================COMMENT
MEDIUMtests/test_shardformer/test_model/test_shard_mixtral.py123 # ===================================================================================COMMENT
MEDIUMtests/kit/model_zoo/timm/timm.py6## ==============COMMENT
MEDIUMtests/kit/model_zoo/timm/timm.py8## ==============COMMENT
MEDIUMtests/kit/model_zoo/timm/timm.py150# ==============COMMENT
MEDIUMtests/kit/model_zoo/timm/timm.py152# ==============COMMENT
MEDIUMtests/kit/model_zoo/transformers/command.py14 # ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/command.py16 # ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/mistral.py7# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/mistral.py9# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/t5.py6# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/t5.py8# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/gptj.py8# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/gptj.py10# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/albert.py6# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/albert.py8# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/albert.py69# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/albert.py71# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/opt.py6# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/opt.py8# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/chatglm2.py7# ================================COMMENT
MEDIUMtests/kit/model_zoo/transformers/chatglm2.py9# ================================COMMENT
MEDIUMtests/kit/model_zoo/transformers/whisper.py6# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/whisper.py8# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/llama.py14 # ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/llama.py16 # ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/llama.py23 # -----------------------------------COMMENT
MEDIUMtests/kit/model_zoo/transformers/llama.py28 # -----------------------------------COMMENT
MEDIUMtests/kit/model_zoo/transformers/vit.py6# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/vit.py8# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/deepseek.py8# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/deepseek.py10# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/bert.py6# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/bert.py8# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/qwen3.py14 # ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/qwen3.py16 # ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/qwen3.py23 # -----------------------------------COMMENT
MEDIUMtests/kit/model_zoo/transformers/qwen3.py28 # -----------------------------------COMMENT
MEDIUMtests/kit/model_zoo/transformers/deepseek_v3.py8# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/deepseek_v3.py10# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/falcon.py6# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/falcon.py8# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/sam.py6# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/sam.py8# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/qwen2.py14 # ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/qwen2.py16 # ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/qwen2.py23 # -----------------------------------COMMENT
MEDIUMtests/kit/model_zoo/transformers/qwen2.py28 # -----------------------------------COMMENT
MEDIUMtests/kit/model_zoo/transformers/mixtral.py8# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/mixtral.py10# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/gpt.py8# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/gpt.py10# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/bloom.py6# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/bloom.py8# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/blip2.py6# ===============================COMMENT
MEDIUMtests/kit/model_zoo/transformers/blip2.py8# ===============================COMMENT
MEDIUMtests/test_pipeline/test_schedule/test_zerobubble_pp.py890 # ===================================================================================COMMENT
MEDIUMtests/test_pipeline/test_schedule/test_zerobubble_pp.py1045 # ===================================================================================COMMENT
MEDIUM…/test_fx/test_profiler/test_profiler_meta_info_prop.py67 # =====================================================COMMENT
MEDIUM…/test_fx/test_profiler/test_profiler_meta_info_prop.py83 # =====================================================COMMENT
471 more matches not shown…
Cross-File Repetition352 hits · 1760 pts
SeverityFileLineSnippetContext
HIGHtests/test_shardformer/test_layer/test_dist_log_prob.py0compute the log probabilities from logits for the given labels. args: logits (torch.tensor): the input logits. labels (tSTRING
HIGHapplications/ColossalChat/coati/distributed/utils.py0compute the log probabilities from logits for the given labels. args: logits (torch.tensor): the input logits. labels (tSTRING
HIGHapplications/ColossalChat/coati/models/utils.py0compute the log probabilities from logits for the given labels. args: logits (torch.tensor): the input logits. labels (tSTRING
HIGHtests/test_legacy/test_moe/moe_utils.py0a helper class to handle all-reduce operations in sub parallel groups. a all-reduce collective communication will be opeSTRING
HIGH…egacy/engine/gradient_handler/_moe_gradient_handler.py0a helper class to handle all-reduce operations in sub parallel groups. a all-reduce collective communication will be opeSTRING
HIGH…ne/gradient_handler/_data_parallel_gradient_handler.py0a helper class to handle all-reduce operations in sub parallel groups. a all-reduce collective communication will be opeSTRING
HIGH…radient_handler/_sequence_parallel_gradient_handler.py0a helper class to handle all-reduce operations in sub parallel groups. a all-reduce collective communication will be opeSTRING
HIGH…radient_handler/_pipeline_parallel_gradient_handler.py0a helper class to handle all-reduce operations in sub parallel groups. a all-reduce collective communication will be opeSTRING
HIGH…est_tensor_shard/test_metainfo/test_linear_metainfo.py0this function is for batchnorm memory test test and print real memory cost and estimated, this test will not be executedSTRING
HIGH…st_tensor_shard/test_metainfo/test_pooling_metainfo.py0this function is for batchnorm memory test test and print real memory cost and estimated, this test will not be executedSTRING
HIGH…/test_tensor_shard/test_metainfo/test_conv_metainfo.py0this function is for batchnorm memory test test and print real memory cost and estimated, this test will not be executedSTRING
HIGH…hard/test_metainfo/test_binary_elementwise_metainfo.py0this function is for batchnorm memory test test and print real memory cost and estimated, this test will not be executedSTRING
HIGH…/test_tensor_shard/test_metainfo/test_norm_metainfo.py0this function is for batchnorm memory test test and print real memory cost and estimated, this test will not be executedSTRING
HIGHcolossalai/pipeline/schedule/interleaved_pp.py0load a batch from data iterator. args: data_iter (iterable): data iterator. device (optional[torch.device], optional): tSTRING
HIGHcolossalai/pipeline/schedule/one_f_one_b.py0load a batch from data iterator. args: data_iter (iterable): data iterator. device (optional[torch.device], optional): tSTRING
HIGHcolossalai/pipeline/schedule/generate.py0load a batch from data iterator. args: data_iter (iterable): data iterator. device (optional[torch.device], optional): tSTRING
HIGHcolossalai/pipeline/schedule/zero_bubble_pp.py0load a batch from data iterator. args: data_iter (iterable): data iterator. device (optional[torch.device], optional): tSTRING
HIGHcolossalai/nn/optimizer/lamb.py0performs a single optimization step. args: closure (callable, optional): a closure that reevaluates the model and returnSTRING
HIGHcolossalai/nn/optimizer/lars.py0performs a single optimization step. args: closure (callable, optional): a closure that reevaluates the model and returnSTRING
HIGHcolossalai/nn/optimizer/galore.py0performs a single optimization step. args: closure (callable, optional): a closure that reevaluates the model and returnSTRING
HIGHcolossalai/nn/optimizer/distributed_lamb.py0performs a single optimization step. args: closure (callable, optional): a closure that reevaluates the model and returnSTRING
HIGHcolossalai/nn/optimizer/distributed_galore.py0performs a single optimization step. args: closure (callable, optional): a closure that reevaluates the model and returnSTRING
HIGHcolossalai/nn/optimizer/fused_lamb.py0performs a single optimization step. args: closure (callable, optional): a closure that reevaluates the model and returnSTRING
HIGHcolossalai/nn/optimizer/fused_sgd.py0performs a single optimization step. args: closure (callable, optional): a closure that reevaluates the model and returnSTRING
HIGHcolossalai/nn/optimizer/adafactor.py0performs a single optimization step. args: closure (callable, optional): a closure that reevaluates the model and returnSTRING
HIGHcolossalai/nn/optimizer/distributed_adafactor.py0performs a single optimization step. args: closure (callable, optional): a closure that reevaluates the model and returnSTRING
HIGHcolossalai/nn/optimizer/came.py0performs a single optimization step. args: closure (callable, optional): a closure that reevaluates the model and returnSTRING
HIGHcolossalai/nn/optimizer/distributed_came.py0performs a single optimization step. args: closure (callable, optional): a closure that reevaluates the model and returnSTRING
HIGHexamples/images/diffusion/ldm/util.py0performs a single optimization step. args: closure (callable, optional): a closure that reevaluates the model and returnSTRING
HIGHcolossalai/nn/layer/utils.py0only allow exact division. args: numerator (int): numerator of the division. denominator (int): denominator of the divisSTRING
HIGHcolossalai/legacy/nn/layer/utils/common.py0only allow exact division. args: numerator (int): numerator of the division. denominator (int): denominator of the divisSTRING
HIGHcolossalai/legacy/tensor/dist_spec_mgr.py0only allow exact division. args: numerator (int): numerator of the division. denominator (int): denominator of the divisSTRING
HIGH…tched_bias_addition_function/bias_addition_function.py0this method is used to create the non_bias_func proxy, the node created by this proxy will compute the main computation,STRING
HIGH…ddition_patch/patched_bias_addition_function/addbmm.py0this method is used to create the non_bias_func proxy, the node created by this proxy will compute the main computation,STRING
HIGH…h/patched_bias_addition_module/bias_addition_module.py0this method is used to create the non_bias_func proxy, the node created by this proxy will compute the main computation,STRING
HIGHcolossalai/fx/passes/meta_info_prop.py0run `module` via interpretation and return the result and record the shape of each node. args: *args (tensor): the samplSTRING
HIGHcolossalai/fx/passes/concrete_info_prop.py0run `module` via interpretation and return the result and record the shape of each node. args: *args (tensor): the samplSTRING
HIGHcolossalai/_analyzer/fx/passes/shape_prop.py0run `module` via interpretation and return the result and record the shape of each node. args: *args (tensor): the samplSTRING
HIGHcolossalai/fx/codegen/activation_checkpoint_codegen.py0add an obj to be tracked as a global. we call this for names that reference objects external to the graph, like functionSTRING
HIGHcolossalai/autochunk/autochunk_codegen.py0add an obj to be tracked as a global. we call this for names that reference objects external to the graph, like functionSTRING
HIGHcolossalai/_analyzer/fx/codegen.py0add an obj to be tracked as a global. we call this for names that reference objects external to the graph, like functionSTRING
HIGHcolossalai/fx/codegen/activation_checkpoint_codegen.py0delete values after their last use. this ensures that values that are not used in the remainder of the code are freed anSTRING
HIGHcolossalai/autochunk/autochunk_codegen.py0delete values after their last use. this ensures that values that are not used in the remainder of the code are freed anSTRING
HIGHcolossalai/_analyzer/fx/codegen.py0delete values after their last use. this ensures that values that are not used in the remainder of the code are freed anSTRING
HIGHcolossalai/booster/plugin/gemini_plugin.py0save model to checkpoint but only on master process.STRING
HIGHcolossalai/booster/plugin/torch_fsdp_plugin.py0save model to checkpoint but only on master process.STRING
HIGHcolossalai/booster/plugin/torch_ddp_plugin.py0save model to checkpoint but only on master process.STRING
HIGHcolossalai/booster/plugin/gemini_plugin.py0prepare a dataloader for distributed training. the dataloader will be wrapped by `torch.utils.data.dataloader` and `statSTRING
HIGHcolossalai/booster/plugin/dp_plugin_base.py0prepare a dataloader for distributed training. the dataloader will be wrapped by `torch.utils.data.dataloader` and `statSTRING
HIGHcolossalai/booster/plugin/hybrid_parallel_plugin.py0prepare a dataloader for distributed training. the dataloader will be wrapped by `torch.utils.data.dataloader` and `statSTRING
HIGHexamples/language/data_utils.py0prepare a dataloader for distributed training. the dataloader will be wrapped by `torch.utils.data.dataloader` and `statSTRING
HIGHexamples/language/mixtral/data_utils.py0prepare a dataloader for distributed training. the dataloader will be wrapped by `torch.utils.data.dataloader` and `statSTRING
HIGHexamples/language/deepseek/data_utils.py0prepare a dataloader for distributed training. the dataloader will be wrapped by `torch.utils.data.dataloader` and `statSTRING
HIGHexamples/language/llama/data_utils.py0prepare a dataloader for distributed training. the dataloader will be wrapped by `torch.utils.data.dataloader` and `statSTRING
HIGHcolossalai/shardformer/policies/command.py0if under pipeline parallel setting, replacing the original forward method of huggingface to customized forward method, aSTRING
HIGHcolossalai/shardformer/policies/mistral.py0if under pipeline parallel setting, replacing the original forward method of huggingface to customized forward method, aSTRING
HIGHcolossalai/shardformer/policies/gpt2.py0if under pipeline parallel setting, replacing the original forward method of huggingface to customized forward method, aSTRING
HIGHcolossalai/shardformer/policies/t5.py0if under pipeline parallel setting, replacing the original forward method of huggingface to customized forward method, aSTRING
HIGHcolossalai/shardformer/policies/gptj.py0if under pipeline parallel setting, replacing the original forward method of huggingface to customized forward method, aSTRING
HIGHcolossalai/shardformer/policies/opt.py0if under pipeline parallel setting, replacing the original forward method of huggingface to customized forward method, aSTRING
292 more matches not shown…
Hyper-Verbose Identifiers1068 hits · 1014 pts
SeverityFileLineSnippetContext
LOW…heckpoint_io/test_plugins_huggingface_compatibility.py78def test_huggingface_compatibility(world_size):CODE
LOWtests/test_checkpoint_io/test_gemini_checkpoint_io.py39def exam_state_dict_with_origin(CODE
LOWtests/test_checkpoint_io/test_gemini_checkpoint_io.py193def exam_lazy_from_pretrained():CODE
LOWtests/test_checkpoint_io/test_general_checkpoint_io.py23def test_unsharded_checkpoint(use_safetensors: bool, use_async: bool):CODE
LOWtests/test_checkpoint_io/test_general_checkpoint_io.py77def test_sharded_model_checkpoint(use_safetensors: bool, use_async: bool):CODE
LOWtests/test_checkpoint_io/test_general_checkpoint_io.py117def test_sharded_optimizer_checkpoint(use_async: bool):CODE
LOWtests/test_checkpoint_io/test_general_checkpoint_io.py186def test_sharded_optimizer_multiple_param_groups(use_async: bool):CODE
LOW…ts/test_checkpoint_io/test_gemini_torch_compability.py24def exam_torch_load_from_gemini(shard: bool, model_name: str):CODE
LOW…ts/test_checkpoint_io/test_gemini_torch_compability.py90def exam_gemini_load_from_torch(shard: bool, model_name: str):CODE
LOWtests/test_zero/test_gemini/test_zeroddp_state_dict.py18def ignore_the_first_parameter(model: torch.nn.Module):CODE
LOW…sts/test_zero/test_gemini/test_zerooptim_state_dict.py23def exam_zero_optim_state_dict(placement_config, keep_gathered):CODE
LOWtests/test_shardformer/test_flash_attention.py67def post_process_kwargs_for_raw_attn(attn_kwargs: dict):CODE
LOWtests/test_shardformer/test_with_torch_ddp.py18def check_shardformer_with_ddp(lazy_init: bool):CODE
LOWtests/test_shardformer/test_model/_utils.py120def build_model_from_hybrid_plugin(CODE
LOWtests/test_shardformer/test_model/_utils.py174def build_model_from_low_level_zero_plugin(CODE
LOWtests/test_shardformer/test_model/_utils.py200def run_forward_backward_with_hybrid_plugin(CODE
LOWtests/test_shardformer/test_model/_utils.py263def run_forward_backward_with_low_level_zero_plugin(CODE
LOWtests/test_shardformer/test_model/_utils.py301def check_output_hidden_state(CODE
LOWtests/test_shardformer/test_model/_utils.py368def get_grad_tensors_for_check(CODE
LOW…/test_shardformer/test_layer/test_sequence_parallel.py173def test_all_to_all_attention():CODE
LOWtests/test_shardformer/test_layer/test_linear_1d.py121def check_linear_without_weight_grad_store(lazy_init: bool, seq_parallel_mode: bool):CODE
LOWtests/test_shardformer/test_layer/test_linear_1d.py161def check_linear_with_weight_grad_store(lazy_init: bool, seq_parallel_mode: bool):CODE
LOWtests/test_shardformer/test_layer/test_linear_1d.py208def check_linear_col_plus_row(lazy_init: bool, seq_parallel_mode: bool, overlap: bool):CODE
LOWtests/test_shardformer/test_layer/test_dropout.py10def check_dropout_parallel_input():CODE
LOWtests/test_shardformer/test_layer/test_dropout.py42def check_dropout_replicated_input():CODE
LOWtests/test_legacy/test_context/test_hybrid_parallel.py32def check_pipeline_parallel_rank(rank):CODE
LOWtests/test_legacy/test_context/test_hybrid_parallel.py44def check_model_parallel_rank(rank):CODE
LOWtests/test_legacy/test_context/test_hybrid_parallel.py51def check_tensor_parallel_rank(rank):CODE
LOWtests/test_legacy/test_utils/test_memory.py9def _run_colo_set_process_memory_fraction_and_colo_device_memory_capacity():CODE
LOW…est_legacy/test_utils/test_activation_checkpointing.py45def test_activation_checkpointing(cpu_offload, use_reentrant):CODE
LOWtests/test_legacy/test_layers/test_cache_embedding.py224def run_parallel_freq_aware_embed_tablewise(rank, world_size):CODE
LOWtests/test_legacy/test_layers/test_cache_embedding.py310def run_parallel_freq_aware_embed_columnwise(rank, world_size):CODE
LOWtests/test_legacy/test_layers/test_cache_embedding.py389def test_parallel_freq_aware_embed(world_size):CODE
LOW…legacy/test_layers/test_1d/checks_1d/check_layer_1d.py203def check_vocab_parallel_embed():CODE
LOW…legacy/test_layers/test_1d/checks_1d/check_layer_1d.py245def check_classifier_no_given_weight():CODE
LOW…legacy/test_layers/test_1d/checks_1d/check_layer_1d.py311def check_vocab_parallel_classifier_no_given_weight():CODE
LOW…legacy/test_layers/test_1d/checks_1d/check_layer_1d.py371def check_classifier_given_embed_weight():CODE
LOW…legacy/test_layers/test_1d/checks_1d/check_layer_1d.py422def check_vocab_parallel_classifier_given_embed_weight():CODE
LOW…legacy/test_layers/test_1d/checks_1d/check_layer_1d.py474def check_vocab_parallel_loss():CODE
LOW…legacy/test_layers/test_1d/checks_1d/check_layer_1d.py510def check_linear_row_stream_inference():CODE
LOWtests/test_legacy/test_layers/test_2d/test_2d.py51def check_layer_and_operation(rank, world_size, port):CODE
LOW…legacy/test_layers/test_2d/checks_2d/check_layer_2d.py276def check_vocab_parallel_embed():CODE
LOW…legacy/test_layers/test_2d/checks_2d/check_layer_2d.py323def check_classifier_no_given_weight():CODE
LOW…legacy/test_layers/test_2d/checks_2d/check_layer_2d.py401def check_vocab_parallel_classifier_no_given_weight():CODE
LOW…legacy/test_layers/test_2d/checks_2d/check_layer_2d.py469def check_classifier_given_embed_weight():CODE
LOW…legacy/test_layers/test_2d/checks_2d/check_layer_2d.py521def check_vocab_parallel_classifier_given_embed_weight():CODE
LOW…legacy/test_layers/test_2d/checks_2d/check_layer_2d.py610def check_vocab_parallel_loss():CODE
LOWtests/test_legacy/test_layers/test_2p5d/test_2p5d.py39def check_layer_and_operation(rank, world_size, port):CODE
LOW…/test_layers/test_2p5d/checks_2p5d/check_layer_2p5d.py280def check_vocab_parallel_embed():CODE
LOW…/test_layers/test_2p5d/checks_2p5d/check_layer_2p5d.py328def check_classifier_no_given_weight():CODE
LOW…/test_layers/test_2p5d/checks_2p5d/check_layer_2p5d.py407def check_vocab_parallel_classifier_no_given_weight():CODE
LOW…/test_layers/test_2p5d/checks_2p5d/check_layer_2p5d.py474def check_classifier_given_embed_weight():CODE
LOW…/test_layers/test_2p5d/checks_2p5d/check_layer_2p5d.py526def check_vocab_parallel_classifier_given_embed_weight():CODE
LOW…/test_layers/test_2p5d/checks_2p5d/check_layer_2p5d.py615def check_vocab_parallel_loss():CODE
LOWtests/test_legacy/test_layers/test_3d/test_3d.py45def check_layer_and_operation(rank, world_size, port):CODE
LOW…legacy/test_layers/test_3d/checks_3d/check_layer_3d.py206def check_classifier_no_given_weight():CODE
LOW…legacy/test_layers/test_3d/checks_3d/check_layer_3d.py306def check_vocab_parallel_classifier_no_given_weight():CODE
LOW…legacy/test_layers/test_3d/checks_3d/check_layer_3d.py413def check_classifier_given_embed_weight():CODE
LOW…legacy/test_layers/test_3d/checks_3d/check_layer_3d.py497def check_vocab_parallel_classifier_given_embed_weight():CODE
LOW…legacy/test_layers/test_3d/checks_3d/check_layer_3d.py747def check_vocab_parallel_embed():CODE
1008 more matches not shown…
Unused Imports960 hits · 940 pts
SeverityFileLineSnippetContext
LOWsetup.py8CODE
LOWtests/test_legacy/test_layers/test_1d/test_1d.py6CODE
LOWtests/test_legacy/test_layers/test_2p5d/test_2p5d.py3CODE
LOW…/test_layers/test_2p5d/checks_2p5d/check_layer_2p5d.py22CODE
LOW…t_layers/test_2p5d/checks_2p5d/check_operation_2p5d.py9CODE
LOWtests/test_legacy/test_tensor/common_utils/__init__.py1CODE
LOWtests/test_auto_parallel/test_offload/test_perf.py18CODE
LOWtests/test_auto_parallel/test_offload/test_solver.py11CODE
LOWtests/kit/model_zoo/__init__.py3CODE
LOWtests/kit/model_zoo/__init__.py3CODE
LOWtests/kit/model_zoo/__init__.py3CODE
LOWtests/kit/model_zoo/__init__.py3CODE
LOWtests/kit/model_zoo/__init__.py3CODE
LOWtests/kit/model_zoo/__init__.py3CODE
LOWtests/kit/model_zoo/__init__.py4CODE
LOWtests/kit/model_zoo/__init__.py4CODE
LOWtests/kit/model_zoo/__init__.py5CODE
LOWtests/kit/model_zoo/torchrec/__init__.py1CODE
LOWtests/kit/model_zoo/diffusers/__init__.py1CODE
LOWtests/kit/model_zoo/torchvision/__init__.py1CODE
LOWtests/kit/model_zoo/timm/__init__.py1CODE
LOWtests/kit/model_zoo/transformers/__init__.py1CODE
LOWtests/kit/model_zoo/transformers/__init__.py2CODE
LOWtests/kit/model_zoo/transformers/__init__.py3CODE
LOWtests/kit/model_zoo/transformers/__init__.py4CODE
LOWtests/kit/model_zoo/transformers/__init__.py5CODE
LOWtests/kit/model_zoo/transformers/__init__.py6CODE
LOWtests/kit/model_zoo/transformers/__init__.py7CODE
LOWtests/kit/model_zoo/transformers/__init__.py8CODE
LOWtests/kit/model_zoo/transformers/__init__.py9CODE
LOWtests/kit/model_zoo/transformers/__init__.py10CODE
LOWtests/kit/model_zoo/transformers/__init__.py11CODE
LOWtests/kit/model_zoo/transformers/__init__.py12CODE
LOWtests/kit/model_zoo/transformers/__init__.py13CODE
LOWtests/kit/model_zoo/transformers/__init__.py14CODE
LOWtests/kit/model_zoo/transformers/__init__.py15CODE
LOWtests/kit/model_zoo/transformers/__init__.py16CODE
LOWtests/kit/model_zoo/transformers/__init__.py17CODE
LOWtests/kit/model_zoo/transformers/__init__.py18CODE
LOWtests/kit/model_zoo/transformers/__init__.py19CODE
LOWtests/kit/model_zoo/transformers/__init__.py20CODE
LOWtests/kit/model_zoo/torchaudio/__init__.py1CODE
LOWtests/kit/model_zoo/custom/__init__.py1CODE
LOWtests/kit/model_zoo/custom/__init__.py2CODE
LOWtests/kit/model_zoo/custom/__init__.py3CODE
LOWtests/kit/model_zoo/custom/__init__.py4CODE
LOWtests/kit/model_zoo/custom/__init__.py5CODE
LOW…/test_infer/test_kernels/triton/test_rmsnorm_triton.py10CODE
LOWtests/test_infer/test_kernels/triton/test_xine_copy.py8CODE
LOW…nfer/test_kernels/triton/test_rotary_embdding_unpad.py13CODE
LOW…s/test_infer/test_kernels/triton/test_decoding_attn.py19CODE
LOW…ts/test_infer/test_kernels/triton/test_kvcache_copy.py14CODE
LOW…t_infer/test_kernels/triton/test_context_attn_unpad.py15CODE
LOW…fer/test_kernels/triton/test_fused_rotary_embedding.py12CODE
LOWextensions/pybind/optimizer/__init__.py1CODE
LOWextensions/pybind/layernorm/__init__.py1CODE
LOWextensions/pybind/cpu_adam/__init__.py1CODE
LOWextensions/pybind/cpu_adam/__init__.py2CODE
LOWextensions/pybind/softmax/__init__.py1CODE
LOWextensions/pybind/softmax/__init__.py2CODE
900 more matches not shown…
Deep Nesting545 hits · 540 pts
SeverityFileLineSnippetContext
LOWtests/test_checkpoint_io/test_safetensors_async_io.py118CODE
LOW…ts/test_checkpoint_io/test_torch_fsdp_checkpoint_io.py18CODE
LOW…/test_shardformer/test_model/test_shard_deepseek_v3.py26CODE
LOWtests/test_legacy/test_comm/test_object_list_p2p_v2.py85CODE
LOWtests/test_lora/test_lora.py20CODE
LOW…arallel/test_ckpt_solvers/test_C_solver_consistency.py29CODE
LOW…test_auto_parallel/test_ckpt_solvers/test_linearize.py32CODE
LOW…test_auto_parallel/test_ckpt_solvers/test_linearize.py96CODE
LOW…_tensor_shard/test_node_handler/test_matmul_handler.py50CODE
LOW…_parallel/test_tensor_shard/test_node_handler/utils.py66CODE
LOWtests/kit/model_zoo/registry.py64CODE
LOWtests/test_pipeline/test_schedule/test_zerobubble_pp.py114CODE
LOWtests/test_tensor/test_dtensor/test_dtensor.py22CODE
LOWtests/test_optimizer/test_dist_came.py63CODE
LOWtests/test_optimizer/test_dist_adafactor.py196CODE
LOWtests/test_optimizer/_utils.py230CODE
LOWtests/test_moe/test_moe_checkpoint.py42CODE
LOWtests/test_moe/test_moe_checkpoint.py90CODE
LOW…ochunk_transformer/test_autochunk_transformer_utils.py81CODE
LOWtests/test_fp8/test_fp8_fsdp_comm_hook.py33CODE
LOWextensions/utils.py154CODE
LOWcolossalai/pipeline/weight_grad_store.py19CODE
LOWcolossalai/pipeline/schedule/v_schedule.py81CODE
LOWcolossalai/pipeline/schedule/v_schedule.py290CODE
LOWcolossalai/pipeline/schedule/v_schedule.py318CODE
LOWcolossalai/pipeline/schedule/generate.py203CODE
LOWcolossalai/pipeline/schedule/generate.py348CODE
LOWcolossalai/pipeline/schedule/zero_bubble_pp.py239CODE
LOWcolossalai/pipeline/schedule/zero_bubble_pp.py297CODE
LOWcolossalai/pipeline/schedule/zero_bubble_pp.py871CODE
LOWcolossalai/interface/model.py9CODE
LOWcolossalai/nn/optimizer/adafactor.py86CODE
LOWcolossalai/nn/optimizer/lars.py40CODE
LOWcolossalai/nn/optimizer/galore.py118CODE
LOWcolossalai/nn/optimizer/galore.py235CODE
LOWcolossalai/nn/optimizer/distributed_adafactor.py286CODE
LOWcolossalai/nn/optimizer/came.py64CODE
LOWcolossalai/nn/optimizer/cpu_adam.py127CODE
LOWcolossalai/nn/optimizer/distributed_lamb.py104CODE
LOWcolossalai/nn/optimizer/distributed_galore.py143CODE
LOWcolossalai/nn/optimizer/fused_lamb.py105CODE
LOWcolossalai/nn/optimizer/distributed_came.py68CODE
LOWcolossalai/nn/optimizer/distributed_came.py333CODE
LOWcolossalai/nn/optimizer/hybrid_adam.py93CODE
LOWcolossalai/nn/layer/scaled_softmax.py132CODE
LOWcolossalai/fx/_meta_regist_12.py435CODE
LOWcolossalai/fx/tracer/experimental.py497CODE
LOWcolossalai/fx/tracer/experimental.py525CODE
LOWcolossalai/fx/tracer/experimental.py595CODE
LOWcolossalai/fx/tracer/experimental.py201CODE
LOWcolossalai/fx/tracer/experimental.py334CODE
LOWcolossalai/fx/tracer/experimental.py366CODE
LOWcolossalai/fx/tracer/experimental.py370CODE
LOWcolossalai/fx/tracer/tracer.py83CODE
LOWcolossalai/fx/tracer/tracer.py152CODE
LOWcolossalai/fx/tracer/tracer.py218CODE
LOWcolossalai/fx/tracer/tracer.py317CODE
LOWcolossalai/fx/tracer/tracer.py157CODE
LOW…ai/fx/tracer/meta_patch/patched_function/arithmetic.py8CODE
LOW…lai/fx/tracer/meta_patch/patched_function/torch_ops.py146CODE
485 more matches not shown…
Modern Structural Boilerplate360 hits · 355 pts
SeverityFileLineSnippetContext
LOWtests/test_lazy/lazy_init_utils.py21def set_seed(seed: int) -> None:CODE
LOWtests/kit/model_zoo/registry.py5__all__ = ["ModelZooRegistry", "ModelAttribute", "model_zoo"]CODE
LOWtests/kit/model_zoo/__init__.py33__all__ = ["model_zoo", "run_fwd", "run_fwd_bwd", "COMMON_MODELS", "IS_FAST_TEST"]CODE
LOWtests/test_optimizer/test_adam_optim.py30def set_grad(model: nn.Module, torch_model: nn.Module, g_dtype: torch.dtype) -> None:CODE
LOWextensions/triton_extension.py3__all__ = ["_TritonExtension"]CODE
LOWextensions/base_extension.py6__all__ = ["_Extension"]CODE
LOWextensions/__init__.py27__all__ = [CODE
LOWextensions/cpp_extension.py10__all__ = ["_CppExtension"]CODE
LOWextensions/cuda_extension.py11__all__ = ["_CudaExtension"]CODE
LOWextensions/pybind/optimizer/__init__.py3__all__ = ["FusedOptimizerCudaExtension"]CODE
LOWextensions/pybind/layernorm/__init__.py3__all__ = ["LayerNormCudaExtension"]CODE
LOWextensions/pybind/cpu_adam/__init__.py4__all__ = ["CpuAdamArmExtension", "CpuAdamX86Extension"]CODE
LOWextensions/pybind/softmax/__init__.py4__all__ = ["ScaledMaskedSoftmaxCudaExtension", "ScaledUpperTriangleMaskedSoftmaxCudaExtension"]CODE
LOWextensions/pybind/inference/__init__.py3__all__ = ["InferenceOpsCudaExtension"]CODE
LOWextensions/pybind/moe/__init__.py3__all__ = ["MoeCudaExtension"]CODE
LOWextensions/pybind/flash_attention/__init__.py14__all__ = ["FlashAttentionDaoCudaExtension", "FlashAttentionSdpaCudaExtension", "FlashAttentionNpuExtension"]CODE
LOWcolossalai/__init__.py13__all__ = ["launch", "launch_from_openmpi", "launch_from_slurm", "launch_from_torch", "__version__"]CODE
LOWcolossalai/pipeline/__init__.py5__all__ = [CODE
LOWcolossalai/pipeline/schedule/__init__.py6__all__ = [CODE
LOWcolossalai/cluster/__init__.py6__all__ = ["DistCoordinator", "ProcessGroupManager", "DeviceMeshManager", "ProcessGroupMesh"]CODE
LOWcolossalai/interface/pretrained.py5__all__ = [CODE
LOWcolossalai/interface/pretrained.py15def set_pretrained_path(model: Module, path: str) -> None:CODE
LOWcolossalai/interface/__init__.py4__all__ = ["OptimizerWrapper", "ModelWrapper", "AMPModelMixin"]CODE
LOWcolossalai/context/__init__.py3__all__ = [CODE
LOWcolossalai/nn/optimizer/adafactor.py21__all__ = ["Adafactor"]CODE
LOWcolossalai/nn/optimizer/__init__.py21__all__ = [CODE
LOWcolossalai/nn/optimizer/distributed_adafactor.py12__all__ = ["DistributedAdaFactor"]CODE
LOWcolossalai/nn/optimizer/distributed_lamb.py12__all__ = ["DistributedLamb"]CODE
LOWcolossalai/nn/optimizer/distributed_galore.py17__all__ = ["DistributedGalore"]CODE
LOWcolossalai/nn/lr_scheduler/__init__.py8__all__ = [CODE
LOWcolossalai/fx/proxy.py8__all__ = ["ColoProxy"]CODE
LOWcolossalai/fx/tracer/_tracer_utils.py8__all__ = ["is_element_in_list", "extract_meta"]CODE
LOWcolossalai/fx/tracer/tracer.py32__all__ = ["ColoTracer"]CODE
LOWcolossalai/fx/profiler/constants.py3__all__ = ["ALIAS_ATEN", "INPLACE_NEW", "INPLACE_MATH_ATEN", "CLONE_ATEN", "RELU_LIKE_OPS", "RELU_LIKE_MOD"]CODE
LOWcolossalai/fx/profiler/tensor.py10__all__ = ["MetaTensor"]CODE
LOWcolossalai/fx/profiler/profiler.py18__all__ = ["profile_function", "profile_module", "profile_method"]CODE
LOWcolossalai/fx/profiler/shard_utils.py10__all__ = ["calculate_fwd_in", "calculate_fwd_tmp", "calculate_fwd_out"]CODE
LOWcolossalai/fx/profiler/memory_utils.py8__all__ = ["activation_size", "parameter_size", "is_inplace"]CODE
LOWcolossalai/fx/profiler/experimental/constants.py5__all__ = ["INPLACE_OPS", "INPLACE_METHOD", "NON_INPLACE_METHOD"]CODE
LOWcolossalai/fx/profiler/experimental/profiler.py12__all__ = ["profile_function", "profile_module", "profile_method"]CODE
LOWcolossalai/fx/profiler/experimental/shard_utils.py7__all__ = ["calculate_fwd_in", "calculate_fwd_tmp", "calculate_fwd_out"]CODE
LOWcolossalai/booster/accelerator.py4__all__ = ["Accelerator"]CODE
LOWcolossalai/booster/booster.py30__all__ = ["Booster"]CODE
LOWcolossalai/booster/plugin/gemini_plugin.py37__all__ = ["GeminiPlugin"]CODE
LOWcolossalai/booster/plugin/plugin_base.py12__all__ = ["Plugin"]CODE
LOWcolossalai/booster/plugin/torch_fsdp_plugin.py37__all__ = ["TorchFSDPPlugin"]CODE
LOWcolossalai/booster/plugin/__init__.py8__all__ = [CODE
LOWcolossalai/booster/plugin/low_level_zero_plugin.py49__all__ = ["LowLevelZeroPlugin"]CODE
LOWcolossalai/booster/plugin/torch_ddp_plugin.py22__all__ = ["TorchDDPPlugin"]CODE
LOWcolossalai/booster/mixed_precision/__init__.py8__all__ = [CODE
LOWcolossalai/booster/mixed_precision/fp16_torch.py13__all__ = ["FP16_Torch_MixedPrecision", "TorchAMPOptimizer", "TorchAMPModule"]CODE
LOWcolossalai/shardformer/policies/command.py26__all__ = ["CommandPolicy", "CommandForCausalLMPolicy"]CODE
LOWcolossalai/shardformer/policies/command.py278 def set_pipeline_forward(self, model_cls: nn.Module, new_forward: Callable, policy: Dict) -> None:CODE
LOWcolossalai/shardformer/policies/mistral.py28__all__ = ["MistralPolicy", "MistralModelPolicy", "MistralForCausalLMPolicy", "MistralForSequenceClassificationPolicy"]CODE
LOWcolossalai/shardformer/policies/mistral.py264 def set_pipeline_forward(self, model_cls: nn.Module, new_forward: Callable, policy: Dict) -> None:CODE
LOWcolossalai/shardformer/policies/gpt2.py12__all__ = [CODE
LOWcolossalai/shardformer/policies/gpt2.py325 def set_pipeline_forward(self, model_cls: nn.Module, new_forward: Callable, policy: Dict) -> None:CODE
LOWcolossalai/shardformer/policies/t5.py35__all__ = [CODE
LOWcolossalai/shardformer/policies/t5.py575 def set_pipeline_forward(self, model_cls: nn.Module, new_forward: Callable, policy: Dict) -> None:CODE
LOWcolossalai/shardformer/policies/gptj.py16__all__ = [CODE
300 more matches not shown…
AI Structural Patterns354 hits · 352 pts
SeverityFileLineSnippetContext
LOW…/test_shardformer/test_layer/test_sequence_parallel.py91CODE
LOW…est_legacy/test_utils/test_activation_checkpointing.py30CODE
LOW…est_legacy/test_utils/test_activation_checkpointing.py39CODE
LOW…o_parallel/test_tensor_shard/test_liveness_analysis.py26CODE
LOW…t_tensor_shard/test_node_handler/test_where_handler.py20CODE
LOW…d/test_node_handler/test_binary_elementwise_handler.py132CODE
LOW…d/test_node_handler/test_binary_elementwise_handler.py143CODE
LOW…d/test_node_handler/test_binary_elementwise_handler.py28CODE
LOWtests/test_infer/test_kernels/triton/kernel_utils.py78CODE
LOWtests/test_smoothquant/test_llama_attention.py49CODE
LOWextensions/pybind/optimizer/fused_optimizer_cuda.py20CODE
LOWextensions/pybind/layernorm/layernorm_cuda.py13CODE
LOWextensions/pybind/layernorm/layernorm_cuda.py17CODE
LOWextensions/pybind/cpu_adam/cpu_adam_arm.py26CODE
LOWextensions/pybind/cpu_adam/cpu_adam_x86.py26CODE
LOW…d/softmax/scaled_upper_triangle_masked_softmax_cuda.py16CODE
LOWextensions/pybind/softmax/scaled_masked_softmax_cuda.py13CODE
LOWextensions/pybind/inference/inference_ops_cuda.py23CODE
LOWextensions/pybind/moe/moe_cuda.py13CODE
LOW…ons/pybind/flash_attention/flash_attention_dao_cuda.py45CODE
LOW…tensions/pybind/flash_attention/flash_attention_npu.py37CODE
LOW…ns/pybind/flash_attention/flash_attention_sdpa_cuda.py32CODE
LOWcolossalai/nn/optimizer/fused_adam.py54CODE
LOWcolossalai/nn/optimizer/adafactor.py26CODE
LOWcolossalai/nn/optimizer/galore.py192CODE
LOWcolossalai/nn/optimizer/distributed_adafactor.py16CODE
LOWcolossalai/nn/optimizer/cpu_adam.py65CODE
LOWcolossalai/nn/optimizer/distributed_galore.py49CODE
LOWcolossalai/nn/optimizer/fused_lamb.py52CODE
LOWcolossalai/nn/optimizer/hybrid_adam.py63CODE
LOWcolossalai/nn/lr_scheduler/onecycle.py68CODE
LOWcolossalai/fx/_meta_regist_12.py161CODE
LOWcolossalai/fx/_meta_regist_12.py178CODE
LOWcolossalai/fx/profiler/profiler.py214CODE
LOWcolossalai/booster/plugin/gemini_plugin.py442CODE
LOWcolossalai/booster/plugin/moe_hybrid_parallel_plugin.py40CODE
LOWcolossalai/booster/plugin/moe_hybrid_parallel_plugin.py177CODE
LOWcolossalai/booster/plugin/torch_fsdp_plugin.py472CODE
LOWcolossalai/booster/plugin/low_level_zero_plugin.py407CODE
LOWcolossalai/booster/plugin/hybrid_parallel_plugin.py491CODE
LOWcolossalai/booster/plugin/hybrid_parallel_plugin.py667CODE
LOWcolossalai/booster/plugin/hybrid_parallel_plugin.py1000CODE
LOWcolossalai/booster/mixed_precision/fp16_apex.py26CODE
LOWcolossalai/shardformer/modeling/command.py40CODE
LOWcolossalai/shardformer/modeling/command.py244CODE
LOWcolossalai/shardformer/modeling/command.py433CODE
LOWcolossalai/shardformer/modeling/command.py576CODE
LOWcolossalai/shardformer/modeling/mistral.py25CODE
LOWcolossalai/shardformer/modeling/mistral.py187CODE
LOWcolossalai/shardformer/modeling/mistral.py276CODE
LOWcolossalai/shardformer/modeling/mistral.py379CODE
LOWcolossalai/shardformer/modeling/mistral.py560CODE
LOWcolossalai/shardformer/modeling/gpt2.py116CODE
LOWcolossalai/shardformer/modeling/gpt2.py344CODE
LOWcolossalai/shardformer/modeling/gpt2.py433CODE
LOWcolossalai/shardformer/modeling/gpt2.py527CODE
LOWcolossalai/shardformer/modeling/gpt2.py618CODE
LOWcolossalai/shardformer/modeling/gpt2.py693CODE
LOWcolossalai/shardformer/modeling/t5.py32CODE
LOWcolossalai/shardformer/modeling/t5.py264CODE
294 more matches not shown…
Self-Referential Comments75 hits · 192 pts
SeverityFileLineSnippetContext
MEDIUM…t_hybrid_parallel_grad_clip_norm/test_amp_optimizer.py74 # Create a list of parameter-gradient pairs containing working parameters and their gradientsCOMMENT
MEDIUM…est_tensor_shard/test_metainfo/test_linear_metainfo.py23 """This function is for linear memory testSTRING
MEDIUM…est_tensor_shard/test_metainfo/test_linear_metainfo.py61 """This function is for linear memory testSTRING
MEDIUM…st_tensor_shard/test_metainfo/test_pooling_metainfo.py14 """This function is for AdaptiveAvgPool memory testSTRING
MEDIUM…st_tensor_shard/test_metainfo/test_pooling_metainfo.py55 """This function is for MaxPool memory testSTRING
MEDIUM…/test_tensor_shard/test_metainfo/test_conv_metainfo.py23 """This function is for conv memory testSTRING
MEDIUM…/test_tensor_shard/test_metainfo/test_conv_metainfo.py65 """This function is for conv function memory testSTRING
MEDIUM…hard/test_metainfo/test_binary_elementwise_metainfo.py24 """This function is for binary elementwise ops memory testSTRING
MEDIUM…/test_tensor_shard/test_metainfo/test_norm_metainfo.py18 """This function is for batchnorm memory testSTRING
MEDIUMtests/test_infer/test_streamingllm.py115 result_list = manager.list([-1] * 1) # Create a shared listCODE
MEDIUMtests/test_infer/test_inference_engine.py90 result_list = manager.list([-1] * world_size) # Create a shared listCODE
MEDIUMtests/test_infer/test_models/test_baichuan.py82 result_list = manager.list([-1] * world_size) # Create a shared listCODE
MEDIUMtests/test_infer/test_models/test_custom_model.py84 result_list = manager.list([-1] * world_size) # Create a shared listCODE
MEDIUMtests/test_infer/test_kernels/triton/kernel_utils.py7# This function is adapted from src/transformers/models/llama/modeling_llama.pyCOMMENT
MEDIUMcolossalai/fx/codegen/activation_checkpoint_codegen.py168 """This function is to find the offload regionsSTRING
MEDIUMcolossalai/booster/plugin/hybrid_parallel_plugin.py1403 # Create a context for gradient synchronization based on the optimizer type.COMMENT
MEDIUMcolossalai/shardformer/modeling/gpt2.py137 # This function is modified on the basis of transformers.models.gpt2.modeling_gpt2.GPT2Model.forward.COMMENT
MEDIUMcolossalai/shardformer/modeling/gpt2.py555 # This function is modified on the basis of transformers.models.gpt2.modeling_gpt2.GPT2ForQuestionAnswering.forwCOMMENT
MEDIUMcolossalai/shardformer/modeling/gpt2.py643 # This function is modified on the basis of transformers.models.gpt2.modeling_gpt2.GPT2ForTokenClassification.foCOMMENT
MEDIUMcolossalai/shardformer/modeling/gpt2.py718 # This function is modified on the basis of transformers.models.gpt2.modeling_gpt2.GPT2ForSequenceClassificationCOMMENT
MEDIUMcolossalai/shardformer/modeling/t5.py54 # This function is modified on the basis of transformers.models.t5.modeling_t5.T5Stack.forward.COMMENT
MEDIUMcolossalai/shardformer/modeling/t5.py290 # This function is modified on the basis of transformers.models.t5.modeling_t5.T5Model.forward.COMMENT
MEDIUMcolossalai/shardformer/modeling/t5.py430 # This function is modified on the basis of transformers.models.t5.modeling_t5.T5ForConditionalGeneration.forwarSTRING
MEDIUMcolossalai/shardformer/modeling/gptj.py99 # This function is modified on the v4.51.3 transformers.models.gptj.modeling_gptj.GPTJModel.forward.COMMENT
MEDIUMcolossalai/shardformer/modeling/gptj.py288 # This function is modified on the basis of transformers.models.gptj.modeling_gptj.GPTJForCausalLM.forward.COMMENT
MEDIUMcolossalai/shardformer/modeling/gptj.py381 # This function is modified on the v4.51.3 transformers.models.gptj.modeling_gptj.GPTJForSequenceClassification.COMMENT
MEDIUMcolossalai/shardformer/modeling/gptj.py503 # This function is modified on the basis of transformers.models.gptj.modeling_gptj.GPTJForQuestionAnswering.forwCOMMENT
MEDIUMcolossalai/shardformer/modeling/gptj.py583 # This function is modified on the v4.51.3 transformers.models.gptj.modeling_gptj.GPTJAttention.forward.STRING
MEDIUMcolossalai/legacy/pipeline/middleware/topo.py127# This class is a middleware between partition splitterCOMMENT
MEDIUMcolossalai/legacy/context/parallel_context.py25 """This class provides interface functions for users to get the parallel context,STRING
MEDIUMcolossalai/legacy/context/random/seed_manager.py11 """This class is a manager of all random seeds involved in the system.STRING
MEDIUMcolossalai/legacy/nn/loss/loss_1d.py30 # Create a mask of valid vocab ids (1 means it needs to be masked).COMMENT
MEDIUM…lossalai/legacy/zero/sharded_optim/sharded_optim_v2.py189 # This function is called except the last stage of pipeline parallelCOMMENT
MEDIUMcolossalai/legacy/amp/naive_amp/_fp16_optimizer.py128 # Create a fp32 copyCOMMENT
MEDIUMcolossalai/utils/tensor_detector/tensor_detector.py17 """This class is a detector to detect tensor on different devices.STRING
MEDIUMcolossalai/quantization/bnb.py219 # Create a copy of the modelSTRING
MEDIUMcolossalai/zero/gemini/gemini_ddp.py261 """This function is only triggered for inference."""STRING
MEDIUMcolossalai/zero/gemini/gemini_ddp.py296 """This function is only triggered for inference."""STRING
MEDIUMcolossalai/zero/gemini/gemini_optimizer.py306 # This function is called except the last stage of pipeline parallelCOMMENT
MEDIUM…ssalai/zero/gemini/memory_tracer/memstats_collector.py76 # The following code work for ZeroInitContext, which is deprecated in v0.1.12COMMENT
MEDIUMcolossalai/lazy/lazy_init.py80 """This class is only for correctness verification."""STRING
MEDIUMcolossalai/inference/batch_bucket.py483 ########## The following methods are expected to be used in modeling ###########COMMENT
MEDIUMcolossalai/tensor/comm_spec.py423 # Create a new member `logical_process_axes` to distinguish from original flattenCOMMENT
MEDIUM…/tutorial/sequence_parallel/loss_func/cross_entropy.py15 # Create a mask of valid vocab ids (1 means it needs to be masked).COMMENT
MEDIUM…ce_parallel/data/datasets/test/test_indexed_dataset.py1# This file isn't really a formal automated test, it's just a place toCOMMENT
MEDIUMexamples/images/diffusion/main.py56 # Create an ArgumentParser object with specifies kwargsCOMMENT
MEDIUMexamples/images/diffusion/main.py404 # Create a list of exponentially increasing log steps, starting from 1 and ending at batch_frequencyCOMMENT
MEDIUMexamples/images/diffusion/main.py661 # Create the checkpoint and configuration directories within the log directory.COMMENT
MEDIUMexamples/images/diffusion/main.py755 # Create an empty OmegaConf configuration objectCOMMENT
MEDIUMexamples/images/diffusion/main.py796 # Create a Trainer object with the specified command-line arguments and keyword arguments, and set the log direcCOMMENT
MEDIUMexamples/images/diffusion/main.py800 # Create a data module based on the configuration fileCOMMENT
MEDIUMexamples/images/diffusion/ldm/data/lsun.py10# This class is used to create a dataset of images from LSUN dataset for trainingCOMMENT
MEDIUMexamples/images/dreambooth/train_dreambooth.py689 # Create the pipeline using using the trained modules and save it.COMMENT
MEDIUMexamples/images/dreambooth/train_dreambooth_inpaint.py726 # Create the pipeline using using the trained modules and save it.COMMENT
MEDIUMexamples/language/gpt/titans/model/embed.py262 # Create a mask of valid vocab ids (1 means it needs to be masked).COMMENT
MEDIUMexamples/language/gpt/gemini/commons/model_zoo.py5## Define the Model and Loss Based on Huggingface transformers GPT2LMHeadModelCOMMENT
MEDIUM…anguage/gpt/experiments/pipeline_parallel/model_zoo.py5## Define the Model and Loss Based on Huggingface transformers GPT2LMHeadModelCOMMENT
MEDIUM…ples/community/roberta/pretraining/model/deberta_v2.py102 >>> # Create a maskSTRING
MEDIUM…ons/ColossalQA/colossalqa/retrieval_conversation_en.py31 # Define the retrieverCOMMENT
MEDIUM…ons/ColossalQA/colossalqa/retrieval_conversation_en.py34 # Define the chain to preprocess the inputCOMMENT
15 more matches not shown…
Docstring Block Structure25 hits · 125 pts
SeverityFileLineSnippetContext
HIGHcolossalai/pipeline/schedule/_utils.py72Get the batch size (size of dimension-0) of the first tensor in the batch. Args: batch (Any): Batch to be iSTRING
HIGHcolossalai/context/config.py52Reads a python file and constructs a corresponding :class:`Config` object. Args: filename (str): NaSTRING
HIGHcolossalai/shardformer/layer/normalization.py140 Convert a native PyTorch layer normalization module to a specific layer normalization module, and optioSTRING
HIGHcolossalai/legacy/context/parallel_context.py130Returns the local rank of the current device. Args: parallel_mode (:class:`colossalai.legacy.contexSTRING
HIGHcolossalai/legacy/context/parallel_context.py160Returns the global rank of the next device. Args: parallel_mode (:class:`colossalai.legacy.context.STRING
HIGHcolossalai/legacy/context/parallel_context.py182Returns the global rank of the previous device. Args: parallel_mode (:class:`colossalai.legacy.contSTRING
HIGHcolossalai/legacy/context/parallel_context.py204Returns a boolean value indicating whether the current device is the first one among its group for `parallel_modSTRING
HIGHcolossalai/legacy/context/parallel_context.py222Returns a boolean value indicating whether the current device is the last one among its group for `parallel_modeSTRING
HIGHcolossalai/legacy/context/parallel_context.py256Returns the world size for `parallel_mode`. Args: parallel_mode (:class:`colossalai.legacy.context.STRING
HIGHcolossalai/legacy/context/parallel_context.py286Returns the group of the current device for `parallel_mode`. Args: parallel_mode (:class:`colossalaSTRING
HIGHcolossalai/legacy/context/parallel_context.py342Returns the rank of the current device for `parallel_mode` in the group. Args: parallel_mode (:clasSTRING
HIGHcolossalai/legacy/utils/checkpointing.py221Loads training states from a checkpoint file. Args: file: a file-like object (has to implement read(), readSTRING
HIGH…lossalai/legacy/zero/gemini/tensor_placement_policy.py64 Evict tensors from CUDA device. Args: hold_cuda_tensor_list (List[StatefulTensor]): the liSTRING
HIGHcolossalai/legacy/registry/registry.py28Registers a module represented in `module_class`. Args: module_class (class): The module to be regiSTRING
HIGHcolossalai/legacy/registry/registry.py45Retrieves a module with name `module_name` and returns the module if it has already been registered before. STRING
HIGHcolossalai/legacy/builder/builder.py10Returns an object of :class:`module` constructed from `config`. Args: module: A python or user-defined clasSTRING
HIGHcolossalai/legacy/builder/builder.py27Returns an object constructed from `config`, the type of the object is specified by `registry`. Note: tSTRING
HIGHcolossalai/cli/launcher/run.py59Parse an inclusion or exclusion string and filter a hostfile dictionary. Examples: include_str="worker-0,woSTRING
HIGHcolossalai/_analyzer/fx/tracer/symbolic_trace.py37 Traces a ``torch.nn.Module`` or a function and returns a ``GraphModule`` with ``Node``s and ``MetaInfo`` attachSTRING
HIGHcolossalai/_analyzer/fx/passes/graph_profile.py233 Run a specific node ``n`` and profile its execution time and memory usage. Calls into call_function, caSTRING
HIGH…lossalai/auto_parallel/checkpoint/ckpt_solver_rotor.py311Backtrack the cost table and retrieve the optimal checkpointing strategy. Args: chain (Chain): A baSTRING
HIGHcolossalai/zero/gemini/placement_policy.py158 Evict tensors from CUDA device. Args: can_evict_chunks (List[StatefulTensor]): the list ofSTRING
HIGHcolossalai/tensor/d_tensor/layout_converter.py453 Check if the process groups required for sharding have been deleted by torch.distributed.destroy_procesSTRING
HIGH…/ColossalChat/coati/distributed/reward/reward_utils.py80 Modified from: https://gist.github.com/lewtun/9c2ce1937b741404090a3dc4c7c022b3 Retrieves the content from the lSTRING
HIGHapplications/ColossalChat/coati/models/generation.py279 Generates new tokens using a streaming approach. Args: model (Any): The model used for token generatioSTRING
Excessive Try-Catch Wrapping111 hits · 123 pts
SeverityFileLineSnippetContext
LOW…est_checkpoint_io/test_low_level_zero_checkpoint_io.py149 except Exception as e:CODE
MEDIUM…est_checkpoint_io/test_low_level_zero_checkpoint_io.py95def run_fn(stage, shard, offload, model_fn, data_gen_fn, output_transform_fn, loCODE
LOWtests/test_shardformer/test_model/test_shard_gpt2.py226 except Exception as e:CODE
LOWtests/test_shardformer/test_model/test_shard_gpt2.py271 except Exception as e:CODE
LOWtests/test_shardformer/test_model/test_shard_command.py286 except Exception as e:CODE
LOWtests/test_shardformer/test_model/test_shard_llama.py81 except Exception as e:CODE
LOWtests/test_shardformer/test_model/test_shard_llama.py305 except Exception as e:CODE
LOWtests/test_shardformer/test_model/test_shard_llama.py359 except Exception as e:CODE
LOW…sts/test_shardformer/test_model/test_shard_chatglm2.py241 except Exception as e:CODE
LOWtests/test_shardformer/test_model/test_shard_qwen3.py214 except Exception as e:CODE
LOWtests/test_shardformer/test_model/test_shard_qwen3.py263 except Exception as e:CODE
LOWtests/test_shardformer/test_model/test_shard_qwen2.py218 except Exception as e:CODE
LOWtests/test_shardformer/test_model/test_shard_qwen2.py267 except Exception as e:CODE
LOWtests/test_booster/test_plugin/test_gemini_plugin.py62 except Exception as e:CODE
MEDIUMtests/test_booster/test_plugin/test_gemini_plugin.py19def run_fn(init_method, model_fn, data_gen_fn, output_transform_fn, zero_size, tCODE
LOWtests/test_booster/test_plugin/test_3d_plugin.py82 except Exception as e:CODE
MEDIUMtests/test_booster/test_plugin/test_3d_plugin.py48def run_fn(init_method, model_fn, data_gen_fn, output_transform_fn) -> Optional[CODE
LOW…test_booster/test_plugin/test_low_level_zero_plugin.py56 except Exception as e:CODE
LOWtests/test_fx/test_tracer/test_patched_op.py13 except Exception as e:CODE
MEDIUMtests/test_fx/test_tracer/test_patched_op.py9def _run(data, patch_fn):CODE
LOWtests/test_fx/test_tracer/test_patched_module.py16 except Exception as e:CODE
MEDIUMtests/test_fx/test_tracer/test_patched_module.py7def _run(data, module, patch_fn):CODE
LOW…racer/test_torchvision_model/test_torchvision_model.py41 except Exception as e:CODE
LOW…s/test_fx/test_tracer/test_hf_model/hf_tracer_utils.py23 except Exception as e:CODE
LOW…ts/test_fx/test_pipeline/test_timm_model/timm_utils.py29 except Exception as e:CODE
LOWtests/test_fx/test_pipeline/test_topo/topo_utils.py45 except Exception as e:CODE
LOWtests/test_fx/test_pipeline/test_hf_model/hf_utils.py32 except Exception as e:CODE
LOWtests/test_optimizer/test_dist_lamb.py148 except Exception as e:CODE
LOWtests/test_optimizer/test_dist_lamb.py230 except Exception as e:CODE
LOWtests/test_optimizer/test_dist_lamb.py250 except Exception as e:CODE
LOWtests/test_optimizer/test_dist_galore.py80 except Exception as e:CODE
LOWtests/test_optimizer/test_dist_galore.py171 except Exception as e:CODE
LOWtests/test_optimizer/test_dist_galore.py245 except Exception as e:CODE
LOWtests/test_optimizer/test_dist_galore.py264 except Exception as e:CODE
LOWtests/test_optimizer/test_dist_galore.py289 except Exception as e:CODE
LOW…est_autochunk_diffuser/benchmark_autochunk_diffuser.py134 except Exception as e:CODE
LOW…t_autochunk_alphafold/benchmark_autochunk_alphafold.py123 except Exception as e:CODE
LOW…tochunk_transformer/benchmark_autochunk_transformer.py135 except Exception as e:CODE
LOWcolossalai/fx/tracer/experimental.py589 except Exception as e:CODE
LOWcolossalai/fx/tracer/tracer.py312 except Exception as e:CODE
LOWcolossalai/shardformer/layer/parallel_module.py152 except Exception as ex:STRING
LOWcolossalai/shardformer/layer/parallel_module.py330 except Exception as ex:STRING
LOWcolossalai/shardformer/layer/normalization.py22except Exception:CODE
LOWcolossalai/shardformer/shard/sharder.py200 except Exception as e:STRING
LOW…lossalai/legacy/zero/sharded_model/sharded_model_v2.py208 except Exception as e:STRING
LOW…lossalai/legacy/zero/sharded_model/sharded_model_v2.py556 except Exception as ex:STRING
LOWcolossalai/legacy/inference/async_manager.py148 except Exception:CODE
MEDIUMcolossalai/legacy/inference/async_manager.py135def start_dynamic_batching(args, tp_engine, waiting_req_list):CODE
LOWcolossalai/legacy/inference/manager.py293 except Exception:CODE
MEDIUMcolossalai/legacy/inference/manager.py280def start_dynamic_batching(args, tp_engine, waiting_req_list):CODE
LOW…egacy/inference/quant/smoothquant/models/base_model.py228 except Exception as e:CODE
LOWcolossalai/legacy/builder/builder.py55 except Exception as e:STRING
MEDIUMcolossalai/legacy/builder/builder.py56 print(f"An error occurred when building {mod_type} from registry {registry.name}", flush=True)STRING
LOWcolossalai/utils/safetensors.py12except Exception:CODE
LOWcolossalai/cli/launcher/multinode_runner.py55 except Exception as e:CODE
LOWcolossalai/_analyzer/fx/graph_module.py71 except Exception as e:CODE
MEDIUMcolossalai/_analyzer/fx/graph_module.py65def __call__(self, obj, *args, **kwargs):CODE
LOWcolossalai/_analyzer/fx/passes/graph_profile.py257 except Exception as e:CODE
LOWcolossalai/zero/gemini/gemini_ddp.py727 except Exception as ex:STRING
LOWcolossalai/lazy/pretrained.py259 except Exception:CODE
51 more matches not shown…
Fake / Example Data95 hits · 112 pts
SeverityFileLineSnippetContext
LOWtests/test_fx/test_graph_manipulation.py42 if node.op in ("placeholder", "output"):CODE
LOWtests/test_fx/test_meta_info_prop.py33 if node.op == "placeholder":CODE
LOWtests/test_fx/test_pipeline/test_topo/topo_utils.py66 if node.op == "placeholder":CODE
LOWcolossalai/fx/proxy.py68 if self.node.op == "placeholder":CODE
LOWcolossalai/fx/tracer/experimental.py121 if self.node.op == "placeholder":CODE
LOWcolossalai/fx/tracer/experimental.py213 if kind == "placeholder":CODE
LOWcolossalai/fx/tracer/experimental.py338 if node.op == "placeholder":CODE
LOWcolossalai/fx/tracer/experimental.py499 if kind == "placeholder":CODE
LOWcolossalai/fx/tracer/experimental.py526 if kind == "placeholder" and target in meta_args and meta_args[target].is_meta:CODE
LOWcolossalai/fx/tracer/tracer.py219 if kind == "placeholder" and target in self.meta_args and self.meta_args[target].is_meta:CODE
LOWcolossalai/fx/tracer/tracer.py418 if node.op == "placeholder":CODE
LOWcolossalai/fx/tracer/_meta_trace.py61 "placeholder", "placeholder", (graph._root,), name=namespace.create_name(name, tensor)CODE
LOWcolossalai/fx/passes/split_module.py163 if node.op in ["placeholder"]:CODE
LOWcolossalai/fx/passes/split_module.py246 if node.op == "placeholder":CODE
LOWcolossalai/fx/passes/utils.py46 if node.op == "placeholder":CODE
LOWcolossalai/fx/passes/utils.py71 if node.op == "placeholder":CODE
LOWcolossalai/fx/passes/passes_for_gpt2_test.py56 if node.op == "placeholder":CODE
LOWcolossalai/fx/passes/passes_for_gpt2_test.py116 if node.op == "placeholder":CODE
LOWcolossalai/fx/passes/passes_for_gpt2_test.py202 if n.op != "placeholder" and n._fx_partition > partition_name:CODE
LOWcolossalai/fx/passes/passes_for_gpt2_test.py236 if node.op in ["placeholder"]:CODE
LOWcolossalai/fx/passes/passes_for_gpt2_test.py322 if node.op == "placeholder":CODE
LOWcolossalai/fx/passes/adding_split_node_pass.py265 if node.op == "placeholder":CODE
LOW…x/passes/experimental/adding_shape_consistency_pass.py51 if node.op != "placeholder":CODE
LOW…x/passes/experimental/adding_shape_consistency_pass.py53 input_specs_node = mod_graph.create_node("placeholder", target="sharding_spec_convert_dict")CODE
LOW…x/passes/experimental/adding_shape_consistency_pass.py54 origin_specs_node = mod_graph.create_node("placeholder", target="origin_node_sharding_spec_dict")CODE
LOWcolossalai/fx/codegen/activation_checkpoint_codegen.py950 if user.op == "placeholder":CODE
LOWcolossalai/fx/codegen/activation_checkpoint_codegen.py965 if node.op == "placeholder":CODE
LOWcolossalai/fx/codegen/activation_checkpoint_codegen.py598 if input_node.op != "placeholder":STRING
LOWcolossalai/fx/codegen/activation_checkpoint_codegen.py733 if user.op == "placeholder":STRING
LOWcolossalai/fx/codegen/activation_checkpoint_codegen.py748 if node.op == "placeholder":STRING
LOWcolossalai/fx/profiler/profiler.py222 "placeholder",CODE
LOWcolossalai/fx/profiler/profiler.py223 "placeholder",CODE
LOWcolossalai/legacy/pipeline/middleware/adaptor/fx.py102 if node.op == "placeholder":CODE
LOWcolossalai/autochunk/estimate_memory.py35 if n.op == "placeholder":CODE
LOWcolossalai/autochunk/estimate_memory.py51 elif node.op == "placeholder":CODE
LOWcolossalai/autochunk/estimate_memory.py115 if n.op in ["placeholder", "get_attr", "output"]:CODE
LOWcolossalai/autochunk/estimate_memory.py198 if node.op == "placeholder":CODE
LOWcolossalai/autochunk/utils.py7NON_COMPUTE_OP = ["placeholder", "get_attr", "output"]CODE
LOWcolossalai/autochunk/utils.py131 if "placeholder" in node.op:CODE
LOWcolossalai/autochunk/utils.py145 if n.op == "placeholder":CODE
LOWcolossalai/autochunk/trace_indice.py841 if node.op == "placeholder":CODE
LOWcolossalai/autochunk/autochunk_codegen.py484 if user.op == "placeholder":CODE
LOWcolossalai/autochunk/autochunk_codegen.py500 if node.op == "placeholder":CODE
LOWcolossalai/_analyzer/fx/codegen.py351 if user.op == "placeholder":CODE
LOWcolossalai/_analyzer/fx/codegen.py366 if node.op == "placeholder":CODE
LOWcolossalai/_analyzer/fx/tracer/proxy.py78 if self.node.op == "placeholder":CODE
LOWcolossalai/_analyzer/fx/tracer/tracer.py116 if kind == "placeholder":CODE
LOWcolossalai/_analyzer/fx/tracer/tracer.py207 if node.op == "placeholder":CODE
LOWcolossalai/_analyzer/fx/tracer/tracer.py310 if node.op == "placeholder":CODE
LOWcolossalai/auto_parallel/checkpoint/ckpt_solver_base.py162 next(node for node in self.graph.nodes if node.name == name).op == "placeholder"CODE
LOWcolossalai/auto_parallel/checkpoint/ckpt_solver_base.py175 if n.op != "placeholder" and n.op != "output":CODE
LOWcolossalai/auto_parallel/checkpoint/ckpt_solver_base.py177 if n_par.op != "placeholder" and n_par.name not in self.cnode:CODE
LOW…lossalai/auto_parallel/checkpoint/ckpt_solver_rotor.py172 if node.op == "placeholder":CODE
LOWcolossalai/auto_parallel/passes/meta_info_prop.py27 "placeholder": self.placeholder_handler,CODE
LOW…salai/auto_parallel/passes/runtime_preparation_pass.py122 if node.op != "placeholder":CODE
LOW…salai/auto_parallel/passes/runtime_preparation_pass.py124 input_specs_node = mod_graph.create_node("placeholder", target="sharding_spec_convert_dict")CODE
LOW…salai/auto_parallel/passes/runtime_preparation_pass.py125 origin_specs_node = mod_graph.create_node("placeholder", target="origin_node_sharding_spec_dict")CODE
LOW…salai/auto_parallel/passes/runtime_preparation_pass.py126 comm_actions_dict_node = mod_graph.create_node("placeholder", target="comm_actions_dict")CODE
LOW…parallel/tensor_shard/solver/strategies_constructor.py78 if node.op in ("placeholder", "get_attr", "output"):CODE
LOW…parallel/tensor_shard/solver/strategies_constructor.py99 elif node.op == "placeholder":CODE
35 more matches not shown…
Over-Commented Block86 hits · 86 pts
SeverityFileLineSnippetContext
LOW…legacy/test_layers/test_2d/checks_2d/check_layer_2d.py641 out_grad = torch.chunk(out_grad, DEPTH, dim=0)[i]COMMENT
LOW…legacy/test_layers/test_2d/checks_2d/check_layer_2d.py661# )COMMENT
LOW…legacy/test_layers/test_2d/checks_2d/check_layer_2d.py681# out.backward(grad)COMMENT
LOW…legacy/test_layers/test_2d/checks_2d/check_layer_2d.py701# torch.distributed.broadcast(A_master, src=0)COMMENT
LOW…legacy/test_layers/test_2d/checks_2d/check_layer_2d.py721# INPUT_SIZE = HIDDEN_SIZECOMMENT
LOW…legacy/test_layers/test_2d/checks_2d/check_layer_2d.py741# mask_shape = (BATCH_SIZE // DEPTH, NUM_ATTENTION_HEADS // DEPTH, SEQ_LENGTH, SEQ_LENGTH)COMMENT
LOW…/test_layers/test_2p5d/checks_2p5d/check_layer_2p5d.py661COMMENT
LOW…/test_layers/test_2p5d/checks_2p5d/check_layer_2p5d.py681# assert out.shape == (BATCH_SIZE // TESSERACT_DIM, SEQ_LENGTH, INPUT_SIZE // TESSERACT_DIM)COMMENT
LOW…/test_layers/test_2p5d/checks_2p5d/check_layer_2p5d.py701# HIDDEN_SIZE,COMMENT
LOW…/test_layers/test_2p5d/checks_2p5d/check_layer_2p5d.py721# grad = torch.randn(grad_shape, dtype=dtype, device=device)COMMENT
LOW…/test_layers/test_2p5d/checks_2p5d/check_layer_2p5d.py741# attention_dropout_prob=0.5,COMMENT
LOW…/test_layers/test_2p5d/checks_2p5d/check_layer_2p5d.py761# grad_shape = out.shapeCOMMENT
LOW…rallel/test_tensor_shard/test_solver_with_resnet_v2.py21 mesh_shape = (2, 4)COMMENT
LOW…rallel/test_tensor_shard/test_solver_with_resnet_v2.py41 # %layer1_0_conv2 : [#users=1] = call_module[target=layer1.0.conv2](args = (%layer1_0_relu,), kwargs = {})COMMENT
LOWtests/kit/model_zoo/transformers/mistral.py1import torchCOMMENT
LOWtests/kit/model_zoo/transformers/whisper.py1import torchCOMMENT
LOWtests/kit/model_zoo/transformers/deepseek.py1# modified from tests/kit/model_zoo/transformers/mistral.pyCOMMENT
LOWtests/kit/model_zoo/transformers/bert.py1import torchCOMMENT
LOWtests/kit/model_zoo/transformers/deepseek_v3.py1# modified from tests/kit/model_zoo/transformers/mistral.pyCOMMENT
LOWtests/kit/model_zoo/transformers/sam.py1import torchCOMMENT
LOWtests/kit/model_zoo/transformers/mixtral.py1# modified from tests/kit/model_zoo/transformers/mistral.pyCOMMENT
LOWtests/kit/model_zoo/transformers/mixtral.py81# name="transformers_mixtral_for_sequence_classification",COMMENT
LOWtests/kit/model_zoo/transformers/blip2.py1import torchCOMMENT
LOWtests/test_tensor/test_comm_spec_apply.py81 if rank in (0, 1):COMMENT
LOWtests/test_tensor/test_dtensor/test_comm_spec.py61 sharded_tensor_1 = torch.ones(2, 1) * 3COMMENT
LOWtests/test_fx/test_tracer/test_bias_addition_module.py41 # %linear : [#users=1] = call_function[target=torch._C._nn.linear](args = (%x, %linear_weight), kwargs = {})COMMENT
LOW…/test_fx/test_profiler/test_profiler_meta_info_prop.py61 torch.cuda.reset_peak_memory_stats()COMMENT
LOW…/test_fx/test_profiler/test_profiler_meta_info_prop.py101 # If we need to dive deep into the memory usage byCOMMENT
LOW…e/en/features/mixed_precision_training_with_booster.md221```pythonCOMMENT
LOWextensions/csrc/common/micros.h101// https://github.com/pytorch/pytorch/commit/3aeb78079bcd68282fe9117088e138b77318e288COMMENT
LOWextensions/csrc/common/micros.h221 } else { \COMMENT
LOWextensions/csrc/funcs/cast_functor.h1#pragma onceCOMMENT
LOWextensions/csrc/funcs/binary_functor.h1#pragma onceCOMMENT
LOWextensions/csrc/funcs/ternary_functor.h1#pragma onceCOMMENT
LOWextensions/csrc/funcs/unary_functor.h1#pragma onceCOMMENT
LOWextensions/csrc/kernel/cuda/attention/attention_utils.h21COMMENT
LOWextensions/csrc/kernel/x86/cpu_adam.h21*/COMMENT
LOWextensions/csrc/kernel/x86/cpu_adam.h41#define SIMD_WIDTH 16COMMENT
LOWextensions/csrc/kernel/x86/cpu_adam.h61#define SIMD_LOAD(x) _mm256_loadu_ps(x)COMMENT
LOWcolossalai/pipeline/schedule/v_schedule.py1# Refer from Zero Bubble Pipeline Parallelism.COMMENT
LOWcolossalai/nn/optimizer/adafactor.py1# coding=utf-8COMMENT
LOWcolossalai/shardformer/modeling/bert.py1081 # if uni-directional self-attention (decoder) save Tuple(torch.Tensor, torch.Tensor) ofCOMMENT
LOWcolossalai/legacy/pipeline/middleware/topo.py121 return resCOMMENT
LOWcolossalai/legacy/moe/openmoe/train.sh21# epCOMMENT
LOWcolossalai/legacy/moe/openmoe/test_ci.sh1# pip install -r requirements.txtCOMMENT
LOWcolossalai/legacy/moe/openmoe/test_ci.sh21# torchrun --standalone --nproc_per_node 4 train.py \COMMENT
LOW…salai/legacy/moe/openmoe/model/convert_openmoe_ckpt.py1# coding=utf-8COMMENT
LOWcolossalai/legacy/moe/openmoe/model/modeling_openmoe.py1# coding=utf-8COMMENT
LOWcolossalai/_analyzer/fx/graph_module.py21from torch.fx.graph_module import _exec_with_source, _forward_from_srcCOMMENT
LOWcolossalai/auto_parallel/checkpoint/ckpt_solver_base.py101 """COMMENT
LOW…salai/auto_parallel/passes/runtime_preparation_pass.py241 getitem_index = node.args[1]COMMENT
LOWcolossalai/device/device_mesh.py421 # key is axis nameCOMMENT
LOWcolossalai/tensor/colo_parameter.py81COMMENT
LOW…utorial/sequence_parallel/lr_scheduler/annealing_lr.py1# coding=utf-8COMMENT
LOWexamples/tutorial/sequence_parallel/data/__init__.py21 (train_dataloader, valid_dataloader, test_dataloader) = (None, None, None)COMMENT
LOW…/sequence_parallel/data/tokenizer/bert_tokenization.py1# coding=utf-8COMMENT
LOW…/tutorial/sequence_parallel/data/tokenizer/__init__.py1# coding=utf-8COMMENT
LOW…tutorial/sequence_parallel/data/tokenizer/tokenizer.py1# coding=utf-8COMMENT
LOW…l/sequence_parallel/data/datasets/blendable_dataset.py1# coding=utf-8COMMENT
LOW…torial/sequence_parallel/data/datasets/bert_dataset.py1# coding=utf-8COMMENT
26 more matches not shown…
Structural Annotation Overuse27 hits · 49 pts
SeverityFileLineSnippetContext
LOWcolossalai/inference/README.md46# Step 1: create a model in "transformers" wayCOMMENT
LOWcolossalai/inference/README.md51# Step 2: create an inference_configCOMMENT
LOWcolossalai/inference/README.md60# Step 3: create an engine with model and configCOMMENT
LOWcolossalai/inference/README.md63# Step 4: try inferenceCOMMENT
LOWexamples/images/diffusion/README.md41#### Step 1: RequirementsCOMMENT
LOWexamples/images/diffusion/README.md57#### Step 2: Install [Colossal-AI](https://colossalai.org/download/) From Our Official WebsiteCOMMENT
LOWexamples/images/diffusion/README.md83#### Step 3: Accelerate with flash attention by xformers (Optional)COMMENT
LOWapplications/ColossalQA/README.md33#### Step 1: Collect DataCOMMENT
LOWapplications/ColossalQA/README.md37#### Step 2: Split DataCOMMENT
LOWapplications/ColossalQA/README.md41#### Step 3: Construct Vector StoresCOMMENT
LOWapplications/ColossalQA/README.md47#### Step 4: Retrieve Relative TextCOMMENT
LOWapplications/ColossalQA/README.md50#### Step 5: Format PromptCOMMENT
LOWapplications/ColossalQA/README.md53#### Step 6: InferenceCOMMENT
LOWapplications/ColossalQA/README.md56#### Step 7: Update MemoryCOMMENT
LOWapplications/ColossalChat/examples/README.md463#### Step 1: Data CollectionCOMMENT
LOWapplications/ColossalChat/examples/README.md485#### Step 2: PreprocessingCOMMENT
LOWapplications/ColossalChat/examples/README.md537#### Step 3: TrainingCOMMENT
LOWapplications/ColossalChat/examples/README.md547#### Step 1: Data CollectionCOMMENT
LOWapplications/ColossalChat/examples/README.md579#### Step 2: PreprocessingCOMMENT
LOWapplications/ColossalChat/examples/README.md583#### Step 3: TrainingCOMMENT
LOWapplications/ColossalChat/examples/README.md623#### Step 1: Data CollectionCOMMENT
LOWapplications/ColossalChat/examples/README.md654#### Step 2: PreprocessingCOMMENT
LOWapplications/ColossalChat/examples/README.md661#### Step 3: TrainingCOMMENT
LOWapplications/ColossalChat/examples/README.md742### Step 1: Data Collection & PreparationCOMMENT
LOWapplications/ColossalChat/examples/README.md745### Step 2: TrainingCOMMENT
LOWapplications/ColossalChat/examples/README.md800#### Step 1: Data Collection & PreparationCOMMENT
LOWapplications/ColossalChat/examples/README.md804#### Step 2: TrainingCOMMENT
Modern AI Meta-Vocabulary12 hits · 37 pts
SeverityFileLineSnippetContext
MEDIUMapplications/ColossalQA/README.md66Our platform accommodates two kinds of LLMs: API-accessible and locally run models. For the API-style LLMs, we support CCODE
MEDIUM…tions/ColossalQA/examples/retrieval_conversation_en.py42 # Setup embedding model locallyCOMMENT
MEDIUM…tions/ColossalQA/examples/retrieval_conversation_zh.py41 # Setup embedding model locallyCOMMENT
MEDIUM…examples/retrieval_conversation_en_customer_service.py44 # Setup embedding model locallyCOMMENT
MEDIUM…retrieval_intent_classification_zh_customer_service.py36 # Setup embedding model locallyCOMMENT
MEDIUMapplications/ColossalQA/examples/webui_demo/config.py6 "embed_name": "m3e", # embedding model nameCODE
MEDIUMapplications/ColossalQA/examples/webui_demo/config.py7 "embed_model_name_or_path": "moka-ai/m3e-base", # path to embedding model, could be a local path or a huggingfaCODE
MEDIUMapplications/ColossalQA/examples/webui_demo/README.md24## Configure the RAG ChainCOMMENT
MEDIUM…ications/ColossalEval/colossal_eval/dataset/agieval.py102# process few-shot raw_promptsCOMMENT
MEDIUM…ications/ColossalEval/colossal_eval/dataset/agieval.py214 # process demo once if it is few-shot-CoTCOMMENT
MEDIUM…tions/ColossalEval/colossal_eval/utils/conversation.py138 # First few shot data is something like "The following are questions about xxx".COMMENT
MEDIUM…tions/ColossalEval/colossal_eval/utils/conversation.py184 # For few-shot, only need input. Otherwise use instruction (in AGIEval).COMMENT
Redundant / Tautological Comments22 hits · 30 pts
SeverityFileLineSnippetContext
LOWtests/test_shardformer/test_model/_utils.py324 # Check if the output sequence is gathered before cross entropyCOMMENT
LOWcolossalai/booster/plugin/hybrid_parallel_plugin.py173 # Check if the DP group size is 1, meaning no synchronization is needed.STRING
LOWcolossalai/booster/plugin/hybrid_parallel_plugin.py854 # Check if the list of gradients is emptySTRING
LOWcolossalai/booster/plugin/hybrid_parallel_plugin.py1427 # Check if the optimizer is a HybridParallelZeroOptimizer and synchronize data parallelism gradients if so.COMMENT
LOWcolossalai/quantization/bnb.py160 # Check if the current key is not in the `modules_to_not_convert`COMMENT
LOWcolossalai/quantization/bnb.py232 # Check if it is a base modelSTRING
LOWcolossalai/tensor/d_tensor/layout_converter.py472 # Check if each process group is still aliveSTRING
LOWexamples/images/diffusion/main.py257 # Check if the train dataset is iterableCOMMENT
LOWexamples/images/diffusion/main.py274 # Check if the validation dataset is iterableCOMMENT
LOWexamples/images/diffusion/main.py289 # Check if the test dataset is iterableCOMMENT
LOWexamples/images/diffusion/main.py620 # Check if the "resume" option is specified, resume training from the checkpoint if it is trueCOMMENT
LOWexamples/images/diffusion/main.py854 # Assign melk to SIGUSR1 signal and divein to SIGUSR2 signalCOMMENT
LOW…flows/scripts/example_checks/detect_changed_example.py19 # Output the result using print. Then the shell can get the values.COMMENT
LOW…ons/ColossalChat/coati/distributed/reward/reward_fn.py140 # Check if the sequence is over lengthCOMMENT
LOW…ons/ColossalChat/coati/distributed/reward/reward_fn.py205 # Check if the sequence is over lengthCOMMENT
LOW…ons/ColossalChat/coati/distributed/reward/reward_fn.py286 # Check if the sequence is over lengthCOMMENT
LOW…tions/ColossalChat/coati/dataset/tokenization_utils.py112 # Check if all labels are ignored, this may happen when the tokenized length is too longCOMMENT
LOW…tions/ColossalChat/coati/dataset/tokenization_utils.py321 # Check if loss mask is all 0s (no loss), this may happen when the tokenized length is too longCOMMENT
LOWapplications/ColossalChat/tests/test_train.sh406 # Check if the plugin is either "gemini_auto" or "gemini" and set grad_accu to '1'COMMENT
LOWapplications/ColossalChat/tests/test_lora.py59 # Check if model has overfittedCOMMENT
LOW…ns/ColossalChat/examples/training_scripts/train_kto.py182 # Check if the user specified weights fit into the theoratical lower and upper bounds from Eq. (8) of https://arxiv.COMMENT
LOW…t/examples/data_preparation_scripts/prepare_dataset.py173 # Set stop_ids to eos_token_id for other dataset types if not existCOMMENT
AI Slop Vocabulary17 hits · 30 pts
SeverityFileLineSnippetContext
LOWcolossalai/fx/tracer/tracer.py97 # if no extra manipulation is applied, we just pass the origin arguments to create_proxy functionCOMMENT
MEDIUMcolossalai/legacy/context/parallel_mode.py17 # this is added to facilitate amp and grad clipping in hybrid parallelCOMMENT
LOW…lossalai/legacy/zero/sharded_optim/sharded_optim_v2.py339 # We just set p.data = p.colo_attr.saved_grad.payload hereCOMMENT
MEDIUM…egacy/zero/shard_utils/bucket_tensor_shard_strategy.py40 # Ensure we utilize maximum PCIE bandwidthCOMMENT
LOWcolossalai/autochunk/estimate_memory.py197 # if node is placeholder, just add the size of the nodeCOMMENT
LOWcolossalai/cli/launcher/hostinfo.py36 port = 22 # no port specified, lets just use the ssh portCODE
LOWcolossalai/zero/gemini/chunk/chunk.py332 # just use another way for the movementCOMMENT
LOWcolossalai/lazy/lazy_init.py340 # inplace op, just return origin lazy tensorCOMMENT
MEDIUMcolossalai/kernel/triton/context_attn_unpad.py602 # NOTE use `triton.next_power_of_2` here to utilize the cache mechanism of tritonCOMMENT
MEDIUMcolossalai/kernel/triton/flash_decoding.py409 # NOTE use `triton.next_power_of_2` here to utilize the cache mechanism of tritonCOMMENT
LOW…utorial/sequence_parallel/lr_scheduler/annealing_lr.py67 # If the learning rate is constant, just return the initial value.COMMENT
LOWexamples/language/gpt/gemini/train_gpt_demo.py210 # we just use randomly generated data hereCOMMENT
LOW…uage/gpt/experiments/pipeline_parallel/train_gpt_pp.py143 # we just use randomly generated data hereCOMMENT
LOW…pt/experiments/auto_parallel/auto_parallel_with_gpt.py81 # we just use randomly generated data hereCOMMENT
LOW…les/community/roberta/pretraining/utils/global_vars.py113 # pollutes the runs list, so we just add each as a scalarCOMMENT
MEDIUM…/ColossalChat/coati/distributed/reward/reward_utils.py21 """Performs comprehensive validation of response structure.STRING
MEDIUM…cations/ColossalChat/coati/utils/reward_score/utils.py21 """Performs comprehensive validation of response structure.STRING
Slop Phrases6 hits · 14 pts
SeverityFileLineSnippetContext
MEDIUMcolossalai/shardformer/shard/shard_config.py116 # you can add all the optimization flag hereSTRING
MEDIUM…s/tutorial/opt/inference/script/processing_ckpt_66b.py7# you can use whether wget or git lfsCOMMENT
MEDIUM…ples/community/roberta/preprocessing/sentence_split.py143 # if you have multiple server, you can use code below or modify code to openmpiCOMMENT
MEDIUM…mples/community/roberta/preprocessing/tokenize_mask.py239 # if you have multiple server, you can use code below or modify code to openmpiCOMMENT
MEDIUMapplications/ColossalQA/data/tests/test.html1113<p>If you are using {}-formatting (<a class="reference internal" href="https://docs.python.org/3/library/stdtypes.html#sCODE
MEDIUMapplications/ColossalQA/data/tests/test.html1296counterparts in <a class="reference internal" href="https://docs.python.org/3/library/logging.html#logging.Logger" titleCODE
Cross-Language Confusion3 hits · 10 pts
SeverityFileLineSnippetContext
HIGHcolossalai/legacy/nn/layer/parallel_3d/_utils.py90 return _async_grad_bucket.push(op, grad, param_id)CODE
HIGHcolossalai/legacy/moe/layer/layers.py312 | C1 || C2 || C3 || C4 |STRING
HIGHcolossalai/legacy/moe/layer/layers.py313 | A1 || A2 | | R1 | A3 || R2 | A4 || R3 | | R4 |STRING
Hallucination Indicators1 hit · 10 pts
SeverityFileLineSnippetContext
CRITICAL…ns/ColossalChat/coati/distributed/inference_backend.py284 self.llm.llm_engine.model_executor.driver_worker.model_runner.model.load_weights(state_dict.items())CODE
TODO Padding4 hits · 8 pts
SeverityFileLineSnippetContext
LOW…ssalai/booster/mixed_precision/mixed_precision_base.py22 # TODO: implement this methodCOMMENT
LOWcolossalai/legacy/inference/hybridengine/engine.py73 # TODO: implement early_stopping, and various generation optionsCOMMENT
LOWcolossalai/auto_parallel/tensor_shard/initialize.py62 # TODO: implement this functionCOMMENT
LOWcolossalai/auto_parallel/tensor_shard/initialize.py70 # TODO: implement this functionCOMMENT
Verbosity Indicators3 hits · 3 pts
SeverityFileLineSnippetContext
LOWcolossalai/fx/codegen/activation_checkpoint_codegen.py586 # we need to check if the checkpoint need to offload the inputSTRING
LOWcolossalai/fx/codegen/activation_checkpoint_codegen.py593 # we need to check if the checkpoint need use_reentrant=FalseSTRING
LOW.github/workflows/scripts/check_doc_i18n.py6 # First, we need to check if the two directories existCOMMENT
AI Response Leakage1 hit · 2 pts
SeverityFileLineSnippetContext
HIGHcolossalai/legacy/utils/profiler/profiler.py101 # In this example with wait=1, warmup=1, active=2,STRING
Overly Generic Function Names3 hits · 2 pts
SeverityFileLineSnippetContext
LOWcolossalai/auto_parallel/tensor_shard/utils/misc.py20 def do_something():STRING
LOWcolossalai/testing/utils.py235 def do_something(rank, world_size, port):STRING
LOWcolossalai/testing/utils.py241 def do_something(rank, world_size, port, arg1, arg2):STRING