Pattern Findings

The scanner identified 261 distinct pattern matches across 12 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a detector match to inspect in context. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the configured finding weight (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — legacy comment-context matches receive a ×1.5 score weight. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier under the legacy scoring model.

Cross-File Repetition64 hits · 320 pts

Severity	File	Line	Snippet	Context
HIGH	labml_nn/activations/fta/experiment.py	0	## configurations this inherits from [`nlpautoregressionconfigs`](../../experiments/nlp_autoregression.html#nlpautoregre	STRING
HIGH	labml_nn/normalization/deep_norm/experiment.py	0	## configurations this inherits from [`nlpautoregressionconfigs`](../../experiments/nlp_autoregression.html#nlpautoregre	STRING
HIGH	labml_nn/rwkv/experiment.py	0	## configurations this inherits from [`nlpautoregressionconfigs`](../../experiments/nlp_autoregression.html#nlpautoregre	STRING
HIGH	labml_nn/transformers/gpt/__init__.py	0	## configurations this inherits from [`nlpautoregressionconfigs`](../../experiments/nlp_autoregression.html#nlpautoregre	STRING
HIGH	…bml_nn/transformers/basic/autoregressive_experiment.py	0	## configurations this inherits from [`nlpautoregressionconfigs`](../../experiments/nlp_autoregression.html#nlpautoregre	STRING
HIGH	labml_nn/transformers/aft/experiment.py	0	## configurations this inherits from [`nlpautoregressionconfigs`](../../experiments/nlp_autoregression.html#nlpautoregre	STRING
HIGH	labml_nn/hypernetworks/experiment.py	0	## configurations the default configs can and will be over-ridden when we start the experiment	STRING
HIGH	labml_nn/transformers/knn/train_model.py	0	## configurations the default configs can and will be over-ridden when we start the experiment	STRING
HIGH	labml_nn/transformers/fast_weights/experiment.py	0	## configurations the default configs can and will be over-ridden when we start the experiment	STRING
HIGH	labml_nn/transformers/feedback/experiment.py	0	## configurations the default configs can and will be over-ridden when we start the experiment	STRING
HIGH	labml_nn/transformers/glu_variants/experiment.py	0	## configurations the default configs can and will be over-ridden when we start the experiment	STRING
HIGH	labml_nn/transformers/xl/experiment.py	0	## configurations the default configs can and will be over-ridden when we start the experiment	STRING
HIGH	…_nn/normalization/weight_standardization/experiment.py	0	### vgg style model for cifar-10 classification this derives from the [generic vgg style architecture](../experiments/ci	STRING
HIGH	labml_nn/normalization/instance_norm/experiment.py	0	### vgg style model for cifar-10 classification this derives from the [generic vgg style architecture](../experiments/ci	STRING
HIGH	labml_nn/normalization/group_norm/experiment.py	0	### vgg style model for cifar-10 classification this derives from the [generic vgg style architecture](../experiments/ci	STRING
HIGH	labml_nn/normalization/batch_norm/cifar10.py	0	### vgg style model for cifar-10 classification this derives from the [generic vgg style architecture](../experiments/ci	STRING
HIGH	labml_nn/distillation/small.py	0	### vgg style model for cifar-10 classification this derives from the [generic vgg style architecture](../experiments/ci	STRING
HIGH	labml_nn/distillation/large.py	0	### vgg style model for cifar-10 classification this derives from the [generic vgg style architecture](../experiments/ci	STRING
HIGH	labml_nn/normalization/instance_norm/__init__.py	0	`x` is a tensor of shape `[batch_size, channels, ]`. `` denotes any number of (possibly 0) dimensions. for example, in	STRING
HIGH	labml_nn/normalization/batch_channel_norm/__init__.py	0	`x` is a tensor of shape `[batch_size, channels, ]`. `` denotes any number of (possibly 0) dimensions. for example, in	STRING
HIGH	labml_nn/normalization/group_norm/__init__.py	0	`x` is a tensor of shape `[batch_size, channels, ]`. `` denotes any number of (possibly 0) dimensions. for example, in	STRING
HIGH	labml_nn/normalization/batch_norm/__init__.py	0	`x` is a tensor of shape `[batch_size, channels, ]`. `` denotes any number of (possibly 0) dimensions. for example, in	STRING
HIGH	labml_nn/experiments/nlp_autoregression.py	0	### sampling function to generate samples periodically while training	STRING
HIGH	labml_nn/transformers/glu_variants/simple.py	0	### sampling function to generate samples periodically while training	STRING
HIGH	labml_nn/transformers/mlm/experiment.py	0	### sampling function to generate samples periodically while training	STRING
HIGH	labml_nn/transformers/compressive/experiment.py	0	### sampling function to generate samples periodically while training	STRING
HIGH	labml_nn/transformers/xl/experiment.py	0	### sampling function to generate samples periodically while training	STRING
HIGH	labml_nn/experiments/nlp_autoregression.py	0	### basic english tokenizer we use character level tokenizer in this experiment. you can switch by setting, ``` 'tokeniz	STRING
HIGH	labml_nn/experiments/nlp_classification.py	0	### basic english tokenizer we use character level tokenizer in this experiment. you can switch by setting, ``` 'tokeniz	STRING
HIGH	labml_nn/utils/tokenizer.py	0	### basic english tokenizer we use character level tokenizer in this experiment. you can switch by setting, ``` 'tokeniz	STRING
HIGH	labml_nn/resnet/experiment.py	0	## configurations we use [`cifar10configs`](../../experiments/cifar10.html) which defines all the dataset related config	STRING
HIGH	labml_nn/distillation/small.py	0	## configurations we use [`cifar10configs`](../../experiments/cifar10.html) which defines all the dataset related config	STRING
HIGH	labml_nn/distillation/large.py	0	## configurations we use [`cifar10configs`](../../experiments/cifar10.html) which defines all the dataset related config	STRING
HIGH	labml_nn/conv_mixer/experiment.py	0	## configurations we use [`cifar10configs`](../../experiments/cifar10.html) which defines all the dataset related config	STRING
HIGH	labml_nn/transformers/vit/experiment.py	0	## configurations we use [`cifar10configs`](../../experiments/cifar10.html) which defines all the dataset related config	STRING
HIGH	labml_nn/optimizers/adam_warmup.py	0	### initialize the optimizer * `params` is the list of parameters * `lr` is the learning rate $\alpha$ * `betas` is a tu	STRING
HIGH	labml_nn/optimizers/ada_belief.py	0	### initialize the optimizer * `params` is the list of parameters * `lr` is the learning rate $\alpha$ * `betas` is a tu	STRING
HIGH	labml_nn/optimizers/noam.py	0	### initialize the optimizer * `params` is the list of parameters * `lr` is the learning rate $\alpha$ * `betas` is a tu	STRING
HIGH	labml_nn/optimizers/radam.py	0	### initialize the optimizer * `params` is the list of parameters * `lr` is the learning rate $\alpha$ * `betas` is a tu	STRING
HIGH	labml_nn/optimizers/adam.py	0	### initialize the optimizer * `params` is the list of parameters * `lr` is the learning rate $\alpha$ * `betas` is a tu	STRING
HIGH	labml_nn/optimizers/amsgrad.py	0	### initialize the optimizer * `params` is the list of parameters * `lr` is the learning rate $\alpha$ * `betas` is a tu	STRING
HIGH	labml_nn/optimizers/adam_fp16.py	0	### take an update step for a given parameter tensor * `state` is the optimizer state of the parameter (tensor) * `group	STRING
HIGH	labml_nn/optimizers/ada_belief.py	0	### take an update step for a given parameter tensor * `state` is the optimizer state of the parameter (tensor) * `group	STRING
HIGH	labml_nn/optimizers/radam.py	0	### take an update step for a given parameter tensor * `state` is the optimizer state of the parameter (tensor) * `group	STRING
HIGH	labml_nn/optimizers/adam.py	0	### take an update step for a given parameter tensor * `state` is the optimizer state of the parameter (tensor) * `group	STRING
HIGH	labml_nn/optimizers/ada_belief.py	0	### initialize a parameter state * `state` is the optimizer state of the parameter (tensor) * `group` stores optimizer a	STRING
HIGH	labml_nn/optimizers/adam.py	0	### initialize a parameter state * `state` is the optimizer state of the parameter (tensor) * `group` stores optimizer a	STRING
HIGH	labml_nn/optimizers/sophia.py	0	### initialize a parameter state * `state` is the optimizer state of the parameter (tensor) * `group` stores optimizer a	STRING
HIGH	labml_nn/optimizers/amsgrad.py	0	### initialize a parameter state * `state` is the optimizer state of the parameter (tensor) * `group` stores optimizer a	STRING
HIGH	labml_nn/transformers/mha.py	0	`query`, `key` and `value` are the tensors that store collection of token embeddings for query, key and value. the	STRING
HIGH	labml_nn/transformers/alibi/__init__.py	0	`query`, `key` and `value` are the tensors that store collection of token embeddings for query, key and value. the	STRING
HIGH	labml_nn/transformers/rope/value_pe/__init__.py	0	`query`, `key` and `value` are the tensors that store collection of token embeddings for query, key and value. the	STRING
HIGH	labml_nn/transformers/aft/__init__.py	0	`query`, `key` and `value` are the tensors that store collection of token embeddings for query, key and value. the	STRING
HIGH	labml_nn/transformers/gpt/__init__.py	0	* `encoder` is the transformer [encoder](../models.html#encoder) * `src_embed` is the token [embedding module (with posi	STRING
HIGH	…bml_nn/transformers/basic/autoregressive_experiment.py	0	* `encoder` is the transformer [encoder](../models.html#encoder) * `src_embed` is the token [embedding module (with posi	STRING
HIGH	labml_nn/transformers/mlm/experiment.py	0	* `encoder` is the transformer [encoder](../models.html#encoder) * `src_embed` is the token [embedding module (with posi	STRING
HIGH	labml_nn/transformers/aft/experiment.py	0	* `encoder` is the transformer [encoder](../models.html#encoder) * `src_embed` is the token [embedding module (with posi	STRING
HIGH	labml_nn/transformers/fnet/experiment.py	0	* `encoder` is the transformer [encoder](../models.html#encoder) * `src_embed` is the token [embedding module (with posi	STRING
HIGH	labml_nn/transformers/primer_ez/efficient.py	0	## multi-per-head-depth-wise-conv-head attention we extend our original implementation of [multi-head attention](../mha.	STRING
HIGH	labml_nn/transformers/primer_ez/__init__.py	0	## multi-per-head-depth-wise-conv-head attention we extend our original implementation of [multi-head attention](../mha.	STRING
4 more matches not shown…

Self-Referential Comments61 hits · 178 pts

Severity	File	Line	Snippet	Context
MEDIUM	labml_nn/gan/stylegan/__init__.py	180	# Create the MLP	COMMENT
MEDIUM	labml_nn/gan/stylegan/experiment.py	432	# Create an experiment	COMMENT
MEDIUM	labml_nn/gan/cycle_gan/__init__.py	89	# This module is defined below.	COMMENT
MEDIUM	labml_nn/gan/cycle_gan/__init__.py	108	# Create a sequential module with the layers	COMMENT
MEDIUM⚡	labml_nn/gan/cycle_gan/__init__.py	390	# Create the models	COMMENT
MEDIUM⚡	labml_nn/gan/cycle_gan/__init__.py	396	# Create the optmizers	COMMENT
MEDIUM⚡	labml_nn/gan/cycle_gan/__init__.py	404	# Create the learning rate schedules.	COMMENT
MEDIUM	labml_nn/gan/cycle_gan/__init__.py	669	# Create an experiment	COMMENT
MEDIUM	labml_nn/activations/fta/experiment.py	157	# Create the transformer.	COMMENT
MEDIUM	labml_nn/neox/utils/llm_int8.py	55	# Create an empty Linear8bitLt module	COMMENT
MEDIUM	labml_nn/neox/utils/cache.py	43	# Create an empty queue if it's not present	COMMENT
MEDIUM	labml_nn/neox/utils/text_dataset.py	74	# Create a PyTorch tensor	COMMENT
MEDIUM	labml_nn/neox/samples/finetune.py	70	# Create the Pipe module	COMMENT
MEDIUM	labml_nn/neox/evaluation/__init__.py	142	# Create a tensor	COMMENT
MEDIUM	labml_nn/neox/evaluation/__init__.py	258	# Create the adapter	COMMENT
MEDIUM	labml_nn/experiments/arithmetic_dataset.py	169	# Create a dataset to generate problems	COMMENT
MEDIUM	labml_nn/experiments/arithmetic_dataset.py	176	# Create a tensor with only the initial token	COMMENT
MEDIUM	labml_nn/experiments/nlp_classification.py	270	# Create a counter	COMMENT
MEDIUM	labml_nn/experiments/cifar10.py	100	# Create a sequential model with the layers	COMMENT
MEDIUM	labml_nn/distillation/small.py	52	# Create a model with given convolution sizes (channels)	COMMENT
MEDIUM	labml_nn/distillation/large.py	54	# Create a model with given convolution sizes (channels)	COMMENT
MEDIUM	labml_nn/sampling/temperature.py	40	# Create a categorical distribution with temperature adjusted logits	COMMENT
MEDIUM	labml_nn/cfr/kuhn/__init__.py	202	# Create a new information set object	COMMENT
MEDIUM	labml_nn/cfr/kuhn/__init__.py	231	# Create an experiment, we only write tracking information to `sqlite` to speed things up.	COMMENT
MEDIUM	labml_nn/diffusion/stable_diffusion/util.py	140	# Create the destination folder	COMMENT
MEDIUM	labml_nn/diffusion/ddpm/evaluate.py	322	# Create an interpolation animation	COMMENT
MEDIUM	labml_nn/scaling/zero3/__init__.py	130	# Create an empty tensor to receive the sizes	COMMENT
MEDIUM	labml_nn/scaling/zero3/__init__.py	181	# Create an empty padding tensor	COMMENT
MEDIUM	labml_nn/scaling/zero3/__init__.py	201	#### Create an empty tensor of the given shape.	STRING
MEDIUM	labml_nn/scaling/zero3/__init__.py	251	# Create an empty tensor to receive the parameters	COMMENT
MEDIUM	labml_nn/scaling/zero3/finetune_neox.py	46	#### Create the model with Zero3 memory optimizer	STRING
MEDIUM	labml_nn/scaling/zero3/finetune_neox.py	59	# Create a sequential model	COMMENT
MEDIUM	labml_nn/scaling/zero3/finetune_neox.py	82	# Create the experiment	COMMENT
MEDIUM	labml_nn/transformers/hour_glass/__init__.py	226	# Create a mask if we haven't created or sizes have changed	COMMENT
MEDIUM	labml_nn/transformers/hour_glass/experiment.py	107	# Create the auto-regressive wrapper	COMMENT
MEDIUM	labml_nn/transformers/gmlp/experiment.py	42	### Create a gMLP block	STRING
MEDIUM	labml_nn/transformers/basic/with_sophia.py	48	# Create a categorical distribution from logits	COMMENT
MEDIUM	labml_nn/transformers/jax_transformer/__init__.py	679	# Create a transformer layer	STRING
MEDIUM	labml_nn/transformers/jax_transformer/__init__.py	953	# Create the model	STRING
MEDIUM	labml_nn/transformers/feedback/__init__.py	399	# Create a tensor for the stack	STRING
MEDIUM	labml_nn/transformers/feedback/__init__.py	401	# Create a tensor to accumulate the gradients	STRING
MEDIUM	labml_nn/transformers/vit/experiment.py	55	# Create a vision transformer	COMMENT
MEDIUM	labml_nn/transformers/retro/model.py	134	# Create a triangular mask	COMMENT
MEDIUM	labml_nn/transformers/retro/train.py	159	# Create an experiment	COMMENT
MEDIUM⚡	labml_nn/transformers/retro/train.py	186	# Create the nearest neighbor encoder	COMMENT
MEDIUM⚡	labml_nn/transformers/retro/train.py	188	# Create the model	COMMENT
MEDIUM⚡	labml_nn/transformers/retro/train.py	195	# Create the optimizer	COMMENT
MEDIUM	labml_nn/transformers/compressive/experiment.py	60	# Create a subsequent mask for tokens	COMMENT
MEDIUM	labml_nn/transformers/compressive/experiment.py	64	# Create an all ones (full visibility) mask for memory	COMMENT
MEDIUM	labml_nn/transformers/xl/experiment.py	44	# Create a subsequent mask for tokens	COMMENT
MEDIUM	labml_nn/transformers/xl/experiment.py	48	# Create an all ones (full visibility) mask for memory	COMMENT
MEDIUM	labml_nn/capsule_networks/mnist.py	86	# Create a mask to maskout all the other capsules	COMMENT
MEDIUM	labml_nn/graphs/gat/experiment.py	291	# Create an experiment	COMMENT
MEDIUM	labml_nn/graphs/gatv2/experiment.py	93	# Create an experiment	COMMENT
MEDIUM	labml_nn/sketch_rnn/__init__.py	205	# Create a bidirectional LSTM taking a sequence of	COMMENT
MEDIUM	labml_nn/sketch_rnn/__init__.py	293	# Create a bi-variate Gaussian mixture	COMMENT
MEDIUM	labml_nn/sketch_rnn/__init__.py	404	# Create a PyTorch tensor of the sequence of strokes	COMMENT
MEDIUM	labml_nn/sketch_rnn/__init__.py	427	# Create an empty stroke $(\Delta x, \Delta y, q_1, q_2, q_3)$	COMMENT
MEDIUM	labml_nn/sketch_rnn/__init__.py	440	# Create a new numpy array of the form $(x, y, q_2)$	COMMENT
MEDIUM	labml_nn/rl/dqn/experiment.py	255	# Create the experiment	COMMENT
1 more matches not shown…

Over-Commented Block38 hits · 38 pts

Severity	File	Line	Snippet	Context
LOW	labml_nn/gan/cycle_gan/__init__.py	721	# so that it only loads those and their dependencies.	COMMENT
LOW	labml_nn/hypernetworks/hyper_lstm.py	101	📝 Since the computation of $z$ and $d$ are two sequential linear transformations	COMMENT
LOW	labml_nn/neox/model.py	141		COMMENT
LOW	labml_nn/normalization/batch_channel_norm/__init__.py	141	# Variance for each feature	COMMENT
LOW	labml_nn/cfr/__init__.py	421	key: str	COMMENT
LOW	labml_nn/cfr/__init__.py	641	# \pi^{\sigma^t}_{-i}(h + a) &= \pi^{\sigma^t}_{-i}(h)	COMMENT
LOW	labml_nn/cfr/__init__.py	661	# $$\sum_{t=1}^T \pi_i^{\sigma^t}(I)\textcolor{lightgreen}{\sigma^t(I)(a)}	COMMENT
LOW	labml_nn/diffusion/ddpm/unet.py	61	# Create sinusoidal position embeddings	COMMENT
LOW	labml_nn/diffusion/ddpm/evaluate.py	41	# $T$	COMMENT
LOW	labml_nn/optimizers/amsgrad.py	81	* `grad` is the current gradient tensor $g_t$ for the parameter $\theta_{t-1}$	COMMENT
LOW	labml_nn/transformers/primer_ez/__init__.py	121	#	COMMENT
LOW	labml_nn/transformers/rope/__init__.py	181		COMMENT
LOW	labml_nn/transformers/rope/value_pe/__init__.py	141		COMMENT
LOW	labml_nn/transformers/basic/with_sophia.py	61	# Update EMA Hessian diagonal	COMMENT
LOW	labml_nn/transformers/feedback/__init__.py	401	# Create a tensor to accumulate the gradients	COMMENT
LOW	labml_nn/transformers/flash/__init__.py	761		COMMENT
LOW	labml_nn/transformers/switch/experiment.py	101	# Update global step (number of tokens processed) when in training mode	COMMENT
LOW	labml_nn/transformers/retro/model.py	61	idx_theta2 = torch.cat([idx_theta, idx_theta], dim=1)	COMMENT
LOW	labml_nn/transformers/aft/__init__.py	161		COMMENT
LOW	labml_nn/transformers/aft/__init__.py	181		COMMENT
LOW	labml_nn/transformers/fnet/__init__.py	81	#	COMMENT
LOW	labml_nn/recurrent_highway_networks/__init__.py	81	for d in range(self.depth):	COMMENT
LOW	labml_nn/graphs/gat/__init__.py	101		COMMENT
LOW	labml_nn/graphs/gat/__init__.py	121	# \Big] \Big)$$	COMMENT
LOW	labml_nn/graphs/gat/__init__.py	141	# \overrightarrow{g_2} \Vert \overrightarrow{g_1},	COMMENT
LOW	labml_nn/graphs/gat/__init__.py	161	assert adj_mat.shape[1] == 1 or adj_mat.shape[1] == n_nodes	COMMENT
LOW	labml_nn/graphs/gat/__init__.py	181	# $$\overrightarrow{h'^k_i} = \sum_{j \in \mathcal{N}_i} \alpha^k_{ij} \overrightarrow{g^k_j}$$	COMMENT
LOW	labml_nn/graphs/gatv2/__init__.py	141	#	COMMENT
LOW	labml_nn/graphs/gatv2/__init__.py	161	# Note: The paper desrcibes $e_{ij}$ as	COMMENT
LOW	labml_nn/graphs/gatv2/__init__.py	181	g_r_repeat_interleave = g_r.repeat_interleave(n_nodes, dim=0)	COMMENT
LOW	labml_nn/graphs/gatv2/__init__.py	201	e = e.squeeze(-1)	COMMENT
LOW	labml_nn/sketch_rnn/__init__.py	81	# since the mean is anyway close to $0$.	COMMENT
LOW	labml_nn/sketch_rnn/__init__.py	281	outputs, state = self.lstm(x, state)	COMMENT
LOW	labml_nn/sketch_rnn/__init__.py	321	# $(\Delta x, \Delta y, p_1, p_2, p_3)$.	COMMENT
LOW	labml_nn/rl/dqn/__init__.py	121	tracker.add('q_sampled_action', q_sampled_action)	COMMENT
LOW	labml_nn/rl/dqn/__init__.py	141	# Calculate the desired Q value.	COMMENT
LOW	labml_nn/rl/ppo/__init__.py	141	# this is different from rewards $r_t$.	COMMENT
LOW	labml_nn/rl/ppo/experiment.py	361	# You can change this while the experiment is running.	COMMENT

AI Slop Vocabulary17 hits · 30 pts

Severity	File	Line	Snippet	Context
LOW	docs/transformers/alibi/index.html	239	<p>Since it's causal mask we can just use <span ><span class="katex"><span aria-hidden="true" class="katex-html"><s	CODE
LOW	translate_cache/transformers/alibi/__init__.si.json	19	"<p>Calculate distances <span translate=no>_^_0_^_</span> Here we calculate the distances using the mask.</p>\n<p>Since	CODE
LOW	translate_cache/transformers/alibi/__init__.zh.json	19	"<p>Calculate distances <span translate=no>_^_0_^_</span> Here we calculate the distances using the mask.</p>\n<p>Since	CODE
LOW	translate_cache/transformers/alibi/__init__.ja.json	19	"<p>Calculate distances <span translate=no>_^_0_^_</span> Here we calculate the distances using the mask.</p>\n<p>Since	CODE
MEDIUM	translate_cache/transformers/primer_ez/readme.si.json	2	"<h1><a href=\"https://nn.labml.ai/transformers/primer_ez/index.html\">Primer: Searching for Efficient Transformers for	CODE
MEDIUM	translate_cache/transformers/primer_ez/__init__.si.json	2	"<h1>Primer: Searching for Efficient Transformers for Language Modeling</h1>\n<p>This is a <a href=\"https://pytorch.or	CODE
MEDIUM	translate_cache/transformers/primer_ez/readme.ja.json	2	"<h1><a href=\"https://nn.labml.ai/transformers/primer_ez/index.html\">Primer: Searching for Efficient Transformers for	CODE
MEDIUM	translate_cache/transformers/primer_ez/__init__.zh.json	2	"<h1>Primer: Searching for Efficient Transformers for Language Modeling</h1>\n<p>This is a <a href=\"https://pytorch.or	CODE
MEDIUM	translate_cache/transformers/primer_ez/__init__.ja.json	2	"<h1>Primer: Searching for Efficient Transformers for Language Modeling</h1>\n<p>This is a <a href=\"https://pytorch.or	CODE
MEDIUM	translate_cache/transformers/primer_ez/readme.zh.json	2	"<h1><a href=\"https://nn.labml.ai/transformers/primer_ez/index.html\">Primer: Searching for Efficient Transformers for	CODE
LOW	labml_nn/neox/model.py	280	# No cache - simply add RoPE embeddings	COMMENT
MEDIUM	labml_nn/neox/evaluation/llm_int8.py	48	# Run [evaluation harness](index.html)	COMMENT
MEDIUM	labml_nn/neox/evaluation/half_precision.py	42	# Run [evaluation harness](index.html)	COMMENT
MEDIUM	labml_nn/neox/evaluation/__init__.py	195	# Run [EleutherAI/lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) evaluator	COMMENT
MEDIUM	labml_nn/neox/evaluation/__init__.py	237	## Run evaluation harness with a given model	STRING
LOW	labml_nn/transformers/alibi/__init__.py	100	# Since it's causal mask we can just use $[0, 1, \dots, N]$ too.	COMMENT
LOW	labml_nn/transformers/compressive/__init__.py	131	# If there is no memory just return the token embeddings	COMMENT

Unused Imports18 hits · 18 pts

Severity	File	Line	Context
LOW	labml_nn/activations/__init__.py	14	CODE
LOW	labml_nn/neox/utils/__init__.py	25	CODE
LOW	labml_nn/cfr/__init__.py	330	CODE
LOW	…nn/diffusion/stable_diffusion/scripts/text_to_image.py	12	CODE
LOW	labml_nn/optimizers/adam_fp16.py	16	CODE
LOW	labml_nn/optimizers/adam.py	44	CODE
LOW	labml_nn/transformers/__init__.py	112	CODE
LOW	labml_nn/transformers/__init__.py	113	CODE
LOW	labml_nn/transformers/__init__.py	113	CODE
LOW	labml_nn/transformers/__init__.py	113	CODE
LOW	labml_nn/transformers/__init__.py	113	CODE
LOW	labml_nn/transformers/__init__.py	113	CODE
LOW	labml_nn/transformers/__init__.py	114	CODE
LOW	labml_nn/transformers/__init__.py	115	CODE
LOW	labml_nn/transformers/alibi/__init__.py	37	CODE
LOW	labml_nn/transformers/mlp_mixer/experiment.py	16	CODE
LOW	labml_nn/rl/dqn/__init__.py	30	CODE
LOW	labml_nn/rl/ppo/__init__.py	29	CODE

Redundant / Tautological Comments14 hits · 17 pts

Severity	File	Line	Snippet	Context
LOW	labml_nn/gan/stylegan/__init__.py	634	# Check if the batch size is divisible by the group size	COMMENT
LOW	labml_nn/gan/cycle_gan/__init__.py	545	# Loop through the dataset	COMMENT
LOW	labml_nn/neox/utils/trainer.py	108	# Set gradients to zero	COMMENT
LOW	labml_nn/diffusion/ddpm/experiment.py	128	# Iterate through the dataset	COMMENT
LOW	labml_nn/unet/experiment.py	109	# Iterate through the dataset.	COMMENT
LOW	labml_nn/transformers/knn/build_index.py	76	# Loop through data	COMMENT
LOW	labml_nn/transformers/knn/build_index.py	146	# Set model to evaluation mode	COMMENT
LOW	labml_nn/transformers/knn/eval_knn.py	142	# Set model to evaluation mode	COMMENT
LOW	labml_nn/transformers/jax_transformer/__init__.py	189	# Return the result	COMMENT
LOW	labml_nn/transformers/jax_transformer/__init__.py	876	# Read the file	STRING
LOW	labml_nn/transformers/feedback/__init__.py	296	# Append them to the list of layer outputs	STRING
LOW	labml_nn/transformers/feedback/__init__.py	507	# Append them to the list of layer outputs	STRING
LOW	labml_nn/graphs/gat/experiment.py	238	# Set mode to evaluation mode for validation	COMMENT
LOW	labml_nn/rl/dqn/replay_buffer.py	123	### Add sample to queue	STRING

Hyper-Verbose Identifiers14 hits · 17 pts

Severity	File	Line	Snippet	Context
LOW	labml_nn/neox/checkpoint.py	28	def get_checkpoints_download_path():	CODE
LOW	labml_nn/neox/model.py	647	def _create_transformer_layer(self):	CODE
LOW	labml_nn/neox/utils/finetune.py	20	def get_layer_trainable_params(self, layer: NeoXModule, prefix: str) -> Dict[str, nn.Parameter]:	CODE
LOW	labml_nn/neox/utils/finetune.py	44	def get_layer_trainable_params(self, layer: NeoXModule, prefix: str) -> Dict[str, nn.Parameter]:	CODE
LOW⚡	labml_nn/transformers/configs.py	277	def _src_embed_with_positional(c: TransformerConfigs):	CODE
LOW⚡	labml_nn/transformers/configs.py	285	def _tgt_embed_with_positional(c: TransformerConfigs):	CODE
LOW⚡	labml_nn/transformers/configs.py	294	def _src_embed_with_learned_positional(c: TransformerConfigs):	CODE
LOW⚡	labml_nn/transformers/configs.py	302	def _tgt_embed_with_learned_positional(c: TransformerConfigs):	CODE
LOW⚡	labml_nn/transformers/configs.py	311	def _src_embed_without_positional(c: TransformerConfigs):	CODE
LOW⚡	labml_nn/transformers/configs.py	319	def _tgt_embed_without_positional(c: TransformerConfigs):	CODE
LOW	labml_nn/transformers/positional_encoding.py	64	def _test_positional_encoding():	CODE
LOW	labml_nn/transformers/alibi/experiment.py	67	def shuffled_longer_valid_loader(c: Configs):	CODE
LOW	labml_nn/transformers/retro/train.py	48	def retrieve_nearest_neighbours(self, chunk: str):	CODE
LOW	labml_nn/transformers/compressive/experiment.py	299	def attention_reconstruction_loss(c: Configs):	CODE

Slop Phrases8 hits · 15 pts

Severity	File	Line	Snippet	Context
MEDIUM	docs/transformers/retro/bert_embeddings.html	162	, with partial words (partial sub-word tokens) on the ends. We strip them off to get better BERT embeddings. As mentione	CODE
MEDIUM	translate_cache/transformers/alibi/__init__.si.json	2	"<h1>Attention with Linear Biases (ALiBi)</h1>\n<p>This is an implementation of Attention with Linear Biases (ALiBi) fr	CODE
MEDIUM	translate_cache/transformers/alibi/__init__.zh.json	2	"<h1>Attention with Linear Biases (ALiBi)</h1>\n<p>This is an implementation of Attention with Linear Biases (ALiBi) fr	CODE
MEDIUM	translate_cache/transformers/alibi/__init__.ja.json	2	"<h1>Attention with Linear Biases (ALiBi)</h1>\n<p>This is an implementation of Attention with Linear Biases (ALiBi) fr	CODE
MEDIUM	…slate_cache/transformers/retro/bert_embeddings.si.json	7	"<p> In this implementation, we do not make chunks with a fixed number of tokens. One of the reasons is that this imple	CODE
MEDIUM	…slate_cache/transformers/retro/bert_embeddings.zh.json	7	"<p> In this implementation, we do not make chunks with a fixed number of tokens. One of the reasons is that this imple	CODE
MEDIUM	…slate_cache/transformers/retro/bert_embeddings.ja.json	7	"<p> In this implementation, we do not make chunks with a fixed number of tokens. One of the reasons is that this imple	CODE
MEDIUM	labml_nn/transformers/retro/bert_embeddings.py	58	As mentioned earlier this is not necessary if we broke chunks after tokenizing.	STRING

AI Response Leakage3 hits · 15 pts

Severity	File	Line	Snippet	Context
HIGH	…ache/gan/wasserstein/gradient_penalty/__init__.si.json	2	"<h1>Gradient Penalty for Wasserstein GAN (WGAN-GP)</h1>\n<p>This is an implementation of <a href=\"https://arxiv.org/a	CODE
HIGH	…ache/gan/wasserstein/gradient_penalty/__init__.zh.json	2	"<h1>Gradient Penalty for Wasserstein GAN (WGAN-GP)</h1>\n<p>This is an implementation of <a href=\"https://arxiv.org/a	CODE
HIGH	…ache/gan/wasserstein/gradient_penalty/__init__.ja.json	2	"<h1>Gradient Penalty for Wasserstein GAN (WGAN-GP)</h1>\n<p>This is an implementation of <a href=\"https://arxiv.org/a	CODE

Deep Nesting12 hits · 12 pts

Severity	File	Line	Context
LOW	utils/papers_list.py	40	CODE
LOW	labml_nn/gan/original/experiment.py	123	CODE
LOW	labml_nn/gan/stylegan/experiment.py	306	CODE
LOW	labml_nn/neox/utils/trainer.py	101	CODE
LOW	labml_nn/optimizers/adam_fp16.py	101	CODE
LOW	labml_nn/transformers/glu_variants/simple.py	130	CODE
LOW	labml_nn/transformers/glu_variants/simple.py	235	CODE
LOW	labml_nn/transformers/flash/__init__.py	313	CODE
LOW	labml_nn/transformers/mlm/experiment.py	177	CODE
LOW	labml_nn/rl/game.py	150	CODE
LOW	labml_nn/rl/dqn/experiment.py	129	CODE
LOW	labml_nn/rl/ppo/experiment.py	157	CODE

AI Structural Patterns10 hits · 10 pts

Severity	File	Line	Context
LOW	labml_nn/neox/model.py	317	CODE
LOW	labml_nn/neox/model.py	343	CODE
LOW	labml_nn/neox/model.py	506	CODE
LOW	labml_nn/diffusion/ddpm/unet.py	206	CODE
LOW	labml_nn/optimizers/adam_warmup.py	24	CODE
LOW	labml_nn/optimizers/adam_warmup_cosine_decay.py	27	CODE
LOW	labml_nn/optimizers/ada_belief.py	52	CODE
LOW	labml_nn/optimizers/noam.py	27	CODE
LOW	labml_nn/optimizers/radam.py	155	CODE
LOW	labml_nn/transformers/jax_transformer/__init__.py	190	CODE

Empty Error Handlers2 hits · 10 pts

Severity	File	Line	Snippet	Context
HIGH	labml_nn/helpers/trainer.py	92	except ValueError:	CODE
HIGH	labml_nn/helpers/trainer.py	109	except ValueError:	CODE

labmlai/annotated_deep_learning_paper_implementations

Open-Source Code Quality, Security and Technical Debt Review

Risk Review · Payload v1

What These Metrics Mean

Pattern-Signal History

Severity Breakdown

Directory Score Breakdown

Pattern Findings