apache/datafusion

8.7

Adjusted Score

8.7

Raw Score

100%

Time Factor

2026-07-14

Last Push

9.0K

Stars

Rust

Language

917.4K

Lines of Code

2.5K

Files

5.5K

Pattern Hits

2026-07-14

Scan Date

0.00

HC Hit Rate

What These Metrics Mean

Adjusted Score: Primary synthetic code indicator. Raw score normalised per 1,000 lines of code and multiplied by the temporal discount factor. This is the definitive comparative metric — use it to rank repositories by AI authorship density.
Raw Score: The unmodified sum of all severity-weighted, context-multiplied pattern match scores before temporal discounting. Reflects the absolute signal strength independent of when the repository was last active.
Time Factor: The temporal discount multiplier (0–100%) applied to the raw score. Repositories last updated before ChatGPT's launch (Nov 2022) receive a 5% factor. Full signal is only assigned to repositories active in the post-adoption era (Jan 2024+).
Pattern Hits: Total count of individual pattern matches across all files and categories. A high hit count with a low score may indicate a very large codebase with isolated AI snippets; a low count with a high score indicates dense, concentrated AI signatures.
HC Hit Rate: High+Critical pattern hits per file, averaged across the repository. This orthogonal signal catches repositories where a few files are densely packed with high-severity AI tells — a strong indicator even when the normalised score appears moderate due to codebase size.
Lines of Code / Files: Total lines and files analysed. The scanner examines 94 file extensions. These denominators are used to normalise the score, enabling fair comparison between repositories of vastly different sizes.

Score History

This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.

Severity Breakdown

Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.

CRITICAL 0HIGH 2MEDIUM 963LOW 4563

Directory Score Breakdown

This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.

Pattern Findings

The scanner identified 5528 distinct pattern matches across 17 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.

Over-Commented Block4378 hits · 4080 pts

Severity	File	Line	Snippet	Context
LOW	.asf.yaml	1	# Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	Cargo.toml	1	# Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	Cargo.toml	241	# - Debuggability: how much debug information is preserved for debugging and profiling	COMMENT
LOW	lychee.toml	1	# Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	rustfmt.toml	1	# Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	rustfmt.toml	21	# ignore generated files	COMMENT
LOW	taplo.toml	1	# Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	pre-commit.sh	1	#!/bin/bash	COMMENT
LOW	licenserc.toml	1	# Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	rust-toolchain.toml	1	# Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	test-utils/Cargo.toml	1	# Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	test-utils/src/tpch.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	test-utils/src/data_gen.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	test-utils/src/data_gen.rs	241	/// use cases.	COMMENT
LOW	test-utils/src/lib.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	test-utils/src/lib.rs	81	/// multiple batches	COMMENT
LOW	test-utils/src/string_gen.rs	1	use crate::array_gen::StringArrayGenerator;	COMMENT
LOW	test-utils/src/tpcds.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	test-utils/src/array_gen/decimal.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	test-utils/src/array_gen/boolean.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	test-utils/src/array_gen/binary.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	test-utils/src/array_gen/mod.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	test-utils/src/array_gen/string.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	test-utils/src/array_gen/random_data.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	test-utils/src/array_gen/primitive.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	datafusion-cli/Cargo.toml	1	# Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	datafusion-cli/tests/cli_integration.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	datafusion-cli/examples/cli-session-context.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	datafusion-cli/src/print_format.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	datafusion-cli/src/exec.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	datafusion-cli/src/exec.rs	461	/// in the session context.	COMMENT
LOW	datafusion-cli/src/command.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	datafusion-cli/src/lib.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	datafusion-cli/src/functions.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	datafusion-cli/src/functions.rs	721	/// a List of Structs, with one Struct for each object. DataFusion uses these	COMMENT
LOW	datafusion-cli/src/object_storage.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	datafusion-cli/src/object_storage.rs	321	}	COMMENT
LOW	datafusion-cli/src/cli_context.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	datafusion-cli/src/catalog.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	datafusion-cli/src/pool_type.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	datafusion-cli/src/main.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	datafusion-cli/src/highlighter.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	datafusion-cli/src/helper.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	datafusion-cli/src/print_options.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	datafusion-cli/src/object_storage/instrumented.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	datafusion-cli/src/object_storage/instrumented.rs	581	summaries.sort_by_key(\|s\| s.operation);	COMMENT
LOW	datafusion-cli/src/object_storage/stdin.rs	1	// Licensed to the Apache Software Foundation (ASF) under one	COMMENT
LOW	datafusion-cli/src/object_storage/stdin.rs	41	/// [`SessionConfig`]: datafusion::execution::context::SessionConfig	COMMENT
LOW	datafusion-cli/src/object_storage/stdin.rs	61	/// Utilities for exposing the process's standard input as an object store.	COMMENT
LOW	datafusion-cli/src/object_storage/stdin.rs	101		COMMENT
LOW	datafusion-cli/src/object_storage/stdin.rs	261	.await?;	COMMENT
LOW	ci/scripts/rust_fmt.sh	1	#!/usr/bin/env bash	COMMENT
LOW	ci/scripts/check_asf_yaml_status_checks.py	1	#!/usr/bin/env python3	COMMENT
LOW	ci/scripts/typos_check.sh	1	#!/usr/bin/env bash	COMMENT
LOW	ci/scripts/check_no_cargo_install_in_workflows.sh	1	#!/usr/bin/env bash	COMMENT
LOW	ci/scripts/rust_toml_fmt.sh	1	#!/usr/bin/env bash	COMMENT
LOW	ci/scripts/rust_clippy.sh	1	#!/usr/bin/env bash	COMMENT
LOW	ci/scripts/changed_crates.sh	1	#!/usr/bin/env bash	COMMENT
LOW	ci/scripts/changed_crates.sh	21	# Subcommands:	COMMENT
LOW	ci/scripts/rust_example.sh	1	#!/usr/bin/env bash	COMMENT
4318 more matches not shown…

Decorative Section Separators922 hits · 3496 pts

Severity	File	Line	Snippet	Context
MEDIUM	Cargo.toml	234	# --------------------	COMMENT
MEDIUM	Cargo.toml	236	# --------------------	COMMENT
MEDIUM	ci/scripts/changed_crates.sh	34	# ── changed-crates ──────────────────────────────────────────────────	COMMENT
MEDIUM	ci/scripts/changed_crates.sh	62	# ── semver-check ────────────────────────────────────────────────────	COMMENT
MEDIUM	ci/scripts/changed_crates.sh	75	# ── main ────────────────────────────────────────────────────────────	COMMENT
MEDIUM⚡	datafusion/datasource/src/source.rs	79	/// ┌─────────────────────┐ -----► execute path	COMMENT
MEDIUM⚡	datafusion/datasource/src/source.rs	80	/// │ │ ┄┄┄┄┄► init path	COMMENT
MEDIUM⚡	datafusion/datasource/src/source.rs	83	/// └───────▲─────────────┘	COMMENT
MEDIUM⚡	datafusion/datasource/src/source.rs	86	/// ┌──────────▼──────────┐ ┌──────────-──────────┐	COMMENT
MEDIUM⚡	datafusion/datasource/src/source.rs	90	/// └───────▲─────────────┘ └─────────────────────┘	COMMENT
MEDIUM⚡	datafusion/datasource/src/source.rs	92	/// ┌───────────────┿──┴────────────────┐ ┊	COMMENT
MEDIUM⚡	datafusion/datasource/src/source.rs	93	/// \| ┌┄┄┄┄┄┄┄┄┄┄┄┘ \| ┊	COMMENT
MEDIUM⚡	datafusion/datasource/src/source.rs	95	/// ┌──────────▼──────────┐ ┌──────────▼──────────┐ ┊	COMMENT
MEDIUM⚡	datafusion/datasource/src/source.rs	96	/// │ │ │ │ ┌──────────▼──────────┐	COMMENT
MEDIUM⚡	datafusion/datasource/src/source.rs	99	/// └──────────────▲──────┘ └─────────────────────┘ \| \|	COMMENT
MEDIUM⚡	datafusion/datasource/src/source.rs	100	/// │ ┊ └─────────────────────┘	COMMENT
MEDIUM⚡	datafusion/datasource/src/source.rs	103	/// ┌──────────▼──────────┐ ┌──────────▼──────────┐	COMMENT
MEDIUM⚡	datafusion/datasource/src/source.rs	105	/// │ FileSource(trait) ◄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄│ ... │	COMMENT
MEDIUM⚡	datafusion/datasource/src/source.rs	107	/// └─────────────────────┘ └─────────────────────┘	COMMENT
MEDIUM⚡	datafusion/datasource/src/source.rs	112	/// ┌──────────▼──────────┐	COMMENT
MEDIUM⚡	datafusion/datasource/src/source.rs	116	/// └─────────────────────┘	COMMENT
MEDIUM⚡	datafusion/datasource/src/source.rs	120	/// ┌──────────▼──────────┐	COMMENT
MEDIUM⚡	datafusion/datasource/src/source.rs	124	/// └─────────────────────┘	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	48	/// ┌─────────────────┐	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	53	/// │ └─────────────────┘ │	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	55	/// ┌─────────────────┐ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	56	/// │ │ ┌─────────────────┐	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	61	/// │ │ │ └─────────────────┘ │	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	62	/// │ File A (7MB) │ ────────▶ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	64	/// │ │ ┌─────────────────┐	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	69	/// │ │ │ └─────────────────┘ │	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	70	/// └─────────────────┘ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	71	/// ┌─────────────────┐ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	72	/// │ File B (1MB) │ ┌─────────────────┐	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	74	/// └─────────────────┘ │ Range: 6-7MB │	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	75	/// │ └─────────────────┘ │	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	76	/// ┌─────────────────┐	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	79	/// │ └─────────────────┘ │	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	97	/// ┌─────────────────┐	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	102	/// ┌─────────────────┐ │ └─────────────────┘ │	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	105	/// │ │ ┌─────────────────┐	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	109	/// │ File A (6MB) │ ────────▶ │ │	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	110	/// │ (ordered) │ │ └─────────────────┘ │	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	113	/// │ │ ┌─────────────────┐	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	117	/// └─────────────────┘ │ │	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	118	/// ┌─────────────────┐ │ └─────────────────┘ │	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	121	/// └─────────────────┘ ┌─────────────────┐	COMMENT
MEDIUM⚡	datafusion/datasource/src/file_groups.rs	124	/// │ └─────────────────┘ │	COMMENT
MEDIUM⚡	datafusion/datasource/src/write/demux.rs	87	/// ┌───────────┐ ┌──────────	COMMENT
MEDIUM⚡	datafusion/datasource/src/write/demux.rs	88	/// ┌──────▶ │ batch 1 ├────▶...──────▶│ Batch a	COMMENT
MEDIUM⚡	datafusion/datasource/src/write/demux.rs	89	/// │ └───────────┘ └──────────	COMMENT
MEDIUM⚡	datafusion/datasource/src/write/demux.rs	91	/// ┌──────────┐ │ ┌───────────┐ ┌──────────	COMMENT
MEDIUM⚡	datafusion/datasource/src/write/demux.rs	92	/// ┌───────────┐ ┌────────────┐ │ │ ├──────▶ │ batch a+1├────▶...──────▶│ Batch b	COMMENT
MEDIUM⚡	datafusion/datasource/src/write/demux.rs	93	/// │ batch 1 ├────▶...──────▶│ Batch N ├─────▶│ Demux ├────────┤ ... └───────────┘ └──────────	COMMENT
MEDIUM⚡	datafusion/datasource/src/write/demux.rs	94	/// └───────────┘ └────────────┘ │ │ │	COMMENT
MEDIUM⚡	datafusion/datasource/src/write/demux.rs	95	/// └──────────┘ │ ┌───────────┐ ┌──────────	COMMENT
MEDIUM⚡	datafusion/datasource/src/write/demux.rs	96	/// └──────▶ │ batch d ├────▶...──────▶│ Batch n	COMMENT
MEDIUM⚡	datafusion/datasource/src/write/demux.rs	97	/// └───────────┘ └──────────	COMMENT
MEDIUM⚡	…usion/datasource/src/file_scan_config/sort_pushdown.rs	541	/// ┃ ┌───────────────┐ ┌──────────────┐ │	COMMENT
MEDIUM⚡	…usion/datasource/src/file_scan_config/sort_pushdown.rs	544	/// │ └───────────────┘ │ │ └──────────────┘ ┃	COMMENT
862 more matches not shown…

Structural Annotation Overuse73 hits · 121 pts

Severity	File	Line	Snippet	Context
LOW	datafusion/physical-plan/src/aggregates/mod.rs	6262	// Step 1: Partial aggregation on partition 1	COMMENT
LOW	datafusion/physical-plan/src/aggregates/mod.rs	6274	// Step 2: Partial aggregation on partition 2	COMMENT
LOW	datafusion/physical-plan/src/aggregates/mod.rs	6296	// Step 3: PartialReduce — combine partial results, still producing state	COMMENT
LOW	datafusion/physical-plan/src/aggregates/mod.rs	6323	// Step 4: Final aggregation on the PartialReduce output	COMMENT
LOW	…usion/physical-plan/src/aggregates/aggregate_stream.rs	160	// Step 1: Update each partition's current bound	COMMENT
LOW	…usion/physical-plan/src/aggregates/aggregate_stream.rs	197	// Step 2: Sync the dynamic filter physical expression with reader,	COMMENT
LOW⚡	datafusion/physical-plan/src/spill/spill_pool.rs	1454	// Step 1: Create a test batch and measure its size	COMMENT
LOW⚡	datafusion/physical-plan/src/spill/spill_pool.rs	1458	// Step 2: Configure file rotation to approximately 1 batch per file	COMMENT
LOW⚡	datafusion/physical-plan/src/spill/spill_pool.rs	1482	// Step 4: Verify initial disk usage reflects all files	COMMENT
LOW⚡	datafusion/physical-plan/src/spill/spill_pool.rs	1489	// Step 5: Read NUM_BATCHES - 1 batches (all but 1)	COMMENT
LOW⚡	datafusion/physical-plan/src/spill/spill_pool.rs	1515	// Step 7: Read the final batch	COMMENT
LOW⚡	datafusion/physical-plan/src/spill/spill_pool.rs	1519	// Step 8: Drop writer first to signal no more data will be written	COMMENT
LOW⚡	datafusion/physical-plan/src/spill/spill_pool.rs	1530	// Step 9: Drop reader to release all references	COMMENT
LOW⚡	datafusion/physical-plan/src/spill/spill_pool.rs	1533	// Step 10: Verify complete cleanup - disk usage should be 0	COMMENT
LOW	datafusion/physical-plan/src/spill/spill_pool.rs	538	// Step 1: Lock shared state and check coordination	COMMENT
LOW	datafusion/physical-plan/src/spill/spill_pool.rs	563	// Step 2: Lazy-create reader stream if needed	COMMENT
LOW	datafusion/physical-plan/src/spill/spill_pool.rs	588	// Step 3: Poll the reader stream (no lock held)	COMMENT
LOW	datafusion/physical-plan/src/spill/spill_pool.rs	1469	// Step 3: Write NUM_BATCHES batches to create approximately NUM_BATCHES files	COMMENT
LOW	datafusion/physical-plan/src/spill/spill_pool.rs	1503	// Step 6: Verify disk usage decreased but is not zero (at least 1 batch remains)	COMMENT
LOW	datafusion/expr-common/src/interval_arithmetic.rs	462	/// NOTE: This function only works with intervals of the same data type.	COMMENT
LOW	datafusion/expr-common/src/interval_arithmetic.rs	496	/// NOTE: This function only works with intervals of the same data type.	COMMENT
LOW	datafusion/expr-common/src/interval_arithmetic.rs	530	/// NOTE: This function only works with intervals of the same data type.	COMMENT
LOW	datafusion/expr-common/src/interval_arithmetic.rs	541	/// NOTE: This function only works with intervals of the same data type.	COMMENT
LOW	datafusion/expr-common/src/interval_arithmetic.rs	552	/// NOTE: This function only works with intervals of the same data type.	COMMENT
LOW	datafusion/expr-common/src/interval_arithmetic.rs	767	/// NOTE: This function only works with intervals of the same data type.	COMMENT
LOW	datafusion/expr-common/src/interval_arithmetic.rs	1394	/// NOTE: This function only works with intervals of the same data type.	COMMENT
LOW	datafusion/expr-common/src/interval_arithmetic.rs	2175	/// NOTE: This function only works with intervals of the same data type.	COMMENT
LOW	datafusion/functions-aggregate/src/array_agg.rs	661	// Step 1: Count entries per group. For EmitTo::First(n), only groups	COMMENT
LOW	datafusion/functions-aggregate/src/array_agg.rs	673	// Step 2: Do a prefix sum over the counts and use it to build ListArray	COMMENT
LOW	datafusion/functions-aggregate/src/array_agg.rs	692	// Step 3: Scatter entries into group order using the counting sort. The	COMMENT
LOW	datafusion/functions-aggregate/src/array_agg.rs	714	// Step 4: Release state for emitted groups.	COMMENT
LOW⚡	datafusion/physical-optimizer/src/window_topn.rs	132	// Step 1: Match FilterExec at the top	COMMENT
LOW⚡	datafusion/physical-optimizer/src/window_topn.rs	140	// Step 2: Extract limit from predicate (rn <= K, rn < K, etc.)	COMMENT
LOW⚡	datafusion/physical-optimizer/src/window_topn.rs	143	// Step 3: Walk through optional ProjectionExec to find BoundedWindowAggExec	COMMENT
LOW⚡	datafusion/physical-optimizer/src/window_topn.rs	147	// Step 4: Verify col_idx references a supported window function output column	COMMENT
LOW⚡	datafusion/physical-optimizer/src/window_topn.rs	159	// Step 5: Verify child of window is SortExec	COMMENT
LOW⚡	datafusion/physical-optimizer/src/window_topn.rs	163	// Step 6: Determine partition_prefix_len from the window expression	COMMENT
LOW⚡	datafusion/physical-optimizer/src/window_topn.rs	182	// Step 7: Build PartitionedTopKExec using SortExec's expressions	COMMENT
LOW⚡	datafusion/physical-optimizer/src/window_topn.rs	192	// Step 8: Rebuild window with new child	COMMENT
LOW⚡	datafusion/physical-optimizer/src/window_topn.rs	197	// Step 9: If ProjectionExec was between Filter and Window, rebuild it	COMMENT
LOW	datafusion/datasource-parquet/src/sort.rs	590	// Step 1: Create PreparedAccessPlan	COMMENT
LOW	datafusion/datasource-parquet/src/sort.rs	607	// Step 2: Reverse the plan (this is the production code path)	COMMENT
LOW	datafusion/datasource-parquet/src/sort.rs	808	// Step 1: Create PreparedAccessPlan	COMMENT
LOW	datafusion/datasource-parquet/src/sort.rs	854	// Step 2: Reverse the plan (this is the production code path)	COMMENT
LOW	datafusion/datasource-parquet/src/push_decoder.rs	296	// Step 1: drain a batch from the active reader if any.	COMMENT
LOW	datafusion/datasource-parquet/src/push_decoder.rs	318	// Step 2: when the decoder is sitting on a row-group boundary,	COMMENT
LOW	datafusion/datasource-parquet/src/push_decoder.rs	372	// Step 3: drive the decoder.	COMMENT
LOW	datafusion/spark/src/function/string/concat.rs	122	// Step 1: Check for NULL mask in incoming args	COMMENT
LOW	datafusion/spark/src/function/string/concat.rs	133	// Step 2: Delegate to DataFusion's concat	COMMENT
LOW	datafusion/spark/src/function/string/concat.rs	145	// Step 3: Apply NULL mask to result	COMMENT
LOW	datafusion/sql/src/expr/mod.rs	290	// NOTE: This function is called recursively, so each match arm body should be as	COMMENT
LOW	docs/source/contributor-guide/inviting.md	49	### Step 1: Start a Discussion Thread	COMMENT
LOW	docs/source/contributor-guide/inviting.md	75	### Step 2: Formal Vote	COMMENT
LOW	docs/source/contributor-guide/inviting.md	108	### Step 3: Send Invitation to the Candidate	COMMENT
LOW	docs/source/contributor-guide/inviting.md	126	### Step 4: Announce and Celebrate the New Committer	COMMENT
LOW	docs/source/contributor-guide/inviting.md	289	### Step 1: Start a Discussion Thread	COMMENT
LOW	docs/source/contributor-guide/inviting.md	313	### Step 2: Formal Vote	COMMENT
LOW	docs/source/contributor-guide/inviting.md	347	### Step 3: Send invitation email	COMMENT
LOW	docs/source/contributor-guide/inviting.md	399	### Step 4: Chair Promotes the Committer to PMC	COMMENT
LOW	docs/source/contributor-guide/inviting.md	403	### Step 5: Announce and Celebrate the New PMC Member	COMMENT
13 more matches not shown…

Verbosity Indicators53 hits · 90 pts

Severity	File	Line	Snippet	Context
LOW	datafusion/physical-plan/src/aggregates/mod.rs	6262	// Step 1: Partial aggregation on partition 1	COMMENT
LOW	datafusion/physical-plan/src/aggregates/mod.rs	6274	// Step 2: Partial aggregation on partition 2	COMMENT
LOW	datafusion/physical-plan/src/aggregates/mod.rs	6296	// Step 3: PartialReduce — combine partial results, still producing state	COMMENT
LOW	datafusion/physical-plan/src/aggregates/mod.rs	6323	// Step 4: Final aggregation on the PartialReduce output	COMMENT
LOW	…usion/physical-plan/src/aggregates/aggregate_stream.rs	160	// Step 1: Update each partition's current bound	COMMENT
LOW	…usion/physical-plan/src/aggregates/aggregate_stream.rs	197	// Step 2: Sync the dynamic filter physical expression with reader,	COMMENT
LOW	…plan/src/aggregates/group_values/multi_group_by/mod.rs	1135	// In non-streaming case, we need to check if the `group index view`	COMMENT
LOW⚡	datafusion/physical-plan/src/spill/spill_pool.rs	1454	// Step 1: Create a test batch and measure its size	COMMENT
LOW⚡	datafusion/physical-plan/src/spill/spill_pool.rs	1458	// Step 2: Configure file rotation to approximately 1 batch per file	COMMENT
LOW⚡	datafusion/physical-plan/src/spill/spill_pool.rs	1482	// Step 4: Verify initial disk usage reflects all files	COMMENT
LOW⚡	datafusion/physical-plan/src/spill/spill_pool.rs	1489	// Step 5: Read NUM_BATCHES - 1 batches (all but 1)	COMMENT
LOW⚡	datafusion/physical-plan/src/spill/spill_pool.rs	1515	// Step 7: Read the final batch	COMMENT
LOW⚡	datafusion/physical-plan/src/spill/spill_pool.rs	1519	// Step 8: Drop writer first to signal no more data will be written	COMMENT
LOW⚡	datafusion/physical-plan/src/spill/spill_pool.rs	1530	// Step 9: Drop reader to release all references	COMMENT
LOW⚡	datafusion/physical-plan/src/spill/spill_pool.rs	1533	// Step 10: Verify complete cleanup - disk usage should be 0	COMMENT
LOW	datafusion/physical-plan/src/spill/spill_pool.rs	538	// Step 1: Lock shared state and check coordination	COMMENT
LOW	datafusion/physical-plan/src/spill/spill_pool.rs	563	// Step 2: Lazy-create reader stream if needed	COMMENT
LOW	datafusion/physical-plan/src/spill/spill_pool.rs	588	// Step 3: Poll the reader stream (no lock held)	COMMENT
LOW	datafusion/physical-plan/src/spill/spill_pool.rs	1469	// Step 3: Write NUM_BATCHES batches to create approximately NUM_BATCHES files	COMMENT
LOW	datafusion/physical-plan/src/spill/spill_pool.rs	1503	// Step 6: Verify disk usage decreased but is not zero (at least 1 batch remains)	COMMENT
LOW	datafusion/functions-aggregate/src/array_agg.rs	661	// Step 1: Count entries per group. For EmitTo::First(n), only groups	COMMENT
LOW	datafusion/functions-aggregate/src/array_agg.rs	673	// Step 2: Do a prefix sum over the counts and use it to build ListArray	COMMENT
LOW	datafusion/functions-aggregate/src/array_agg.rs	692	// Step 3: Scatter entries into group order using the counting sort. The	COMMENT
LOW	datafusion/functions-aggregate/src/array_agg.rs	714	// Step 4: Release state for emitted groups.	COMMENT
LOW⚡	datafusion/physical-optimizer/src/window_topn.rs	132	// Step 1: Match FilterExec at the top	COMMENT
LOW⚡	datafusion/physical-optimizer/src/window_topn.rs	140	// Step 2: Extract limit from predicate (rn <= K, rn < K, etc.)	COMMENT
LOW⚡	datafusion/physical-optimizer/src/window_topn.rs	143	// Step 3: Walk through optional ProjectionExec to find BoundedWindowAggExec	COMMENT
LOW⚡	datafusion/physical-optimizer/src/window_topn.rs	147	// Step 4: Verify col_idx references a supported window function output column	COMMENT
LOW⚡	datafusion/physical-optimizer/src/window_topn.rs	159	// Step 5: Verify child of window is SortExec	COMMENT
LOW⚡	datafusion/physical-optimizer/src/window_topn.rs	163	// Step 6: Determine partition_prefix_len from the window expression	COMMENT
LOW⚡	datafusion/physical-optimizer/src/window_topn.rs	182	// Step 7: Build PartitionedTopKExec using SortExec's expressions	COMMENT
LOW⚡	datafusion/physical-optimizer/src/window_topn.rs	192	// Step 8: Rebuild window with new child	COMMENT
LOW⚡	datafusion/physical-optimizer/src/window_topn.rs	197	// Step 9: If ProjectionExec was between Filter and Window, rebuild it	COMMENT
LOW	datafusion/datasource-parquet/src/sort.rs	590	// Step 1: Create PreparedAccessPlan	COMMENT
LOW	datafusion/datasource-parquet/src/sort.rs	607	// Step 2: Reverse the plan (this is the production code path)	COMMENT
LOW	datafusion/datasource-parquet/src/sort.rs	808	// Step 1: Create PreparedAccessPlan	COMMENT
LOW	datafusion/datasource-parquet/src/sort.rs	854	// Step 2: Reverse the plan (this is the production code path)	COMMENT
LOW	datafusion/datasource-parquet/src/push_decoder.rs	296	// Step 1: drain a batch from the active reader if any.	COMMENT
LOW	datafusion/datasource-parquet/src/push_decoder.rs	318	// Step 2: when the decoder is sitting on a row-group boundary,	COMMENT
LOW	datafusion/datasource-parquet/src/push_decoder.rs	372	// Step 3: drive the decoder.	COMMENT
LOW	datafusion/spark/src/function/string/concat.rs	122	// Step 1: Check for NULL mask in incoming args	COMMENT
LOW	datafusion/spark/src/function/string/concat.rs	133	// Step 2: Delegate to DataFusion's concat	COMMENT
LOW	datafusion/spark/src/function/string/concat.rs	145	// Step 3: Apply NULL mask to result	COMMENT
LOW⚡	…s/examples/custom_data_source/adapter_serialization.rs	87	// Step 1: Create sample Parquet data in memory	COMMENT
LOW⚡	…s/examples/custom_data_source/adapter_serialization.rs	94	// Step 2: Set up session with custom adapter	COMMENT
LOW⚡	…s/examples/custom_data_source/adapter_serialization.rs	120	// Step 3: Create physical plan with filter	COMMENT
LOW⚡	…s/examples/custom_data_source/adapter_serialization.rs	129	// Step 4: Serialize with our custom codec	COMMENT
LOW⚡	…s/examples/custom_data_source/adapter_serialization.rs	140	// Step 5: Deserialize with our custom codec	COMMENT
LOW⚡	…s/examples/custom_data_source/adapter_serialization.rs	150	// Step 6: Execute and compare results	COMMENT
LOW	benchmarks/bench.sh	1348	# Step 1: Write a single sorted file with small (100K-row) RGs	COMMENT
LOW	benchmarks/bench.sh	1361	# Step 2: Redistribute RGs into 3 files with scrambled RG order.	COMMENT
LOW	benchmarks/bench.sh	1411	# Step 1: Write a single sorted file with small (100K-row) RGs	COMMENT
LOW	benchmarks/bench.sh	1424	# Step 2: Redistribute into 5 files with scrambled RG order.	COMMENT

AI Slop Vocabulary21 hits · 62 pts

Severity	File	Line	Snippet	Context
MEDIUM	…afusion/physical-plan/src/joins/symmetric_hash_join.rs	122	/// Prune build side when the new RecordBatch comes to the probe side. We utilize interval arithmetic	COMMENT
MEDIUM	datafusion/physical-plan/src/joins/stream_join_utils.rs	399	/// we can utilize sorting information while traversing the filter expression	COMMENT
MEDIUM	datafusion/physical-plan/src/windows/mod.rs	478	// we might be able to leverage this property.	COMMENT
MEDIUM	datafusion/physical-plan/src/windows/mod.rs	482	// monotonicity, we can utilize it to introduce new orderings.	COMMENT
MEDIUM	datafusion/physical-plan/src/windows/mod.rs	508	// utilize set-monotonicity since the set shrinks as the frame	COMMENT
MEDIUM	datafusion/physical-plan/src/sorts/partitioned_topk.rs	172	/// cases, the sort-based plan is more robust. Therefore, this optimization	COMMENT
MEDIUM	datafusion/physical-expr-common/src/physical_expr.rs	251	/// Expressions (should) implement this function and utilize the independence	COMMENT
MEDIUM	datafusion/core/tests/parquet/page_pruning.rs	1073	// to be robust against future changes in defaults as requested by reviewers.	COMMENT
MEDIUM	datafusion/core/src/lib.rs	560	//! If your system does not fully utilize either the CPU or network bandwidth	COMMENT
MEDIUM	datafusion/expr/src/udwf.rs	233	/// implemented directly. Instead, implement [`Eq`] and [`Hash`] and leverage the	COMMENT
MEDIUM	datafusion/expr/src/type_coercion/functions.rs	842	// f64 is chosen since most of the math functions utilize Signature::numeric,	COMMENT
MEDIUM	datafusion/common/src/config.rs	1373	/// repartitioning to increase parallelism to leverage more CPU cores	COMMENT
MEDIUM	datafusion/physical-optimizer/src/join_selection.rs	19	//! accommodate infinite sources and utilize statistical information (if there	COMMENT
MEDIUM	…imizer/src/ensure_requirements/enforce_distribution.rs	939	/// i.e. the operator can actually utilize parallelism.	COMMENT
MEDIUM	datafusion/execution/src/config.rs	418	/// add round robin repartition to increase parallelism to leverage more CPU cores.	COMMENT
MEDIUM	datafusion/functions/src/regex/regexpreplace.rs	619	// leverage it in order to determine whether the scalarity	COMMENT
MEDIUM	datafusion/sql/src/planner.rs	244	/// Struct to store the states used by the Planner. The Planner will leverage the states	COMMENT
MEDIUM	datafusion-examples/examples/query_planning/expr_api.rs	361	/// This function shows how to think about and leverage the analysis API	COMMENT
MEDIUM	datafusion-examples/examples/dataframe/cache_factory.rs	43	/// This example demonstrates how to leverage [CacheFactory] to implement custom caching strategies for dataframes in Da	COMMENT
MEDIUM	…xamples/examples/builtin_functions/function_factory.rs	37	/// This example shows how to utilize [FunctionFactory] to implement simple	COMMENT
MEDIUM	benchmarks/bench.sh	828	# its native DATAFUSION_* config before invoking bench.sh -- the harness reads	COMMENT

Redundant / Tautological Comments24 hits · 38 pts

Severity	File	Line	Snippet	Context
LOW	ci/scripts/check_asf_yaml_status_checks.py	67	# Check if pull_request trigger has path filters	COMMENT
LOW	ci/scripts/check_asf_yaml_status_checks.py	112	# Check if ALL workflows providing this job have path filters	COMMENT
LOW	datafusion/optimizer/src/common_subexpr_eliminate.rs	446	/// # Return value	COMMENT
LOW	datafusion/expr/src/udwf.rs	407	/// # Return value	COMMENT
LOW	datafusion/expr/src/udaf.rs	795	/// # Return value	COMMENT
LOW	datafusion/expr/src/higher_order_function.rs	675	/// # Return value	COMMENT
LOW	datafusion/expr/src/higher_order_function.rs	803	/// # Return value	COMMENT
LOW	datafusion/expr/src/udf.rs	1006	/// # Return value	COMMENT
LOW	datafusion/expr/src/utils.rs	1261	/// # Return value	COMMENT
LOW	datafusion/expr/src/utils.rs	1288	/// # Return value	COMMENT
LOW	datafusion/expr/src/logical_plan/plan.rs	621	/// # Return value	COMMENT
LOW	benchmarks/bench.sh	706	# Check if `web_site.parquet` exists in the TPCDS data directory to verify data presence	COMMENT
LOW	benchmarks/bench.sh	870	# Check if TPCDS data directory and representative file exists	COMMENT
LOW	benchmarks/bench.sh	1033	# Check if required files exist	COMMENT
LOW	benchmarks/queries/clickbench/update_queries.sh	47	# Check if download was successful	COMMENT
LOW	benchmarks/queries/clickbench/update_queries.sh	61	# Read the file line by line and create individual query files	COMMENT
LOW	benchmarks/queries/clickbench/update_queries.sh	74	# Increment counter	COMMENT
LOW	dev/release/add-branch-protection.sh	40	# Check if release number is provided	COMMENT
LOW	dev/release/add-branch-protection.sh	60	# Check if .asf.yaml exists	COMMENT
LOW	dev/release/add-branch-protection.sh	67	# Check if the branch exists in the official Apache DataFusion repository	COMMENT
LOW⚡	dev/release/add-branch-protection.sh	80	# Check if branch protection already exists for this release	COMMENT
LOW⚡	dev/release/add-branch-protection.sh	89	# Read the file and insert the new branch protection block	COMMENT
LOW	dev/release/add-branch-protection.sh	115	# Check if this belongs to the last branch block by looking ahead	COMMENT
LOW	dev/release/add-branch-protection.sh	137	# Check if awk succeeded	COMMENT

Self-Referential Comments8 hits · 30 pts

Severity	File	Line	Snippet	Context
MEDIUM	pre-commit.sh	20	# This file is git pre-commit hook.	COMMENT
MEDIUM	ci/scripts/utils/tool_versions.sh	20	# This file defines centralized tool versions used by CI and development scripts.	COMMENT
MEDIUM⚡	datafusion/expr/src/logical_plan/plan.rs	96	/// ## Creating a LogicalPlan from SQL:	COMMENT
MEDIUM⚡	datafusion/expr/src/logical_plan/plan.rs	100	/// ## Creating a LogicalPlan from the DataFrame API:	COMMENT
MEDIUM⚡	datafusion/expr/src/logical_plan/plan.rs	104	/// ## Creating a LogicalPlan programmatically:	COMMENT
MEDIUM	docs/scripts/update_committer_list.py	165	# Create a map of existing committers by apache id	COMMENT
MEDIUM	benchmarks/bench.sh	67	# Create the datasets for all benchmarks in $DATA_DIR	COMMENT
MEDIUM⚡	dev/release/add-branch-protection.sh	86	# Create a temporary file	COMMENT

Synthetic Comment Markers2 hits · 15 pts

Severity	File	Line	Snippet	Context
HIGH	datafusion/core/tests/parquet/page_pruning.rs	1073	// to be robust against future changes in defaults as requested by reviewers.	COMMENT
HIGH	docs/source/contributor-guide/index.md	206	### Why fully AI-generated PRs without understanding are not helpful	COMMENT

Excessive Try-Catch Wrapping9 hits · 15 pts

Severity	File	Line	Snippet	Context
LOW	docs/scripts/update_committer_list.py	63	except Exception as e:	CODE
MEDIUM	docs/scripts/update_committer_list.py	64	print(f"Error fetching ASF roster: {e}")	CODE
LOW	docs/scripts/update_committer_list.py	90	except Exception as e:	CODE
MEDIUM	docs/scripts/update_committer_list.py	91	print(f"Error fetching ASF people: {e}")	CODE
MEDIUM	docs/scripts/update_committer_list.py	153	print(f"Error: File {file_path} not found")	CODE
MEDIUM	docs/scripts/update_committer_list.py	219	print("Error: Could not find table markers in file")	CODE
LOW	docs/scripts/update_committer_list.py	237	except Exception as e:	CODE
MEDIUM	docs/scripts/update_committer_list.py	238	print(f"Error writing file: {e}")	CODE
MEDIUM	docs/scripts/update_committer_list.py	253	print(f"Error: Governance file not found at {governance_file}")	CODE

TODO Padding9 hits · 14 pts

Severity	File	Line	Snippet	Context
LOW	datafusion/physical-plan/src/aggregates/mod.rs	1095	// TODO: implement memory-limited path and remove this limitation	COMMENT
LOW	datafusion/physical-plan/src/aggregates/mod.rs	1108	// TODO: implement memory-limited path and remove this limitation	COMMENT
LOW	datafusion/physical-plan/src/aggregates/mod.rs	1121	// TODO: implement memory-limited path and remove this limitation	COMMENT
LOW	datafusion/physical-plan/src/aggregates/mod.rs	1136	// TODO: implement memory-limited path and remove this limitation	COMMENT
LOW	datafusion/physical-plan/src/aggregates/mod.rs	1149	// TODO: implement memory-limited path and remove this limitation	COMMENT
LOW	datafusion/physical-plan/src/aggregates/mod.rs	1164	// TODO: implement memory-limited path and remove this limitation	COMMENT
LOW	datafusion/physical-expr/src/async_scalar_function.rs	222	// TODO: implement this for scalar value input	COMMENT
LOW	datafusion/expr-common/src/signature.rs	1012	// TODO: implement for nested types	COMMENT
LOW	datafusion/expr-common/src/interval_arithmetic.rs	4092	// TODO: Add tests for non-exponential boundary aligned intervals too.	COMMENT

Slop Phrases5 hits · 14 pts

Severity	File	Line	Snippet	Context
MEDIUM	…ples/examples/custom_data_source/custom_file_format.rs	94	/// for demonstration purposes.	COMMENT
MEDIUM	…ples/examples/custom_data_source/custom_file_format.rs	175	/// for demonstration purposes.	COMMENT
MEDIUM	datafusion-examples/examples/udf/advanced_udf.rs	110	// For demonstration purposes we also implement the scalar / scalar	COMMENT
MEDIUM	datafusion-examples/examples/udf/advanced_udf.rs	237	// we will return an error for demonstration purposes	COMMENT
LOW	.github/workflows/rust.yml	68	# date with the manifest. When this fails, please make sure to commit	COMMENT

Unused Imports6 hits · 6 pts

Severity	File	Line	Context
LOW	docs/scripts/update_committer_list.py	26	CODE
LOW	docs/scripts/update_committer_list.py	30	CODE
LOW	docs/scripts/update_committer_list.py	30	CODE
LOW	benchmarks/compile_profile.py	30	CODE
LOW	benchmarks/compare.py	19	CODE
LOW	benchmarks/lineprotocol.py	79	CODE

Modern AI Meta-Vocabulary2 hits · 6 pts

Severity	File	Line	Snippet	Context
MEDIUM	benchmarks/bench.sh	21	# etc and orchestrates gathering data and run the benchmark binary in	COMMENT
MEDIUM	benchmarks/queries/clickbench/update_queries.sh	21	# etc and orchestrates gathering data and run the benchmark binary in	COMMENT

Hyper-Verbose Identifiers5 hits · 5 pts

Severity	File	Line	Snippet	Context
LOW	docs/rustdoc_trim.py	25	def remove_hashtag_lines_in_rust_blocks(markdown_content):	CODE
LOW	dev/update_arrow_deps.py	57	def update_commit_dependencies(dependencies, new_sha):	CODE
LOW	dev/update_arrow_deps.py	81	def update_version_cargo_toml(cargo_toml, new_version):	CODE
LOW	dev/update_datafusion_versions.py	78	def update_datafusion_version(cargo_toml: str, new_version: str):	CODE
LOW	dev/update_datafusion_versions.py	91	def update_downstream_versions(cargo_toml: str, new_version: str):	CODE

Example Usage Blocks3 hits · 4 pts

Severity	File	Line	Snippet	Context
LOW	datafusion/expr-common/src/interval_arithmetic.rs	1376	/// Example usage:	COMMENT
LOW	datafusion/common/src/metadata.rs	221	/// # Example usage	COMMENT
LOW	dev/release/add-branch-protection.sh	27	# Usage:	COMMENT

Fake / Example Data4 hits · 4 pts

Severity	File	Line	Snippet	Context
LOW	datafusion/catalog-listing/src/helpers.rs	595	"John Doe",	CODE
LOW	datafusion/catalog-listing/src/helpers.rs	869	evaluate_partition_prefix(partitions, &[col("a").eq(lit("John Doe"))]),	CODE
LOW	datafusion/catalog-listing/src/helpers.rs	876	&[col("a").eq(lit("foo")).and(col("b").eq(lit("John Doe")))],	CODE
LOW	dev/release/README.md	68	(gpg --list-sigs "John Doe" && gpg --armor --export "John Doe") >> KEYS	CODE

Deep Nesting4 hits · 4 pts

Severity	File	Line	Context
LOW	docs/scripts/update_committer_list.py	98	CODE
LOW	dev/update_arrow_deps.py	81	CODE
LOW	dev/release/download-python-wheels.py	35	CODE
LOW	dev/release/generate-changelog.py	36	CODE

Analysis Overview

What These Metrics Mean

Score History

Severity Breakdown

Directory Score Breakdown

Pattern Findings