29.0

Adjusted Signal

29.0

Raw Pattern Score

100%

Time Factor

2026-11-14

Last Push

16.3K

Stars

Python

Language

110.7K

Lines of Code

593

Files

2.1K

Pattern Hits

2026-11-14

Scan Date

0.25

HC Hit Rate

Risk Review · Payload v1

Use each vector to guide manual verification of the underlying findings.

Payload schema: 1.0.0 · Engine: 2.0.0 · Scoring: heuristic-density-v1 · Scan date: 2026-11-14.

Cognitive Ownership Risk 57.4/100 · HIGH

Can maintainers explain intent, assumptions, and trade-offs? 1301 finding(s).

Hallucinated Dependency & Supply Chain Risk 18.4/100 · LOW

Are imports declared and package/API names verified? 379 finding(s).

Agent Over-Engineering Ratio 54.9/100 · HIGH

Are wrappers, guards, and repeated structures necessary? 1298 finding(s).

PR Review Velocity Impact 77.3/100 · CRITICAL

Which changes need focused human review? 2111 finding(s).

What These Metrics Mean

Adjusted Signal: Legacy pattern score normalized per 1,000 scanned lines and multiplied by the repository-level time factor. It is not an estimate of AI authorship.
Raw Pattern Score: Sum of severity- and context-weighted pattern findings before applying the time factor.
Time Factor: A coarse multiplier based on repository last-push metadata. It does not identify when individual code was written.
Pattern Hits: Count of configured pattern findings. Review each location and surrounding code before acting.
HC Hit Rate: High- and critical-severity findings per scanned file. Severity is a configured detector weight, not proof of a vulnerability or authorship.
Lines / Files: Scan coverage denominators used by the legacy normalized score.

Pattern-Signal History

Run additional scans to build a history of the legacy pattern signal. Score changes are not a measure of AI adoption or code quality.

No multi-scan history yet — run the scanner again to build trend data.

Severity Breakdown

Groups findings by configured severity weight. Severity is not a confidence estimate, security rating, or proof that a package is missing. Verify imports against the project's manifests and inspect each finding in its local design context.

CRITICAL 2HIGH 147MEDIUM 211LOW 1751

Directory Score Breakdown

This horizontal bar chart shows where configured pattern scores accumulate across directories. Treat the bars as locations to inspect, not as evidence of authorship or code quality. Use the file-level findings to decide whether a directory needs a focused ownership, dependency, complexity, or review check.

Pattern Findings

The scanner identified 2111 distinct pattern matches across 23 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a detector match to inspect in context. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the configured finding weight (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — legacy comment-context matches receive a ×1.5 score weight. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier under the legacy scoring model.

Hyper-Verbose Identifiers843 hits · 864 pts

Severity	File	Line	Snippet	Context
LOW	tests/test_quoted_spans.py	36	def test_mixed_case_and_whitespace():	CODE
LOW	tests/test_quoted_spans.py	44	def test_no_quotes_returns_zero_with_zero_denominator():	CODE
LOW⚡	tests/unit/test_tool_call_accuracy.py	27	def test_is_sequence_aligned_perfect_match(self, tool_call_accuracy):	CODE
LOW⚡	tests/unit/test_tool_call_accuracy.py	33	def test_is_sequence_aligned_different_order(self, tool_call_accuracy):	CODE
LOW⚡	tests/unit/test_tool_call_accuracy.py	39	def test_is_sequence_aligned_different_length(self, tool_call_accuracy):	CODE
LOW⚡	tests/unit/test_tool_call_accuracy.py	45	def test_is_sequence_aligned_empty_sequences(self, tool_call_accuracy):	CODE
LOW⚡	tests/unit/test_tool_call_accuracy.py	50	async def test_perfect_match_scenario(self, tool_call_accuracy, mock_callbacks):	CODE
LOW	tests/unit/test_tool_call_accuracy.py	81	async def test_no_predicted_tool_calls(self, tool_call_accuracy, mock_callbacks):	CODE
LOW	tests/unit/test_tool_call_accuracy.py	95	async def test_sequence_misalignment(self, tool_call_accuracy, mock_callbacks):	CODE
LOW	tests/unit/test_tool_call_accuracy.py	121	async def test_length_mismatch_more_predicted(	CODE
LOW	tests/unit/test_tool_call_accuracy.py	148	async def test_length_mismatch_fewer_predicted(	CODE
LOW	tests/unit/test_tool_call_accuracy.py	175	async def test_partial_argument_match(self, tool_call_accuracy, mock_callbacks):	CODE
LOW	tests/unit/test_tool_call_accuracy.py	221	async def test_multiple_ai_messages(self, tool_call_accuracy, mock_callbacks):	CODE
LOW	tests/unit/test_tool_call_accuracy.py	251	async def test_empty_reference_tool_calls(self, tool_call_accuracy, mock_callbacks):	CODE
LOW	tests/unit/test_tool_call_accuracy.py	262	async def test_empty_reference_with_predictions(	CODE
LOW⚡	tests/unit/test_tool_call_accuracy.py	292	def test_strict_order_parameter_default(self):	CODE
LOW⚡	tests/unit/test_tool_call_accuracy.py	297	def test_strict_order_parameter_explicit(self):	CODE
LOW⚡	tests/unit/test_tool_call_accuracy.py	305	def test_is_sequence_aligned_flexible_mode(self):	CODE
LOW	tests/unit/test_tool_call_accuracy.py	319	def test_flexible_order_sorting_behavior(self):	CODE
LOW	tests/unit/test_tool_call_accuracy.py	353	def test_sorted_key_for_tool_call(self):	CODE
LOW	tests/unit/test_async_evaluation.py	11	def test_run_with_nest_asyncio_default(self):	CODE
LOW	tests/unit/test_async_evaluation.py	24	def test_run_without_nest_asyncio(self):	CODE
LOW	tests/unit/test_async_evaluation.py	41	def test_evaluate_with_nest_asyncio_default(self):	CODE
LOW	tests/unit/test_async_evaluation.py	64	def test_evaluate_allow_nest_asyncio_true(self):	CODE
LOW	tests/unit/test_async_evaluation.py	88	def test_evaluate_allow_nest_asyncio_false(self):	CODE
LOW	tests/unit/test_async_evaluation.py	119	def test_aevaluate_importable(self):	CODE
LOW	tests/unit/test_async_evaluation.py	126	def test_evaluate_has_allow_nest_asyncio_param(self):	CODE
LOW	tests/unit/test_async_evaluation.py	141	async def test_aevaluate_no_nest_asyncio_applied(self):	CODE
LOW	tests/unit/test_async_evaluation.py	191	async def test_aevaluate_in_running_loop(self):	CODE
LOW	tests/unit/test_metric.py	99	def test_metrics_with_definition(metric):	CODE
LOW	tests/unit/test_metric.py	190	def test_ignored_columns_validation():	CODE
LOW	tests/unit/test_metric.py	281	def test_metric_get_variables():	CODE
LOW	tests/unit/test_metric.py	328	def test_metric_response_model():	CODE
LOW	tests/unit/test_metric.py	335	def test_metric_prompt_conversion():	CODE
LOW	tests/unit/test_tool_call_f1_collections.py	42	async def test_partial_match_missing_prediction(self, tool_call_f1):	CODE
LOW	tests/unit/test_tool_call_f1_collections.py	67	async def test_partial_match_extra_prediction(self, tool_call_f1):	CODE
LOW	tests/unit/test_tool_call_f1_collections.py	157	async def test_only_predicted_no_reference(self, tool_call_f1):	CODE
LOW	tests/unit/test_tool_call_f1_collections.py	177	async def test_only_reference_no_predicted(self, tool_call_f1):	CODE
LOW	tests/unit/test_tool_call_f1_collections.py	220	async def test_duplicate_tool_calls_in_prediction(self, tool_call_f1):	CODE
LOW	tests/unit/test_run_config.py	41	def test_random_num_generator(	CODE
LOW	tests/unit/test_engine.py	50	def test_parallel_stores_transformations():	CODE
LOW	tests/unit/test_engine.py	57	def test_parallel_generate_execution_plan_aggregates(kg):	CODE
LOW⚡	tests/unit/test_engine.py	88	def test_get_desc_parallel_and_single():	CODE
LOW⚡	tests/unit/test_engine.py	97	def test_apply_transforms_single(kg):	CODE
LOW⚡	tests/unit/test_engine.py	106	def test_apply_transforms_list(kg):	CODE
LOW⚡	tests/unit/test_engine.py	116	def test_apply_transforms_parallel(kg):	CODE
LOW	tests/unit/test_engine.py	127	def test_apply_transforms_invalid():	CODE
LOW	tests/unit/test_datacompy_score_collections.py	112	async def test_columns_mode_partial_match(self):	CODE
LOW	tests/unit/test_datacompy_score_collections.py	125	async def test_invalid_reference_type(self):	CODE
LOW	tests/unit/test_datacompy_score_collections.py	133	async def test_invalid_response_type(self):	CODE
LOW	tests/unit/test_datacompy_score_collections.py	153	async def test_result_reason_contains_info(self):	CODE
LOW	tests/unit/test_datacompy_score_collections.py	178	async def test_csv_parse_error_returns_nan(self):	CODE
LOW	tests/unit/test_utils.py	27	def test_check_if_sum_is_close(values, close_to, num_places):	CODE
LOW⚡	tests/unit/test_utils.py	106	def test_create_nano_id_default_size(self):	CODE
LOW⚡	tests/unit/test_utils.py	112	def test_create_nano_id_custom_size(self):	CODE
LOW⚡	tests/unit/test_utils.py	119	def test_create_nano_id_uniqueness(self):	CODE
LOW⚡	tests/unit/test_utils.py	127	def test_create_nano_id_alphanumeric(self):	CODE
LOW	tests/unit/test_utils.py	148	def test_async_to_sync_with_kwargs(self):	CODE
LOW	tests/unit/test_utils.py	159	def test_async_to_sync_exception_handling(self):	CODE
LOW	tests/unit/test_utils.py	170	def test_async_to_sync_return_types(self):	CODE
783 more matches not shown…

Unused Imports358 hits · 344 pts

Severity	File	Line	Context
LOW	tests/conftest.py	1	CODE
LOW	tests/unit/test_datacompy_score_collections.py	10	CODE
LOW	tests/unit/test_datacompy_score_collections.py	13	CODE
LOW	tests/unit/test_tokenizers.py	3	CODE
LOW	tests/unit/test_chrf_score_collections.py	6	CODE
LOW	tests/unit/test_embeddings.py	1	CODE
LOW	tests/unit/test_simple.py	1	CODE
LOW	tests/unit/test_analytics.py	1	CODE
LOW	tests/unit/test_dspy_adapter.py	15	CODE
LOW	tests/unit/test_import.py	1	CODE
LOW	tests/unit/test_dspy_optimizer.py	9	CODE
LOW	tests/unit/integrations/test_ag_ui.py	3	CODE
LOW	tests/unit/integrations/test_ag_ui.py	598	CODE
LOW	tests/unit/llms/test_llm.py	1	CODE
LOW	tests/utils/__init__.py	7	CODE
LOW	tests/utils/__init__.py	7	CODE
LOW	tests/utils/__init__.py	7	CODE
LOW	tests/utils/__init__.py	7	CODE
LOW	tests/utils/__init__.py	7	CODE
LOW	tests/utils/__init__.py	14	CODE
LOW	tests/utils/__init__.py	14	CODE
LOW	tests/utils/__init__.py	14	CODE
LOW	tests/utils/__init__.py	14	CODE
LOW	tests/utils/__init__.py	14	CODE
LOW	tests/benchmarks/utils.py	1	CODE
LOW	tests/e2e/test_amnesty_in_ci.py	16	CODE
LOW	tests/e2e/test_fullflow.py	12	CODE
LOW	docs/alfred.py	1	CODE
LOW	examples/ragas_examples/__init__.py	17	CODE
LOW	examples/ragas_examples/text2sql/__init__.py	39	CODE
LOW	examples/ragas_examples/text2sql/__init__.py	39	CODE
LOW	examples/ragas_examples/text2sql/__init__.py	40	CODE
LOW	examples/ragas_examples/text2sql/__init__.py	40	CODE
LOW	examples/ragas_examples/text2sql/__init__.py	41	CODE
LOW	examples/ragas_examples/text2sql/__init__.py	42	CODE
LOW	examples/ragas_examples/text2sql/__init__.py	42	CODE
LOW	examples/ragas_examples/text2sql/__init__.py	42	CODE
LOW	examples/ragas_examples/text2sql/data_utils.py	14	CODE
LOW	examples/ragas_examples/judge_alignment/__init__.py	13	CODE
LOW	examples/ragas_examples/judge_alignment/__init__.py	13	CODE
LOW	examples/ragas_examples/judge_alignment/__init__.py	13	CODE
LOW	examples/ragas_examples/judge_alignment/__init__.py	13	CODE
LOW	examples/ragas_examples/judge_alignment/__init__.py	13	CODE
LOW	src/ragas/config.py	1	CODE
LOW	src/ragas/evaluation.py	1	CODE
LOW	src/ragas/__init__.py	1	CODE
LOW	src/ragas/__init__.py	2	CODE
LOW	src/ragas/__init__.py	2	CODE
LOW	src/ragas/__init__.py	2	CODE
LOW	src/ragas/__init__.py	3	CODE
LOW	src/ragas/__init__.py	3	CODE
LOW	src/ragas/__init__.py	4	CODE
LOW	src/ragas/__init__.py	4	CODE
LOW	src/ragas/__init__.py	4	CODE
LOW	src/ragas/__init__.py	5	CODE
LOW	src/ragas/__init__.py	5	CODE
LOW	src/ragas/__init__.py	6	CODE
LOW	src/ragas/__init__.py	6	CODE
LOW	src/ragas/__init__.py	6	CODE
LOW	src/ragas/__init__.py	7	CODE
298 more matches not shown…

Cross-File Repetition68 hits · 340 pts

Severity	File	Line	Snippet	Context
HIGH	tests/unit/test_tool_call_accuracy.py	0	test perfect match scenario with identical tool calls.	STRING
HIGH	tests/unit/test_tool_call_f1_collections.py	0	test perfect match scenario with identical tool calls.	STRING
HIGH	tests/unit/test_tool_call_accuracy_collections.py	0	test perfect match scenario with identical tool calls.	STRING
HIGH	…sts/unit/test_instance_specific_rubrics_collections.py	0	mock implementation of instructorbaseragasllm for testing.	STRING
HIGH	tests/unit/test_sql_semantic_equivalence_collections.py	0	mock implementation of instructorbaseragasllm for testing.	STRING
HIGH	tests/unit/test_domain_specific_rubrics_collections.py	0	mock implementation of instructorbaseragasllm for testing.	STRING
HIGH	…2e/metrics_migration/test_answer_accuracy_migration.py	0	create a modern instructor llm for v2 implementation.	STRING
HIGH	…/metrics_migration/test_noise_sensitivity_migration.py	0	create a modern instructor llm for v2 implementation.	STRING
HIGH	…/e2e/metrics_migration/test_summary_score_migration.py	0	create a modern instructor llm for v2 implementation.	STRING
HIGH	…/metrics_migration/test_context_precision_migration.py	0	create a modern instructor llm for v2 implementation.	STRING
HIGH	…e/metrics_migration/test_answer_relevancy_migration.py	0	create a modern instructor llm for v2 implementation.	STRING
HIGH	…rics_migration/test_response_groundedness_migration.py	0	create a modern instructor llm for v2 implementation.	STRING
HIGH	…metrics_migration/test_answer_correctness_migration.py	0	create a modern instructor llm for v2 implementation.	STRING
HIGH	…etrics_migration/test_factual_correctness_migration.py	0	create a modern instructor llm for v2 implementation.	STRING
HIGH	…/metrics_migration/test_context_relevance_migration.py	0	create a modern instructor llm for v2 implementation.	STRING
HIGH	…s/e2e/metrics_migration/test_faithfulness_migration.py	0	create a modern instructor llm for v2 implementation.	STRING
HIGH	…2e/metrics_migration/test_answer_accuracy_migration.py	0	e2e test that legacy and v2 produce similar scores.	STRING
HIGH	…/metrics_migration/test_noise_sensitivity_migration.py	0	e2e test that legacy and v2 produce similar scores.	STRING
HIGH	…/e2e/metrics_migration/test_summary_score_migration.py	0	e2e test that legacy and v2 produce similar scores.	STRING
HIGH	…rics_migration/test_response_groundedness_migration.py	0	e2e test that legacy and v2 produce similar scores.	STRING
HIGH	…etrics_migration/test_factual_correctness_migration.py	0	e2e test that legacy and v2 produce similar scores.	STRING
HIGH	…/metrics_migration/test_context_relevance_migration.py	0	e2e test that legacy and v2 produce similar scores.	STRING
HIGH	…s/e2e/metrics_migration/test_faithfulness_migration.py	0	e2e test that legacy and v2 produce similar scores.	STRING
HIGH	…/metrics_migration/test_context_precision_migration.py	0	e2e test that legacy and v2 produce similar scores.	STRING
HIGH	…ts/e2e/metrics_migration/plan-for-metrics-migration.md	0	e2e test that legacy and v2 produce similar scores.	STRING
HIGH	…2e/metrics_migration/test_answer_accuracy_migration.py	0	test that migration requirements are properly documented.	STRING
HIGH	…/metrics_migration/test_noise_sensitivity_migration.py	0	test that migration requirements are properly documented.	STRING
HIGH	…/e2e/metrics_migration/test_summary_score_migration.py	0	test that migration requirements are properly documented.	STRING
HIGH	…/metrics_migration/test_context_precision_migration.py	0	test that migration requirements are properly documented.	STRING
HIGH	…rics_migration/test_response_groundedness_migration.py	0	test that migration requirements are properly documented.	STRING
HIGH	…etrics_migration/test_factual_correctness_migration.py	0	test that migration requirements are properly documented.	STRING
HIGH	…/metrics_migration/test_context_relevance_migration.py	0	test that migration requirements are properly documented.	STRING
HIGH	…s/e2e/metrics_migration/test_faithfulness_migration.py	0	test that migration requirements are properly documented.	STRING
HIGH	…e2e/metrics_migration/test_context_recall_migration.py	0	e2e test that legacy and v2 implementations produce similar scores with real llm.	STRING
HIGH	…rics_migration/test_context_entity_recall_migration.py	0	e2e test that legacy and v2 implementations produce similar scores with real llm.	STRING
HIGH	…e/metrics_migration/test_answer_relevancy_migration.py	0	e2e test that legacy and v2 implementations produce similar scores with real llm.	STRING
HIGH	…metrics_migration/test_answer_correctness_migration.py	0	e2e test that legacy and v2 implementations produce similar scores with real llm.	STRING
HIGH	tests/e2e/metrics_migration/test_rouge_migration.py	0	real-world sample reference and response texts for testing.	STRING
HIGH	tests/e2e/metrics_migration/test_bleu_migration.py	0	real-world sample reference and response texts for testing.	STRING
HIGH	tests/e2e/metrics_migration/test_string_migration.py	0	real-world sample reference and response texts for testing.	STRING
HIGH	tests/e2e/metrics_migration/test_rouge_migration.py	0	e2e test that legacy and v2 class implementations produce identical scores.	STRING
HIGH	tests/e2e/metrics_migration/test_bleu_migration.py	0	e2e test that legacy and v2 class implementations produce identical scores.	STRING
HIGH	tests/e2e/metrics_migration/test_string_migration.py	0	e2e test that legacy and v2 class implementations produce identical scores.	STRING
HIGH	tests/e2e/metrics_migration/test_rouge_migration.py	0	compare performance characteristics between legacy and v2 class.	STRING
HIGH	tests/e2e/metrics_migration/test_bleu_migration.py	0	compare performance characteristics between legacy and v2 class.	STRING
HIGH	tests/e2e/metrics_migration/test_string_migration.py	0	compare performance characteristics between legacy and v2 class.	STRING
HIGH	…e/metrics_migration/test_answer_relevancy_migration.py	0	create legacy embeddings for legacy implementation.	STRING
HIGH	…metrics_migration/test_answer_correctness_migration.py	0	create legacy embeddings for legacy implementation.	STRING
HIGH	…etrics_migration/test_semantic_similarity_migration.py	0	create legacy embeddings for legacy implementation.	STRING
HIGH	src/ragas/cache.py	0	define how pydantic generates a schema for baseragasembeddings.	STRING
HIGH	src/ragas/losses.py	0	define how pydantic generates a schema for baseragasembeddings.	STRING
HIGH	src/ragas/embeddings/base.py	0	define how pydantic generates a schema for baseragasembeddings.	STRING
HIGH	src/ragas/metrics/discrete.py	0	calculate the correlation between gold labels and predictions. this is a placeholder method and should be implemented ba	STRING
HIGH	src/ragas/metrics/numeric.py	0	calculate the correlation between gold labels and predictions. this is a placeholder method and should be implemented ba	STRING
HIGH	src/ragas/metrics/ranking.py	0	calculate the correlation between gold labels and predictions. this is a placeholder method and should be implemented ba	STRING
HIGH	src/ragas/metrics/discrete.py	0	load a rankingmetric from a json file. parameters: ----------- path : str file path to load from. supports .gz compresse	STRING
HIGH	src/ragas/metrics/numeric.py	0	load a rankingmetric from a json file. parameters: ----------- path : str file path to load from. supports .gz compresse	STRING
HIGH	src/ragas/metrics/ranking.py	0	load a rankingmetric from a json file. parameters: ----------- path : str file path to load from. supports .gz compresse	STRING
HIGH	src/ragas/metrics/collections/_rouge_score.py	0	calculate chrf score asynchronously. args: reference: the reference/ground truth text response: the response text to eva	STRING
HIGH	src/ragas/metrics/collections/_string.py	0	calculate chrf score asynchronously. args: reference: the reference/ground truth text response: the response text to eva	STRING
8 more matches not shown…

Modern AI Meta-Vocabulary102 hits · 290 pts

Severity	File	Line	Snippet	Context
MEDIUM	README.md	81	# Create a RAG evaluation project	COMMENT
MEDIUM⚡	tests/unit/test_simple_llm_metric_persistence.py	171	# Create a mock embedding model for testing	COMMENT
MEDIUM	tests/unit/test_simple_llm_metric_persistence.py	208	# Save (should warn about embedding model)	COMMENT
MEDIUM	tests/unit/test_simple_llm_metric_persistence.py	212	# Load (provide embedding model)	COMMENT
MEDIUM	tests/unit/prompt/test_dynamic_few_shot_prompt.py	178	# Test save with warning about embedding model	COMMENT
MEDIUM	tests/unit/prompt/test_dynamic_few_shot_prompt.py	192	# Test load with embedding model provided	COMMENT
MEDIUM	tests/unit/prompt/test_dynamic_few_shot_prompt.py	225	# Load with new embedding model	COMMENT
MEDIUM	tests/unit/prompt/test_dynamic_few_shot_prompt.py	237	# Verify embedding model was called during initialization	COMMENT
MEDIUM	tests/unit/prompt/test_dynamic_few_shot_prompt.py	340	# Test load without embedding model shows warning (when embedding_model_info exists but no model provided)	COMMENT
MEDIUM	tests/unit/prompt/test_dynamic_few_shot_prompt.py	341	# First save a prompt with only embedding model info (no response model to avoid error)	COMMENT
MEDIUM	tests/unit/prompt/test_dynamic_few_shot_prompt.py	352	# Now test load without providing embedding model - should show warning	COMMENT
MEDIUM	docs/getstarted/quickstart.md	127	├── rag.py # Your RAG application	CODE
MEDIUM	docs/getstarted/rag_testset_generation.md	1	# Testset Generation for RAG	COMMENT
MEDIUM	docs/getstarted/rag_eval.md	1	# Evaluate a simple RAG system	COMMENT
MEDIUM	docs/getstarted/rag_eval.md	23	### Build a Simple RAG System	COMMENT
MEDIUM	docs/getstarted/rag_eval.md	94	# Initialize RAG instance	COMMENT
MEDIUM	docs/getstarted/evals.md	34	├── rag.py # Your RAG/LLM application	CODE
MEDIUM	docs/getstarted/evals.md	69	response="", # Will be filled by querying RAG	CODE
MEDIUM	docs/extra/components/choose_evaluator_llm.md	179	model="models/embedding-001", # Google's text embedding model	CODE
MEDIUM	docs/extra/components/choose_generator_llm.md	180	model="models/embedding-001", # Google's text embedding model	CODE
MEDIUM	docs/concepts/index.md	28	Metrics for evaluating [RAG](metrics/available_metrics/index.md#retrieval-augmented-generation), [Agentic workflows]	CODE
MEDIUM	docs/concepts/test_data_generation/rag.md	1	# Testset Generation for RAG	COMMENT
MEDIUM	docs/concepts/test_data_generation/rag.md	5	## Query types in RAG	COMMENT
MEDIUM	docs/concepts/test_data_generation/rag.md	53	### Specific vs. Abstract Queries in a RAG	COMMENT
MEDIUM	docs/tutorials/rag.md	1	# Evaluate a simple RAG system	COMMENT
MEDIUM	docs/howtos/observability.md	106	### 5. Build Your RAG Application With LlamaIndex	COMMENT
MEDIUM	docs/howtos/customizations/customize_models.md	75	# system_prompt="You are a helpful assistant that evaluates RAG systems."	COMMENT
MEDIUM	docs/howtos/customizations/customize_models.md	120	# system_prompt="You are a helpful assistant that evaluates RAG systems."	COMMENT
MEDIUM	docs/howtos/customizations/customize_models.md	147	"embeddings": "amazon.titan-embed-text-v2:0", # Your embedding model ID	CODE
MEDIUM	docs/howtos/customizations/customize_models.md	166	# system_prompt="You are a helpful assistant that evaluates RAG systems."	COMMENT
MEDIUM	docs/howtos/customizations/optimizers/index.md	115	max_bootstrapped_demos=10, # Auto-generated few-shot examples	CODE
MEDIUM	docs/howtos/cli/improve_rag.md	1	# Improve RAG Quickstart	COMMENT
MEDIUM	docs/howtos/cli/improve_rag.md	37	### Naive RAG Mode (Default)	COMMENT
MEDIUM	docs/howtos/cli/improve_rag.md	43	### Agentic RAG Mode	COMMENT
MEDIUM⚡	docs/howtos/cli/improve_rag.md	71	├── rag.py # RAG implementation (naive & agentic)	CODE
MEDIUM⚡	docs/howtos/cli/improve_rag.md	71	├── rag.py # RAG implementation (naive & agentic)	CODE
MEDIUM⚡	docs/howtos/cli/improve_rag.md	80	## Understanding the RAG Modes	COMMENT
MEDIUM⚡	docs/howtos/cli/improve_rag.md	82	### Naive RAG	COMMENT
MEDIUM	docs/howtos/cli/improve_rag.md	107	### Agentic RAG	COMMENT
MEDIUM	docs/howtos/cli/improve_rag.md	150	### The RAG Implementation (`rag.py`)	COMMENT
MEDIUM	docs/howtos/cli/improve_rag.md	167	#### RAG Class	COMMENT
MEDIUM	docs/howtos/cli/improve_rag.md	284	# Run agentic mode	COMMENT
MEDIUM	docs/howtos/cli/rag_eval.md	1	# RAG Evaluation Quickstart	COMMENT
MEDIUM	docs/howtos/cli/rag_eval.md	94	├── rag.py # RAG application implementation	CODE
MEDIUM	docs/howtos/cli/rag_eval.md	105	### The RAG Application (`rag.py`)	COMMENT
MEDIUM	docs/howtos/cli/rag_eval.md	120	# Query the RAG system	COMMENT
MEDIUM	docs/howtos/cli/rag_eval.md	227	### Use Your Own RAG System	COMMENT
MEDIUM	docs/howtos/cli/rag_eval.md	240	# Call your RAG system	COMMENT
MEDIUM	docs/howtos/cli/index.md	43	# Create a RAG evaluation project	COMMENT
MEDIUM	docs/howtos/cli/index.md	89	### RAG & Retrieval	COMMENT
MEDIUM	docs/howtos/integrations/griptape.md	57	### Setting up the RAG Engine	COMMENT
MEDIUM	docs/howtos/integrations/griptape.md	71	# Create a basic RAG pipeline	COMMENT
MEDIUM	docs/howtos/integrations/griptape.md	92	### Testing Our RAG Pipeline	COMMENT
MEDIUM	docs/howtos/integrations/griptape.md	305	- [ResponseGroundedness](../../concepts/metrics/available_metrics/nvidia_metrics.md#response-groundedness): Measures whe	CODE
MEDIUM	docs/howtos/integrations/_arize.md	101	## 5. Build Your RAG Application With LlamaIndex	COMMENT
MEDIUM	docs/howtos/integrations/oci_genai.md	76	embeddings=None, # You can use any embedding model	CODE
MEDIUM	docs/howtos/integrations/_openlayer.md	2	## Evaluating RAG pipelines with Openlayer and Ragas	COMMENT
MEDIUM	docs/howtos/integrations/_openlayer.md	50	## Building RAG	COMMENT
MEDIUM	docs/howtos/integrations/llamaindex_agents.md	669	### Evaluating using Ragas RAG Metrics	COMMENT
MEDIUM	docs/howtos/integrations/_haystack.md	36	## Initialize components for RAG pipeline	COMMENT
42 more matches not shown…

Excessive Try-Catch Wrapping198 hits · 218 pts

Severity	File	Line	Snippet	Context
LOW	tests/unit/test_async_evaluation.py	180	except Exception:	CODE
LOW	tests/unit/test_async_evaluation.py	214	except Exception as e:	CODE
LOW⚡	tests/unit/test_datatable_inheritance.py	269	except Exception as e:	CODE
LOW⚡	tests/unit/test_datatable_inheritance.py	284	except Exception as e:	CODE
LOW	tests/unit/test_cancellation.py	242	except Exception as e:	CODE
LOW	tests/unit/backends/test_local_csv.py	330	except Exception as e:	CODE
LOW	tests/unit/backends/test_local_csv.py	467	except Exception as e:	CODE
MEDIUM	tests/unit/backends/test_inmemory.py	702	def save_dataset(dataset_name, data):	CODE
MEDIUM	tests/unit/backends/test_inmemory.py	752	def read_dataset():	CODE
MEDIUM	tests/unit/backends/test_inmemory.py	799	def read_operation():	CODE
MEDIUM	tests/unit/backends/test_inmemory.py	806	def write_operation(dataset_name, data):	CODE
LOW	tests/unit/backends/test_inmemory.py	706	except Exception as e:	CODE
LOW	tests/unit/backends/test_inmemory.py	756	except Exception as e:	CODE
LOW	tests/unit/backends/test_inmemory.py	803	except Exception as e:	CODE
LOW	tests/unit/backends/test_inmemory.py	810	except Exception as e:	CODE
LOW	tests/unit/backends/test_local_jsonl.py	470	except Exception as e:	CODE
LOW	tests/unit/backends/test_local_jsonl.py	512	except Exception as e:	CODE
LOW	tests/utils/llm_setup.py	68	except Exception as e:	CODE
LOW	tests/utils/llm_setup.py	107	except Exception as e:	CODE
LOW	tests/utils/llm_setup.py	131	except Exception as e:	CODE
LOW	tests/utils/llm_setup.py	181	except Exception as e:	CODE
LOW	tests/utils/metric_comparison.py	130	except Exception as e:	CODE
MEDIUM	tests/utils/metric_comparison.py	131	print(f"Error processing sample: {e}")	CODE
LOW⚡	tests/utils/metric_comparison.py	287	except Exception as e:	CODE
LOW⚡	tests/utils/metric_comparison.py	296	except Exception as e2:	CODE
LOW	tests/e2e/test_dataset_utils.py	82	except Exception as e:	CODE
LOW	tests/e2e/test_dataset_utils.py	107	except Exception as e:	CODE
LOW	tests/e2e/test_dspy_integration.py	109	except Exception as e:	CODE
LOW	…2e/metrics_migration/test_answer_accuracy_migration.py	23	except Exception as e:	CODE
LOW	…2e/metrics_migration/test_answer_accuracy_migration.py	40	except Exception as e:	CODE
LOW	…2e/metrics_migration/test_answer_accuracy_migration.py	86	except Exception as e:	CODE
LOW	…2e/metrics_migration/test_answer_accuracy_migration.py	105	except Exception as e:	CODE
LOW	…/metrics_migration/test_noise_sensitivity_migration.py	54	except Exception as e:	CODE
LOW	…/metrics_migration/test_noise_sensitivity_migration.py	69	except Exception as e:	CODE
LOW	…/e2e/metrics_migration/test_summary_score_migration.py	52	except Exception as e:	CODE
LOW	…/e2e/metrics_migration/test_summary_score_migration.py	67	except Exception as e:	CODE
LOW	tests/e2e/metrics_migration/conftest.py	26	except Exception as e:	CODE
LOW	tests/e2e/metrics_migration/conftest.py	39	except Exception as e:	CODE
LOW	tests/e2e/metrics_migration/conftest.py	52	except Exception as e:	CODE
LOW	tests/e2e/metrics_migration/conftest.py	68	except Exception as e:	CODE
LOW	…rics_migration/test_context_entity_recall_migration.py	70	except Exception as e:	CODE
LOW	…rics_migration/test_context_entity_recall_migration.py	85	except Exception as e:	CODE
LOW	…/metrics_migration/test_context_precision_migration.py	71	except Exception as e:	CODE
LOW	…/metrics_migration/test_context_precision_migration.py	86	except Exception as e:	CODE
LOW	…e/metrics_migration/test_answer_relevancy_migration.py	54	except Exception as e:	CODE
LOW	…e/metrics_migration/test_answer_relevancy_migration.py	69	except Exception as e:	CODE
LOW	…e/metrics_migration/test_answer_relevancy_migration.py	82	except Exception as e:	CODE
LOW	…e/metrics_migration/test_answer_relevancy_migration.py	107	except Exception as e:	CODE
LOW	…rics_migration/test_response_groundedness_migration.py	67	except Exception as e:	CODE
LOW	…rics_migration/test_response_groundedness_migration.py	83	except Exception as e:	CODE
LOW	…metrics_migration/test_answer_correctness_migration.py	62	except Exception as e:	CODE
LOW	…metrics_migration/test_answer_correctness_migration.py	77	except Exception as e:	CODE
LOW	…metrics_migration/test_answer_correctness_migration.py	89	except Exception as e:	CODE
LOW	…metrics_migration/test_answer_correctness_migration.py	111	except Exception as e:	CODE
LOW	…etrics_migration/test_factual_correctness_migration.py	59	except Exception as e:	CODE
LOW	…etrics_migration/test_factual_correctness_migration.py	74	except Exception as e:	CODE
LOW	…/metrics_migration/test_context_relevance_migration.py	23	except Exception as e:	CODE
LOW	…/metrics_migration/test_context_relevance_migration.py	40	except Exception as e:	CODE
LOW	…/metrics_migration/test_context_relevance_migration.py	93	except Exception as e:	CODE
LOW	…/metrics_migration/test_context_relevance_migration.py	112	except Exception as e:	CODE
138 more matches not shown…

Docstring Block Structure34 hits · 170 pts

Severity	File	Line	Snippet	Context
HIGH	tests/utils/llm_setup.py	13	Check if required API key is set. Args: provider: The provider to check for (default: "openai") Return	STRING
HIGH	tests/utils/llm_setup.py	44	Create an LLM instance using the unified llm_factory. Args: model: The model name to use **kwargs:	STRING
HIGH	tests/utils/llm_setup.py	78	Create an LLM instance using the unified llm_factory. Args: provider: The LLM provider (default: "openai")	STRING
HIGH	tests/utils/llm_setup.py	112	Create legacy embeddings for old-style metrics. Args: model: The embedding model name to use **kwar	STRING
HIGH	tests/utils/llm_setup.py	144	Create modern embeddings for v2 metrics. Args: provider: The embeddings provider (e.g., "openai") m	STRING
HIGH	docs/howtos/integrations/_langgraph_agent_evaluation.md	94	Fetches the current per gram price of the specified metal. Args: metal_name : The name of the metal (e.g.,	STRING
HIGH	examples/ragas_examples/text2sql/data_utils.py	150	Load JSON data and remove duplicates. Args: input_file: Path to the BookSQL train.json file	STRING
HIGH	src/ragas/dataset.py	111	Resolve backend from string or return existing BaseBackend instance. Args: backend: Either a BaseBa	STRING
HIGH	src/ragas/dataset.py	163	Load dataset with optional validation. Args: name: Name of the dataset to load backend:	STRING
HIGH	src/ragas/dataset.py	210	Create a DataTable from a pandas DataFrame. Args: dataframe: The pandas DataFrame to convert	STRING
HIGH	src/ragas/metrics/discrete.py	95	Load a DiscreteMetric from a JSON file. Parameters: ----------- path : str	STRING
HIGH	src/ragas/metrics/numeric.py	99	Load a NumericMetric from a JSON file. Parameters: ----------- path : str	STRING
HIGH	src/ragas/metrics/ranking.py	102	Load a RankingMetric from a JSON file. Parameters: ----------- path : str	STRING
HIGH	src/ragas/metrics/base.py	1085	Read just the metric type from a saved metric file. Parameters: ----------- path : str	STRING
HIGH	src/ragas/metrics/base.py	1152	Load a metric from a JSON file. Parameters: ----------- path : str File pa	STRING
HIGH	…as/metrics/collections/multi_modal_relevance/metric.py	91	Calculate multimodal relevance score. Args: user_input: The user's question or input	STRING
HIGH	src/ragas/metrics/collections/summary_score/metric.py	112	Calculate summary score using multi-step pipeline. Args: reference_contexts: The original	STRING
HIGH	…etrics/collections/instance_specific_rubrics/metric.py	86	Score a response using instance-specific rubric criteria. Args: rubrics: Dictionary mappin	STRING
HIGH	…ragas/metrics/collections/answer_correctness/metric.py	87	Initialize AnswerCorrectness metric with required components. Args: llm: Modern instructor	STRING
HIGH	…metrics/collections/multi_modal_faithfulness/metric.py	89	Calculate multimodal faithfulness score. Args: response: The response to evaluate for fait	STRING
HIGH	src/ragas/embeddings/utils.py	10	Run an async coroutine in the current event loop if possible. This handles Jupyter environments correctly by using	STRING
HIGH	src/ragas/embeddings/utils.py	134	Validate and normalize text inputs. Args: texts: Single text or list of texts Returns: List of	STRING
HIGH	src/ragas/embeddings/utils.py	164	Safely import a module with helpful error message. Args: module_name: Name of the module to import	STRING
HIGH	src/ragas/embeddings/google_provider.py	74	Resolve the client to use for embeddings. For Vertex AI: Returns the client as-is (must be provided). F	STRING
HIGH	src/ragas/backends/inmemory.py	39	Load dataset by name. Args: name: Dataset identifier Returns: List of dictiona	STRING
HIGH	src/ragas/backends/inmemory.py	57	Load experiment by name. Args: name: Experiment identifier Returns: List of di	STRING
HIGH	src/ragas/backends/base.py	48	Load dataset by name. Args: name: Dataset identifier (alphanumeric, hyphens, underscores recommende	STRING
HIGH	src/ragas/backends/base.py	68	Load experiment by name. Args: name: Experiment identifier (alphanumeric, hyphens, underscores reco	STRING
HIGH	src/ragas/prompt/simple_prompt.py	204	Load a prompt from a JSON file. Parameters: ----------- path : str File pa	STRING
HIGH	src/ragas/prompt/dynamic_few_shot.py	344	Load a DynamicFewShotPrompt from a JSON file. Parameters: ----------- path : str	STRING
HIGH	src/ragas/llms/oci_genai_wrapper.py	322	Factory function to create an OCI Gen AI LLM instance. Args: model_id: The OCI model ID to use for gen	STRING
HIGH	src/ragas/llms/base.py	633	Create an LLM instance for structured output generation with automatic adapter selection. Supports multiple LL	STRING
HIGH	src/ragas/llms/adapters/__init__.py	13	Get adapter by name. Args: name: Adapter name ("instructor" or "litellm") Returns: Struct	STRING
HIGH	src/ragas/llms/adapters/instructor.py	21	Create InstructorLLM instance by patching client with Instructor. Args: client: Pre-initia	STRING

Self-Referential Comments61 hits · 169 pts

Severity	File	Line	Snippet	Context
MEDIUM	tests/unit/test_multi_modal_relevance_collections.py	45	# Create a temporary image	COMMENT
MEDIUM	tests/unit/test_metric_decorators.py	661	# Create a mock custom class similar to InstructorLLM	COMMENT
MEDIUM⚡	tests/unit/test_multi_modal_faithfulness_collections.py	67	# Create a temporary image file	COMMENT
MEDIUM	tests/unit/test_multi_modal_faithfulness_collections.py	96	# Create a small valid PNG in base64	COMMENT
MEDIUM	tests/unit/test_multi_modal_faithfulness_collections.py	128	# Create a temporary image	COMMENT
MEDIUM	tests/unit/test_multi_modal_faithfulness_collections.py	209	# Create a mock LLM that won't be called	COMMENT
MEDIUM⚡	tests/unit/test_experiment.py	378	# Create a fresh instance to avoid state from other tests	COMMENT
MEDIUM	tests/unit/test_experiment.py	402	# Create a nested directory structure with .git at the top	COMMENT
MEDIUM	tests/unit/test_knowledge_graph_save.py	5	# Create a knowledge graph with special characters	COMMENT
MEDIUM⚡	tests/unit/test_simple_llm_metric_persistence.py	171	# Create a mock embedding model for testing	COMMENT
MEDIUM	tests/unit/test_single_hop_query_synthesizer.py	91	# Create a node with tuple-formatted entities (the problematic case)	COMMENT
MEDIUM	tests/unit/test_single_hop_query_synthesizer.py	164	# Create a node with string-formatted entities (the normal case)	COMMENT
MEDIUM	tests/unit/test_graph.py	7	# Create a knowledge graph with special characters	COMMENT
MEDIUM	tests/unit/test_knowledge_graph_clusters.py	19	# Create a random UUID internally	COMMENT
MEDIUM	tests/unit/backends/test_local_csv.py	454	# Create a malformed CSV file manually	COMMENT
MEDIUM	tests/unit/backends/test_inmemory.py	378	# Create a large dataset (1000 items)	COMMENT
MEDIUM	tests/unit/backends/test_gdrive_backend.py	15	# Create a mock HttpError for testing when Google API isn't available	COMMENT
MEDIUM	tests/unit/backends/test_local_jsonl.py	456	# Create a malformed JSONL file manually	COMMENT
MEDIUM	tests/unit/integrations/test_ag_ui.py	558	# Create a mock event with dict delta (bypassing Pydantic validation)	COMMENT
MEDIUM	tests/unit/prompt/test_prompt_save_load.py	27	# Create a prompt with examples	COMMENT
MEDIUM	tests/unit/prompt/test_prompt_save_load.py	143	# Create a different model with different schema	COMMENT
MEDIUM	tests/unit/llms/test_llm.py	204	# Create a mock LLM that would normally support multiple completions	COMMENT
MEDIUM	tests/unit/llms/test_instructor_factory.py	350	# Create a mock client with OpenAI-style API (chat.completions.create)	COMMENT
MEDIUM⚡	tests/unit/llms/test_adapters.py	133	# Create a mock client that appears to be from litellm module	COMMENT
MEDIUM	tests/e2e/test_dataset_utils.py	86	# Create a local dataset from sample data	COMMENT
MEDIUM	tests/e2e/test_dataset_utils.py	111	# Create a local dataset from sample data	COMMENT
MEDIUM	docs/alfred.py	60	# Create an argument parser	STRING
MEDIUM	examples/ragas_examples/llamaIndex_agent_evals/evals.py	143	# Create a dataset	STRING
MEDIUM	examples/ragas_examples/agent_evals/evals.py	22	# Create a dataset	COMMENT
MEDIUM	examples/ragas_examples/prompt_evals/evals.py	32	# Create a dataset	COMMENT
MEDIUM	src/ragas/_analytics.py	286	# Create a global batcher instance	COMMENT
MEDIUM	src/ragas/cli.py	26	# Create a callback for the main app to make it a group	COMMENT
MEDIUM	src/ragas/cli.py	540	# Create a table of templates	COMMENT
MEDIUM	src/ragas/cli.py	699	# Create a README.md with setup instructions	COMMENT
MEDIUM	src/ragas/cli.py	478	ragas quickstart rag_eval # Create a RAG evaluation project	STRING
MEDIUM	src/ragas/utils.py	70	# Create a handler for the asyncio logger	COMMENT
MEDIUM	src/ragas/utils.py	350	# Create a formatter with the custom formatter	COMMENT
MEDIUM	src/ragas/utils.py	353	# Create a console handler and set its level	COMMENT
MEDIUM	src/ragas/utils.py	693	# Create a directory in the system temp directory	COMMENT
MEDIUM	src/ragas/metrics/decorator.py	190	# Create the dynamic model with arbitrary types allowed	COMMENT
MEDIUM	src/ragas/metrics/decorator.py	339	# Create the metric instance with all parameters	COMMENT
MEDIUM⚡	src/ragas/metrics/discrete.py	42	>>> # Create an LLM instance	STRING
MEDIUM⚡	src/ragas/metrics/discrete.py	46	>>> # Create a custom discrete metric	STRING
MEDIUM	src/ragas/metrics/_factual_correctness.py	39	# Define an enum for decomposition types	COMMENT
MEDIUM	src/ragas/metrics/_factual_correctness.py	52	# Define the examples using the Pydantic structure	COMMENT
MEDIUM⚡	src/ragas/metrics/numeric.py	42	>>> # Create an LLM instance	STRING
MEDIUM⚡	src/ragas/metrics/numeric.py	46	>>> # Create a custom numeric metric with 0-10 range	STRING
MEDIUM⚡	src/ragas/metrics/ranking.py	43	>>> # Create an LLM instance	STRING
MEDIUM⚡	src/ragas/metrics/ranking.py	47	>>> # Create a ranking metric that returns top 3 items	STRING
MEDIUM	src/ragas/metrics/base.py	1205	# Create the metric instance	COMMENT
MEDIUM	src/ragas/metrics/base.py	1348	# Create a wrapper that implements BaseRagasEmbedding interface	COMMENT
MEDIUM	…as/metrics/collections/context_entity_recall/metric.py	75	self.prompt = ExtractEntitiesPrompt() # Initialize prompt class once	CODE
MEDIUM	src/ragas/metrics/collections/context_recall/metric.py	77	self.prompt = ContextRecallPrompt() # Initialize prompt class once	CODE
MEDIUM	…/ragas/metrics/collections/factual_correctness/util.py	197	# Create a new instance	COMMENT
MEDIUM	…c/ragas/metrics/collections/answer_relevancy/metric.py	88	self.prompt = AnswerRelevancePrompt() # Initialize prompt class once	CODE
MEDIUM	…/ragas/metrics/collections/context_precision/metric.py	76	self.prompt = ContextPrecisionPrompt() # Initialize prompt class once	CODE
MEDIUM	…/ragas/metrics/collections/context_precision/metric.py	196	self.prompt = ContextPrecisionPrompt() # Initialize prompt class once	CODE
MEDIUM	src/ragas/embeddings/utils.py	34	# Create a new event loop for this thread	COMMENT
MEDIUM	src/ragas/testset/graph.py	684	# Create a deep copy and modify it	COMMENT
MEDIUM	src/ragas/llms/litellm_llm.py	137	# Create a new event loop for this thread	COMMENT
1 more matches not shown…

Empty Error Handlers26 hits · 130 pts

Severity	File	Line	Snippet	Context
HIGH	tests/unit/test_async_evaluation.py	180	except Exception:	CODE
HIGH	tests/unit/test_experiment.py	421	except ValueError:	CODE
HIGH	tests/unit/test_prechunked_generation.py	80	except ValueError:	CODE
HIGH	tests/unit/test_prechunked_generation.py	105	except ValueError:	CODE
HIGH	tests/unit/test_prechunked_generation.py	138	except ValueError:	CODE
HIGH	tests/unit/test_prechunked_generation.py	162	except ValueError:	CODE
HIGH	tests/unit/integrations/test_tracing_simple.py	59	except ImportError:	CODE
HIGH	examples/ragas_examples/benchmark_llm/evals.py	181	except Exception:	CODE
HIGH	examples/ragas_examples/text2sql/data_utils.py	23	except ImportError:	CODE
HIGH	examples/ragas_examples/improve_rag/rag.py	107	except Exception:	CODE
HIGH	examples/iterate_prompt/evals.py	244	except Exception:	CODE
HIGH	src/ragas/_safe_fetch.py	86	except ValueError:	CODE
HIGH	src/ragas/cli.py	583	except ImportError:	CODE
HIGH	src/ragas/cli.py	885	except Exception:	CODE
HIGH	…s/metrics/collections/multi_modal_faithfulness/util.py	76	except ValueError:	CODE
HIGH	src/ragas/embeddings/google_provider.py	185	except ImportError:	CODE
HIGH	src/ragas/embeddings/google_provider.py	187	except Exception:	CODE
HIGH	src/ragas/embeddings/google_provider.py	197	except ImportError:	CODE
HIGH	src/ragas/embeddings/google_provider.py	140	except ImportError:	CODE
HIGH	src/ragas/backends/gdrive_backend.py	297	except ValueError:	CODE
HIGH	src/ragas/backends/local_jsonl.py	85	except (ValueError, TypeError):	CODE
HIGH	src/ragas/prompt/multi_modal_prompt.py	361	except ValueError:	CODE
HIGH	src/ragas/llms/oci_genai_wrapper.py	135	except Exception:	CODE
HIGH	src/ragas/llms/litellm_llm.py	110	except (ValueError, AttributeError):	CODE
HIGH	src/ragas/llms/base.py	938	except ValueError:	CODE
HIGH	src/ragas/llms/base.py	1062	except (ValueError, AttributeError):	CODE

Modern Structural Boilerplate120 hits · 120 pts

Severity	File	Line	Snippet	Context
LOW	tests/utils/__init__.py	22	__all__ = [	CODE
LOW	tests/e2e/test_dataset_utils.py	7	logger = logging.getLogger(__name__)	CODE
LOW	examples/ragas_examples/text2sql/text2sql_agent.py	18	logger = logging.getLogger(__name__)	CODE
LOW	examples/ragas_examples/text2sql/__init__.py	44	__all__ = [	CODE
LOW	examples/ragas_examples/text2sql/evals.py	25	logger = logging.getLogger(__name__)	CODE
LOW	examples/ragas_examples/text2sql/data_utils.py	32	logger = logging.getLogger(__name__)	CODE
LOW	…/ragas_examples/ag_ui_agent_experiments/experiments.py	55	logger = logging.getLogger(__name__)	CODE
LOW	examples/ragas_examples/judge_alignment/__init__.py	21	__all__ = [	CODE
LOW	examples/ragas_examples/judge_alignment/evals.py	28	logger = logging.getLogger(__name__)	CODE
LOW	examples/ragas_examples/improve_rag/evals.py	35	logger = logging.getLogger(__name__)	CODE
LOW	examples/ragas_examples/improve_rag/rag.py	27	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/async_utils.py	7	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/cache.py	13	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/__init__.py	21	__all__ = [	CODE
LOW	src/ragas/_safe_fetch.py	41	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/dataset.py	3	__all__ = [	CODE
LOW	src/ragas/experiment.py	3	__all__ = ["Experiment", "experiment", "version_experiment"]	CODE
LOW	src/ragas/_analytics.py	34	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/cost.py	12	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/executor.py	15	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/validation.py	11	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/metrics/_simple_criteria.py	25	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/metrics/_context_recall.py	26	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/metrics/decorator.py	3	__all__ = [	CODE
LOW	src/ragas/metrics/_answer_similarity.py	22	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/metrics/validators.py	3	__all__ = [	CODE
LOW	src/ragas/metrics/discrete.py	3	__all__ = ["discrete_metric", "DiscreteMetric"]	CODE
LOW	src/ragas/metrics/_datacompy_score.py	13	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/metrics/__init__.py	101	__all__ = [	CODE
LOW	src/ragas/metrics/_factual_correctness.py	28	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/metrics/_domain_specific_rubrics.py	23	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/metrics/_summarization.py	22	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/metrics/_answer_relevance.py	20	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/metrics/result.py	3	__all__ = ["MetricResult"]	CODE
LOW	src/ragas/metrics/_aspect_critic.py	24	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/metrics/_nv_metrics.py	15	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/metrics/_faithfulness.py	22	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/metrics/numeric.py	3	__all__ = ["numeric_metric", "NumericMetric"]	CODE
LOW	src/ragas/metrics/_sql_semantic_equivalence.py	22	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/metrics/ranking.py	3	__all__ = ["ranking_metric", "RankingMetric"]	CODE
LOW	src/ragas/metrics/_topic_adherence.py	22	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/metrics/_answer_correctness.py	31	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/metrics/_noise_sensitivity.py	28	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/metrics/_context_entities_recall.py	22	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/metrics/base.py	38	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/metrics/_context_precision.py	25	logger = logging.getLogger(__name__)	CODE
LOW	src/ragas/metrics/collections/__init__.py	51	__all__ = [	CODE
LOW	…agas/metrics/collections/multi_modal_relevance/util.py	115	__all__ = [	CODE
LOW	…/metrics/collections/multi_modal_relevance/__init__.py	5	__all__ = [	CODE
LOW	…/metrics/collections/context_entity_recall/__init__.py	5	__all__ = [	CODE
LOW	…/ragas/metrics/collections/answer_accuracy/__init__.py	5	__all__ = [	CODE
LOW	…trics/collections/sql_semantic_equivalence/__init__.py	7	__all__ = ["SQLSemanticEquivalence"]	CODE
LOW	…/ragas/metrics/collections/datacompy_score/__init__.py	5	__all__ = ["DataCompyScore"]	CODE
LOW	src/ragas/metrics/collections/datacompy_score/metric.py	12	logger = logging.getLogger(__name__)	CODE
LOW	…agas/metrics/collections/context_relevance/__init__.py	5	__all__ = [	CODE
LOW	…c/ragas/metrics/collections/context_recall/__init__.py	5	__all__ = [	CODE
LOW	…as/metrics/collections/factual_correctness/__init__.py	5	__all__ = [	CODE
LOW	src/ragas/metrics/collections/summary_score/__init__.py	5	__all__ = [	CODE
LOW	…rics/collections/instance_specific_rubrics/__init__.py	7	__all__ = ["InstanceSpecificRubrics"]	CODE
LOW	…gas/metrics/collections/tool_call_accuracy/__init__.py	5	__all__ = [	CODE
60 more matches not shown…

Structural Annotation Overuse48 hits · 104 pts

Severity	File	Line	Snippet	Context
LOW⚡	tests/unit/test_cosine_relationship_builders.py	45	# Step 1: Create a base vector	COMMENT
LOW⚡	tests/unit/test_cosine_relationship_builders.py	49	# Step 2: Generate similar vectors	COMMENT
LOW⚡	tests/unit/test_cosine_relationship_builders.py	62	# Step 3: Generate additional random unit vectors	COMMENT
LOW⚡	tests/unit/test_cosine_relationship_builders.py	69	# Step 4: Combine and shuffle	COMMENT
LOW	docs/getstarted/quickstart.md	5	## Step 1: Create Your Project	COMMENT
LOW	docs/getstarted/quickstart.md	26	## Step 2: Install Dependencies	COMMENT
LOW	docs/getstarted/quickstart.md	40	## Step 3: Set Your API Key	COMMENT
LOW	docs/getstarted/quickstart.md	136	## Step 4: Run Your Evaluation	COMMENT
LOW⚡	docs/howtos/applications/align-llm-as-judge.md	162	# Step 1: Get response (in production, this is where you'd call your LLM app)	COMMENT
LOW⚡	docs/howtos/applications/align-llm-as-judge.md	166	# Step 2: Judge evaluates the response	COMMENT
LOW⚡	docs/howtos/applications/align-llm-as-judge.md	174	# Step 3: Compare judge decision with human target	COMMENT
LOW⚡	examples/ragas_examples/workflow_eval/workflow.py	557	# Step 1: Classify email	COMMENT
LOW⚡	examples/ragas_examples/workflow_eval/workflow.py	560	# Step 2: Extract relevant information based on category	COMMENT
LOW⚡	examples/ragas_examples/workflow_eval/workflow.py	563	# Step 3: Generate response template	COMMENT
LOW⚡	examples/ragas_examples/text2sql/data_utils.py	357	# Step 1: Load and clean data	COMMENT
LOW⚡	examples/ragas_examples/text2sql/data_utils.py	360	# Step 2: Sample data for each difficulty level	COMMENT
LOW⚡	examples/ragas_examples/text2sql/data_utils.py	381	# Step 3: Combine all sampled data	COMMENT
LOW⚡	examples/ragas_examples/text2sql/data_utils.py	384	# Step 4: Save results	COMMENT
LOW⚡	examples/ragas_examples/judge_alignment/evals.py	140	# Step 1: Get response (in production, this is where you'd call your LLM app)	COMMENT
LOW⚡	examples/ragas_examples/judge_alignment/evals.py	144	# Step 2: Judge evaluates the response	COMMENT
LOW⚡	examples/ragas_examples/judge_alignment/evals.py	152	# Step 3: Compare judge decision with human target	COMMENT
LOW	…agas/metrics/collections/factual_correctness/metric.py	135	# Step 1: Get claim verifications to match legacy behavior exactly	COMMENT
LOW⚡	…agas/metrics/collections/factual_correctness/metric.py	149	# Step 2: Compute TP, FP, FN exactly like legacy	COMMENT
LOW⚡	…agas/metrics/collections/factual_correctness/metric.py	157	# Step 3: Compute final score based on mode	COMMENT
LOW⚡	src/ragas/metrics/collections/summary_score/metric.py	134	# Step 1: Combine contexts and extract keyphrases	COMMENT
LOW⚡	src/ragas/metrics/collections/summary_score/metric.py	143	# Step 2: Generate questions from keyphrases	COMMENT
LOW⚡	src/ragas/metrics/collections/summary_score/metric.py	151	# Step 3: Check if summary can answer the questions	COMMENT
LOW⚡	src/ragas/metrics/collections/summary_score/metric.py	154	# Step 4: Calculate QA score	COMMENT
LOW⚡	src/ragas/metrics/collections/summary_score/metric.py	157	# Step 5: Calculate final score (with optional conciseness penalty)	COMMENT
LOW⚡	src/ragas/metrics/collections/topic_adherence/metric.py	113	# Step 1: Extract topics from the conversation	COMMENT
LOW⚡	src/ragas/metrics/collections/topic_adherence/metric.py	118	# Step 2: Check which topics the AI answered vs refused	COMMENT
LOW⚡	src/ragas/metrics/collections/topic_adherence/metric.py	121	# Step 3: Classify topics against reference topics	COMMENT
LOW⚡	src/ragas/metrics/collections/topic_adherence/metric.py	124	# Step 4: Compute score based on mode	COMMENT
LOW⚡	src/ragas/metrics/collections/faithfulness/metric.py	116	# Step 1: Break response into atomic statements	COMMENT
LOW⚡	src/ragas/metrics/collections/faithfulness/metric.py	123	# Step 2: Join all contexts and evaluate statements against them	COMMENT
LOW⚡	src/ragas/metrics/collections/faithfulness/metric.py	127	# Step 3: Compute faithfulness score	COMMENT
LOW⚡	…ragas/metrics/collections/answer_correctness/metric.py	170	# Step 1: Generate statements from both response and reference	COMMENT
LOW⚡	…ragas/metrics/collections/answer_correctness/metric.py	174	# Step 2: Calculate factuality score via TP/FP/FN classification	COMMENT
LOW⚡	…ragas/metrics/collections/answer_correctness/metric.py	184	# Step 3: Calculate semantic similarity score	COMMENT
LOW⚡	…ragas/metrics/collections/answer_correctness/metric.py	190	# Step 4: Combine scores with weighted average	COMMENT
LOW⚡	…agas/metrics/collections/agent_goal_accuracy/metric.py	99	# Step 1: Infer the end state from the workflow	COMMENT
LOW⚡	…agas/metrics/collections/agent_goal_accuracy/metric.py	102	# Step 2: Compare the end state with reference	COMMENT
LOW⚡	…agas/metrics/collections/agent_goal_accuracy/metric.py	202	# Step 1: Infer the user goal and end state from the workflow	COMMENT
LOW⚡	…agas/metrics/collections/agent_goal_accuracy/metric.py	205	# Step 2: Compare the inferred goal with the end state	COMMENT
LOW⚡	…/ragas/metrics/collections/noise_sensitivity/metric.py	141	# Step 1: Decompose reference and response into statements	COMMENT
LOW⚡	…/ragas/metrics/collections/noise_sensitivity/metric.py	149	# Step 2: Evaluate statement faithfulness against each retrieved context	COMMENT
LOW	…/ragas/metrics/collections/noise_sensitivity/metric.py	166	# Step 3: Build matrices for computation (exact legacy shape handling)	COMMENT
LOW⚡	…/ragas/metrics/collections/noise_sensitivity/metric.py	182	# Step 4: Compute noise sensitivity score	COMMENT

Verbosity Indicators41 hits · 91 pts

Severity	File	Line	Snippet	Context
LOW⚡	tests/unit/test_cosine_relationship_builders.py	45	# Step 1: Create a base vector	COMMENT
LOW⚡	tests/unit/test_cosine_relationship_builders.py	49	# Step 2: Generate similar vectors	COMMENT
LOW⚡	tests/unit/test_cosine_relationship_builders.py	62	# Step 3: Generate additional random unit vectors	COMMENT
LOW⚡	tests/unit/test_cosine_relationship_builders.py	69	# Step 4: Combine and shuffle	COMMENT
LOW⚡	examples/ragas_examples/workflow_eval/workflow.py	557	# Step 1: Classify email	COMMENT
LOW⚡	examples/ragas_examples/workflow_eval/workflow.py	560	# Step 2: Extract relevant information based on category	COMMENT
LOW⚡	examples/ragas_examples/workflow_eval/workflow.py	563	# Step 3: Generate response template	COMMENT
LOW⚡	examples/ragas_examples/text2sql/data_utils.py	357	# Step 1: Load and clean data	COMMENT
LOW⚡	examples/ragas_examples/text2sql/data_utils.py	360	# Step 2: Sample data for each difficulty level	COMMENT
LOW⚡	examples/ragas_examples/text2sql/data_utils.py	381	# Step 3: Combine all sampled data	COMMENT
LOW⚡	examples/ragas_examples/text2sql/data_utils.py	384	# Step 4: Save results	COMMENT
LOW⚡	examples/ragas_examples/judge_alignment/evals.py	140	# Step 1: Get response (in production, this is where you'd call your LLM app)	COMMENT
LOW⚡	examples/ragas_examples/judge_alignment/evals.py	144	# Step 2: Judge evaluates the response	COMMENT
LOW⚡	examples/ragas_examples/judge_alignment/evals.py	152	# Step 3: Compare judge decision with human target	COMMENT
LOW	…agas/metrics/collections/factual_correctness/metric.py	135	# Step 1: Get claim verifications to match legacy behavior exactly	COMMENT
LOW⚡	…agas/metrics/collections/factual_correctness/metric.py	149	# Step 2: Compute TP, FP, FN exactly like legacy	COMMENT
LOW⚡	…agas/metrics/collections/factual_correctness/metric.py	157	# Step 3: Compute final score based on mode	COMMENT
LOW⚡	src/ragas/metrics/collections/summary_score/metric.py	134	# Step 1: Combine contexts and extract keyphrases	COMMENT
LOW⚡	src/ragas/metrics/collections/summary_score/metric.py	143	# Step 2: Generate questions from keyphrases	COMMENT
LOW⚡	src/ragas/metrics/collections/summary_score/metric.py	151	# Step 3: Check if summary can answer the questions	COMMENT
LOW⚡	src/ragas/metrics/collections/summary_score/metric.py	154	# Step 4: Calculate QA score	COMMENT
LOW⚡	src/ragas/metrics/collections/summary_score/metric.py	157	# Step 5: Calculate final score (with optional conciseness penalty)	COMMENT
LOW⚡	src/ragas/metrics/collections/topic_adherence/metric.py	113	# Step 1: Extract topics from the conversation	COMMENT
LOW⚡	src/ragas/metrics/collections/topic_adherence/metric.py	118	# Step 2: Check which topics the AI answered vs refused	COMMENT
LOW⚡	src/ragas/metrics/collections/topic_adherence/metric.py	121	# Step 3: Classify topics against reference topics	COMMENT
LOW⚡	src/ragas/metrics/collections/topic_adherence/metric.py	124	# Step 4: Compute score based on mode	COMMENT
LOW⚡	src/ragas/metrics/collections/faithfulness/metric.py	116	# Step 1: Break response into atomic statements	COMMENT
LOW⚡	src/ragas/metrics/collections/faithfulness/metric.py	123	# Step 2: Join all contexts and evaluate statements against them	COMMENT
LOW⚡	src/ragas/metrics/collections/faithfulness/metric.py	127	# Step 3: Compute faithfulness score	COMMENT
LOW⚡	…ragas/metrics/collections/answer_correctness/metric.py	170	# Step 1: Generate statements from both response and reference	COMMENT
LOW⚡	…ragas/metrics/collections/answer_correctness/metric.py	174	# Step 2: Calculate factuality score via TP/FP/FN classification	COMMENT
LOW⚡	…ragas/metrics/collections/answer_correctness/metric.py	184	# Step 3: Calculate semantic similarity score	COMMENT
LOW⚡	…ragas/metrics/collections/answer_correctness/metric.py	190	# Step 4: Combine scores with weighted average	COMMENT
LOW⚡	…agas/metrics/collections/agent_goal_accuracy/metric.py	99	# Step 1: Infer the end state from the workflow	COMMENT
LOW⚡	…agas/metrics/collections/agent_goal_accuracy/metric.py	102	# Step 2: Compare the end state with reference	COMMENT
LOW⚡	…agas/metrics/collections/agent_goal_accuracy/metric.py	202	# Step 1: Infer the user goal and end state from the workflow	COMMENT
LOW⚡	…agas/metrics/collections/agent_goal_accuracy/metric.py	205	# Step 2: Compare the inferred goal with the end state	COMMENT
LOW⚡	…/ragas/metrics/collections/noise_sensitivity/metric.py	141	# Step 1: Decompose reference and response into statements	COMMENT
LOW⚡	…/ragas/metrics/collections/noise_sensitivity/metric.py	149	# Step 2: Evaluate statement faithfulness against each retrieved context	COMMENT
LOW	…/ragas/metrics/collections/noise_sensitivity/metric.py	166	# Step 3: Build matrices for computation (exact legacy shape handling)	COMMENT
LOW⚡	…/ragas/metrics/collections/noise_sensitivity/metric.py	182	# Step 4: Compute noise sensitivity score	COMMENT

Magic Placeholder Names17 hits · 76 pts

Severity	File	Line	Snippet	Context
HIGH	tests/utils/llm_setup.py	37	f" export {env_var}='your-api-key-here'"	CODE
HIGH	docs/getstarted/quickstart.md	113	client = OpenAI(api_key="your-api-key", base_url="https://your-api-endpoint")	CODE
HIGH	docs/getstarted/evals.md	161	client = OpenAI(api_key="your-api-key", base_url="https://your-api-endpoint")	CODE
HIGH	docs/howtos/observability.md	380	export LANGCHAIN_API_KEY=<your-api-key>	CODE
HIGH	docs/howtos/customizations/customize_models.md	56	"api_key": "your-api-key",	CODE
HIGH	docs/howtos/customizations/metrics/_cost.md	17	os.environ["OPENAI_API_KEY"] = "your-api-key"	CODE
HIGH	docs/howtos/customizations/metrics/tracing.md	10	export LANGCHAIN_API_KEY=<your-api-key>	CODE
HIGH	docs/howtos/integrations/_langsmith.md	20	export LANGCHAIN_API_KEY=<your-api-key>	CODE
HIGH	docs/howtos/integrations/langsmith.md	14	export LANGCHAIN_API_KEY=<your-api-key>	CODE
HIGH	docs/howtos/applications/align-llm-as-judge.md	22	export OPENAI_API_KEY="your-api-key-here"	CODE
HIGH	docs/howtos/applications/text2sql.md	31	os.environ["OPENAI_API_KEY"] = "your-api-key-here"	CODE
HIGH	examples/ragas_examples/text2sql/__init__.py	20	client = AsyncOpenAI(api_key="your-api-key")	STRING
HIGH	…mples/ragas_examples/ag_ui_agent_experiments/README.md	253	export OPENAI_API_KEY='your-api-key-here'	CODE
HIGH	src/ragas/cli.py	893	console.print(" export OPENAI_API_KEY='your-api-key'")	STRING
HIGH⚡	src/ragas/metrics/discrete.py	43	>>> client = OpenAI(api_key="your-api-key")	STRING
HIGH⚡	src/ragas/metrics/numeric.py	43	>>> client = OpenAI(api_key="your-api-key")	STRING
HIGH⚡	src/ragas/metrics/ranking.py	44	>>> client = OpenAI(api_key="your-api-key")	STRING

Redundant / Tautological Comments48 hits · 74 pts

Severity	File	Line	Snippet	Context
LOW	tests/unit/test_embeddings.py	16	# Check if BaseRagasEmbedding is in the module	COMMENT
LOW	tests/unit/test_knowledge_graph_clusters.py	861	# Check if time complexity is reasonable	COMMENT
LOW	tests/unit/test_knowledge_graph_clusters.py	961	# Check if time complexity is reasonable	COMMENT
LOW	tests/unit/integrations/test_ag_ui.py	11	# Check if ag_ui is available	COMMENT
LOW	tests/utils/metric_comparison.py	264	# Check if metric supports batching	COMMENT
LOW	tests/e2e/metrics_migration/base_migration_test.py	50	# Check if required components are available	COMMENT
LOW	tests/e2e/metrics_migration/base_migration_test.py	131	# Check if required components are available	COMMENT
LOW	examples/oci_genai_example.py	125	# Check if OCI configuration is available	COMMENT
LOW	…amples/ragas_examples/text2sql/validate_sql_dataset.py	114	# Check if all values in the first row are null/None	COMMENT
LOW	examples/ragas_examples/agent_evals/agent.py	323	# Check if the model wants to call functions	COMMENT
LOW	.github/workflows/ci.yaml	185	# Check if code is properly formatted (without making changes)	COMMENT
LOW	src/ragas/experiment.py	66	# Check if there are any changes to the repo	COMMENT
LOW	src/ragas/experiment.py	77	# Check if there are uncommitted changes	COMMENT
LOW	src/ragas/utils.py	78	# Set propagate to False if you don't want it to log to the root logger's handlers as well	COMMENT
LOW	src/ragas/utils.py	627	# Check if the current directory is a git repository	COMMENT
LOW	src/ragas/utils.py	670	# Check if we're already in an event loop	COMMENT
LOW	src/ragas/metrics/decorator.py	179	# Check if it's an optional type	COMMENT
LOW	src/ragas/metrics/decorator.py	260	# Check if we're already in an event loop	COMMENT
LOW	src/ragas/metrics/collections/base.py	42	# Check if this instance has these attributes after initialization	COMMENT
LOW	src/ragas/metrics/collections/base.py	72	# Check if we're already in an async context	COMMENT
LOW	src/ragas/metrics/collections/base.py	101	# Check if we're already in an async context	COMMENT
LOW	…c/ragas/metrics/collections/answer_relevancy/metric.py	129	# Check if all responses are noncommittal	COMMENT
LOW	…s/metrics/collections/multi_modal_faithfulness/util.py	211	# Check if file exists	COMMENT
LOW⚡	src/ragas/embeddings/google_provider.py	100	# Check if it's the new google-genai SDK Client	COMMENT
LOW⚡	src/ragas/embeddings/google_provider.py	110	# Check if client has embed_content method (it's the old genai module)	COMMENT
LOW⚡	src/ragas/embeddings/google_provider.py	117	# Check if it's a GenerativeModel instance - extract genai module from it	COMMENT
LOW	src/ragas/backends/registry.py	212	# Check if it's an alias first	COMMENT
LOW	src/ragas/backends/gdrive_backend.py	158	# Check if main folder exists	COMMENT
LOW	src/ragas/integrations/ag_ui.py	361	# Check if there are completed tool calls for this message	COMMENT
LOW	src/ragas/integrations/ag_ui.py	449	# Check if there are unclaimed tool calls	COMMENT
LOW	src/ragas/integrations/ag_ui.py	544	# Check if there are completed tool calls for this message	COMMENT
LOW	src/ragas/prompt/simple_prompt.py	244	# Check if response_model is required but not provided	COMMENT
LOW	src/ragas/prompt/dynamic_few_shot.py	387	# Check if models are required but not provided	COMMENT
LOW	src/ragas/prompt/multi_modal_prompt.py	296	# Check if the string ends with one of the common image extensions (case-insensitive)	COMMENT
LOW	src/ragas/prompt/multi_modal_prompt.py	462	# Check if the path exists and is a file	COMMENT
LOW	src/ragas/testset/graph.py	366	# Check if graph has enough nodes for meaningful paths	COMMENT
LOW	src/ragas/testset/graph.py	403	# Check if graph is empty after node removal	COMMENT
LOW	src/ragas/testset/graph.py	620	# Check if the new cluster is a subset of any existing cluster	COMMENT
LOW	src/ragas/testset/transforms/extractors/embeddings.py	50	# Check if the client supports async operations by checking if is_async exists and is True	COMMENT
LOW	src/ragas/llms/litellm_llm.py	51	# Check if client is async-capable at initialization	COMMENT
LOW	src/ragas/llms/litellm_llm.py	67	# Check if this is an AsyncInstructor wrapper (instructor.AsyncInstructor)	COMMENT
LOW	src/ragas/llms/litellm_llm.py	106	# Check if the closure object has acompletion (e.g., litellm Router)	COMMENT
LOW	src/ragas/llms/base.py	840	# Check if client is async-capable at initialization	COMMENT
LOW	src/ragas/llms/base.py	1002	# Check if this is an AsyncInstructor wrapper	COMMENT
LOW	src/ragas/llms/base.py	1006	# Check if this is a sync Instructor wrapper that wraps an async client	COMMENT
LOW	src/ragas/llms/base.py	1023	# Check if this is an async client by checking for a coroutine method	COMMENT
LOW	src/ragas/llms/base.py	1047	# Check if the closure object is an async client	COMMENT
LOW	src/ragas/llms/adapters/__init__.py	71	# Check if client is LiteLLM	COMMENT

Deep Nesting77 hits · 72 pts

Severity	File	Line	Context
LOW	tests/unit/test_async_evaluation.py	141	CODE
LOW	tests/unit/test_async_evaluation.py	191	CODE
LOW	tests/utils/metric_comparison.py	234	CODE
LOW	tests/e2e/metrics_migration/base_migration_test.py	25	CODE
LOW	docs/ipynb_to_md.py	36	CODE
LOW	examples/ragas_examples/text2sql/analyze_errors.py	76	CODE
LOW	…amples/ragas_examples/text2sql/validate_sql_dataset.py	58	CODE
LOW	examples/ragas_examples/text2sql/evals.py	32	CODE
LOW	examples/ragas_examples/text2sql/db_utils.py	226	CODE
LOW	examples/ragas_examples/agent_evals/agent.py	169	CODE
LOW	examples/ragas_examples/agent_evals/agent.py	242	CODE
LOW	src/ragas/async_utils.py	58	CODE
LOW	src/ragas/async_utils.py	159	CODE
LOW	src/ragas/async_utils.py	86	CODE
LOW	src/ragas/async_utils.py	181	CODE
LOW	src/ragas/evaluation.py	59	CODE
LOW	src/ragas/cache.py	135	CODE
LOW	src/ragas/_safe_fetch.py	53	CODE
LOW	src/ragas/_safe_fetch.py	197	CODE
LOW	src/ragas/dataset.py	337	CODE
LOW	src/ragas/experiment.py	141	CODE
LOW	src/ragas/cli.py	101	CODE
LOW	src/ragas/cli.py	371	CODE
LOW	src/ragas/cli.py	461	CODE
LOW	src/ragas/cost.py	78	CODE
LOW	src/ragas/cost.py	112	CODE
LOW	src/ragas/dataset_schema.py	126	CODE
LOW	src/ragas/dataset_schema.py	377	CODE
LOW	src/ragas/dataset_schema.py	761	CODE
LOW	src/ragas/metrics/_topic_adherence.py	152	CODE
LOW	src/ragas/metrics/_topic_adherence.py	189	CODE
LOW	src/ragas/metrics/base.py	120	CODE
LOW	src/ragas/metrics/base.py	249	CODE
LOW	src/ragas/metrics/base.py	1215	CODE
LOW	src/ragas/metrics/collections/answer_accuracy/metric.py	132	CODE
LOW	…/ragas/metrics/collections/context_relevance/metric.py	142	CODE
LOW	src/ragas/metrics/collections/topic_adherence/metric.py	180	CODE
LOW	…as/metrics/collections/response_groundedness/metric.py	134	CODE
LOW	…s/metrics/collections/multi_modal_faithfulness/util.py	126	CODE
LOW	…s/metrics/collections/multi_modal_faithfulness/util.py	179	CODE
LOW	src/ragas/embeddings/utils.py	108	CODE
LOW	src/ragas/backends/gdrive_backend.py	112	CODE
LOW	src/ragas/backends/gdrive_backend.py	246	CODE
LOW	src/ragas/backends/local_jsonl.py	61	CODE
LOW	src/ragas/backends/local_jsonl.py	74	CODE
LOW	src/ragas/integrations/swarm.py	7	CODE
LOW	src/ragas/integrations/amazon_bedrock.py	74	CODE
LOW	src/ragas/integrations/llama_index.py	120	CODE
LOW	src/ragas/integrations/ag_ui.py	850	CODE
LOW	src/ragas/integrations/ag_ui.py	251	CODE
LOW	src/ragas/integrations/ag_ui.py	599	CODE
LOW	src/ragas/integrations/tracing/__init__.py	53	CODE
LOW	src/ragas/prompt/utils.py	7	CODE
LOW	src/ragas/prompt/utils.py	28	CODE
LOW	src/ragas/prompt/multi_modal_prompt.py	366	CODE
LOW	src/ragas/prompt/multi_modal_prompt.py	421	CODE
LOW	src/ragas/testset/graph.py	471	CODE
LOW	src/ragas/testset/graph.py	559	CODE
LOW	src/ragas/testset/synthesizers/__init__.py	24	CODE
LOW	src/ragas/testset/synthesizers/single_hop/specific.py	46	CODE
17 more matches not shown…

Decorative Section Separators19 hits · 66 pts

Severity	File	Line	Snippet	Context
MEDIUM	mkdocs-pdf.yml	32	# -------------------------------------------	COMMENT
MEDIUM	tests/conftest.py	79	# ====================	COMMENT
MEDIUM	tests/conftest.py	81	# ====================	COMMENT
MEDIUM	tests/unit/test_metric.py	243	# ====================	COMMENT
MEDIUM	tests/unit/test_metric.py	245	# ====================	COMMENT
MEDIUM⚡	tests/unit/integrations/test_ag_ui.py	753	# ============================================================================	COMMENT
MEDIUM⚡	tests/unit/integrations/test_ag_ui.py	755	# ============================================================================	COMMENT
MEDIUM⚡	tests/unit/integrations/test_ag_ui.py	895	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/unit/integrations/test_ag_ui.py	897	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/unit/integrations/test_ag_ui.py	968	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/unit/integrations/test_ag_ui.py	970	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	tests/unit/integrations/test_ag_ui.py	793	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	tests/unit/integrations/test_ag_ui.py	795	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	src/ragas/integrations/ag_ui.py	1075	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	src/ragas/integrations/ag_ui.py	1077	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	src/ragas/integrations/ag_ui.py	1159	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	src/ragas/integrations/ag_ui.py	1161	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	src/ragas/integrations/ag_ui.py	1261	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	src/ragas/integrations/ag_ui.py	1263	# ---------------------------------------------------------------------------	COMMENT

AI Structural Patterns24 hits · 24 pts

Severity	File	Line	Context
LOW	tests/unit/test_experiment.py	315	CODE
LOW	tests/unit/test_uvloop_compatibility.py	21	CODE
LOW	tests/unit/test_uvloop_compatibility.py	75	CODE
LOW	tests/unit/test_uvloop_compatibility.py	89	CODE
LOW	examples/ragas_examples/benchmark_llm/prompt.py	57	CODE
LOW	examples/ragas_examples/agent_evals/agent.py	150	CODE
LOW	examples/ragas_examples/agent_evals/agent.py	155	CODE
LOW	examples/ragas_examples/agent_evals/agent.py	160	CODE
LOW	examples/ragas_examples/agent_evals/agent.py	167	CODE
LOW	examples/ragas_examples/improve_rag/evals.py	170	CODE
LOW	examples/ragas_examples/prompt_evals/prompt.py	27	CODE
LOW	examples/iterate_prompt/run_prompt.py	32	CODE
LOW	src/ragas/evaluation.py	59	CODE
LOW	src/ragas/evaluation.py	349	CODE
LOW	src/ragas/utils.py	197	CODE
LOW	src/ragas/metrics/_domain_specific_rubrics.py	86	CODE
LOW	…agas/metrics/collections/factual_correctness/metric.py	183	CODE
LOW	src/ragas/metrics/collections/faithfulness/metric.py	146	CODE
LOW	src/ragas/integrations/llama_index.py	31	CODE
LOW	src/ragas/integrations/tracing/langfuse.py	28	CODE
LOW	src/ragas/testset/synthesizers/generate.py	114	CODE
LOW	src/ragas/testset/synthesizers/generate.py	222	CODE
LOW	src/ragas/testset/synthesizers/generate.py	302	CODE
LOW	src/ragas/testset/synthesizers/generate.py	414	CODE

Hallucination Indicators2 hits · 20 pts

Severity	File	Line	Snippet	Context
CRITICAL	…as/metrics/collections/multi_modal_relevance/metric.py	162	result = await self.llm.client.chat.completions.create(	CODE
CRITICAL	…metrics/collections/multi_modal_faithfulness/metric.py	154	result = await self.llm.client.chat.completions.create(	CODE

AI Slop Vocabulary7 hits · 12 pts

Severity	File	Line	Snippet	Context
MEDIUM	tests/unit/test_metric_decorators.py	407	"""Test comprehensive error handling and validation."""	STRING
MEDIUM	tests/unit/test_metric_decorators.py	608	"""Test comprehensive type validation with Pydantic for all complex types."""	STRING
MEDIUM	.github/workflows/claude-code-review.yml	75	# - For React components: Performance, accessibility, and best practices	COMMENT
MEDIUM	.github/workflows/claude-code-review.yml	82	# 'Please provide a thorough code review focusing on our coding standards and best practices.' }}	COMMENT
LOW	src/ragas/metrics/decorator.py	318	# For sync functions, just call directly	COMMENT
LOW	src/ragas/metrics/collections/example_metric.py	38	# Example logic - just return a simple score based on response length	COMMENT
MEDIUM	src/ragas/metrics/collections/context_relevance/util.py	29	instruction = """You are a world class expert designed to evaluate the relevance score of a Context in order to answ	CODE

Over-Commented Block11 hits · 11 pts

Severity	File	Line	Snippet	Context
LOW	tests/unit/test_graph.py	201	]	COMMENT
LOW	tests/unit/test_graph.py	221	# sorted(expected_clusters, key=lambda c: (len(c), get_cluster_names(c)))	COMMENT
LOW	tests/docs/__init__.py	1	# Tests for documentation code examples	COMMENT
LOW	examples/gdrive_backend_example.py	41		COMMENT
LOW	.github/workflows/claude-code-review.yml	61	- Performance considerations	COMMENT
LOW	.github/workflows/claude-code-review.yml	81	# 'Welcome! Please review this PR from a first-time contributor. Be encouraging and provide detailed explana	COMMENT
LOW	.github/workflows/claude.yml	41	actions: read	COMMENT
LOW	.github/workflows/claude.yml	61	# Optional: Custom environment variables for Claude	COMMENT
LOW	src/ragas/embeddings/base.py	261	"""	COMMENT
LOW	src/ragas/embeddings/base.py	521	assert isinstance(predictions, Tensor)	COMMENT
LOW	src/ragas/integrations/__init__.py	21	# Tracing integrations are available as a submodule	COMMENT

Cross-Language Confusion2 hits · 10 pts

Severity	File	Line	Snippet	Context
HIGH	examples/ragas_examples/workflow_eval/evals.py	32	"pass_criteria": "category Bug Report; error_code ERR_MEMORY_OVERFLOW; product_version null; response handle	CODE
HIGH	examples/ragas_examples/workflow_eval/evals.py	44	"pass_criteria": "category Bug Report; error_code API-500; product_version latest/null; response acknowledge	CODE

Slop Phrases2 hits · 3 pts

Severity	File	Line	Snippet	Context
LOW	examples/ragas_examples/prompt_evals/evals.py	60	# make sure to save it	COMMENT
LOW	examples/ragas_examples/rag_eval/evals.py	46	# make sure to save it	COMMENT

Overly Generic Function Names2 hits · 2 pts

Severity	File	Line	Snippet	Context
LOW	tests/unit/integrations/test_tracing_simple.py	24	def test_function():	CODE
LOW⚡	tests/unit/integrations/test_tracing.py	52	def test_function():	CODE

Example Usage Blocks1 hit · 2 pts

Severity	File	Line	Snippet	Context
LOW	examples/ragas_examples/agent_evals/agent.py	401	# Example usage	COMMENT

vibrantlabsai/ragas

Open-Source Code Quality, Security and Technical Debt Review