This repository showcases various advanced techniques for Retrieval-Augmented Generation (RAG) systems. Each technique has a detailed notebook tutorial.
This report summarizes scanner evidence for NirDiamant/RAG_Techniques, a Jupyter Notebook project with 29,770 GitHub stars. SynthScan examined 10,812 lines of code across 38 source files, recording 158 pattern matches distributed across 12 syntactic categories. The legacy adjusted pattern signal is 27.9 (Elevated pattern signal); it is not an authorship estimate.
Use the findings to ask whether maintainers can explain intent, verify dependencies, simplify unnecessary scaffolding, and focus review effort. Risk Payload v1 adds ownership, supply-chain, over-engineering, and review-effort vectors when available. These are prompts for human verification, not probabilities, security findings, or measured maintenance costs.
Use each vector to guide manual verification of the underlying findings.
Can maintainers explain intent, assumptions, and trade-offs? 103 finding(s).
Are imports declared and package/API names verified? 50 finding(s).
Are wrappers, guards, and repeated structures necessary? 28 finding(s).
Which changes need focused human review? 158 finding(s).
This chart tracks the legacy adjusted pattern score across scan runs. Changes can reflect source changes, scan coverage, or detector/scoring versions; they do not measure AI adoption or prove that a team refactored code. Compare runs only when their scoring versions are compatible. The dashed line tracks the raw pattern-hit count.
Groups findings by configured severity weight. Severity is not a confidence estimate, security rating, or proof that a package is missing. Verify imports against the project's manifests and inspect each finding in its local design context.
This horizontal bar chart shows where configured pattern scores accumulate across directories. Treat the bars as locations to inspect, not as evidence of authorship or code quality. Use the file-level findings to decide whether a directory needs a focused ownership, dependency, complexity, or review check.
The scanner identified 158 distinct pattern matches across 12 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a detector match to inspect in context. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the configured finding weight (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — legacy comment-context matches receive a ×1.5 score weight. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier under the legacy scoring model.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM⚡ | helper_functions.py | 166 | # Define the prompt template for chain-of-thought reasoning | COMMENT |
| MEDIUM⚡ | helper_functions.py | 174 | # Create a PromptTemplate object with the specified template and input variables | STRING |
| MEDIUM⚡ | helper_functions.py | 180 | # Create a chain by combining the prompt template and the language model | STRING |
| MEDIUM | …unnable_scripts/HyPE_Hypothetical_Prompt_Embeddings.py | 46 | # Create a retriever from the vector store | COMMENT |
| MEDIUM | …l_rag_techniques_runnable_scripts/choose_chunk_size.py | 75 | # Define the main class for the RAG method | COMMENT |
| MEDIUM | …nnable_scripts/HyDe_Hypothetical_Document_Embedding.py | 18 | # Define the HyDe retriever class - creating vector store, generating hypothetical document, and retrieving | COMMENT |
| MEDIUM | …l_rag_techniques_runnable_scripts/semantic_chunking.py | 56 | # Create a vector store and retriever from the semantic chunks | COMMENT |
| MEDIUM | …iques_runnable_scripts/retrieval_with_feedback_loop.py | 25 | # Define the Response class | COMMENT |
| MEDIUM | …iques_runnable_scripts/retrieval_with_feedback_loop.py | 104 | # Define the main RAG class | STRING |
| MEDIUM⚡ | all_rag_techniques_runnable_scripts/graph_rag.py | 108 | # Define the knowledge graph class | COMMENT |
| MEDIUM⚡ | all_rag_techniques_runnable_scripts/graph_rag.py | 109 | # Define the Concepts class | COMMENT |
| MEDIUM⚡ | all_rag_techniques_runnable_scripts/graph_rag.py | 114 | # Define the KnowledgeGraph class | COMMENT |
| MEDIUM⚡ | all_rag_techniques_runnable_scripts/graph_rag.py | 315 | # Define the Query Engine class | COMMENT |
| MEDIUM⚡ | all_rag_techniques_runnable_scripts/graph_rag.py | 316 | # Define the AnswerCheck class | COMMENT |
| MEDIUM⚡ | all_rag_techniques_runnable_scripts/graph_rag.py | 322 | # Define the QueryEngine class | COMMENT |
| MEDIUM⚡ | all_rag_techniques_runnable_scripts/graph_rag.py | 809 | # Create a graph RAG instance | COMMENT |
| MEDIUM | all_rag_techniques_runnable_scripts/graph_rag.py | 47 | # Define the document processor class | COMMENT |
| MEDIUM | all_rag_techniques_runnable_scripts/graph_rag.py | 48 | # Define the DocumentProcessor class | COMMENT |
| MEDIUM | all_rag_techniques_runnable_scripts/graph_rag.py | 584 | # Import necessary libraries | COMMENT |
| MEDIUM | all_rag_techniques_runnable_scripts/graph_rag.py | 590 | # Define the Visualizer class | COMMENT |
| MEDIUM | all_rag_techniques_runnable_scripts/graph_rag.py | 731 | # Define the graph RAG class | COMMENT |
| MEDIUM | all_rag_techniques_runnable_scripts/self_rag.py | 70 | # Define main class | COMMENT |
| MEDIUM | all_rag_techniques_runnable_scripts/simple_rag.py | 41 | # Create a retriever from the vector store | COMMENT |
| MEDIUM | …_rag_techniques_runnable_scripts/adaptive_retrieval.py | 193 | # Define the main Adaptive RAG class | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | llms.txt | 1 | # RAG Techniques | COMMENT |
| MEDIUM | llms.txt | 13 | ## Foundational RAG | COMMENT |
| MEDIUM | llms.txt | 52 | ## Iterative and Self-Correcting RAG | COMMENT |
| MEDIUM | llms.txt | 61 | ## Graph RAG | COMMENT |
| MEDIUM⚡ | helper_functions.py | 166 | # Define the prompt template for chain-of-thought reasoning | COMMENT |
| MEDIUM | README.md | 3 | # Advanced RAG Techniques 🚀 | COMMENT |
| MEDIUM | README.md | 255 | ### 🌱 Foundational RAG Techniques | COMMENT |
| MEDIUM | CONTRIBUTING.md | 1 | # Contributing to RAG Techniques | COMMENT |
| MEDIUM | CONTRIBUTING.md | 41 | ## Adding a New RAG Method | COMMENT |
| MEDIUM | CONTRIBUTING.md | 80 | ### 1. [Simple RAG 🌱](https://colab.research.google.com/github/NirDiamant/RAG_Techniques/blob/main/all_rag_techniques/si | COMMENT |
| MEDIUM | …unnable_scripts/HyPE_Hypothetical_Prompt_Embeddings.py | 185 | # Initialize the HyPE-based RAG Retriever | COMMENT |
| MEDIUM | …l_rag_techniques_runnable_scripts/choose_chunk_size.py | 75 | # Define the main class for the RAG method | COMMENT |
| MEDIUM | …_techniques_runnable_scripts/contextual_compression.py | 109 | # Main function to run the RAG pipeline | COMMENT |
| MEDIUM | …nnable_scripts/HyDe_Hypothetical_Document_Embedding.py | 76 | # Create and run the RAG method instance | COMMENT |
| MEDIUM | …iques_runnable_scripts/retrieval_with_feedback_loop.py | 104 | # Define the main RAG class | STRING |
| MEDIUM | …able_scripts/context_enrichment_window_around_chunk.py | 74 | # Main class that encapsulates the RAG method | COMMENT |
| MEDIUM | …able_scripts/context_enrichment_window_around_chunk.py | 139 | # Initialize and run the RAG method | STRING |
| MEDIUM | …g_techniques_runnable_scripts/query_transformations.py | 62 | # Main class for the RAG method | COMMENT |
| MEDIUM⚡ | all_rag_techniques_runnable_scripts/graph_rag.py | 809 | # Create a graph RAG instance | COMMENT |
| MEDIUM⚡ | all_rag_techniques_runnable_scripts/graph_rag.py | 815 | # Input a query and get the retrieved information from the graph RAG | COMMENT |
| MEDIUM | all_rag_techniques_runnable_scripts/graph_rag.py | 731 | # Define the graph RAG class | COMMENT |
| MEDIUM | …_rag_techniques_runnable_scripts/adaptive_retrieval.py | 193 | # Define the main Adaptive RAG class | COMMENT |
| MEDIUM | all_rag_techniques/README.md | 3 | ## RAG Techniques - notebook index | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_imports.py | 2 | CODE | |
| LOW | …unnable_scripts/HyPE_Hypothetical_Prompt_Embeddings.py | 14 | CODE | |
| LOW | …unnable_scripts/HyPE_Hypothetical_Prompt_Embeddings.py | 15 | CODE | |
| LOW | …_techniques_runnable_scripts/contextual_compression.py | 9 | CODE | |
| LOW | …_techniques_runnable_scripts/contextual_compression.py | 10 | CODE | |
| LOW | …nnable_scripts/HyDe_Hypothetical_Document_Embedding.py | 9 | CODE | |
| LOW | …nnable_scripts/HyDe_Hypothetical_Document_Embedding.py | 10 | CODE | |
| LOW | all_rag_techniques_runnable_scripts/raptor.py | 22 | CODE | |
| LOW | all_rag_techniques_runnable_scripts/raptor.py | 23 | CODE | |
| LOW | all_rag_techniques_runnable_scripts/light_rag.py | 9 | CODE | |
| LOW | …l_rag_techniques_runnable_scripts/semantic_chunking.py | 6 | CODE | |
| LOW | …g_techniques_runnable_scripts/document_augmentation.py | 9 | CODE | |
| LOW | …g_techniques_runnable_scripts/document_augmentation.py | 9 | CODE | |
| LOW | …g_techniques_runnable_scripts/document_augmentation.py | 21 | CODE | |
| LOW | …iques_runnable_scripts/retrieval_with_feedback_loop.py | 8 | CODE | |
| LOW | …iques_runnable_scripts/retrieval_with_feedback_loop.py | 14 | CODE | |
| LOW | …iques_runnable_scripts/retrieval_with_feedback_loop.py | 15 | CODE | |
| LOW | …able_scripts/context_enrichment_window_around_chunk.py | 2 | CODE | |
| LOW | …able_scripts/context_enrichment_window_around_chunk.py | 5 | CODE | |
| LOW | …able_scripts/context_enrichment_window_around_chunk.py | 6 | CODE | |
| LOW | …g_techniques_runnable_scripts/explainable_retrieval.py | 6 | CODE | |
| LOW | …g_techniques_runnable_scripts/explainable_retrieval.py | 7 | CODE | |
| LOW | all_rag_techniques_runnable_scripts/reranking.py | 14 | CODE | |
| LOW | all_rag_techniques_runnable_scripts/reranking.py | 15 | CODE | |
| LOW | all_rag_techniques_runnable_scripts/graph_rag.py | 18 | CODE | |
| LOW | all_rag_techniques_runnable_scripts/graph_rag.py | 29 | CODE | |
| LOW | all_rag_techniques_runnable_scripts/graph_rag.py | 33 | CODE | |
| LOW | all_rag_techniques_runnable_scripts/graph_rag.py | 34 | CODE | |
| LOW | all_rag_techniques_runnable_scripts/self_rag.py | 10 | CODE | |
| LOW | all_rag_techniques_runnable_scripts/self_rag.py | 11 | CODE | |
| LOW | …ag_techniques_runnable_scripts/hierarchical_indices.py | 8 | CODE | |
| LOW | …ag_techniques_runnable_scripts/hierarchical_indices.py | 8 | CODE | |
| LOW | …ag_techniques_runnable_scripts/hierarchical_indices.py | 11 | CODE | |
| LOW | …ag_techniques_runnable_scripts/hierarchical_indices.py | 12 | CODE | |
| LOW | all_rag_techniques_runnable_scripts/simple_rag.py | 10 | CODE | |
| LOW | all_rag_techniques_runnable_scripts/simple_rag.py | 11 | CODE | |
| LOW | all_rag_techniques_runnable_scripts/fusion_retrieval.py | 11 | CODE | |
| LOW | all_rag_techniques_runnable_scripts/fusion_retrieval.py | 12 | CODE | |
| LOW | …_rag_techniques_runnable_scripts/adaptive_retrieval.py | 9 | CODE | |
| LOW | …_rag_techniques_runnable_scripts/adaptive_retrieval.py | 10 | CODE | |
| LOW | …_rag_techniques_runnable_scripts/adaptive_retrieval.py | 10 | CODE | |
| LOW | …_rag_techniques_runnable_scripts/adaptive_retrieval.py | 11 | CODE | |
| LOW | …_rag_techniques_runnable_scripts/adaptive_retrieval.py | 17 | CODE | |
| LOW | …_rag_techniques_runnable_scripts/adaptive_retrieval.py | 18 | CODE | |
| LOW | evaluation/evalute_rag.py | 16 | CODE | |
| LOW | evaluation/evalute_rag.py | 17 | CODE | |
| LOW | evaluation/evalute_rag.py | 19 | CODE | |
| LOW | evaluation/evalute_rag.py | 34 | CODE | |
| LOW | evaluation/evalute_rag.py | 34 | CODE | |
| LOW | evaluation/evalute_rag.py | 34 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | …_techniques_runnable_scripts/contextual_compression.py | 42 | # Step 1: Create a vector store | COMMENT |
| LOW⚡ | …_techniques_runnable_scripts/contextual_compression.py | 45 | # Step 2: Create a retriever | COMMENT |
| LOW⚡ | …_techniques_runnable_scripts/contextual_compression.py | 48 | # Step 3: Initialize language model and create a contextual compressor | COMMENT |
| LOW⚡ | …_techniques_runnable_scripts/contextual_compression.py | 52 | # Step 4: Combine the retriever with the compressor | COMMENT |
| LOW⚡ | …_techniques_runnable_scripts/contextual_compression.py | 58 | # Step 5: Create a QA chain with the compressed retriever | COMMENT |
| LOW⚡ | all_rag_techniques_runnable_scripts/self_rag.py | 88 | # Step 1: Determine if retrieval is necessary | COMMENT |
| LOW⚡ | all_rag_techniques_runnable_scripts/self_rag.py | 95 | # Step 2: Retrieve relevant documents | COMMENT |
| LOW⚡ | all_rag_techniques_runnable_scripts/self_rag.py | 101 | # Step 3: Evaluate relevance of retrieved documents | COMMENT |
| LOW⚡ | all_rag_techniques_runnable_scripts/self_rag.py | 119 | # Step 4: Generate response using relevant contexts | COMMENT |
| LOW⚡ | all_rag_techniques_runnable_scripts/self_rag.py | 127 | # Step 5: Assess support | COMMENT |
| LOW⚡ | all_rag_techniques_runnable_scripts/self_rag.py | 133 | # Step 6: Evaluate utility | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | …_techniques_runnable_scripts/contextual_compression.py | 42 | # Step 1: Create a vector store | COMMENT |
| LOW⚡ | …_techniques_runnable_scripts/contextual_compression.py | 45 | # Step 2: Create a retriever | COMMENT |
| LOW⚡ | …_techniques_runnable_scripts/contextual_compression.py | 48 | # Step 3: Initialize language model and create a contextual compressor | COMMENT |
| LOW⚡ | …_techniques_runnable_scripts/contextual_compression.py | 52 | # Step 4: Combine the retriever with the compressor | COMMENT |
| LOW⚡ | …_techniques_runnable_scripts/contextual_compression.py | 58 | # Step 5: Create a QA chain with the compressed retriever | COMMENT |
| LOW⚡ | all_rag_techniques_runnable_scripts/self_rag.py | 88 | # Step 1: Determine if retrieval is necessary | COMMENT |
| LOW⚡ | all_rag_techniques_runnable_scripts/self_rag.py | 95 | # Step 2: Retrieve relevant documents | COMMENT |
| LOW⚡ | all_rag_techniques_runnable_scripts/self_rag.py | 101 | # Step 3: Evaluate relevance of retrieved documents | COMMENT |
| LOW⚡ | all_rag_techniques_runnable_scripts/self_rag.py | 119 | # Step 4: Generate response using relevant contexts | COMMENT |
| LOW⚡ | all_rag_techniques_runnable_scripts/self_rag.py | 127 | # Step 5: Assess support | COMMENT |
| LOW⚡ | all_rag_techniques_runnable_scripts/self_rag.py | 133 | # Step 6: Evaluate utility | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | helper_functions.py | 129 | def retrieve_context_per_question(question, chunks_query_retriever): | CODE |
| LOW⚡ | helper_functions.py | 162 | def create_question_answer_from_context_chain(llm): | CODE |
| LOW⚡ | helper_functions.py | 186 | def answer_question_from_context(question, context, question_answer_from_context_chain): | STRING |
| LOW | helper_functions.py | 294 | async def retry_with_exponential_backoff(coroutine, max_retries=5): | STRING |
| LOW | helper_functions.py | 338 | def get_langchain_embedding_provider(provider: EmbeddingProvider, model_id: str = None): | STRING |
| LOW | tests/test_imports.py | 8 | def execute_imports_from_notebook(notebook_path) -> None: | CODE |
| LOW | tests/test_imports.py | 42 | def execute_imports_from_script_files(script_path) -> None: | CODE |
| LOW | …unnable_scripts/HyPE_Hypothetical_Prompt_Embeddings.py | 49 | def generate_hypothetical_prompt_embeddings(self, chunk_text): | CODE |
| LOW | …l_rag_techniques_runnable_scripts/choose_chunk_size.py | 23 | def evaluate_response_time_and_accuracy(chunk_size, eval_questions, eval_documents, faithfulness_evaluator, | CODE |
| LOW | …l_rag_techniques_runnable_scripts/choose_chunk_size.py | 99 | def create_faithfulness_evaluator(self): | CODE |
| LOW | …l_rag_techniques_runnable_scripts/choose_chunk_size.py | 110 | def create_relevancy_evaluator(self): | STRING |
| LOW | …nnable_scripts/HyDe_Hypothetical_Document_Embedding.py | 34 | def generate_hypothetical_document(self, query): | CODE |
| LOW | …g_techniques_runnable_scripts/document_augmentation.py | 52 | def clean_and_filter_questions(questions: List[str]) -> List[str]: | CODE |
| LOW | …able_scripts/context_enrichment_window_around_chunk.py | 17 | def split_text_to_chunks_with_indices(text: str, chunk_size: int, chunk_overlap: int) -> List[Document]: | CODE |
| LOW | …able_scripts/context_enrichment_window_around_chunk.py | 38 | def retrieve_with_context_overlap(vectorstore, retriever, query: str, num_neighbors: int = 1, chunk_size: int = 200, | CODE |
| LOW⚡ | all_rag_techniques_runnable_scripts/graph_rag.py | 331 | def _create_answer_check_chain(self): | CODE |
| LOW | all_rag_techniques_runnable_scripts/graph_rag.py | 95 | def compute_similarity_matrix(self, embeddings): | CODE |
| LOW | all_rag_techniques_runnable_scripts/graph_rag.py | 206 | def _extract_concepts_and_entities(self, content, llm): | CODE |
| LOW | all_rag_techniques_runnable_scripts/graph_rag.py | 567 | def _retrieve_relevant_documents(self, query: str): | CODE |
| LOW | all_rag_techniques_runnable_scripts/fusion_retrieval.py | 20 | def encode_pdf_and_get_split_documents(path, chunk_size=1000, chunk_overlap=200): | CODE |
| LOW | …techniques_runnable_scripts/multi_faceted_filtering.py | 57 | def apply_similarity_threshold( | CODE |
| LOW | evaluation/evalute_rag.py | 40 | def create_deep_eval_test_cases( | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | helper_functions.py | 80 | Encodes a string into a vector store using OpenAI embeddings. Args: content (str): The text content to | STRING |
| HIGH | helper_functions.py | 295 | Retries a coroutine using exponential backoff upon encountering a RateLimitError. Args: coroutine: | STRING |
| HIGH | helper_functions.py | 339 | Returns an embedding provider based on the specified provider and model ID. Args: provider (EmbeddingP | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | helper_functions.py | 115 | # Assign metadata to each chunk | COMMENT |
| LOW | …_techniques_runnable_scripts/contextual_compression.py | 85 | # Display the result and the source documents | COMMENT |
| LOW | all_rag_techniques_runnable_scripts/graph_rag.py | 456 | # Check if we have a complete answer with the current context | COMMENT |
| LOW | all_rag_techniques_runnable_scripts/graph_rag.py | 497 | # Check if we have a complete answer after adding the neighbor's content | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | helper_functions.py | 123 | except Exception as e: | CODE |
| LOW | tests/test_imports.py | 33 | except Exception as e: | CODE |
| LOW | tests/test_imports.py | 64 | except Exception as e: | CODE |
| MEDIUM | all_rag_techniques_runnable_scripts/crag.py | 119 | print("Error parsing search results. Returning empty list.") | CODE |
| LOW | all_rag_techniques_runnable_scripts/light_rag.py | 156 | except Exception as e: | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_imports.py | 8 | CODE | |
| LOW | all_rag_techniques_runnable_scripts/light_rag.py | 149 | CODE | |
| LOW | all_rag_techniques_runnable_scripts/graph_rag.py | 363 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | …l_rag_techniques_runnable_scripts/choose_chunk_size.py | 26 | Evaluate the average response time, faithfulness, and relevancy of responses generated by GPT-3.5-turbo for a given | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | all_rag_techniques_runnable_scripts/crag.py | 91 | CODE |