Repository Analysis

pathwaycom/pathway

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

13.4 Low AI signal View on GitHub

Analysis Overview

This report presents the forensic synthetic code analysis of pathwaycom/pathway, a Python project with 62,670 GitHub stars. SynthScan v2.0 examined 373,868 lines of code across 1393 source files, recording 4169 pattern matches distributed across 22 syntactic categories. The overall adjusted score of 13.4 places this repository in the Low AI signal band.

The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).

13.4
Adjusted Score
13.4
Raw Score
100%
Time Factor
2026-07-14
Last Push
62.7K
Stars
Python
Language
373.9K
Lines of Code
1.4K
Files
4.2K
Pattern Hits
2026-07-14
Scan Date
0.09
HC Hit Rate

What These Metrics Mean

Adjusted Score
Primary synthetic code indicator. Raw score normalised per 1,000 lines of code and multiplied by the temporal discount factor. This is the definitive comparative metric — use it to rank repositories by AI authorship density.
Raw Score
The unmodified sum of all severity-weighted, context-multiplied pattern match scores before temporal discounting. Reflects the absolute signal strength independent of when the repository was last active.
Time Factor
The temporal discount multiplier (0–100%) applied to the raw score. Repositories last updated before ChatGPT's launch (Nov 2022) receive a 5% factor. Full signal is only assigned to repositories active in the post-adoption era (Jan 2024+).
Pattern Hits
Total count of individual pattern matches across all files and categories. A high hit count with a low score may indicate a very large codebase with isolated AI snippets; a low count with a high score indicates dense, concentrated AI signatures.
HC Hit Rate
High+Critical pattern hits per file, averaged across the repository. This orthogonal signal catches repositories where a few files are densely packed with high-severity AI tells — a strong indicator even when the normalised score appears moderate due to codebase size.
Lines of Code / Files
Total lines and files analysed. The scanner examines 94 file extensions. These denominators are used to normalise the score, enabling fair comparison between repositories of vastly different sizes.

Score History

This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.

Severity Breakdown

Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.

CRITICAL 6HIGH 121MEDIUM 382LOW 3660

Directory Score Breakdown

This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.

Pattern Findings

The scanner identified 4169 distinct pattern matches across 22 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.

Hyper-Verbose Identifiers1701 hits · 1339 pts
SeverityFileLineSnippetContext
LOWpython/pathway/conftest.py91def tmp_path_with_airbyte_config(tmp_path):CODE
LOWpython/pathway/cli.py53def validate_and_resolve_spawn_args(CODE
LOWpython/pathway/cli.py186def terminate_process_handles(process_handles: list[subprocess.Popen]):CODE
LOWpython/pathway/udfs.py23... async def long_running_async_function(a: int, b: int) -> int:STRING
LOWpython/pathway/io/_synchronization.py59def register_input_synchronization_group(CODE
LOWpython/pathway/io/_utils.py268def construct_raw_data_schema_by_flags(CODE
LOWpython/pathway/io/_utils.py285def construct_schema_and_data_format(CODE
LOWpython/pathway/io/_utils.py410def check_raw_and_plaintext_only_kwargs_for_message_queues(f):CODE
LOWpython/pathway/io/_utils.py619 def add_column_reference_to_extract(CODE
LOWpython/pathway/io/_utils.py660def maybe_schema_registry_settings(CODE
LOWpython/pathway/io/python/__init__.py331def _create_python_datasource(CODE
LOWpython/pathway/io/deltalake/__init__.py63 def _ensure_versions_are_consecutive(self):CODE
LOWpython/pathway/io/deltalake/__init__.py83 def _create_no_op_version_in_between(self, prev_version: int, current_version: int):CODE
LOWpython/pathway/io/deltalake/__init__.py273 def _get_actual_parquet_block_paths(self) -> set[str]:CODE
LOWpython/pathway/io/deltalake/__init__.py284 def _remove_obsolete_versions(self, tlog_file_names: list[str]):CODE
LOWpython/pathway/io/deltalake/__init__.py504def _read_table_schema_from_metadata(CODE
LOWpython/pathway/io/bigquery/__init__.py42 def _convert_to_compatible_format(self, row: dict[str, Any]):CODE
LOWpython/pathway/io/gdrive/__init__.py137 def _deduce_list_objects_strategy(self) -> _ListObjectsStrategy:CODE
LOWpython/pathway/io/gdrive/__init__.py170 def _traverse_objects_with_limit(self) -> list[GDriveFile]:CODE
LOWpython/pathway/io/gdrive/__init__.py211 def _detect_objects_with_full_scan(self) -> list[GDriveFile]:CODE
LOWpython/pathway/io/gdrive/__init__.py328 def _prepare_download_request(self, file: GDriveFile) -> Any:CODE
LOWpython/pathway/io/postgres/__init__.py131def _augment_postgres_settings(settings: dict, unique_name: str | None) -> dict:CODE
LOWpython/pathway/io/postgres/__init__.py145def _connection_string_from_settings(settings: dict):CODE
LOWpython/pathway/io/postgres/__init__.py154def _replication_connection_string_from_settings(settings: dict):CODE
LOWpython/pathway/io/postgres/__init__.py242def _construct_replication_settings(CODE
LOWpython/pathway/io/http/_common.py50 def wait_duration_before_retry(self):CODE
LOWpython/pathway/io/http/_server.py217 def _add_optional_traits_if_present(self, field_description, props):CODE
LOWpython/pathway/io/http/_server.py223 def _construct_openapi_plaintext_schema(self, schema) -> dict:CODE
LOWpython/pathway/io/http/_server.py242 def _construct_openapi_get_request_schema(self, schema) -> list:CODE
LOWpython/pathway/io/http/_server.py265 def _construct_openapi_json_schema(self, schema) -> dict:CODE
LOWpython/pathway/io/http/_server.py445 def _create_request_processor(self):CODE
LOWpython/pathway/io/http/_server.py572 def _wrap_handler_with_logger(CODE
LOWpython/pathway/io/airbyte/logic.py89 def _handle_stream_state_inner(self, stream):CODE
LOWpython/pathway/io/milvus/__init__.py69def _is_milvus_transient_connect_error(e: Exception) -> bool:CODE
LOWpython/pathway/tests/test_async_transformer.py304def test_with_instance_work_after_restart():STRING
LOWpython/pathway/tests/test_async_transformer.py348def test_fails_whole_instance():STRING
LOWpython/pathway/tests/test_async_transformer.py391def test_fails_on_too_many_columns():STRING
LOWpython/pathway/tests/test_async_transformer.py414def test_fails_on_not_enough_columns():STRING
LOWpython/pathway/tests/test_async_transformer.py470def test_consistent_when_instance_for_key_changes():STRING
LOWpython/pathway/tests/test_async_transformer.py503def test_requires_hashable_instance():STRING
LOWpython/pathway/tests/test_async_transformer.py568def test_commits_even_if_blocked_on_processing(tmp_path):STRING
LOWpython/pathway/tests/test_async_transformer.py597def test_commits_even_if_nothing_to_process(tmp_path):STRING
LOWpython/pathway/tests/test_deduplicate.py14def test_deduplicate_keeps_state(tmp_path: pathlib.Path):CODE
LOWpython/pathway/tests/test_deduplicate.py103def test_deduplicate_keeps_state_after_two_restarts(tmp_path: pathlib.Path):STRING
LOWpython/pathway/tests/test_deduplicate.py225def test_deduplicate_with_instance_keeps_state(tmp_path: pathlib.Path):STRING
LOWpython/pathway/tests/test_deduplicate.py303def test_deduplicate_keeps_state_after_code_change(tmp_path: pathlib.Path):STRING
LOWpython/pathway/tests/test_deduplicate.py369def test_deduplicate_keeps_state_with_regular_persistence(tmp_path: pathlib.Path):STRING
LOWpython/pathway/tests/test_deduplicate.py414def test_selective_persistence_name_set(STRING
LOWpython/pathway/tests/test_deduplicate.py485def test_selective_persistence_no_name_set_or_different_names_set(STRING
LOWpython/pathway/tests/test_deduplicate.py559def test_deduplicate_python_tuple():STRING
LOWpython/pathway/tests/test_io_leann.py51def test_on_change_deletion_nonexistent_key(tmp_path: pathlib.Path):CODE
LOWpython/pathway/tests/test_io_leann.py61def test_on_change_skips_empty_text(tmp_path: pathlib.Path):CODE
LOWpython/pathway/tests/test_io_leann.py86def test_on_change_with_metadata(tmp_path: pathlib.Path):CODE
LOWpython/pathway/tests/test_io_leann.py104def test_on_time_end_rebuilds_when_dirty(tmp_path: pathlib.Path):CODE
LOWpython/pathway/tests/test_io_leann.py115def test_on_end_builds_index_when_dirty(tmp_path: pathlib.Path):CODE
LOWpython/pathway/tests/test_io_leann.py125def test_on_end_skips_build_when_not_dirty_and_index_exists(tmp_path: pathlib.Path):CODE
LOWpython/pathway/tests/test_io_leann.py135def test_build_index_skips_when_no_documents(tmp_path: pathlib.Path):CODE
LOWpython/pathway/tests/test_io_leann.py144def test_build_index_with_documents(tmp_path: pathlib.Path):CODE
LOWpython/pathway/tests/test_io_leann.py182def test_observer_initialization_via_write_params(tmp_path: pathlib.Path):CODE
LOWpython/pathway/tests/test_io_leann.py203def test_check_leann_available_raises_on_missing_package():CODE
1641 more matches not shown…
Over-Commented Block915 hits · 874 pts
SeverityFileLineSnippetContext
LOWpython/pathway/io/gdrive/__init__.py41 "application/vnd.google-apps.document": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",COMMENT
LOWpython/pathway/io/gdrive/__init__.py61# 1. Traverse the directory using any graph-traversal algorithm, basically making oneCOMMENT
LOWpython/pathway/io/postgres/__init__.py61 return prefixCOMMENT
LOWpython/pathway/io/postgres/__init__.py81# Detection-time math: with ``keepalives_idle=300`` andCOMMENT
LOWpython/pathway/io/postgres/__init__.py161 def enc(v) -> str:COMMENT
LOWpython/pathway/tests/test_graphs.py221COMMENT
LOWpython/pathway/tests/test_persistence.py161# A scenario is executed as follows:COMMENT
LOWpython/pathway/tests/test_deltalake.py1461 time_start_5 = int(time.time() * 1000)COMMENT
LOWpython/pathway/tests/test_persistence_iterate.py301COMMENT
LOWpython/pathway/tests/test_persistence_iterate.py321#COMMENT
LOWpython/pathway/tests/test_persistence_iterate.py401 # Scenario: append new events AFTER all existing ones.COMMENT
LOWpython/pathway/tests/test_persistence_iterate.py421 # chunk=10 ────────────────────COMMENT
LOWpython/pathway/tests/test_persistence_iterate.py441 #COMMENT
LOWpython/pathway/tests/test_persistence_iterate.py461 # chunk=5 chunk=10 ─────────────COMMENT
LOWpython/pathway/tests/test_persistence_iterate.py481 # Events 30, 40 switch from chunk 30 → chunk 10.COMMENT
LOWpython/pathway/tests/test_persistence_iterate.py501 "flip_false_to_true": [COMMENT
LOWpython/pathway/tests/test_persistence_iterate.py521 "d": (40, False, "d")},COMMENT
LOWpython/pathway/tests/test_persistence_iterate.py541 {"c": (30, True, "c")},COMMENT
LOWpython/pathway/tests/test_persistence_iterate.py561 {"x": (55, True, "x")}, # splitCOMMENT
LOWpython/pathway/tests/test_persistence_iterate.py581 {"c": (30, False, "c"), "d": (40, True, "d")},COMMENT
LOWintegration_tests/kafka/test_nats.py21COMMENT
LOWintegration_tests/kafka/test_simple.py2481# ---------------------------------------------------------------------------COMMENT
LOWintegration_tests/kafka/test_backfilling.py41 # The input topic is fully populated before the program starts and doesCOMMENT
LOWintegration_tests/kafka/utils.py61# always transient, so the timeout is generous on purpose.COMMENT
LOWintegration_tests/kafka/utils.py81# request timed out (rstream's request timeout) before the broker answered;COMMENT
LOWintegration_tests/kafka/utils.py321 def read_topic(self, topic, poll_timeout_ms: int = 1000) -> list[ConsumerRecord]:COMMENT
LOWintegration_tests/kafka/utils.py761 await asyncio.sleep(RABBITMQ_RETRY_INTERVAL)COMMENT
LOWintegration_tests/kafka/utils.py881 # confirmation, so the message is durably committed — in its own streamCOMMENT
LOWintegration_tests/db_connectors/conftest.py241COMMENT
LOWintegration_tests/db_connectors/test_milvus_types.py1# Copyright © 2026 PathwayCOMMENT
LOWintegration_tests/db_connectors/test_mysql.py441 elapsed = time.monotonic() - startCOMMENT
LOWintegration_tests/db_connectors/test_mongodb.py821 # knows to delete the persistence directory rather than getting silentCOMMENT
LOWintegration_tests/db_connectors/utils.py61# NeonDB is exposed through the official ``neondatabase/neon_local`` proxyCOMMENT
LOWintegration_tests/db_connectors/utils.py181# Cross-process reader/writer lock for the shared binary log.COMMENT
LOWintegration_tests/db_connectors/utils.py261COMMENT
LOWintegration_tests/db_connectors/utils.py321# tightest bound; PostgreSQL handles concurrent replication better, so its capCOMMENT
LOWintegration_tests/db_connectors/utils.py341# catch-up (which only settles on two empty getMores) can't be kept spinning byCOMMENT
LOWintegration_tests/db_connectors/utils.py1201 # The native connector talks to Qdrant over gRPC, so the URL handed toCOMMENT
LOWintegration_tests/db_connectors/test_leann.py141COMMENT
LOWintegration_tests/db_connectors/test_leann.py161 # )COMMENT
LOWintegration_tests/db_connectors/test_postgres.py4961 augmented = _augment_postgres_settings(user_settings, unique_name="ignored")COMMENT
LOWtests/integration/test_polling.rs1// Copyright © 2026 PathwayCOMMENT
LOWtests/integration/test_cached_object_storage.rs81COMMENT
LOW…/4.user-guide/40.temporal-data/.window-join/article.py1# ---COMMENT
LOW…/4.user-guide/40.temporal-data/.window-join/article.py21# src: '/assets/content/tutorials/clickstream_window_join/window_join_illustrative.png'COMMENT
LOW…/4.user-guide/40.temporal-data/.window-join/article.py101COMMENT
LOW…poral-data/.interval_join_temporal_behavior/article.py1# ---COMMENT
LOW…poral-data/.interval_join_temporal_behavior/article.py81COMMENT
LOW…poral-data/.interval_join_temporal_behavior/article.py121# However, the maximal seen time in the `orders` stream when Frank's order arrived was $128$COMMENT
LOW…poral-data/.interval_join_temporal_behavior/article.py221 discounts_with_updates.price,COMMENT
LOW…s/4.user-guide/40.temporal-data/.asof-join/.article.py1# ---COMMENT
LOW…s/4.user-guide/40.temporal-data/.asof-join/.article.py21# * `Forward`: It finds the next row in the other table where the time is greater than or equal to the time in the curreCOMMENT
LOW…s/4.user-guide/40.temporal-data/.asof-join/.article.py41#COMMENT
LOW…s/4.user-guide/40.temporal-data/.asof-join/.article.py161preview_table(merged, time_col="event_t")COMMENT
LOW…40.temporal-data/.windows_temporal_behavior/article.py1# ---COMMENT
LOW…40.temporal-data/.windows_temporal_behavior/article.py21#COMMENT
LOW…de/40.temporal-data/.windowby-reduce-manual/article.py1# ---COMMENT
LOW….user-guide/40.temporal-data/.interval-join/article.py1# ---COMMENT
LOW….user-guide/40.temporal-data/.interval-join/article.py101# Pathway Live Data Framework has a function called `interval_join` (resp. `interval_join_{outer/left/right}`) which youCOMMENT
LOW…uide/20.connect/99.connectors/.fs-connector/article.py1# ---COMMENT
855 more matches not shown…
Unused Imports626 hits · 528 pts
SeverityFileLineSnippetContext
LOWpython/pathway/conftest.py3CODE
LOWpython/pathway/reducers.py23CODE
LOWpython/pathway/reducers.py23CODE
LOWpython/pathway/reducers.py23CODE
LOWpython/pathway/reducers.py28CODE
LOWpython/pathway/reducers.py28CODE
LOWpython/pathway/reducers.py28CODE
LOWpython/pathway/reducers.py28CODE
LOWpython/pathway/reducers.py28CODE
LOWpython/pathway/reducers.py28CODE
LOWpython/pathway/reducers.py28CODE
LOWpython/pathway/reducers.py28CODE
LOWpython/pathway/reducers.py28CODE
LOWpython/pathway/reducers.py28CODE
LOWpython/pathway/reducers.py28CODE
LOWpython/pathway/reducers.py28CODE
LOWpython/pathway/reducers.py28CODE
LOWpython/pathway/reducers.py28CODE
LOWpython/pathway/reducers.py28CODE
LOWpython/pathway/reducers.py28CODE
LOWpython/pathway/universes.py34CODE
LOWpython/pathway/universes.py34CODE
LOWpython/pathway/universes.py34CODE
LOWpython/pathway/__init__.py3CODE
LOWpython/pathway/__init__.py5CODE
LOWpython/pathway/__init__.py6CODE
LOWpython/pathway/__init__.py8CODE
LOWpython/pathway/__init__.py9CODE
LOWpython/pathway/__init__.py10CODE
LOWpython/pathway/__init__.py10CODE
LOWpython/pathway/__init__.py10CODE
LOWpython/pathway/__init__.py10CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
LOWpython/pathway/__init__.py11CODE
566 more matches not shown…
Modern AI Meta-Vocabulary152 hits · 472 pts
SeverityFileLineSnippetContext
MEDIUMCargo.toml114tantivy = "0.22.1" # Note: don't bump this dependency before the RAG integration test failure is investigatedCODE
MEDIUMpython/pathway/io/mongodb/__init__.py584 # vectors, for example the output of an embedder UDF in a RAG pipeline.STRING
MEDIUMpython/pathway/tests/test_io_duckdb.py431# Destination guardrails (Rust-side preflight)COMMENT
MEDIUMpython/pathway/tests/test_io_duckdb.py559# Rich types & RAG vectorsCOMMENT
MEDIUMpython/pathway/xpacks/llm/question_answering.py774 ... # define your agent, or custom RAG using any framework or plain PythonSTRING
MEDIUMintegration_tests/rag_evals/README.md1# Pathway Live Data Framework RAG evalsCOMMENT
MEDIUMintegration_tests/db_connectors/test_pinecone.py636# --- Configuration / type guardrails (validated up front, before any request) ---COMMENT
MEDIUM…pers/4.user-guide/90.development/10.troubleshooting.md106## Common errors with RAG applicationsCOMMENT
MEDIUM…s/2.developers/4.user-guide/50.llm-xpack/20.llm-app.md11# Create your own real-time RAG with Pathway Live Data FrameworkCOMMENT
MEDIUM…s/2.developers/4.user-guide/50.llm-xpack/20.llm-app.md33## RAG architectureCOMMENT
MEDIUM…s/2.developers/4.user-guide/50.llm-xpack/20.llm-app.md37- [**User Query**](#user-queries): A user inputs a query, which could be a question or a request for information. This qCODE
MEDIUM…s/2.developers/4.user-guide/50.llm-xpack/50.parsers.md35[`Utf8Parser`](/developers/api-docs/pathway-xpacks-llm/parsers#pathway.xpacks.llm.parsers.Utf8Parser) is a simple parserCODE
MEDIUM…s/2.developers/4.user-guide/50.llm-xpack/50.parsers.md248[`TwelveLabsVideoParser`](/developers/api-docs/pathway-xpacks-llm/parsers#pathway.xpacks.llm.parsers.TwelveLabsVideoParsCODE
MEDIUM…2.developers/4.user-guide/50.llm-xpack/70.embedders.md20- [`MarengoEmbedder`](#marengoembedder) - Embed text with TwelveLabs' multimodal Marengo model, for Video RAG pipelinesCODE
MEDIUM…2.developers/4.user-guide/50.llm-xpack/70.embedders.md141[`MarengoEmbedder`](/developers/api-docs/pathway-xpacks-llm/embedders#pathway.xpacks.llm.embedders.MarengoEmbedder) embeSTRING
MEDIUM…/2.developers/4.user-guide/50.llm-xpack/10.overview.md108To combine all the pieces into a [RAG](https://en.wikipedia.org/wiki/Retrieval-augmented_generation) (Retrieval AugmenteCODE
MEDIUM…velopers/4.user-guide/50.llm-xpack/.parsers/parsers.md35[`Utf8Parser`](/developers/api-docs/pathway-xpacks-llm/parsers#pathway.xpacks.llm.parsers.Utf8Parser) is a simple parserCODE
MEDIUM…velopers/4.user-guide/50.llm-xpack/.parsers/parsers.md248[`TwelveLabsVideoParser`](/developers/api-docs/pathway-xpacks-llm/parsers#pathway.xpacks.llm.parsers.TwelveLabsVideoParsCODE
MEDIUM…pers/4.user-guide/50.llm-xpack/.embedders/embedders.md20- [`MarengoEmbedder`](#marengoembedder) - Embed text with TwelveLabs' multimodal Marengo model, for Video RAG pipelinesCODE
MEDIUM…pers/4.user-guide/50.llm-xpack/.embedders/embedders.md141[`MarengoEmbedder`](/developers/api-docs/pathway-xpacks-llm/embedders#pathway.xpacks.llm.embedders.MarengoEmbedder) embeSTRING
MEDIUMdocs/2.developers/7.templates/30.configure-yaml.md8# Customizing Pathway Live Data Framework RAG Templates with YAML configuration filesCOMMENT
MEDIUMdocs/2.developers/7.templates/30.configure-yaml.md96## Example: Question-Answering RAGCOMMENT
MEDIUM…mplates/39.yaml-snippets/30.full-pipelines-examples.md19## Adaptive RAGCOMMENT
MEDIUM…mplates/39.yaml-snippets/30.full-pipelines-examples.md65# Manages the storage and retrieval of documents for the RAG template.COMMENT
MEDIUM…mplates/39.yaml-snippets/30.full-pipelines-examples.md72# Configures the question-answering component using the RAG approach.COMMENT
MEDIUM…mplates/39.yaml-snippets/30.full-pipelines-examples.md122# Manages the storage and retrieval of documents for the RAG template.COMMENT
MEDIUM…mplates/39.yaml-snippets/30.full-pipelines-examples.md129# Configures the question-answering component using the RAG approach.COMMENT
MEDIUM…mplates/39.yaml-snippets/30.full-pipelines-examples.md179# Manages the storage and retrieval of documents for the RAG template.COMMENT
MEDIUM…mplates/39.yaml-snippets/30.full-pipelines-examples.md186# Configures the question-answering component using the RAG approach.COMMENT
MEDIUM…mplates/39.yaml-snippets/30.full-pipelines-examples.md235# Manages the storage and retrieval of documents for the RAG template.COMMENT
MEDIUM…mplates/39.yaml-snippets/30.full-pipelines-examples.md242# Configures the question-answering component using the RAG approach.COMMENT
MEDIUM…ates/39.yaml-snippets/20.rag-configuration-examples.md9# RAG configuration YAML ExamplesCOMMENT
MEDIUM…ates/39.yaml-snippets/20.rag-configuration-examples.md39#### Standard RAGCOMMENT
MEDIUM…ates/39.yaml-snippets/20.rag-configuration-examples.md59#### Adaptive RAGCOMMENT
MEDIUM…ates/39.yaml-snippets/20.rag-configuration-examples.md370[`Utf8Parser`](/developers/api-docs/pathway-xpacks-llm/parsers#pathway.xpacks.llm.parsers.Utf8Parser) is a simple parserCODE
MEDIUM…ers/7.templates/ETL/_readmes/question-answering-rag.md1# Retrieval-Augmented Generation (RAG) Pipeline with PathwayCOMMENT
MEDIUM…ers/7.templates/ETL/_readmes/question-answering-rag.md28## RAG architectureCOMMENT
MEDIUM…elopers/7.templates/40.rag-customization/30.parsers.md35[`Utf8Parser`](/developers/api-docs/pathway-xpacks-llm/parsers#pathway.xpacks.llm.parsers.Utf8Parser) is a simple parserCODE
MEDIUM…elopers/7.templates/40.rag-customization/30.parsers.md248[`TwelveLabsVideoParser`](/developers/api-docs/pathway-xpacks-llm/parsers#pathway.xpacks.llm.parsers.TwelveLabsVideoParsCODE
MEDIUM…lopers/7.templates/40.rag-customization/10.REST-API.md9# Pathway Live Data Framework RAG REST APICOMMENT
MEDIUM…lopers/7.templates/40.rag-customization/10.REST-API.md24| [Answer with RAG](#answer-with-rag) | POST | `/v2/answer` | Generate an answer to a query using CODE
MEDIUM…lopers/7.templates/40.rag-customization/10.REST-API.md76## Answer with RAGCOMMENT
MEDIUM…opers/7.templates/40.rag-customization/50.embedders.md20- [`MarengoEmbedder`](#marengoembedder) - Embed text with TwelveLabs' multimodal Marengo model, for Video RAG pipelinesCODE
MEDIUM…opers/7.templates/40.rag-customization/50.embedders.md141[`MarengoEmbedder`](/developers/api-docs/pathway-xpacks-llm/embedders#pathway.xpacks.llm.embedders.MarengoEmbedder) embeSTRING
MEDIUM…s/7.templates/40.rag-customization/20.custom-prompt.md8# Customizing prompts in RAG templatesCOMMENT
MEDIUM…/2.developers/7.templates/rag/.adaptive-rag/article.py18# keywords: ['LLM', 'RAG', 'Adaptive RAG', 'prompt engineering', 'prompt', 'explainability', 'notebook', 'Docker']COMMENT
MEDIUM…/2.developers/7.templates/rag/.adaptive-rag/article.py23# # Adaptive RAG: cut your LLM costs without sacrificing accuracyCOMMENT
MEDIUM…/2.developers/7.templates/rag/.adaptive-rag/article.py26# <!-- Source for plots: https://github.com/pathwaycom/IoT-Pathway/blob/develop/experimental/szymon/geometric-RAG-plots/COMMENT
MEDIUM…/2.developers/7.templates/rag/.adaptive-rag/article.py28# **TLDR**: We demonstrate how to dynamically adapt the number of documents in a RAG prompt using feedback from the LLM.COMMENT
MEDIUM…/2.developers/7.templates/rag/.adaptive-rag/article.py40# Retrieval Augmented Generation (RAG) allows Large Language Models (LLMs) to answer questions based on knowledge not prCOMMENT
MEDIUM…/2.developers/7.templates/rag/.adaptive-rag/article.py42# A typical RAG Question Answering procedure works in two steps. First the question is analyzed and a number of relevantCOMMENT
MEDIUM…/2.developers/7.templates/rag/.adaptive-rag/article.py44# Practical implementations of the RAG procedure need to specify the number of documents put into the prompt. A large nuCOMMENT
MEDIUM…/2.developers/7.templates/rag/.adaptive-rag/article.py46# Thus the context size, given by the number of considered documents in a RAG setup, must be chosen to balance costs, deCOMMENT
MEDIUM…/2.developers/7.templates/rag/.adaptive-rag/article.py54# We base our evaluation on the Stanford Question Answering Dataset ([SQUAD](https://rajpurkar.github.io/SQuAD-explorer/COMMENT
MEDIUM…/2.developers/7.templates/rag/.adaptive-rag/article.py56# We form a document knowledge base from all wikipedia paragraphs indexed using their vector representations computed usCOMMENT
MEDIUM…/2.developers/7.templates/rag/.adaptive-rag/article.py78# The LLM can respond using a paraphrase of the ground-truth answer. To properly score paraphrases we canonicalize answeCOMMENT
MEDIUM…/2.developers/7.templates/rag/.adaptive-rag/article.py81# The chart below shows a typical relation between accuracy and supporting context size for a RAG question answering syCOMMENT
MEDIUM…/2.developers/7.templates/rag/.adaptive-rag/article.py83# ![Accuracy of base RAG](/assets/content/blog/adaptive-rag-plots/base-rag-accuracy.png)COMMENT
MEDIUM…/2.developers/7.templates/rag/.adaptive-rag/article.py87# We can get further insights into the operation of the RAG LLM by performing error analysis: categorizing each error asCOMMENT
MEDIUM…/2.developers/7.templates/rag/.adaptive-rag/article.py87# We can get further insights into the operation of the RAG LLM by performing error analysis: categorizing each error asCOMMENT
92 more matches not shown…
Decorative Section Separators100 hits · 347 pts
SeverityFileLineSnippetContext
MEDIUMpython/pathway/tests/test_persistence_iterate.py400 # -----------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py404 # chunk=10 ────────────── chunk=40 ──────────────COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py407 # chunk=70 ─────────────COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py410 # -----------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py417 # -----------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py421 # chunk=10 ────────────────────COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py425 # chunk=10 ────── chunk=25 ─────────────COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py429 # -----------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py436 # -----------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py440 # chunk=10 ─────────────COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py444 # chunk=10 ──────────────────COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py447 # -----------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py453 # -----------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py457 # chunk=10 ─────────────COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py461 # chunk=5 chunk=10 ─────────────COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py465 # -----------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py471 # -----------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py475 # chunk=10 ────── chunk=30 ──────COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py479 # chunk=10 ────────────────────COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py482 # -----------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py489 # -----------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py493 # chunk=10 ────────────────────COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py497 # chunk=10 ────── chunk=30 ──────COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py500 # -----------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py507 # -----------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py511 # chunk=10 ────────────────────COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py518 # -----------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py526 # -----------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py530 # chunk=10 ────────────────────────────COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py537 # -----------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py545 # -----------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py549 # chunk=10 ────── chunk=30 ────── chunk=50 ──────COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py556 # -----------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py564 # -----------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py568 # chunk=10 ──────COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py578 # -----------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py586 # -----------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py590 # chunk=10 ──────────────COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py600 # -----------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py303# ---------------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py305# ---------------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py331# ---------------------------------------------------------------------------COMMENT
MEDIUMpython/pathway/tests/test_persistence_iterate.py1402# ---------------------------------------------------------------------------STRING
MEDIUMpython/pathway/tests/test_persistence_iterate.py1406# ---------------------------------------------------------------------------STRING
MEDIUMintegration_tests/iceberg/test_iceberg.py1322# ----------------------------------------------------------------------------COMMENT
MEDIUMintegration_tests/iceberg/test_iceberg.py1326# ----------------------------------------------------------------------------COMMENT
MEDIUMintegration_tests/iceberg/test_iceberg.py595# ----------------------------------------------------------------------------COMMENT
MEDIUMintegration_tests/iceberg/test_iceberg.py603# ----------------------------------------------------------------------------COMMENT
MEDIUMintegration_tests/kafka/test_simple.py2321# ---------------------------------------------------------------------------COMMENT
MEDIUMintegration_tests/kafka/test_simple.py2328# ---------------------------------------------------------------------------COMMENT
MEDIUMintegration_tests/kafka/test_simple.py2374# ---------------------------------------------------------------------------COMMENT
MEDIUMintegration_tests/kafka/test_simple.py2382# ---------------------------------------------------------------------------COMMENT
MEDIUMintegration_tests/kafka/test_simple.py2427# ---------------------------------------------------------------------------COMMENT
MEDIUMintegration_tests/kafka/test_simple.py2434# ---------------------------------------------------------------------------COMMENT
MEDIUMintegration_tests/kafka/test_simple.py2481# ---------------------------------------------------------------------------COMMENT
MEDIUMintegration_tests/kafka/test_simple.py2491# ---------------------------------------------------------------------------COMMENT
MEDIUMintegration_tests/kafka/test_simple.py2537# ---------------------------------------------------------------------------COMMENT
MEDIUMintegration_tests/kafka/test_simple.py2547# ---------------------------------------------------------------------------COMMENT
MEDIUMintegration_tests/kafka/test_simple.py2581# ---------------------------------------------------------------------------COMMENT
MEDIUMintegration_tests/kafka/test_simple.py2589# ---------------------------------------------------------------------------COMMENT
40 more matches not shown…
Cross-File Repetition50 hits · 250 pts
SeverityFileLineSnippetContext
HIGHpython/pathway/tests/test_deduplicate.py0a | __time__ | __diff__ 1 | 2 | 1 1 | 4 | -1 1 | 6 | 1 2 | 8 | 1 1 | 10 | -1STRING
HIGHpython/pathway/tests/test_common.py0a | __time__ | __diff__ 1 | 2 | 1 1 | 4 | -1 1 | 6 | 1 2 | 8 | 1 1 | 10 | -1STRING
HIGHpython/pathway/tests/test_errors.py0a | __time__ | __diff__ 1 | 2 | 1 1 | 4 | -1 1 | 6 | 1 2 | 8 | 1 1 | 10 | -1STRING
HIGHpython/pathway/tests/test_udf.py0a | __time__ | __diff__ 1 | 2 | 1 1 | 4 | -1 1 | 6 | 1 2 | 8 | 1 1 | 10 | -1STRING
HIGHpython/pathway/tests/test_streaming_test_utils.py0a | __time__ | __diff__ 1 | 2 | 1 1 | 4 | -1 1 | 6 | 1 2 | 8 | 1 1 | 10 | -1STRING
HIGHpython/pathway/tests/test_utils.py0key | genus | epithet 1 | upupa | epops 2 | acherontia | atropos 3 | bubo | scandiacusSTRING
HIGHpython/pathway/tests/test_common.py0key | genus | epithet 1 | upupa | epops 2 | acherontia | atropos 3 | bubo | scandiacusSTRING
HIGHpython/pathway/tests/test_io.py0key | genus | epithet 1 | upupa | epops 2 | acherontia | atropos 3 | bubo | scandiacusSTRING
HIGHpython/pathway/tests/test_common.py0| pet | owner | age 1 | 7 | bob | 11 2 | 1 | bob | 9 3 | 2 | alice | 8 4 | 1 | bob | 7STRING
HIGHpython/pathway/tests/test_api.py0| pet | owner | age 1 | 7 | bob | 11 2 | 1 | bob | 9 3 | 2 | alice | 8 4 | 1 | bob | 7STRING
HIGHpython/pathway/tests/test_error_messages.py0| pet | owner | age 1 | 7 | bob | 11 2 | 1 | bob | 9 3 | 2 | alice | 8 4 | 1 | bob | 7STRING
HIGHpython/pathway/tests/test_build_and_run.py0| pet | owner | age 1 | 7 | bob | 11 2 | 1 | bob | 9 3 | 2 | alice | 8 4 | 1 | bob | 7STRING
HIGHpython/pathway/tests/test_common.py0| a | b | k 0 | 3 | 1 | 1 1 | 4 | 1 | 1 2 | 5 | 3 | 1 3 | 5 | 4 | 1 4 | 1 | 1 | 2STRING
HIGHpython/pathway/tests/test_errors.py0| a | b | k 0 | 3 | 1 | 1 1 | 4 | 1 | 1 2 | 5 | 3 | 1 3 | 5 | 4 | 1 4 | 1 | 1 | 2STRING
HIGHpython/pathway/tests/temporal/test_interval_joins.py0| a | b | k 0 | 3 | 1 | 1 1 | 4 | 1 | 1 2 | 5 | 3 | 1 3 | 5 | 4 | 1 4 | 1 | 1 | 2STRING
HIGHpython/pathway/tests/temporal/test_window_joins.py0| a | b | k 0 | 3 | 1 | 1 1 | 4 | 1 | 1 2 | 5 | 3 | 1 3 | 5 | 4 | 1 4 | 1 | 1 | 2STRING
HIGHpython/pathway/tests/test_operators.py0a | __time__ 1 | 2 2 | 2 3 | 2 4 | 2 5 | 2 6 | 2 7 | 2 8 | 2 9 | 6 10 | 8STRING
HIGHpython/pathway/tests/temporal/test_operators.py0a | __time__ 1 | 2 2 | 2 3 | 2 4 | 2 5 | 2 6 | 2 7 | 2 8 | 2 9 | 6 10 | 8STRING
HIGHpython/pathway/tests/temporal/test_asof_joins_stream.py0a | __time__ 1 | 2 2 | 2 3 | 2 4 | 2 5 | 2 6 | 2 7 | 2 8 | 2 9 | 6 10 | 8STRING
HIGHpython/pathway/tests/programs/debug.py0a | __time__ 1 | 2 2 | 2 3 | 2 4 | 2 5 | 2 6 | 2 7 | 2 8 | 2 9 | 6 10 | 8STRING
HIGH….user-guide/30.data-transformation/.iterate/article.py0a | __time__ 1 | 2 2 | 2 3 | 2 4 | 2 5 | 2 6 | 2 7 | 2 8 | 2 9 | 6 10 | 8STRING
HIGHpython/pathway/internals/joins.py0computes a result of a window join. args: args: column references. kwargs: column expressions with their new assigned naSTRING
HIGHpython/pathway/stdlib/temporal/_asof_now_join.py0computes a result of a window join. args: args: column references. kwargs: column expressions with their new assigned naSTRING
HIGHpython/pathway/stdlib/temporal/_interval_join.py0computes a result of a window join. args: args: column references. kwargs: column expressions with their new assigned naSTRING
HIGHpython/pathway/stdlib/temporal/_window_join.py0computes a result of a window join. args: args: column references. kwargs: column expressions with their new assigned naSTRING
HIGHpython/pathway/internals/expressions/numerical.py0a module containing methods related to string. they can be called using a `str` attribute of an expression. typical use:STRING
HIGHpython/pathway/internals/expressions/date_time.py0a module containing methods related to string. they can be called using a `str` attribute of an expression. typical use:STRING
HIGHpython/pathway/internals/expressions/string.py0a module containing methods related to string. they can be called using a `str` attribute of an expression. typical use:STRING
HIGHintegration_tests/db_connectors/test_chroma.py0rows with an existing primary key should be replaced, not duplicated.STRING
HIGHintegration_tests/db_connectors/test_pinecone.py0rows with an existing primary key should be replaced, not duplicated.STRING
HIGHintegration_tests/db_connectors/test_milvus.py0rows with an existing primary key should be replaced, not duplicated.STRING
HIGHintegration_tests/db_connectors/test_qdrant.py0rows with an existing primary key should be replaced, not duplicated.STRING
HIGHintegration_tests/db_connectors/test_weaviate.py0rows with an existing primary key should be replaced, not duplicated.STRING
HIGHintegration_tests/db_connectors/test_chroma.py0rows with diff=-1 should be removed from the collection.STRING
HIGHintegration_tests/db_connectors/test_milvus.py0rows with diff=-1 should be removed from the collection.STRING
HIGHintegration_tests/db_connectors/test_qdrant.py0rows with diff=-1 should be removed from the collection.STRING
HIGHintegration_tests/db_connectors/test_weaviate.py0rows with diff=-1 should be removed from the collection.STRING
HIGHintegration_tests/db_connectors/test_chroma.py0writing an empty table should succeed without errors.STRING
HIGHintegration_tests/db_connectors/test_pinecone.py0writing an empty table should succeed without errors.STRING
HIGHintegration_tests/db_connectors/test_milvus.py0writing an empty table should succeed without errors.STRING
HIGHintegration_tests/db_connectors/test_qdrant.py0writing an empty table should succeed without errors.STRING
HIGHintegration_tests/db_connectors/test_weaviate.py0writing an empty table should succeed without errors.STRING
HIGHintegration_tests/db_connectors/test_chroma.py0passing a vector column from a different table raises a clear valueerror.STRING
HIGHintegration_tests/db_connectors/test_milvus.py0passing a vector column from a different table raises a clear valueerror.STRING
HIGHintegration_tests/db_connectors/test_pinecone.py0passing a vector column from a different table raises a clear valueerror.STRING
HIGHintegration_tests/db_connectors/test_weaviate.py0passing a vector column from a different table raises a clear valueerror.STRING
HIGHintegration_tests/db_connectors/test_qdrant.py0passing a vector column from a different table raises a clear valueerror.STRING
HIGH…2.developers/4.user-guide/50.llm-xpack/80.llm-chats.md0questions | max_tokens how many 'r' there are in 'strawberry'? | 400STRING
HIGH…velopers/4.user-guide/50.llm-xpack/.chats/llm-chats.md0questions | max_tokens how many 'r' there are in 'strawberry'? | 400STRING
HIGH…opers/7.templates/40.rag-customization/60.llm-chats.md0questions | max_tokens how many 'r' there are in 'strawberry'? | 400STRING
Excessive Try-Catch Wrapping146 hits · 162 pts
SeverityFileLineSnippetContext
LOWpython/pathway/io/deltalake/__init__.py76 except Exception:CODE
LOWpython/pathway/io/deltalake/__init__.py235 except Exception:CODE
LOWpython/pathway/io/deltalake/__init__.py307 except Exception:CODE
LOWpython/pathway/io/http/_server.py483 except Exception:CODE
LOWpython/pathway/io/http/_server.py586 except Exception:CODE
LOWpython/pathway/io/http/_server.py696 except Exception as e:CODE
LOWpython/pathway/io/pubsub/__init__.py48 except Exception:CODE
LOWpython/pathway/io/airbyte/logic.py184 except Exception:CODE
LOWpython/pathway/io/milvus/__init__.py98 except Exception as e:CODE
MEDIUMpython/pathway/tests/utils.py421def __call__(self):CODE
MEDIUMpython/pathway/tests/utils.py490def checker():CODE
MEDIUMpython/pathway/tests/utils.py517def __call__(self):CODE
MEDIUMpython/pathway/tests/utils.py541def __call__(self):CODE
LOWpython/pathway/tests/utils.py122 except Exception as e:CODE
LOWpython/pathway/tests/utils.py424 except Exception:CODE
LOWpython/pathway/tests/utils.py498 except Exception:CODE
LOWpython/pathway/tests/utils.py528 except Exception as exception:CODE
LOWpython/pathway/tests/utils.py544 except Exception as exception:CODE
LOW…/expressions/test_non_existing_date_time_conversion.py303 except Exception: # FIXME: make conversions work in all casesCODE
LOWpython/pathway/xpacks/connectors/sharepoint/__init__.py213 except Exception as e:CODE
LOWpython/pathway/xpacks/llm/parsers.py984 except Exception:CODE
LOWpython/pathway/xpacks/llm/parsers.py1287 except Exception as e:CODE
LOWpython/pathway/xpacks/llm/parsers.py1319 except Exception as e:CODE
LOWpython/pathway/xpacks/llm/parsers.py1552 except Exception:CODE
LOWpython/pathway/xpacks/llm/parsers.py1590 except Exception: # noqa: BLE001CODE
LOWpython/pathway/xpacks/llm/embedders.py64 except Exception:CODE
LOWpython/pathway/xpacks/llm/embedders.py70 except Exception:CODE
MEDIUMpython/pathway/xpacks/llm/embedders.py32def _safe_aclose(self):CODE
LOWpython/pathway/xpacks/llm/prompts.py319 except Exception:STRING
LOWpython/pathway/xpacks/llm/tests/test_parsers.py23 except Exception:CODE
LOWpython/pathway/internals/trace.py133 except Exception as e:CODE
LOWpython/pathway/internals/trace.py143 except Exception as e:CODE
MEDIUMpython/pathway/internals/trace.py130def _pathway_trace_marker(*args, **kwargs):CODE
LOWpython/pathway/internals/udfs/retries.py94 except Exception:CODE
LOWpython/pathway/stdlib/utils/async_transformer.py166 except Exception:CODE
LOWintegration_tests/s3/base.py58 except Exception as exc:CODE
LOWintegration_tests/wordcount/base.py470 except Exception:CODE
LOWintegration_tests/wordcount/base.py479 except Exception:CODE
LOWintegration_tests/rag_evals/experiment.py34 except Exception:CODE
LOWintegration_tests/rag_evals/evaluator.py303 except Exception:CODE
MEDIUMintegration_tests/iceberg/test_iceberg.py112def __call__(self) -> bool:CODE
MEDIUMintegration_tests/iceberg/test_iceberg.py119def provide_information_on_failure(self) -> str:CODE
LOWintegration_tests/iceberg/test_iceberg.py116 except Exception:CODE
LOWintegration_tests/iceberg/test_iceberg.py129 except Exception as e:CODE
LOWintegration_tests/webserver/test_rest_connector.py332 except Exception:CODE
LOWintegration_tests/webserver/test_rest_connector.py559 except Exception:CODE
MEDIUMintegration_tests/webserver/test_rest_connector.py317def checker() -> bool:CODE
MEDIUMintegration_tests/webserver/test_rest_connector.py552def checker() -> bool:CODE
LOWintegration_tests/webserver/test_llm_xpack.py75 except Exception:CODE
LOWintegration_tests/webserver/test_llm_xpack.py289 except Exception:CODE
LOWintegration_tests/webserver/test_llm_xpack.py343 except Exception:CODE
LOWintegration_tests/webserver/test_llm_xpack.py417 except Exception:CODE
LOWintegration_tests/webserver/test_llm_xpack.py461 except Exception:CODE
LOWintegration_tests/webserver/test_llm_xpack.py513 except Exception:CODE
LOWintegration_tests/webserver/test_llm_xpack.py551 except Exception:CODE
MEDIUMintegration_tests/webserver/test_llm_xpack.py283def checker() -> bool:CODE
MEDIUMintegration_tests/webserver/test_llm_xpack.py333def checker() -> bool:CODE
MEDIUMintegration_tests/webserver/test_llm_xpack.py395def checker() -> bool:CODE
LOWintegration_tests/kafka/test_nats.py83 except Exception:CODE
LOWintegration_tests/kafka/test_nats.py143 except Exception:CODE
86 more matches not shown…
Slop Phrases55 hits · 151 pts
SeverityFileLineSnippetContext
LOWpython/pathway/io/iceberg/__init__.py183 Don't forget to run your program with ``pw.run`` once you define all necessarySTRING
LOWpython/pathway/io/iceberg/__init__.py309 Don't forget to run your program with ``pw.run`` once you define all necessarySTRING
MEDIUMpython/pathway/io/kafka/__init__.py129 Consider a Kafka queue running locally on port 9092. For demonstration purposes, ourSTRING
MEDIUMpython/pathway/io/kafka/__init__.py618 Consider a Kafka queue running locally on port 9092. For demonstration purposes, ourSTRING
LOWpython/pathway/io/pubsub/__init__.py122 At last, don't forget to add ``pw.run()`` to run your pipeline.STRING
MEDIUM…/4.user-guide/40.temporal-data/.window-join/article.py124# Here's how you can do it:COMMENT
MEDIUM…poral-data/.interval_join_temporal_behavior/article.py22# To deliver streaming data in a production system, you can use a connector like Kafka or Redpanda (you can read more abCOMMENT
MEDIUM…poral-data/.interval_join_temporal_behavior/article.py88# As such, their orders are not present in the `result` table. If you want to include all orders (also those that can't COMMENT
MEDIUM…40.temporal-data/.windows_temporal_behavior/article.py16# The examples in this article use the sliding windows, but you can also use behaviors with tumbling windows.COMMENT
MEDIUM…40.temporal-data/.windows_temporal_behavior/article.py58# To that end, you can use sliding windows. To keep things simple, start with a piece of code that only groups data intoCOMMENT
MEDIUM….user-guide/40.temporal-data/.interval-join/article.py101# Pathway Live Data Framework has a function called `interval_join` (resp. `interval_join_{outer/left/right}`) which youCOMMENT
MEDIUM….user-guide/40.temporal-data/.interval-join/article.py101# Pathway Live Data Framework has a function called `interval_join` (resp. `interval_join_{outer/left/right}`) which youCOMMENT
MEDIUM…uide/20.connect/99.connectors/.fs-connector/article.py121# In most cases, in order to read the data, you need to define its schema and pass it to the connector. Furthermore, forCOMMENT
MEDIUM…velopers/4.user-guide/20.connect/.datatypes/article.py80# Sometimes you may want to compute a value of a column, using e.g. a function from an external library, that does not dCOMMENT
MEDIUM…velopers/4.user-guide/20.connect/.datatypes/article.py156# Similarly, as in the conversion between `str` and `bytes`, you can use `apply_with_type` to convert a column of type `COMMENT
MEDIUM…velopers/4.user-guide/20.connect/.datatypes/article.py187# As in the case of conversion from `int` to `float`, you can use `pw.cast` to convert data from type `float` to `int`.COMMENT
MEDIUM…/4.user-guide/20.connect/.schema_generation/article.py80# If you have a CSV file with sample data, you can use it to generate a schema in Pathway Live Data Framework. The name COMMENT
MEDIUM…/4.user-guide/20.connect/.schema_generation/article.py101# For the example, you can use the default values of the optional arguments, so the Python code is:COMMENT
MEDIUM…/4.user-guide/20.connect/.schema_generation/article.py116# To do that, you can use a method in the Schema class called `generate_class` to generate a string with a class definitCOMMENT
MEDIUM…10.introduction/.streaming-and-static-modes/article.py33# In practice, you would need two input connectors to create T1 and T2, you can use the previous sample of code to buildCOMMENT
MEDIUM…pers/4.user-guide/70.migrating/.from_pandas/article.py115# The arguments to `select` are dynamic; you can use them to define the desired column names and then pass the content. COMMENT
MEDIUMdocs/2.developers/4.user-guide/.sql/article.py76# With the `SELECT ...` and `WHERE ...` clauses, you can use the following operators:COMMENT
MEDIUMdocs/2.developers/4.user-guide/.sql/article.py133# With `GROUP BY`, you can use the following aggregation functions:COMMENT
MEDIUM….user-guide/30.data-transformation/.iterate/article.py315# Now you can use the `bellman_ford` in iterate. Note the [`pw.iterate_universe`](/developers/api-docs/pathway#pathway.iCOMMENT
MEDIUM….user-guide/30.data-transformation/.iterate/article.py324# If you wish, you can try simulating changes in the graph over time by adding `__time__` and `__diff__` columns to `pw.COMMENT
MEDIUM…/30.data-transformation/.indexes-in-pathway/article.py109# Monitoring changes of answers to your queries might not be what you want. Especially if you have **a lot of** queries.COMMENT
MEDIUM…/30.data-transformation/.indexes-in-pathway/article.py157# In contrast to an ordinary [`join`](/developers/user-guide/data-transformation/join-manual), `asof_now_join` is not syCOMMENT
MEDIUM…/30.data-transformation/.indexes-in-pathway/article.py177# If you don't want answers to your queries to be updated, you can use [`get_nearest_items_asof_now`](/developers/api-doCOMMENT
MEDIUM…/30.data-transformation/.indexes-in-pathway/article.py247# To receive the queries, you can use the [`rest_connector`](/developers/api-docs/pathway-io/http#pathway.io.http.rest_cCOMMENT
MEDIUM…/30.data-transformation/.indexes-in-pathway/article.py274# To answer the queries, you can use the `get_nearest_items_asof_now` method of the index. It'll find the nearest pointsCOMMENT
MEDIUM…/30.data-transformation/.indexes-in-pathway/article.py370# To intercept messages in a stream, you can use the [`pw.io.subscribe`](/developers/api-docs/pathway-io#pathway.io.subsCOMMENT
MEDIUM…e/30.data-transformation/.async-transformer/article.py14# However, if the flexibility of the UDFs is still not enough, you can use even more general and flexible `AsyncTransforCOMMENT
MEDIUM…e/30.data-transformation/.async-transformer/article.py203# If you want to guarantee an order within a group of records, you can use the `instance` parameter of the `AsyncTransfoCOMMENT
MEDIUM…data-transformation/.user-defined-functions/article.py35# Now you can use it as an ordinary Pathway Live Data Framework expression, as in the example shown below.COMMENT
MEDIUM…data-transformation/.user-defined-functions/article.py77# For example, you can't add a column with type `Any` to a column of type `int` (you also can't add `Any` to `Any`), butCOMMENT
MEDIUM…data-transformation/.user-defined-functions/article.py107# Also note that the `inc` function is only present in this guide for demonstration purposes.COMMENT
MEDIUM…data-transformation/.user-defined-functions/article.py411# Note that accidentally you created a sleepsort. Values in the `finishing` messages are sorted! As an exercise, you canCOMMENT
MEDIUM…r-guide/30.data-transformation/.join-manual/article.py125# that here, you can use columns of both tables as arguments, e.g.COMMENT
MEDIUM…r-guide/30.data-transformation/.join-manual/article.py479# To handle joins using those columns, you can use [`pointer_from`](/developers/api-docs/pathway-table#pathway.Table.poiCOMMENT
MEDIUM…r-guide/30.data-transformation/.join-manual/article.py499# `join_left` in `people_allergies`, you can use the following:COMMENT
MEDIUM…2.developers/7.templates/ETL/.option-greeks/article.py28# The Pathway Live Data Framework comes with an easy-to-use Python API, syntax that is simple and intuitive, and you canCOMMENT
MEDIUM…2.developers/7.templates/ETL/.option-greeks/article.py373# To compute the average prices for all bids, you can use a simple [groupby/reduce](https://pathway.com/developers/user-COMMENT
MEDIUM…2.developers/7.templates/ETL/.option-greeks/article.py686# Furthermore, you can use _[Databento live APIs](https://databento.com/docs/api-reference-live?historical=python&live=pCOMMENT
LOW…2.developers/7.templates/ETL/.option-greeks/article.py127# To continue, make sure to install all the needed packages.COMMENT
MEDIUM…lopers/7.templates/ETL/.merging-time-series/article.py261# The value 1224744825 is hand-picked to fit this dataset; you can replace it by any desired value.COMMENT
MEDIUM…lopers/7.templates/ETL/.merging-time-series/article.py363# To compute the distance, you can use the `pyproj` package, which you can install by uncommenting and executing the folCOMMENT
MEDIUM…lopers/7.templates/ETL/.merging-time-series/article.py386# Note this is the simple 2-dimensional distance, but you can use your favorite distance metric.COMMENT
MEDIUM…lopers/7.templates/ETL/.merging-time-series/article.py398# Then, you can use `ix` method to fetch values from corresponding `prev` and `next` rows, and use them to compute the iCOMMENT
MEDIUM…lopers/7.templates/ETL/.fuzzy_join_chapter1/article.py204# If you would like to get some more experience with Pathway Live Data Framework, you can try those two challenges:COMMENT
MEDIUM…velopers/7.templates/ETL/.live_data_jupyter/article.py43# Because the Pathway Live Data Framework [unites static and stream processing](/developers/user-guide/connect/switch-frCOMMENT
MEDIUM…velopers/7.templates/ETL/.live_data_jupyter/article.py154# The code is analogous to the 20-minute statistics but simpler: you can use a `tumbling window` and don't have to compuCOMMENT
MEDIUM…emplates/ETL/.interval_over_gaussian_filter/article.py211# To do so, you can use `windowby`, that takes three arguments:COMMENT
MEDIUM….developers/7.templates/rag/.multimodal-rag/article.py299# For implementing this example using open source LLMs, here’s a [private RAG app template](https://pathway.com/developeCOMMENT
MEDIUM….developers/7.templates/rag/.multimodal-rag/article.py210# default file you can use to testCOMMENT
MEDIUM…7.templates/rag/.private_rag_ollama_mistral/article.py53# To tackle this, you can use a **private RAG: locally deployed LLMs and embedders in your RAG pipeline**.COMMENT
Cross-Language Confusion34 hits · 145 pts
SeverityFileLineSnippetContext
HIGHpython/pathway/io/__init__.py31 null,CODE
HIGHpython/pathway/tests/test_errors.py1282{"a": 7, "b": 1, "c": null}CODE
HIGHpython/pathway/tests/test_io.py584 {"k": "b", "b": 2, "c": null }CODE
HIGHpython/pathway/tests/test_io.py261 assert result.equals(expected)STRING
HIGHpython/pathway/tests/test_io.py439 assert result.equals(expected)STRING
HIGHpython/pathway/tests/test_io.py2984 "_airbyte_data": '{"type": "STREAM", "stream": {"stream_descriptor": {"name": "commits", "namespace": nuSTRING
HIGHpython/pathway/tests/test_io.py3002 "_airbyte_data": '{"type": "STREAM", "stream": {"stream_descriptor": {"name": "not_commits", "namespace"STRING
HIGHpython/pathway/tests/test_io.py3024 "_airbyte_data": '{"type": "STREAM", "stream": {"stream_descriptor": {"name": "commits", "namespace": nuSTRING
HIGHpython/pathway/tests/test_io.py3468 assert result.equals(expected)STRING
HIGHpython/pathway/tests/test_io.py3480 assert result.equals(expected)STRING
HIGHpython/pathway/tests/test_deltalake.py140 assert final.equals(original)STRING
HIGHpython/pathway/tests/test_deltalake.py659 '{"append_only": false, "description": null, "dtype": {"type": "INT"}, 'STRING
HIGHpython/pathway/tests/utils.py500 return expected.equals(result)CODE
HIGHpython/pathway/tests/test_json.py674 ('"{""a"": 1,""b"": ""foo"", ""c"": null, ""d"": [1,2,3]}"', dict),CODE
HIGHpython/pathway/tests/test_json.py691 ('{"field": 1, "b": "foo", "c": null, "d": [1,2,3]}', dict),CODE
HIGHpython/pathway/internals/expression.py333 - returns `json(null)` for JsonSTRING
HIGHpython/pathway/stdlib/utils/pandas_transformer.py70 if output_universe_arg_index is not None and not result.index.equals(CODE
HIGHintegration_tests/s3/test_s3_deltalake.py196 assert final.equals(original)CODE
HIGHintegration_tests/s3/test_s3_deltalake.py238 assert final.equals(original)STRING
HIGHintegration_tests/s3/test_s3_generic.py102 assert result.equals(expected)CODE
HIGHintegration_tests/s3/test_s3_generic.py218 assert result.equals(expected)CODE
HIGHintegration_tests/s3/test_s3_generic.py275 assert result.equals(expected)CODE
HIGHintegration_tests/s3/test_s3_generic.py308 assert result.equals(expected)CODE
HIGHintegration_tests/iceberg/test_iceberg.py1403 # `important` is required (non-null) — Pathway must supply it.COMMENT
HIGHintegration_tests/iceberg/test_iceberg.py1417 ), f"error should mention that the column is required / non-null, got: {msg}"CODE
HIGHintegration_tests/iceberg/test_iceberg.py1390 """Existing iceberg table has a required (non-null) column that the user'sSTRING
HIGHintegration_tests/kafka/test_rabbitmq.py140 # AMQP 1.0 property fields exist (may be null)COMMENT
HIGH…gration_tests/db_connectors/test_clickhouse_parsing.py186 "quote'semicolon;null,./;'l][p!@#!@#!@#%$#$#]",CODE
HIGHintegration_tests/db_connectors/test_mssql_parsing.py396 "quote'semicolon;null,./;'l][p!@#!@#!@#%$#$#]",CODE
HIGHintegration_tests/db_connectors/test_mongodb_parsing.py314 "quote'semicolon;null,./;'l][p!@#!@#!@#%$#$#]",CODE
HIGH…tegration_tests/db_connectors/test_postgres_parsing.py260 "quote'semicolon;null,./;'l][p!@#!@#!@#%$#$#]",CODE
HIGH…tion_tests/db_connectors/test_elasticsearch_parsing.py248 "quote'semicolon;null,./;'l][p!@#!@#!@#%$#$#]",CODE
HIGHintegration_tests/db_connectors/test_mysql_parsing.py316 "quote'semicolon;null,./;'l][p!@#!@#!@#%$#$#]",CODE
HIGHintegration_tests/db_connectors/test_postgres.py1088 null,CODE
Docstring Block Structure28 hits · 140 pts
SeverityFileLineSnippetContext
HIGHpython/pathway/io/_subscribe.py26 Calls a callback function ``on_change`` on every change happening in table. Args: table: the table to STRING
HIGHpython/pathway/io/http/_server.py101 Adds an example to the collection. Args: id: Short and unique ID for the example. It is usSTRING
HIGHpython/pathway/io/kafka/__init__.py556Write a table to a given topic on a Kafka instance. The produced messages consist of the key, corresponding to row'STRING
HIGHpython/pathway/io/pubsub/__init__.py62Publish the ``table``'s stream of changes into the specified PubSub topic. Please note that ``table`` must consist oSTRING
HIGHpython/pathway/io/duckdb/__init__.py55Writes ``table`` into a table of a `DuckDB <https://duckdb.org/>`_ database file. DuckDB is an in-process analyticalSTRING
HIGHpython/pathway/internals/table_subscription.py189 Calls a callback function on_change on every change happening in table. This method is similar to the one we exSTRING
HIGHpython/pathway/internals/expressions/date_time.py385Returns a number of (nano,micro,milli)seconds from 1970-01-01 for naive DateTime and from 1970-01-01 UTC for timSTRING
HIGHpython/pathway/internals/expressions/date_time.py663Converts DateTimeNaive to UTC from time zone provided as `from_timezone` argument. If the given DateTime doesn'tSTRING
HIGHpython/pathway/internals/expressions/date_time.py753Converts DateTimeUtc to time zone specified as `timezone` argument. Args: timezone: The time zone tSTRING
HIGHpython/pathway/internals/expressions/date_time.py978Rounds DateTime to precision specified by `duration` argument. Args: duration: rounding precision STRING
HIGHpython/pathway/internals/expressions/date_time.py1050Truncates DateTime to precision specified by `duration` argument. Args: duration: truncation precisSTRING
HIGHpython/pathway/stdlib/temporal/_interval_join.py65Allows testing whether two times are within a certain distance. Note: Usually used as an argument of `.inteSTRING
HIGHpython/pathway/stdlib/temporal/_interval_join.py589Performs an interval join of self with other using a time difference and join expressions. If `self_time + lower_bouSTRING
HIGHpython/pathway/stdlib/temporal/_interval_join.py793Performs an interval join of self with other using a time difference and join expressions. If `self_time + lower_bouSTRING
HIGHpython/pathway/stdlib/temporal/_interval_join.py992Performs an interval left join of self with other using a time difference and join expressions. If `self_time + loweSTRING
HIGHpython/pathway/stdlib/temporal/_interval_join.py1209Performs an interval right join of self with other using a time difference and join expressions. If `self_time + lowSTRING
HIGHpython/pathway/stdlib/temporal/_interval_join.py1415Performs an interval outer join of self with other using a time difference and join expressions. If `self_time + lowSTRING
HIGHpython/pathway/stdlib/temporal/_window.py506Allows grouping together elements within a window across ordered time-like data column by locally grouping adjacent STRING
HIGHpython/pathway/stdlib/temporal/_window.py571Allows grouping together elements within a window of a given length sliding across ordered time-like data column accSTRING
HIGHpython/pathway/stdlib/temporal/_window.py645Allows grouping together elements within a window of a given length tumbling across ordered time-like data column stSTRING
HIGHpython/pathway/stdlib/temporal/_window.py706Allows grouping together elements within a window. Windows are created for each time t in at, by taking values withSTRING
HIGHpython/pathway/stdlib/temporal/_window_join.py167Performs a window join of self with other using a window and join expressions. If two records belong to the same winSTRING
HIGHpython/pathway/stdlib/temporal/_window_join.py370Performs a window join of self with other using a window and join expressions. If two records belong to the same winSTRING
HIGHpython/pathway/stdlib/temporal/_window_join.py571Performs a window left join of self with other using a window and join expressions. If two records belong to the samSTRING
HIGHpython/pathway/stdlib/temporal/_window_join.py791Performs a window right join of self with other using a window and join expressions. If two records belong to the saSTRING
HIGHpython/pathway/stdlib/temporal/_window_join.py1006Performs a window outer join of self with other using a window and join expressions. If two records belong to the saSTRING
HIGHpython/pathway/stdlib/statistical/_interpolate.py62 Interpolates missing values in a column using the previous and next values based on a timestamps column. Args:STRING
HIGHpython/pathway/stdlib/ordered/diff.py16 Compute the difference between the values in the ``values`` columns and the previous values according to the orSTRING
AI Slop Vocabulary38 hits · 122 pts
SeverityFileLineSnippetContext
MEDIUMintegration_tests/db_connectors/utils.py329# server-side behavior; a fully robust fix would need a resumable (non-temporary)COMMENT
MEDIUMintegration_tests/db_connectors/test_mongodb_parsing.py33# MSSQL CDC twin of this harness. The streaming reader can live neither on aCOMMENT
MEDIUM…/4.user-guide/40.temporal-data/.window-join/article.py122# The next step is to calculate the conversion rate, i.e., the proportion of product page visits that resulted in a purcCOMMENT
MEDIUM…/4.user-guide/40.temporal-data/.window-join/article.py170# And that concludes the demonstration of the comprehensive, data-driven approach to analyzing user behavior and conversCOMMENT
MEDIUM…s/4.user-guide/40.temporal-data/.asof-join/.article.py36# 1. **Temporal Alignment**: Realtime data is time-stamped and is often required to be analyzed in conjunction with othCOMMENT
MEDIUM…s/4.user-guide/40.temporal-data/.asof-join/.article.py51# In the scope of this tutorial, you will be leveraging data from a public API in a static mode purely for illustrative COMMENT
MEDIUM…s/4.user-guide/40.temporal-data/.asof-join/.article.py125# During the development and testing stages of your data processing pipeline, it's essential to validate that the data iCOMMENT
MEDIUM…de/40.temporal-data/.windowby-reduce-manual/article.py10# Pathway Live Data Framework offers powerful features for time series data manipulation. One such feature is the `windoCOMMENT
MEDIUM…de/40.temporal-data/.windowby-reduce-manual/article.py135# In this guide, you've mastered the use of the windowby-reduce scheme in the Pathway Live Data Framework library, a robCOMMENT
MEDIUM…de/40.temporal-data/.windowby-reduce-manual/article.py135# In this guide, you've mastered the use of the windowby-reduce scheme in the Pathway Live Data Framework library, a robCOMMENT
MEDIUM…de/40.temporal-data/.windowby-reduce-manual/article.py135# In this guide, you've mastered the use of the windowby-reduce scheme in the Pathway Live Data Framework library, a robCOMMENT
MEDIUM….user-guide/40.temporal-data/.interval-join/article.py12# This article offers a comprehensive tutorial on how to use the interval join function in Pathway Live Data Framework tCOMMENT
MEDIUM…velopers/4.user-guide/20.connect/.datatypes/article.py398# Below are a few examples demonstrating the `diff` operator. Essentially, it sorts the table with respect to one columnSTRING
MEDIUM…velopers/4.user-guide/20.connect/.json_type/article.py144# You can utilize the [`flatten()`](/developers/api-docs/pathway-table#pathway.internals.table.Table.flatten) operator sCOMMENT
MEDIUM…uide/80.advanced/.declarative_vs_imperative/article.py14# Many real-world data processing tasks — such as those in logistics, supply chain management, or event stream analysis—COMMENT
MEDIUM…uide/80.advanced/.declarative_vs_imperative/article.py16# In data processing, imperative pipelines require you to specify step-by-step instructions for how data should be transCOMMENT
MEDIUM…opers/4.user-guide/80.advanced/.consistency/article.py117# Given that money was transferred from one account to another without any external funds entering the system, the totalCOMMENT
MEDIUM…r-guide/30.data-transformation/.join-manual/article.py508# Essentially, for a join `left.join(right, *on)`, `pw.left` allows us to address the `left` tableCOMMENT
MEDIUM…lopers/7.templates/ETL/.merging-time-series/article.py97# Combining time series is the process of combining different time series into a single time series based on a common tiCOMMENT
MEDIUM…lopers/7.templates/ETL/.merging-time-series/article.py106# - Environmental monitoring 🌡️: Combining time series data from air quality sensors, weather stations, and pollutant lCOMMENT
MEDIUM…lopers/7.templates/ETL/.merging-time-series/article.py108# - Analyzing stock market data 📈: Combining time series of stock prices, trading volumes, and financial indicators canCOMMENT
MEDIUM…lopers/7.templates/ETL/.merging-time-series/article.py460# By combining these time series, you could create a more comprehensive picture of the conditions affecting your area's COMMENT
MEDIUM…s/7.templates/ETL/.interval_over_upsampling/article.py324# Standard digital signal processing systems assume robust sampling and would deal with out-of-order and irregularly samCOMMENT
MEDIUM…emplates/ETL/.interval_over_gaussian_filter/article.py120# I will not delve into details: if you are interested you can check out the [Wikipedia article](https://en.wikipedia.orCOMMENT
MEDIUM…emplates/ETL/.interval_over_gaussian_filter/article.py496# Now that you understand the differences between idealized signal processing and data stream processing, I'm sure you cCOMMENT
MEDIUM…/2.developers/7.templates/rag/.adaptive-rag/article.py78# The LLM can respond using a paraphrase of the ground-truth answer. To properly score paraphrases we canonicalize answeCOMMENT
MEDIUM…/2.developers/7.templates/rag/.adaptive-rag/article.py78# The LLM can respond using a paraphrase of the ground-truth answer. To properly score paraphrases we canonicalize answeCOMMENT
MEDIUM….developers/7.templates/rag/.multimodal-rag/article.py62# Currently, most [RAG applications](/blog/retrieval-augmented-generation-beginners-guide-rag-apps) are mostly limited tCOMMENT
MEDIUM….developers/7.templates/rag/.multimodal-rag/article.py80# Multimodal search and Multimodal RAG (MM-RAG) both utilize deep learning models to create an embedding space—a map of COMMENT
MEDIUM….developers/7.templates/rag/.multimodal-rag/article.py111# Building a multimodal RAG system for production requires a robust and scalable architecture that can handle diverse daCOMMENT
MEDIUM….developers/7.templates/rag/.multimodal-rag/article.py128# - **Pathway Live Data Framework**: Provides real-time synchronization, secure document handling, and a robust in-memorCOMMENT
MEDIUM….developers/7.templates/rag/.multimodal-rag/article.py142# 2. **Answering Questions**: Questions are sent to the LLM with the relevant context, including parsed tables. This allCOMMENT
MEDIUM….developers/7.templates/rag/.multimodal-rag/article.py182# - **[llm.parsers.DoclingParser](/developers/api-docs/pathway-xpacks-llm/parsers)**: The `DoclingParser` class efficienCOMMENT
LOWexamples/projects/option-greeks/greeks-replay.py212# We now simply add the volatility column, using the `select` operation on a Table.COMMENT
MEDIUMexternal/timely-dataflow/timely/src/worker.rs28/// The `ProgressMode::Demand` variant is the most robust, and leastCOMMENT
MEDIUM…l/timely-dataflow/timely/src/dataflow/operators/map.rs71 // TODO : This would be more robust if it captured an iterator and then pulled an appropriateCOMMENT
MEDIUMsrc/connectors/synchronization.rs9/// implementation but holding a unique source ID and metadata. Essentially, an accessorCOMMENT
MEDIUMsrc/connectors/data_storage/aws/kinesis.rs173 // an equal chance to utilize this extra time.COMMENT
Deep Nesting104 hits · 92 pts
SeverityFileLineSnippetContext
LOWpython/pathway/cli.py150CODE
LOWpython/pathway/cli.py211CODE
LOWpython/pathway/io/_utils.py285CODE
LOWpython/pathway/io/_utils.py437CODE
LOWpython/pathway/io/deltalake/__init__.py193CODE
LOWpython/pathway/io/deltalake/__init__.py203CODE
LOWpython/pathway/io/gdrive/__init__.py474CODE
LOWpython/pathway/io/postgres/__init__.py605CODE
LOWpython/pathway/io/http/_server.py669CODE
LOWpython/pathway/io/pyfilesystem/__init__.py49CODE
LOWpython/pathway/io/airbyte/logic.py58CODE
LOWpython/pathway/io/airbyte/logic.py177CODE
LOWpython/pathway/tests/test_persistence.py222CODE
LOWpython/pathway/tests/test_io.py82CODE
LOWpython/pathway/tests/test_io.py3573CODE
LOWpython/pathway/tests/test_io.py4053CODE
LOWpython/pathway/tests/test_io.py85CODE
LOWpython/pathway/tests/test_io.py4066CODE
LOWpython/pathway/tests/test_io.py3594CODE
LOWpython/pathway/tests/utils.py717CODE
LOWpython/pathway/tests/test_unpack_snapshots.py58CODE
LOWpython/pathway/tests/test_persistence_iterate.py615CODE
LOWpython/pathway/tests/temporal/test_windows_stream.py85CODE
LOWpython/pathway/tests/temporal/test_window_joins.py25CODE
LOWpython/pathway/tests/temporal/test_window_joins.py811CODE
LOWpython/pathway/xpacks/connectors/sharepoint/__init__.py109CODE
LOWpython/pathway/xpacks/llm/parsers.py183CODE
LOWpython/pathway/xpacks/llm/parsers.py525CODE
LOWpython/pathway/xpacks/llm/parsers.py610CODE
LOWpython/pathway/xpacks/llm/llms.py66CODE
LOWpython/pathway/xpacks/llm/llms.py830CODE
LOWpython/pathway/xpacks/llm/llms.py873CODE
LOWpython/pathway/xpacks/llm/_parser_utils.py312CODE
LOWpython/pathway/internals/type_interpreter.py158CODE
LOWpython/pathway/internals/type_interpreter.py502CODE
LOWpython/pathway/internals/dtype.py643CODE
LOWpython/pathway/internals/dtype.py841CODE
LOWpython/pathway/internals/dtype.py881CODE
LOWpython/pathway/internals/dtype.py90CODE
LOWpython/pathway/internals/monitoring.py211CODE
LOWpython/pathway/internals/joins.py621CODE
LOWpython/pathway/internals/joins.py669CODE
LOWpython/pathway/internals/api.py132CODE
LOWpython/pathway/internals/api.py197CODE
LOWpython/pathway/internals/arg_handlers.py72CODE
LOWpython/pathway/internals/arg_handlers.py73CODE
LOWpython/pathway/internals/custom_reducers.py282CODE
LOWpython/pathway/internals/custom_reducers.py324CODE
LOWpython/pathway/internals/custom_reducers.py326CODE
LOWpython/pathway/internals/table.py1192CODE
LOWpython/pathway/internals/expression_printer.py37CODE
LOWpython/pathway/internals/fingerprints.py20CODE
LOWpython/pathway/internals/column.py1064CODE
LOWpython/pathway/internals/schema.py55CODE
LOWpython/pathway/internals/graph_runner/storage_graph.py137CODE
LOWpython/pathway/internals/graph_runner/__init__.py136CODE
LOWpython/pathway/internals/graph_runner/__init__.py201CODE
LOWpython/pathway/internals/graph_runner/telemetry.py163CODE
LOW…hon/pathway/internals/graph_runner/operator_handler.py104CODE
LOW…rnals/graph_runner/row_transformer_operator_handler.py67CODE
44 more matches not shown…
AI Structural Patterns72 hits · 72 pts
SeverityFileLineSnippetContext
LOWpython/pathway/io/_utils.py285CODE
LOWpython/pathway/io/_utils.py437CODE
LOWpython/pathway/io/s3/__init__.py95CODE
LOWpython/pathway/io/s3/__init__.py335CODE
LOWpython/pathway/io/s3/__init__.py489CODE
LOWpython/pathway/io/chroma/__init__.py27CODE
LOWpython/pathway/io/deltalake/__init__.py326CODE
LOWpython/pathway/io/rabbitmq/__init__.py27CODE
LOWpython/pathway/io/gdrive/__init__.py519CODE
LOWpython/pathway/io/minio/__init__.py59CODE
LOWpython/pathway/io/pinecone/__init__.py106CODE
LOWpython/pathway/io/redpanda/__init__.py19CODE
LOWpython/pathway/io/postgres/__init__.py284CODE
LOWpython/pathway/io/postgres/__init__.py605CODE
LOWpython/pathway/io/nats/__init__.py24CODE
LOWpython/pathway/io/http/__init__.py28CODE
LOWpython/pathway/io/http/__init__.py151CODE
LOWpython/pathway/io/http/_server.py723CODE
LOWpython/pathway/io/kafka/__init__.py33CODE
LOWpython/pathway/io/kafka/__init__.py412CODE
LOWpython/pathway/io/kafka/__init__.py541CODE
LOWpython/pathway/io/csv/__init__.py18CODE
LOWpython/pathway/io/jsonlines/__init__.py18CODE
LOWpython/pathway/io/airbyte/__init__.py112CODE
LOWpython/pathway/io/airbyte/logic.py51CODE
LOWpython/pathway/io/mqtt/__init__.py22CODE
LOWpython/pathway/io/fs/__init__.py32CODE
LOWpython/pathway/io/weaviate/__init__.py18CODE
LOWpython/pathway/tests/test_error_messages.py285CODE
LOW…n/pathway/tests/external_index/test_brute_force_knn.py60CODE
LOWpython/pathway/xpacks/connectors/sharepoint/__init__.py267CODE
LOWpython/pathway/xpacks/llm/question_answering.py181CODE
LOWpython/pathway/xpacks/llm/question_answering.py303CODE
LOWpython/pathway/xpacks/llm/question_answering.py891CODE
LOWpython/pathway/xpacks/llm/question_answering.py949CODE
LOWpython/pathway/xpacks/llm/question_answering.py1181CODE
LOWpython/pathway/xpacks/llm/question_answering.py1211CODE
LOWpython/pathway/xpacks/llm/parsers.py714CODE
LOWpython/pathway/xpacks/llm/parsers.py878CODE
LOWpython/pathway/xpacks/llm/parsers.py1154CODE
LOWpython/pathway/xpacks/llm/parsers.py1349CODE
LOWpython/pathway/xpacks/llm/parsers.py1476CODE
LOWpython/pathway/xpacks/llm/embedders.py678CODE
LOWpython/pathway/xpacks/llm/llms.py890CODE
LOWpython/pathway/xpacks/llm/mcp_server.py250CODE
LOWpython/pathway/internals/run.py13CODE
LOWpython/pathway/internals/run.py77CODE
LOWpython/pathway/internals/thisclass.py173CODE
LOWpython/pathway/internals/schema.py756CODE
LOWpython/pathway/internals/schema.py927CODE
LOWpython/pathway/internals/schema.py475CODE
LOWpython/pathway/internals/graph_runner/__init__.py49CODE
LOWpython/pathway/stdlib/temporal/_asof_join.py83CODE
LOWpython/pathway/stdlib/temporal/time_utils.py229CODE
LOWpython/pathway/stdlib/temporal/_window.py842CODE
LOWpython/pathway/stdlib/temporal/_window.py356CODE
LOWpython/pathway/stdlib/temporal/_window.py103CODE
LOWpython/pathway/stdlib/utils/pandas_transformer.py121CODE
LOWpython/pathway/stdlib/utils/col.py188CODE
LOWpython/pathway/stdlib/utils/col.py350CODE
12 more matches not shown…
Structural Annotation Overuse40 hits · 63 pts
SeverityFileLineSnippetContext
LOW…rs/4.user-guide/60.deployment/20.aws-fargate-deploy.md106### Step 1: Login Into AWS CLICOMMENT
LOW…rs/4.user-guide/60.deployment/20.aws-fargate-deploy.md127### Step 2: Register Task Definition In AWSCOMMENT
LOW…rs/4.user-guide/60.deployment/20.aws-fargate-deploy.md189### Step 4: Launch The ContainerCOMMENT
LOW…pers/4.user-guide/60.deployment/25.azure-aci-deploy.md172### Step 1: Performing Azure ConfigurationCOMMENT
LOW…pers/4.user-guide/60.deployment/25.azure-aci-deploy.md226### Step 2: Authenticating in DockerhubCOMMENT
LOW…pers/4.user-guide/60.deployment/25.azure-aci-deploy.md242### Step 3: Configuring Backend for Delta Lake StorageCOMMENT
LOW…pers/4.user-guide/60.deployment/25.azure-aci-deploy.md263### Step 4: Providing Pathway Live Data Framework License Key and Github PATCOMMENT
LOW…pers/4.user-guide/60.deployment/25.azure-aci-deploy.md278### Step 5: Configuring a Container in Azure Container InstancesCOMMENT
LOW…pers/4.user-guide/60.deployment/25.azure-aci-deploy.md387### Step 6: Container Group CreationCOMMENT
LOW…pers/4.user-guide/60.deployment/25.azure-aci-deploy.md421### Step 7: Launch The ContainerCOMMENT
LOW…elopers/7.templates/60.deploy/20.aws-fargate-deploy.md106### Step 1: Login Into AWS CLICOMMENT
LOW…elopers/7.templates/60.deploy/20.aws-fargate-deploy.md127### Step 2: Register Task Definition In AWSCOMMENT
LOW…elopers/7.templates/60.deploy/20.aws-fargate-deploy.md189### Step 4: Launch The ContainerCOMMENT
LOW…evelopers/7.templates/60.deploy/25.azure-aci-deploy.md172### Step 1: Performing Azure ConfigurationCOMMENT
LOW…evelopers/7.templates/60.deploy/25.azure-aci-deploy.md226### Step 2: Authenticating in DockerhubCOMMENT
LOW…evelopers/7.templates/60.deploy/25.azure-aci-deploy.md242### Step 3: Configuring Backend for Delta Lake StorageCOMMENT
LOW…evelopers/7.templates/60.deploy/25.azure-aci-deploy.md263### Step 4: Providing Pathway Live Data Framework License Key and Github PATCOMMENT
LOW…evelopers/7.templates/60.deploy/25.azure-aci-deploy.md278### Step 5: Configuring a Container in Azure Container InstancesCOMMENT
LOW…evelopers/7.templates/60.deploy/25.azure-aci-deploy.md387### Step 6: Container Group CreationCOMMENT
LOW…evelopers/7.templates/60.deploy/25.azure-aci-deploy.md421### Step 7: Launch The ContainerCOMMENT
LOW…al/timely-dataflow/timely/src/progress/reachability.rs570 // Step 0: If logging is enabled, construct and log inbound changes.COMMENT
LOW…al/timely-dataflow/timely/src/progress/reachability.rs594 // Step 1: Drain `self.input_changes` and determine actual frontier changes.COMMENT
LOW…al/timely-dataflow/timely/src/progress/reachability.rs638 // Step 2: Circulate implications of changes to `self.pointstamps`.COMMENT
LOW…ferential-dataflow/mdbook/src/chapter_0/chapter_0_3.md1## Step 3: Make things more exciting.COMMENT
LOW…ferential-dataflow/mdbook/src/chapter_0/chapter_0_2.md1## Step 2: Change its input.COMMENT
LOW…ferential-dataflow/mdbook/src/chapter_0/chapter_0_1.md1## Step 1: Write a program.COMMENT
LOWexternal/differential-dataflow/src/capture.rs349 // Step 1: The MESSAGES operator.COMMENT
LOWexternal/differential-dataflow/src/capture.rs427 // Step 2: The UPDATES operator.COMMENT
LOWexternal/differential-dataflow/src/capture.rs473 // Step 3: The PROGRESS operator.COMMENT
LOWexternal/differential-dataflow/src/capture.rs556 // Step 4: The FEEDBACK operator.COMMENT
LOWsrc/connectors/data_storage/mssql.rs1452 // Step 1: record LSN before snapshot read.COMMENT
LOWsrc/connectors/data_storage/mssql.rs1461 // Step 2: read the full table snapshot.COMMENT
LOWsrc/connectors/data_storage/mssql.rs1464 // Step 3: record LSN after snapshot read.COMMENT
LOWsrc/connectors/data_storage/mssql.rs1471 // Step 4: fetch CDC changes that raced with the snapshot read.COMMENT
LOWsrc/connectors/data_storage/mssql.rs1150 /// Step 1: Record the current CDC max LSN (retry until the capture agentCOMMENT
LOWsrc/connectors/data_storage/mssql.rs1176 /// Step 2: Read the full table snapshot via a plain SELECT.COMMENT
LOWsrc/connectors/data_storage/mssql.rs1190 /// Step 3: Record the CDC max LSN again, immediately after the snapshot SELECT.COMMENT
LOWsrc/connectors/data_storage/mssql.rs1299 /// Step 4: Fetch CDC changes that raced with the snapshot read.COMMENT
LOWsrc/connectors/data_storage/mssql.rs1488 // Step 5: pick the offset that gets stamped on the snapshotCOMMENT
LOWsrc/connectors/data_storage/mssql.rs1548 // Step 6: advance last_lsn so poll_cdc_changes starts strictly afterCOMMENT
Hallucination Indicators6 hits · 60 pts
SeverityFileLineSnippetContext
CRITICAL…pers/4.user-guide/60.deployment/25.azure-aci-deploy.md293from azure.core.exceptions import HttpResponseErrorCODE
CRITICAL…evelopers/7.templates/60.deploy/25.azure-aci-deploy.md293from azure.core.exceptions import HttpResponseErrorCODE
CRITICALexamples/projects/azure-aci-deploy/launch.py6from azure.core.exceptions import HttpResponseErrorCODE
CRITICAL…differential-dataflow/src/trace/implementations/ord.rs315 let starting_updates = self.result.vals.vals.vals.len();CODE
CRITICAL…differential-dataflow/src/trace/implementations/ord.rs323 effort = (self.result.vals.vals.vals.len() - starting_updates) as isize;CODE
CRITICAL…differential-dataflow/src/trace/implementations/ord.rs348 effort = (self.result.vals.vals.vals.len() - starting_updates) as isize;CODE
Modern Structural Boilerplate52 hits · 52 pts
SeverityFileLineSnippetContext
LOWpython/pathway/reducers.py47__all__ = [CODE
LOWpython/pathway/universes.py40__all__ = [CODE
LOWpython/pathway/__init__.py93__all__ = [CODE
LOWpython/pathway/udfs.py69__all__ = [CODE
LOWpython/pathway/io/__init__.py59__all__ = [CODE
LOWpython/pathway/io/s3/__init__.py641__all__ = [CODE
LOWpython/pathway/io/chroma/__init__.py185__all__ = ["write"]CODE
LOWpython/pathway/io/deltalake/__init__.py704__all__ = [CODE
LOWpython/pathway/io/leann/__init__.py21logger = logging.getLogger(__name__)CODE
LOWpython/pathway/io/leann/__init__.py293__all__ = ["write"]CODE
LOWpython/pathway/io/pinecone/__init__.py336__all__ = ["write"]CODE
LOWpython/pathway/io/http/__init__.py290__all__ = [CODE
LOWpython/pathway/io/kafka/__init__.py784__all__ = [CODE
LOWpython/pathway/io/duckdb/__init__.py396__all__ = ["write"]CODE
LOWpython/pathway/io/weaviate/__init__.py182__all__ = ["write"]CODE
LOWpython/pathway/io/elasticsearch/__init__.py378__all__ = [CODE
LOWpython/pathway/xpacks/llm/__init__.py17__all__ = [CODE
LOWpython/pathway/xpacks/llm/parsers.py42logger = logging.getLogger(__name__)CODE
LOWpython/pathway/xpacks/llm/embedders.py22__all__ = [CODE
LOWpython/pathway/xpacks/llm/llms.py27logger = logging.getLogger(__name__)CODE
LOWpython/pathway/xpacks/llm/rerankers.py13logger = logging.getLogger(__name__)CODE
LOWpython/pathway/xpacks/llm/_parser_utils.py38logger = logging.getLogger(__name__)CODE
LOWpython/pathway/internals/config.py142def set_license_key(key: str | None) -> None:CODE
LOWpython/pathway/internals/config.py193__all__ = [CODE
LOWpython/pathway/internals/dtype.py615 def _set_args(self, wrapped: type) -> None:CODE
LOWpython/pathway/internals/monitoring.py206 def update_monitoring(self, data: Any, now: int) -> None:CODE
LOWpython/pathway/internals/operator.py163 def set_error_log(self, error_log: tables.Table | None) -> None:CODE
LOWpython/pathway/internals/__init__.py81__all__ = [CODE
LOWpython/pathway/internals/yaml_loader.py236__all__ = ["load_yaml"]CODE
LOWpython/pathway/internals/json.py256__all__ = ["Json", "JsonValue"]CODE
LOW…pathway/internals/graph_runner/expression_evaluator.py151 def set_temporary_table(self, storage: Storage, table: api.Table) -> None:CODE
LOW…pathway/internals/graph_runner/expression_evaluator.py158 def set_non_deterministic(self) -> None:CODE
LOWpython/pathway/internals/graph_runner/__init__.py285__all__ = [CODE
LOWpython/pathway/internals/graph_runner/state.py148 def set_table(self, storage: Storage, table: api.Table) -> None:CODE
LOWpython/pathway/internals/graph_runner/state.py160 def set_error_log(self, table: table.Table, error_log: api.ErrorLog) -> None:CODE
LOWpython/pathway/internals/udfs/__init__.py44__all__ = [CODE
LOWpython/pathway/internals/expressions/__init__.py7__all__ = ["DateTimeNamespace", "NumericalNamespace", "StringNamespace"]CODE
LOWpython/pathway/stdlib/temporal/__init__.py45__all__ = [CODE
LOWpython/pathway/stdlib/stateful/__init__.py7__all__ = [CODE
LOWpython/pathway/stdlib/utils/__init__.py7__all__ = [CODE
LOWpython/pathway/stdlib/utils/async_transformer.py272 def _set_failure(self, key: Pointer, task_id: Pointer) -> None:CODE
LOWpython/pathway/stdlib/viz/__init__.py6__all__ = ["plot", "show", "_repr_mimebundle_"]CODE
LOWpython/pathway/stdlib/ml/__init__.py7__all__ = ["classifiers", "datasets", "hmm", "smart_table_ops", "utils"]CODE
LOWpython/pathway/stdlib/ml/smart_table_ops/__init__.py19__all__ = [CODE
LOWpython/pathway/stdlib/ml/datasets/__init__.py7__all__ = ["classification"]CODE
LOWpython/pathway/stdlib/ml/classifiers/__init__.py16__all__ = [CODE
LOWpython/pathway/stdlib/statistical/__init__.py6__all__ = [CODE
LOWpython/pathway/stdlib/graphs/__init__.py9__all__ = [CODE
LOWpython/pathway/stdlib/graphs/pagerank/__init__.py7__all__ = ["Result", "pagerank"]CODE
LOWpython/pathway/stdlib/graphs/bellman_ford/__init__.py7__all__ = ["DistFromSource", "Vertex", "bellman_ford"]CODE
LOWpython/pathway/stdlib/indexing/__init__.py30__all__ = [CODE
LOWpython/pathway/stdlib/ordered/__init__.py6__all__ = [CODE
AI Response Leakage6 hits · 42 pts
SeverityFileLineSnippetContext
HIGH…poral-data/.interval_join_temporal_behavior/article.py164# In this example, a special column `__diff__` is used to tell the Pathway Live Data Framework whether a recordCOMMENT
HIGH…10.introduction/.streaming-and-static-modes/article.py52# In this example, T1 and T2 could be obtained using Kafka connectors, and T2bis and T3 could be output using PostgreSQLCOMMENT
HIGH….user-guide/30.data-transformation/.iterate/article.py71# In this example, you are going to use a [Newton method](https://en.wikipedia.org/wiki/Newton%27s_method) to find rootsCOMMENT
HIGH….user-guide/30.data-transformation/.iterate/article.py255# Another interesting use of `pw.iterate` is computing shortest paths in a graph. In this example, you are going to findCOMMENT
HIGHdocs/2.developers/7.templates/30.configure-yaml.md281Alternatively, you may wish to improve indexing capabilities by using the [HybridIndex](/developers/api-docs/indexing#paCODE
HIGH…lopers/7.templates/ETL/.fuzzy_join_chapter1/article.py96# In this example we will use the `smart_fuzzy_join` function from Pathway Live Data Framework's standard library to makCOMMENT
Self-Referential Comments11 hits · 32 pts
SeverityFileLineSnippetContext
MEDIUMpython/pathway/tests/utils.py217# This class is an abstract subclass of OnChangeCallback, which takes a list of entriesCOMMENT
MEDIUMintegration_tests/kafka/utils.py763 # Create a stream and wait until it is fully usable. create_stream returnsCOMMENT
MEDIUM…tegration_tests/monitoring/test_detailed_monitoring.py51 # Create an empty metrics db file so dashboard can startCOMMENT
MEDIUMintegration_tests/db_connectors/test_mssql.py1349 # Create a table inside the new DB via its own connection.COMMENT
MEDIUM…s/4.user-guide/40.temporal-data/.asof-join/.article.py74# Define the list of ticker symbols for the 5 major US companiesCOMMENT
MEDIUM….user-guide/40.temporal-data/.interval-join/article.py52# Create a maintenance data tableCOMMENT
MEDIUM…lopers/7.templates/ETL/.merging-time-series/article.py120# ## Creating the datasetCOMMENT
MEDIUM…velopers/7.templates/ETL/.live_data_jupyter/article.py84# Create a streaming data source that replays data in a CSV file. This is an easy way to simulate a live data stream witCOMMENT
MEDIUM…7.templates/rag/.private_rag_ollama_mistral/article.py147# Create a table with example questionsCOMMENT
MEDIUMexamples/projects/conf42/main.py39# Define the modelCOMMENT
MEDIUMexamples/templates/el-pipeline/app.yaml28# Define the Kafka data source using the schema and the configurations.COMMENT
Verbosity Indicators17 hits · 28 pts
SeverityFileLineSnippetContext
LOW…al/timely-dataflow/timely/src/progress/reachability.rs570 // Step 0: If logging is enabled, construct and log inbound changes.COMMENT
LOW…al/timely-dataflow/timely/src/progress/reachability.rs594 // Step 1: Drain `self.input_changes` and determine actual frontier changes.COMMENT
LOW…al/timely-dataflow/timely/src/progress/reachability.rs638 // Step 2: Circulate implications of changes to `self.pointstamps`.COMMENT
LOWexternal/differential-dataflow/src/capture.rs349 // Step 1: The MESSAGES operator.COMMENT
LOWexternal/differential-dataflow/src/capture.rs427 // Step 2: The UPDATES operator.COMMENT
LOWexternal/differential-dataflow/src/capture.rs473 // Step 3: The PROGRESS operator.COMMENT
LOWexternal/differential-dataflow/src/capture.rs556 // Step 4: The FEEDBACK operator.COMMENT
LOWsrc/connectors/data_storage/mssql.rs1452 // Step 1: record LSN before snapshot read.COMMENT
LOWsrc/connectors/data_storage/mssql.rs1461 // Step 2: read the full table snapshot.COMMENT
LOWsrc/connectors/data_storage/mssql.rs1464 // Step 3: record LSN after snapshot read.COMMENT
LOWsrc/connectors/data_storage/mssql.rs1471 // Step 4: fetch CDC changes that raced with the snapshot read.COMMENT
LOWsrc/connectors/data_storage/mssql.rs1150 /// Step 1: Record the current CDC max LSN (retry until the capture agentCOMMENT
LOWsrc/connectors/data_storage/mssql.rs1176 /// Step 2: Read the full table snapshot via a plain SELECT.COMMENT
LOWsrc/connectors/data_storage/mssql.rs1190 /// Step 3: Record the CDC max LSN again, immediately after the snapshot SELECT.COMMENT
LOWsrc/connectors/data_storage/mssql.rs1299 /// Step 4: Fetch CDC changes that raced with the snapshot read.COMMENT
LOWsrc/connectors/data_storage/mssql.rs1488 // Step 5: pick the offset that gets stamped on the snapshotCOMMENT
LOWsrc/connectors/data_storage/mssql.rs1548 // Step 6: advance last_lsn so poll_cdc_changes starts strictly afterCOMMENT
Magic Placeholder Names3 hits · 10 pts
SeverityFileLineSnippetContext
HIGHpython/pathway/io/pinecone/__init__.py193 >>> pc = Pinecone(api_key="YOUR_API_KEY") # doctest: +SKIPSTRING
HIGHpython/pathway/io/pinecone/__init__.py227 ... api_key="YOUR_API_KEY",STRING
HIGH…e/20.connect/99.connectors/90.websockets-connectors.md128API_KEY = "your-api-key"CODE
Redundant / Tautological Comments5 hits · 8 pts
SeverityFileLineSnippetContext
LOWpython/pathway/internals/schema.py611 # Check if self has all columns of otherCOMMENT
LOWpython/pathway/internals/schema.py616 # Check if types of columns are the sameCOMMENT
LOW…de/40.temporal-data/.windowby-reduce-manual/article.py82# Print the resultCOMMENT
LOW…de/40.temporal-data/.windowby-reduce-manual/article.py104# Print the resultCOMMENT
LOW…de/40.temporal-data/.windowby-reduce-manual/article.py129# Print the resultCOMMENT
Fake / Example Data8 hits · 7 pts
SeverityFileLineSnippetContext
LOWpython/pathway/tests/ml/test_index.py538 Lorem ipsum dolor sit amet, consectetur adipiscing elit. | 1 | 2STRING
LOWpython/pathway/tests/ml/test_index.py538 Lorem ipsum dolor sit amet, consectetur adipiscing elit. | 1 | 2STRING
LOWpython/pathway/xpacks/llm/tests/test_parsers.py84 "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod"CODE
LOWpython/pathway/xpacks/llm/tests/test_parsers.py84 "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod"CODE
LOWintegration_tests/kafka/conftest.py41 monkeypatch.setenv("AWS_ACCESS_KEY_ID", "placeholder")CODE
LOWintegration_tests/kafka/conftest.py42 monkeypatch.setenv("AWS_SECRET_ACCESS_KEY", "placeholder")CODE
LOWintegration_tests/kafka/utils.py473 aws_access_key_id="placeholder",CODE
LOWintegration_tests/kafka/utils.py474 aws_secret_access_key="placeholder",CODE