AI Observability & Evaluation
This report presents the forensic synthetic code analysis of Arize-ai/phoenix, a Python project with 10,557 GitHub stars. SynthScan v2.0 examined 913,583 lines of code across 4739 source files, recording 8733 pattern matches distributed across 26 syntactic categories. The overall adjusted score of 12.1 places this repository in the Low AI signal band.
The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).
This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.
Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.
This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.
The scanner identified 8733 distinct pattern matches across 26 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | evals/pxi/tests/test_gate.py | 56 | def test_zero_scored_rows_with_completed_examples_fails_closed(tmp_path: Path) -> None: | CODE |
| LOW | evals/pxi/tests/test_gate.py | 62 | def test_fewer_scored_rows_than_completed_fails_closed(tmp_path: Path) -> None: | CODE |
| LOW | evals/pxi/tests/test_gate.py | 81 | def test_expected_recording_not_bootstrapped_still_fails_closed(tmp_path: Path) -> None: | CODE |
| LOW | evals/pxi/tests/test_gate.py | 94 | def test_valid_artifact_below_threshold_returns_breach(tmp_path: Path) -> None: | CODE |
| LOW | evals/pxi/evaluators/tools.py | 173 | def tool_call_count_within_limit(output: Any, expected: Any) -> dict[str, Any]: | CODE |
| LOW | evals/pxi/evaluators/tools.py | 197 | def _bash_command_expectations(expected: Any) -> list[dict[str, Any]] | None: | CODE |
| LOW | evals/pxi/evaluators/tools.py | 208 | def evaluate_bash_command_substrings(output: Any, expected: Any) -> dict[str, Any]: | CODE |
| LOW | evals/pxi/evaluators/tools.py | 258 | def bash_command_substrings_match(output: Any, expected: Any) -> dict[str, Any]: | CODE |
| LOW | evals/pxi/evaluators/tools.py | 408 | def _matcher_validation_failures(variants: list[dict[str, Any]]) -> dict[str, str]: | CODE |
| LOW | evals/pxi/evaluators/tools.py | 443 | def _invalid_arg_expectation_reason(expected_for_tool: Any) -> str | None: | CODE |
| LOW | evals/pxi/evaluators/tools.py | 519 | def evaluate_forbidden_tool_call_args(output: Any, expected: Any) -> dict[str, Any]: | CODE |
| LOW | evals/pxi/evaluators/tools.py | 572 | def forbidden_tool_call_args_match(output: Any, expected: Any) -> dict[str, Any]: | CODE |
| LOW | evals/pxi/evaluators/links.py | 27 | def _expects_route_info_before_in_app_links(expected: Any) -> bool: | CODE |
| LOW | evals/pxi/evaluators/links.py | 57 | def _absolute_app_link_reason(href: str, required_paths: list[str]) -> str | None: | CODE |
| LOW | evals/pxi/evaluators/links.py | 76 | def _is_root_relative_app_href(href: str) -> bool: | CODE |
| LOW | evals/pxi/evaluators/links.py | 80 | def _in_app_hrefs_before_tool(output: Any, tool_name: str) -> list[str]: | CODE |
| LOW | evals/pxi/evaluators/text.py | 125 | def evaluate_assistant_text_substrings(output: Any, expected: Any) -> dict[str, Any]: | CODE |
| LOW | evals/pxi/evaluators/text.py | 186 | def assistant_text_substrings_match(output: Any, expected: Any) -> dict[str, Any]: | CODE |
| LOW | evals/pxi/harness/run_experiment.py | 165 | def _warn_if_split_smoke_check_fails(phoenix_dataset: Any, expected_splits: set[str]) -> None: | CODE |
| LOW | evals/pxi/harness/run_experiment.py | 175 | async def _get_split_filtered_dataset( | CODE |
| LOW | evals/pxi/harness/run_experiment.py | 201 | def _rewrite_stable_example_ids(experiment: RanExperiment, experiment_dataset: Any) -> None: | CODE |
| LOW | evals/pxi/harness/agent_task.py | 121 | def _warn_placeholder_api_key(provider: str, base_url: str) -> None: | CODE |
| LOW | evals/pxi/harness/agent_task.py | 201 | def should_build_docs_mcp_server() -> bool: | CODE |
| LOW | evals/pxi/harness/agent_task.py | 210 | def build_shared_docs_mcp_server() -> MCPToolset[Any] | None: | CODE |
| LOW | evals/pxi/harness/agent_task.py | 470 | def _assistant_text_from_messages(messages: list[dict[str, Any]]) -> str | None: | CODE |
| LOW | evals/pxi/harness/reporting.py | 166 | def _has_regression_evaluator_failure( | CODE |
| LOW | evals/pxi/harness/reporting.py | 337 | def _strip_static_instructions(actual_output: Any) -> tuple[Any, bool]: | CODE |
| LOW | app/tests/playground.spec.ts | 4 | function isGraphQLMutationResponse(response: Response, operationName: string) { | CODE |
| LOW | app/tests/settings-general-tables.spec.ts | 3 | async function clickSortableHeaderAndExpect( | CODE |
| LOW | app/tests/datasets.spec.ts | 35 | async function clickSortableHeaderAndExpect( | CODE |
| LOW | app/tests/label-list-management.spec.ts | 4 | function isGraphQLMutationResponse(response: Response, operationName: string) { | CODE |
| LOW | app/tests/projects.spec.ts | 21 | async function clickSortableHeaderAndExpect( | CODE |
| LOW | app/tests/settings-sandboxes.spec.ts | 10 | function isGraphQLMutationResponse(response: Response, operationName: string) { | CODE |
| LOW | app/tests/code-evaluators.spec.ts | 10 | function isGraphQLMutationResponse(response: Response, operationName: string) { | CODE |
| LOW | app/tests/code-evaluators.spec.ts | 260 | async function createCustomCodeEvaluator({ | CODE |
| LOW | app/tests/code-evaluators.spec.ts | 305 | async function expectEvaluatorDetailsPage(page: Page, evaluatorName: string) { | CODE |
| LOW | app/tests/settings-labels-tables.spec.ts | 4 | function isGraphQLMutationResponse(response: Response, operationName: string) { | CODE |
| LOW | app/tests/playground-prompt-configuration.spec.ts | 321 | async function expectToolChoiceOneOrMore({ | CODE |
| LOW | app/tests/playground-prompt-configuration.spec.ts | 368 | async function expectFunctionAndRawTools({ | CODE |
| LOW | app/tests/playground-prompt-configuration.spec.ts | 396 | async function reloadAndExpectPlayground(page: Page): Promise<void> { | CODE |
| LOW | app/tests/settings-secrets.spec.ts | 4 | function isSecretsMutationResponse(response: Response) { | CODE |
| LOW | app/tests/pxi/judge.ts | 28 | export function getRequiredJudgeApiKeyEnv({ | CODE |
| LOW | app/tests/pxi/route-link-smoke.spec.ts | 26 | async function waitForLatestAssistantTurnWithText(page: Page) { | CODE |
| LOW | app/.storybook/preview.tsx | 59 | function getLegacyStorySurfaceLayout( | CODE |
| LOW | app/stories/PlaygroundOutputHeader.stories.tsx | 43 | function RunningPlaygroundProvider({ | CODE |
| LOW | app/stories/FileDropZone.stories.tsx | 289 | function FileListWithRenderFunctionRender(args: FileDropZoneProps) { | CODE |
| LOW | app/stories/FileDropZone.stories.tsx | 471 | function FileInputWithDescriptionRender() { | CODE |
| LOW | app/stories/FileDropZone.stories.tsx | 526 | function DropZoneWithOverlayRender() { | CODE |
| LOW | app/stories/TableColumnOrdering.stories.tsx | 254 | render: function SelectorMenuWithSectionsStory() { | CODE |
| LOW | app/scripts/setup-remote-export.ts | 101 | export function normalizeCollectorEndpoint(value: string): string { | CODE |
| LOW | app/scripts/generate-generative-ui-catalog.ts | 71 | function getToolFriendlyJsonSchema(schema: object): object { | CODE |
| LOW | app/src/typeUtils.ts | 30 | export function isStringOrNullOrUndefined( | CODE |
| LOW | app/src/relayDebugInstrumentation.ts | 172 | export function instrumentRelayEnvironment({ | CODE |
| LOW | app/src/types/code.ts | 56 | export function isTypescriptPackageManager( | CODE |
| LOW | app/src/contexts/createTablePreferencesContext.tsx | 16 | export function createTablePreferencesContext({ | CODE |
| LOW | app/src/contexts/ExperimentCompareContext.tsx | 80 | export function ExperimentCompareDetailsProvider({ | CODE |
| LOW | app/src/contexts/ExperimentCompareContext.tsx | 254 | export function useExperimentCompareDetailsContext() { | CODE |
| LOW | app/src/contexts/ExperimentCompareContext.tsx | 308 | function sortRepetitionsByAnnotation( | CODE |
| LOW | app/src/contexts/ExperimentCompareContext.tsx | 327 | export function areAllExperimentRunsSelected( | CODE |
| LOW | app/src/contexts/ExperimentCompareContext.tsx | 336 | export function areSomeExperimentRunsSelected( | CODE |
| 5537 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | evals/pxi/harness/reporting.py | 232 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | evals/pxi/harness/reporting.py | 239 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | app/stories/BorderAndDividerLines.stories.tsx | 231 | // ───────────────────────────────────────────────────────────────────────────── | COMMENT |
| MEDIUM | app/stories/BorderAndDividerLines.stories.tsx | 233 | // ───────────────────────────────────────────────────────────────────────────── | COMMENT |
| MEDIUM⚡ | js/packages/phoenix-cli/src/commands/formatTable.ts | 23 | * // ┌────────────┬──────────┬──────────┐ | COMMENT |
| MEDIUM⚡ | js/packages/phoenix-cli/src/commands/formatTable.ts | 25 | * // ├────────────┼──────────┼──────────┤ | COMMENT |
| MEDIUM⚡ | js/packages/phoenix-cli/src/commands/formatTable.ts | 28 | * // └────────────┴──────────┴──────────┘ | COMMENT |
| MEDIUM⚡ | …ternal_docs/specs/ldap-authentication/configuration.md | 194 | # ============================================================================== | COMMENT |
| MEDIUM⚡ | …ternal_docs/specs/ldap-authentication/configuration.md | 196 | # ============================================================================== | COMMENT |
| MEDIUM⚡ | …ternal_docs/specs/ldap-authentication/configuration.md | 206 | # ============================================================================== | COMMENT |
| MEDIUM⚡ | …ternal_docs/specs/ldap-authentication/configuration.md | 208 | # ============================================================================== | COMMENT |
| MEDIUM | …ternal_docs/specs/ldap-authentication/configuration.md | 123 | # ============================================================================== | COMMENT |
| MEDIUM | …ternal_docs/specs/ldap-authentication/configuration.md | 125 | # ============================================================================== | COMMENT |
| MEDIUM | …ternal_docs/specs/ldap-authentication/configuration.md | 149 | # ============================================================================== | COMMENT |
| MEDIUM | …ternal_docs/specs/ldap-authentication/configuration.md | 151 | # ============================================================================== | COMMENT |
| MEDIUM | …ternal_docs/specs/ldap-authentication/configuration.md | 162 | # ============================================================================== | COMMENT |
| MEDIUM | …ternal_docs/specs/ldap-authentication/configuration.md | 164 | # ============================================================================== | COMMENT |
| MEDIUM | …ternal_docs/specs/ldap-authentication/configuration.md | 178 | # ============================================================================== | COMMENT |
| MEDIUM | …ternal_docs/specs/ldap-authentication/configuration.md | 180 | # ============================================================================== | COMMENT |
| MEDIUM | …ternal_docs/specs/ldap-authentication/configuration.md | 238 | # ============================================================================== | COMMENT |
| MEDIUM | …ternal_docs/specs/ldap-authentication/configuration.md | 240 | # ============================================================================== | COMMENT |
| MEDIUM | …ternal_docs/specs/ldap-authentication/configuration.md | 281 | # ============================================================================== | COMMENT |
| MEDIUM | …ternal_docs/specs/ldap-authentication/configuration.md | 283 | # ============================================================================== | COMMENT |
| MEDIUM⚡ | tests/unit/trace/gen_ai/test_conversion.py | 46 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unit/trace/gen_ai/test_conversion.py | 48 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unit/trace/gen_ai/test_conversion.py | 168 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unit/trace/gen_ai/test_conversion.py | 170 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unit/trace/gen_ai/test_conversion.py | 218 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unit/trace/gen_ai/test_conversion.py | 220 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unit/trace/gen_ai/test_conversion.py | 318 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unit/trace/gen_ai/test_conversion.py | 320 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unit/trace/gen_ai/test_conversion.py | 358 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unit/trace/gen_ai/test_conversion.py | 360 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unit/trace/gen_ai/test_conversion.py | 716 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unit/trace/gen_ai/test_conversion.py | 718 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unit/trace/gen_ai/test_conversion.py | 751 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unit/trace/gen_ai/test_conversion.py | 753 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unit/trace/gen_ai/test_conversion.py | 923 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unit/trace/gen_ai/test_conversion.py | 925 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unit/trace/gen_ai/test_conversion.py | 1003 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unit/trace/gen_ai/test_conversion.py | 1005 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/unit/trace/gen_ai/test_conversion.py | 119 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/unit/trace/gen_ai/test_conversion.py | 121 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/unit/server/test_app_evaluations.py | 249 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/unit/server/test_app_evaluations.py | 251 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unit/server/daemons/test_experiment_runner.py | 212 | # =========================================================================== | COMMENT |
| MEDIUM⚡ | tests/unit/server/daemons/test_experiment_runner.py | 214 | # =========================================================================== | COMMENT |
| MEDIUM⚡ | tests/unit/server/daemons/test_experiment_runner.py | 248 | # =========================================================================== | COMMENT |
| MEDIUM⚡ | tests/unit/server/daemons/test_experiment_runner.py | 250 | # =========================================================================== | COMMENT |
| MEDIUM⚡ | tests/unit/server/daemons/test_experiment_runner.py | 834 | # =========================================================================== | COMMENT |
| MEDIUM⚡ | tests/unit/server/daemons/test_experiment_runner.py | 836 | # =========================================================================== | COMMENT |
| MEDIUM⚡ | tests/unit/server/daemons/test_experiment_runner.py | 870 | # =========================================================================== | COMMENT |
| MEDIUM⚡ | tests/unit/server/daemons/test_experiment_runner.py | 872 | # =========================================================================== | COMMENT |
| MEDIUM⚡ | tests/unit/server/daemons/test_experiment_runner.py | 890 | # =========================================================================== | COMMENT |
| MEDIUM⚡ | tests/unit/server/daemons/test_experiment_runner.py | 892 | # =========================================================================== | COMMENT |
| MEDIUM⚡ | tests/unit/server/daemons/test_experiment_runner.py | 920 | # =========================================================================== | COMMENT |
| MEDIUM⚡ | tests/unit/server/daemons/test_experiment_runner.py | 922 | # =========================================================================== | COMMENT |
| MEDIUM | tests/unit/server/daemons/test_experiment_runner.py | 28 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/unit/server/daemons/test_experiment_runner.py | 30 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unit/server/sandbox/test_vercel_backend.py | 429 | # --------------------------------------------------------------------------- | COMMENT |
| 405 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | tests/unit/_helpers.py | 188 | Verify that the experiments_dataset_examples junction table contains the expected entries. This function uses | STRING |
| HIGH | tests/integration/_helpers.py | 2002 | If no_wait, run the query once. Otherwise, retry it if it returns None and raise if retries are exhausted. Args | STRING |
| HIGH | tests/integration/db_migrations/__init__.py | 93 | Get schema information for a database table. Retrieves comprehensive schema information for a table, including its | STRING |
| HIGH | …igration_6a88424799fe_update_users_with_auth_method.py | 283 | Create a new local authentication user. Creates a user with: - Local authentication method ('LOCAL') - Pass | STRING |
| HIGH | …igration_6a88424799fe_update_users_with_auth_method.py | 333 | Create a new OAuth2 user. Creates a user with: - External authentication method ('OAUTH2') - OAuth2 credent | STRING |
| HIGH | …generate_data_via_plpgsql/generate_span_annotations.py | 178 | Run a SQL script file using psql. Args: db_name (str): Database name db_user (str): Database user | STRING |
| HIGH | …ripts/llm_token_pricing_tables/litellm_model_prices.py | 41 | Process the model prices JSON file from a URL into a pandas DataFrame. Args: url: URL to fetch the JSON dat | STRING |
| HIGH | packages/phoenix-evals/src/phoenix/evals/utils.py | 180 | Remap eval_input keys based on required_fields and an optional input_mapping. Args: eval_input: The in | STRING |
| HIGH | packages/phoenix-evals/src/phoenix/evals/utils.py | 275 | Extract a value from a nested JSON structure using jsonpath-ng. Args: data: The input dictionary to be | STRING |
| HIGH | packages/phoenix-evals/src/phoenix/evals/utils.py | 415 | Format scores as annotations for logging to Phoenix. This function takes the output of evaluate_dataframe, and a li | STRING |
| HIGH | packages/phoenix-evals/src/phoenix/evals/evaluators.py | 129 | Remap and validate evaluation input. Args: eval_input: The raw evaluation input dictionary. require | STRING |
| HIGH | packages/phoenix-evals/src/phoenix/evals/evaluators.py | 602 | LLM-based evaluator for classification-style judgements. Supports label-only or label+score mappings, and retu | STRING |
| HIGH | packages/phoenix-evals/src/phoenix/evals/evaluators.py | 834 | Decorator that turns a simple function into an Evaluator instance. The decorated function should accept keywor | STRING |
| HIGH | packages/phoenix-evals/src/phoenix/evals/evaluators.py | 1142 | Factory to create a `ClassificationEvaluator`. Note: The evaluator requires the LLM to have tool calling or st | STRING |
| HIGH | packages/phoenix-evals/src/phoenix/evals/evaluators.py | 1226 | Helper to bind an evaluator with a fixed input mapping. This function allows you to create a version of an eva | STRING |
| HIGH | packages/phoenix-evals/src/phoenix/evals/evaluators.py | 1417 | Evaluate a dataframe with a list of evaluators and return an augmented dataframe. This function uses a synchro | STRING |
| HIGH | packages/phoenix-evals/src/phoenix/evals/evaluators.py | 1568 | Evaluate a dataframe with a list of evaluators and return an augmented dataframe. This function uses an asynch | STRING |
| HIGH | …ges/phoenix-evals/src/phoenix/evals/metrics/refusal.py | 14 | An evaluator for detecting when an LLM refuses or declines to answer a query. Args: llm (LLM): The LLM | STRING |
| HIGH | …hoenix-evals/src/phoenix/evals/metrics/faithfulness.py | 14 | A specialized evaluator for detecting faithfulness in grounded LLM responses. Args: llm (LLM): The LLM | STRING |
| HIGH | …phoenix-evals/src/phoenix/evals/metrics/exact_match.py | 6 | Return exact_match score: 1.0 if output == expected else 0.0. Note: No text normalization is performed. Args: | STRING |
| HIGH | …ix-evals/src/phoenix/evals/metrics/precision_recall.py | 67 | Code evaluator that computes precision, recall, and F-beta score given lists of expected and output labels. | STRING |
| HIGH | …phoenix-evals/src/phoenix/evals/metrics/correctness.py | 14 | An evaluator for assessing factual accuracy and completeness of model outputs. Args: llm (LLM): The LL | STRING |
| HIGH | …enix-evals/src/phoenix/evals/metrics/tool_selection.py | 14 | A specialized evaluator for determining if the correct tool was selected for a given context. Args: ll | STRING |
| HIGH | …nix-evals/src/phoenix/evals/metrics/tool_invocation.py | 14 | Determines if a tool was invoked correctly with proper arguments, formatting, and safe content. Args: | STRING |
| HIGH | …-evals/src/phoenix/evals/metrics/document_relevance.py | 14 | A specialized evaluator for determining document relevance to a given question. Args: llm (LLM): T | STRING |
| HIGH | …oenix-evals/src/phoenix/evals/metrics/hallucination.py | 23 | A specialized evaluator for detecting hallucinations in grounded LLM responses. .. deprecated:: Halluc | STRING |
| HIGH | …phoenix-evals/src/phoenix/evals/metrics/conciseness.py | 14 | An evaluator for assessing whether model outputs are concise and free of unnecessary content. Args: ll | STRING |
| HIGH | …ls/src/phoenix/evals/metrics/tool_response_handling.py | 14 | Determines if an AI agent properly handled a tool's response, including error handling, data extraction, transf | STRING |
| HIGH | packages/phoenix-evals/src/phoenix/evals/llm/wrapper.py | 318 | Generate a classification given a prompt and a set of labels. Args: prompt (Union[str, List[Dict[st | STRING |
| HIGH | packages/phoenix-evals/src/phoenix/evals/llm/wrapper.py | 456 | Generate a JSON Schema for an LLM function/structured output call that classifies a single criterion. Args: | STRING |
| HIGH | packages/phoenix-evals/src/phoenix/evals/llm/prompts.py | 608 | Factory function to create appropriate ContentPartTemplate from a dict. Args: content_part: Dictionary with | STRING |
| HIGH | packages/phoenix-evals/src/phoenix/evals/llm/prompts.py | 827 | Render the template with the given variables. Args: variables (Dict[str, Any]): The variables to su | STRING |
| HIGH | packages/phoenix-evals/src/phoenix/evals/llm/prompts.py | 1001 | Render the template with the given variables. Args: variables: The variables to substitute into the | STRING |
| HIGH | packages/phoenix-client/scripts/codegen/transform.py | 349 | Sort class definitions so that each parent class appears before its children. Args: class_nodes: Mappi | STRING |
| HIGH | …enix-client/src/phoenix/client/experiments/__init__.py | 33 | Run an experiment using a given dataset of examples. An experiment is a user-defined task that runs on each ex | STRING |
| HIGH | …enix-client/src/phoenix/client/experiments/__init__.py | 224 | Run an experiment using a given dataset of examples. An experiment is a user-defined task that runs on each ex | STRING |
| HIGH | …enix-client/src/phoenix/client/experiments/__init__.py | 408 | Get a completed experiment by ID. This function retrieves a completed experiment with all its task runs and ev | STRING |
| HIGH | …enix-client/src/phoenix/client/experiments/__init__.py | 475 | Resume an incomplete experiment by running only the missing or failed runs. This method identifies which (exam | STRING |
| HIGH | …enix-client/src/phoenix/client/experiments/__init__.py | 547 | Get a completed experiment by ID. This function retrieves a completed experiment with all its task runs and | STRING |
| HIGH | …enix-client/src/phoenix/client/experiments/__init__.py | 614 | Resume an incomplete experiment by running only the missing or failed runs. This method identifies which (exam | STRING |
| HIGH | …enix-client/src/phoenix/client/experiments/__init__.py | 695 | Run evaluators on a completed experiment. An evaluator is either a synchronous or asynchronous function that r | STRING |
| HIGH | …enix-client/src/phoenix/client/experiments/__init__.py | 889 | Run evaluators on a completed experiment. An `evaluator` is either a synchronous or asynchronous function that | STRING |
| HIGH | …client/src/phoenix/client/resources/traces/__init__.py | 105 | Add a single trace annotation. Args: trace_id (str): The ID of the trace to annotate. a | STRING |
| HIGH | …client/src/phoenix/client/resources/traces/__init__.py | 195 | Log multiple trace annotations. Args: trace_annotations (Iterable[TraceAnnotationData]): An iterabl | STRING |
| HIGH | …client/src/phoenix/client/resources/traces/__init__.py | 263 | Log multiple trace annotations from a pandas DataFrame. This method allows you to create multiple trace annotat | STRING |
| HIGH | …client/src/phoenix/client/resources/traces/__init__.py | 352 | Retrieves traces for a project. Args: project_identifier (str): The project identifier (name or ID) | STRING |
| HIGH | …client/src/phoenix/client/resources/traces/__init__.py | 501 | Add a single trace annotation asynchronously. Args: trace_id (str): The ID of the trace to annotate | STRING |
| HIGH | …client/src/phoenix/client/resources/traces/__init__.py | 591 | Log multiple trace annotations asynchronously. Args: trace_annotations (Iterable[TraceAnnotationDat | STRING |
| HIGH | …client/src/phoenix/client/resources/traces/__init__.py | 659 | Log multiple trace annotations from a pandas DataFrame asynchronously. This method allows you to create multipl | STRING |
| HIGH | …client/src/phoenix/client/resources/traces/__init__.py | 748 | Retrieves traces for a project asynchronously. Args: project_identifier (str): The project identifi | STRING |
| HIGH | …-client/src/phoenix/client/resources/spans/__init__.py | 183 | Retrieves spans based on the provided filter conditions. Args: query (Optional[SpanQuery]): A SpanQ | STRING |
| HIGH | …-client/src/phoenix/client/resources/spans/__init__.py | 262 | Fetches span annotations and returns them as a pandas DataFrame. Exactly one of *spans_dataframe*, *span_ids*, | STRING |
| HIGH | …-client/src/phoenix/client/resources/spans/__init__.py | 387 | Fetches span annotations and returns them as a list of SpanAnnotation objects. Exactly one of *span_ids* or *sp | STRING |
| HIGH | …-client/src/phoenix/client/resources/spans/__init__.py | 491 | Retrieves spans with simple filtering options. Args: project_identifier (str): The project identifi | STRING |
| HIGH | …-client/src/phoenix/client/resources/spans/__init__.py | 593 | Logs spans to a project. If any spans are invalid or duplicates, no spans will be logged and a SpanCrea | STRING |
| HIGH | …-client/src/phoenix/client/resources/spans/__init__.py | 635 | Logs spans to a project from a pandas DataFrame. If any spans are invalid or duplicates, no spans will be logge | STRING |
| HIGH | …-client/src/phoenix/client/resources/spans/__init__.py | 764 | Add a single span annotation. Args: span_id (str): The ID of the span to annotate. anno | STRING |
| HIGH | …-client/src/phoenix/client/resources/spans/__init__.py | 830 | Add a note to a span. Notes are append-only: each call creates a new note with an auto-generated UUIDv4 | STRING |
| HIGH | …-client/src/phoenix/client/resources/spans/__init__.py | 917 | Log multiple span annotations from a pandas DataFrame. This method allows you to create multiple span annotatio | STRING |
| HIGH | …-client/src/phoenix/client/resources/spans/__init__.py | 1034 | Log multiple span annotations. Args: span_annotations (Iterable[SpanAnnotationData]): An iterable o | STRING |
| 138 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | evals/pxi/conftest.py | 10 | CODE | |
| LOW | evals/pxi/gate.py | 14 | CODE | |
| LOW | evals/pxi/tests/test_pxi_evals.py | 10 | CODE | |
| LOW | evals/pxi/tests/test_gate.py | 11 | CODE | |
| LOW | evals/pxi/evaluators/tools.py | 1 | CODE | |
| LOW | evals/pxi/evaluators/links.py | 1 | CODE | |
| LOW | evals/pxi/evaluators/text.py | 1 | CODE | |
| LOW | evals/pxi/harness/datasets.py | 1 | CODE | |
| LOW | evals/pxi/harness/run_experiment.py | 1 | CODE | |
| LOW | evals/pxi/harness/agent_task.py | 1 | CODE | |
| LOW | evals/pxi/harness/reporting.py | 8 | CODE | |
| LOW | …al_docs/vignettes/sqlalchemy/polymorphic_user/users.py | 41 | CODE | |
| LOW | …ernal_docs/vignettes/postgresql_tls/test_connection.py | 54 | CODE | |
| LOW | tests/unit/pxi/evals/test_run_experiment.py | 8 | CODE | |
| LOW | tests/unit/pxi/evals/conftest.py | 3 | CODE | |
| LOW | tests/unit/pxi/evals/test_evaluators.py | 8 | CODE | |
| LOW | tests/unit/pxi/evals/test_datasets.py | 8 | CODE | |
| LOW | tests/unit/pxi/evals/test_evaluator_matchers.py | 8 | CODE | |
| LOW | tests/unit/pxi/evals/test_reporting.py | 10 | CODE | |
| LOW | tests/unit/pxi/evals/test_agent_task_inputs.py | 1 | CODE | |
| LOW | tests/unit/server/test_app_docs_mcp_lifespan.py | 11 | CODE | |
| LOW | tests/unit/server/daemons/test_system_settings.py | 3 | CODE | |
| LOW | tests/unit/server/daemons/test_experiment_runner.py | 1 | CODE | |
| LOW | tests/unit/server/agents/test_agent_factory.py | 1 | CODE | |
| LOW | tests/unit/server/agents/test_server_agents.py | 1 | CODE | |
| LOW | tests/unit/server/agents/test_data_stream_protocol.py | 1 | CODE | |
| LOW | tests/unit/server/agents/test_skill_requests.py | 1 | CODE | |
| LOW | …t/server/agents/capabilities/test_native_tool_retry.py | 1 | CODE | |
| LOW | …ents/capabilities/tools/internal/test_call_subagent.py | 1 | CODE | |
| LOW | …er/agents/capabilities/skills/test_skill_capability.py | 1 | CODE | |
| LOW | …agents/pydantic_ai/test_openinference_model_wrapper.py | 1 | CODE | |
| LOW | …ents/pydantic_ai/test_openinference_toolset_wrapper.py | 1 | CODE | |
| LOW | …s/pydantic_ai/test_openinference_capability_wrapper.py | 1 | CODE | |
| LOW | …agents/pydantic_ai/test_openinference_agent_wrapper.py | 1 | CODE | |
| LOW | tests/unit/server/agents/skills/test_phoenix_graphql.py | 1 | CODE | |
| LOW | tests/unit/server/sandbox/test_modal_backend.py | 1 | CODE | |
| LOW | tests/unit/server/sandbox/test_vercel_backend.py | 1 | CODE | |
| LOW | tests/unit/server/sandbox/test_vercel_backend.py | 9 | CODE | |
| LOW | tests/unit/server/sandbox/test_env_var_resolution.py | 1 | CODE | |
| LOW | tests/unit/server/sandbox/test_wasm_download.py | 7 | CODE | |
| LOW | tests/unit/server/sandbox/test_e2b_backend.py | 1 | CODE | |
| LOW | tests/unit/server/sandbox/test_build_sandbox_backend.py | 1 | CODE | |
| LOW | tests/unit/server/sandbox/test_build_sandbox_backend.py | 25 | CODE | |
| LOW | tests/unit/server/sandbox/test_app_lifecycle.py | 21 | CODE | |
| LOW | tests/unit/server/sandbox/test_app_lifecycle.py | 48 | CODE | |
| LOW | tests/unit/server/sandbox/test_app_lifecycle.py | 55 | CODE | |
| LOW | tests/unit/server/sandbox/test_app_lifecycle.py | 55 | CODE | |
| LOW | tests/unit/server/sandbox/test_app_lifecycle.py | 55 | CODE | |
| LOW | tests/unit/server/sandbox/test_credential_resolution.py | 1 | CODE | |
| LOW | …ts/unit/server/sandbox/test_unified_config_contract.py | 1 | CODE | |
| LOW | tests/unit/server/sandbox/test_daytona_backend.py | 1 | CODE | |
| LOW | tests/unit/server/sandbox/test_session_manager.py | 15 | CODE | |
| LOW | …/unit/server/sandbox/test_sandbox_config_validation.py | 1 | CODE | |
| LOW | tests/unit/server/sandbox/test_deno_backend.py | 1 | CODE | |
| LOW | tests/unit/server/api/test_coerce_output.py | 1 | CODE | |
| LOW | tests/unit/server/api/test_code_evaluator_runner.py | 1 | CODE | |
| LOW | tests/unit/server/api/test_wasm_backend.py | 1 | CODE | |
| LOW | tests/unit/server/api/test_wasm_backend.py | 7 | CODE | |
| LOW | tests/unit/server/api/test_evaluators_timeout.py | 1 | CODE | |
| LOW | tests/unit/server/api/test_capability_advertisement.py | 1 | CODE | |
| 459 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | …/src/components/experiment/RunExperimentCodeDialog.tsx | 43 | `# Define an evaluator. This just an example.\n` + | CODE |
| MEDIUM⚡ | …al_docs/vignettes/sqlalchemy/polymorphic_user/users.py | 24 | # Create a local user | STRING |
| MEDIUM⚡ | …al_docs/vignettes/sqlalchemy/polymorphic_user/users.py | 29 | # Create an external user | STRING |
| MEDIUM⚡ | …al_docs/vignettes/sqlalchemy/polymorphic_user/users.py | 286 | # Create an in-memory SQLite database | COMMENT |
| MEDIUM⚡ | …al_docs/vignettes/sqlalchemy/polymorphic_user/users.py | 295 | # Create a session | COMMENT |
| MEDIUM⚡ | …al_docs/vignettes/sqlalchemy/polymorphic_user/users.py | 299 | # Create a local user with password | COMMENT |
| MEDIUM⚡ | …al_docs/vignettes/sqlalchemy/polymorphic_user/users.py | 307 | # Create an external user | COMMENT |
| MEDIUM | …al_docs/vignettes/sqlalchemy/polymorphic_user/users.py | 323 | # Create a session | COMMENT |
| MEDIUM | tests/unit/test_tracers.py | 710 | # Create a hierarchy: | COMMENT |
| MEDIUM | tests/unit/test_config.py | 2157 | # Create a dummy CA cert file | COMMENT |
| MEDIUM⚡ | tests/unit/server/api/test_cancellation.py | 88 | # Create a tracker that shares the operation_order list | COMMENT |
| MEDIUM⚡ | tests/unit/server/api/test_cancellation.py | 92 | # Create a real async generator | COMMENT |
| MEDIUM⚡ | tests/unit/server/api/test_cancellation.py | 102 | # Create a task that takes time to process cancellation | COMMENT |
| MEDIUM | tests/unit/server/api/test_cancellation.py | 145 | # Create an already-completed task | COMMENT |
| MEDIUM | tests/unit/server/api/test_cancellation.py | 353 | # Create a task for the in-progress generator | COMMENT |
| MEDIUM | tests/unit/server/api/test_queries.py | 2509 | # Create a builtin evaluator without a dataset association | COMMENT |
| MEDIUM | tests/unit/server/api/test_queries.py | 614 | # Create a prompt version for each prompt | STRING |
| MEDIUM | tests/unit/server/api/routers/v1/test_traces.py | 374 | # Create a trace with multiple spans (parent and child) | COMMENT |
| MEDIUM | tests/unit/server/api/routers/v1/test_traces.py | 517 | # Create a relay GlobalID for a non-existent trace | COMMENT |
| MEDIUM | tests/unit/server/api/routers/v1/test_projects.py | 67 | # Create a dictionary of projects by ID for easy lookup | COMMENT |
| MEDIUM | tests/unit/server/api/routers/v1/test_projects.py | 574 | # Create a valid cursor format (base64-encoded project ID) | COMMENT |
| MEDIUM | tests/unit/server/api/routers/v1/test_projects.py | 598 | # Create a small number of test projects | COMMENT |
| MEDIUM | tests/unit/server/api/routers/v1/test_projects.py | 754 | # Create a regular experiment project - this should be filtered by default | COMMENT |
| MEDIUM | tests/unit/server/api/routers/v1/test_projects.py | 774 | # Create an experiment for the regular experiment project | COMMENT |
| MEDIUM | tests/unit/server/api/routers/v1/test_projects.py | 786 | # Create the playground project - this should always be visible | COMMENT |
| MEDIUM | tests/unit/server/api/routers/v1/test_projects.py | 794 | # Create an experiment using the playground project - this shouldn't affect visibility | COMMENT |
| MEDIUM | …unit/server/api/routers/v1/test_list_project_traces.py | 77 | # Create a lightweight Trace object for assertions | COMMENT |
| MEDIUM | tests/unit/server/api/routers/v1/test_experiments.py | 624 | # Create a fresh experiment and verify the create response has correct counts | COMMENT |
| MEDIUM | tests/unit/server/api/routers/v1/test_spans.py | 280 | # Create the hierarchy: parent -> children -> grandchild + sibling | COMMENT |
| MEDIUM | …/unit/server/api/routers/v1/test_annotation_configs.py | 91 | # Create a categorical annotation config | COMMENT |
| MEDIUM | …t/server/api/types/test_ProjectTraceRetentionPolicy.py | 47 | # Create a project with default trace retention policy | COMMENT |
| MEDIUM | …t/server/api/types/test_ProjectTraceRetentionPolicy.py | 50 | # Create a custom trace retention policy | COMMENT |
| MEDIUM | tests/unit/server/api/types/test_Trace.py | 316 | # Create a trace with 5 spans | COMMENT |
| MEDIUM | tests/unit/server/api/types/test_Dataset.py | 1346 | # Create a prompt for the evaluators | STRING |
| MEDIUM | tests/unit/server/api/types/test_Dataset.py | 1351 | # Create a prompt version | STRING |
| MEDIUM | tests/unit/server/api/types/test_Project.py | 2014 | # Create a test project | COMMENT |
| MEDIUM | tests/unit/server/api/types/test_Project.py | 2773 | # Create a test project | COMMENT |
| MEDIUM | …/unit/server/api/mutations/test_evaluator_mutations.py | 1454 | # Create a different prompt with a version | COMMENT |
| MEDIUM | …/unit/server/api/mutations/test_evaluator_mutations.py | 1861 | # Create a second version for the same prompt | COMMENT |
| MEDIUM | …/unit/server/api/mutations/test_evaluator_mutations.py | 1867 | # Create a second version | COMMENT |
| MEDIUM | …/unit/server/api/mutations/test_evaluator_mutations.py | 2866 | # Create a builtin dataset evaluator | COMMENT |
| MEDIUM | …/unit/server/api/mutations/test_evaluator_mutations.py | 2997 | # Create an LLM evaluator with dataset relationship | COMMENT |
| MEDIUM | …/unit/server/api/mutations/test_evaluator_mutations.py | 3104 | # Create a builtin dataset evaluator | COMMENT |
| MEDIUM | …/unit/server/api/mutations/test_evaluator_mutations.py | 3122 | # Create an LLM evaluator | COMMENT |
| MEDIUM | …/unit/server/api/mutations/test_evaluator_mutations.py | 3284 | # Create an LLM evaluator with prompt | COMMENT |
| MEDIUM | …/unit/server/api/mutations/test_evaluator_mutations.py | 3396 | # Create an LLM evaluator with prompt | COMMENT |
| MEDIUM | …/unit/server/api/mutations/test_evaluator_mutations.py | 3940 | # Create the builtin evaluator and verify continuous config is returned | COMMENT |
| MEDIUM | …tions/test_project_trace_retention_policy_mutations.py | 138 | # Create a new retention policy | STRING |
| MEDIUM | …tions/test_project_trace_retention_policy_mutations.py | 324 | # Create a GlobalID for the default policy to use in GraphQL operations | STRING |
| MEDIUM | …rver/api/mutations/test_annotation_config_mutations.py | 269 | # Create a categorical annotation config | STRING |
| MEDIUM | …rver/api/mutations/test_annotation_config_mutations.py | 833 | # Create a categorical annotation config | STRING |
| MEDIUM | …rver/api/mutations/test_annotation_config_mutations.py | 894 | # Create a categorical annotation config | STRING |
| MEDIUM | tests/unit/server/api/mutations/test_trace_mutations.py | 263 | # Create a session | STRING |
| MEDIUM | tests/unit/server/api/mutations/test_trace_mutations.py | 290 | # Create a span for the first trace | STRING |
| MEDIUM | tests/unit/server/api/mutations/test_trace_mutations.py | 330 | # Create a span for the second trace | STRING |
| MEDIUM | tests/unit/server/api/mutations/test_trace_mutations.py | 375 | # Create a span for the third trace | STRING |
| MEDIUM | …ts/unit/server/api/mutations/test_project_mutations.py | 57 | # Create a new project | STRING |
| MEDIUM | …ts/unit/server/api/mutations/test_project_mutations.py | 71 | # Create a project session for the first two traces | STRING |
| MEDIUM | …ts/unit/server/api/mutations/test_project_mutations.py | 83 | # Create a trace | STRING |
| MEDIUM | tests/unit/server/api/helpers/test_evaluators.py | 2104 | # Create a custom output config with different labels | COMMENT |
| 112 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | evals/pxi/harness/run_experiment.py | 330 | except Exception as cleanup_exc: # pragma: no cover - best-effort cleanup | CODE |
| LOW | evals/pxi/harness/agent_task.py | 102 | except Exception as exc: # noqa: BLE001 | CODE |
| LOW | evals/pxi/harness/agent_task.py | 117 | except Exception: # noqa: BLE001 | CODE |
| LOW | evals/pxi/harness/agent_task.py | 596 | except Exception as exc: | CODE |
| LOW | …al_docs/vignettes/postgresql/advisory-lock-analysis.md | 273 | except Exception: | CODE |
| LOW | …al_docs/vignettes/postgresql/advisory-lock-analysis.md | 461 | except Exception: | CODE |
| LOW⚡ | …ernal_docs/vignettes/postgresql_tls/test_connection.py | 642 | except Exception as e: | CODE |
| LOW⚡ | …ernal_docs/vignettes/postgresql_tls/test_connection.py | 648 | except Exception as e: | CODE |
| LOW⚡ | …ernal_docs/vignettes/postgresql_tls/test_connection.py | 654 | except Exception as e: | CODE |
| LOW⚡ | …ernal_docs/vignettes/postgresql_tls/test_connection.py | 660 | except Exception as e: | CODE |
| LOW⚡ | …ernal_docs/vignettes/postgresql_tls/test_connection.py | 676 | except Exception as e: | CODE |
| LOW⚡ | …ernal_docs/vignettes/postgresql_tls/test_connection.py | 682 | except Exception as e: | CODE |
| LOW⚡ | …ernal_docs/vignettes/postgresql_tls/test_connection.py | 688 | except Exception as e: | CODE |
| LOW⚡ | …ernal_docs/vignettes/postgresql_tls/test_connection.py | 694 | except Exception as e: | CODE |
| LOW | …ernal_docs/vignettes/postgresql_tls/test_connection.py | 354 | except Exception as e: | CODE |
| LOW | …ernal_docs/vignettes/postgresql_tls/test_connection.py | 394 | except Exception as e: | CODE |
| LOW | …ernal_docs/vignettes/postgresql_tls/test_connection.py | 431 | except Exception as e: | CODE |
| LOW | …ernal_docs/vignettes/postgresql_tls/test_connection.py | 468 | except Exception as e: | CODE |
| LOW | …ernal_docs/vignettes/postgresql_tls/test_connection.py | 499 | except Exception as e: | CODE |
| LOW | …ernal_docs/vignettes/postgresql_tls/test_connection.py | 532 | except Exception as e: | CODE |
| LOW | …ernal_docs/vignettes/postgresql_tls/test_connection.py | 565 | except Exception as e: | CODE |
| LOW | …ernal_docs/vignettes/postgresql_tls/test_connection.py | 598 | except Exception as e: | CODE |
| MEDIUM | …nal_docs/vignettes/postgresql_tls/generate_pg_certs.py | 60 | print(f"Error writing to file {filename}: {e}") | CODE |
| MEDIUM | …nal_docs/vignettes/postgresql_tls/generate_pg_certs.py | 173 | print("Error: Failed to generate certificates") | STRING |
| LOW | internal_docs/specs/ldap-authentication/security.md | 95 | except Exception: | CODE |
| LOW | …nt-runner-background-process/appendix-rate-limiting.md | 317 | except Exception as e: | CODE |
| LOW | …nt-runner-background-process/appendix-rate-limiting.md | 365 | except Exception as e: | CODE |
| LOW | tests/unit/conftest.py | 255 | except Exception: | CODE |
| LOW | tests/unit/conftest.py | 259 | except Exception: | CODE |
| LOW | tests/integration/conftest.py | 217 | except Exception: | CODE |
| LOW | tests/integration/_mock_ldap_server.py | 319 | except Exception as e: | CODE |
| LOW | tests/integration/_mock_llm_server.py | 451 | except Exception as e: | CODE |
| LOW | tests/integration/_mock_llm_server.py | 636 | except Exception as e: | CODE |
| LOW⚡ | tests/integration/_helpers.py | 1549 | except Exception: | CODE |
| LOW | tests/integration/client/test_experiments.py | 339 | except Exception: | CODE |
| LOW | tests/integration/client/test_experiments.py | 389 | except Exception: | CODE |
| LOW | …ation_e76cbd66ffc3_add_experiments_dataset_examples.py | 264 | except Exception: | CODE |
| LOW | …ation_e76cbd66ffc3_add_experiments_dataset_examples.py | 370 | except Exception: | CODE |
| LOW | examples/rag_agent/tools.py | 159 | except Exception as e: | STRING |
| LOW | examples/agents/run_scaled.py | 89 | except Exception as e: | CODE |
| LOW | examples/agents/run_scaled.py | 161 | except Exception as e: | CODE |
| LOW | examples/agents/run_scaled.py | 237 | except Exception as e: | CODE |
| LOW | examples/agents/tau_bench_openai_agents/run.py | 131 | except Exception as e: | CODE |
| LOW | examples/agents/traject_bench_langgraph/run.py | 141 | except Exception as e: | CODE |
| LOW | examples/agents/traject_bench_langgraph/agent.py | 206 | except Exception as e: | CODE |
| LOW | examples/agents/tau_bench_langgraph/run.py | 135 | except Exception as e: | CODE |
| LOW | examples/computer_use_agent/agent.py | 75 | except Exception as e: | CODE |
| LOW | examples/computer_use_agent/tools/edit.py | 239 | except Exception as e: | CODE |
| LOW | examples/computer_use_agent/tools/edit.py | 246 | except Exception as e: | CODE |
| LOW | examples/code_gen_agent/tools.py | 41 | except Exception as e: | CODE |
| LOW | examples/code_gen_agent/tools.py | 65 | except Exception as e: | STRING |
| LOW | examples/code_gen_agent/tools.py | 89 | except Exception as e: | STRING |
| LOW | examples/code_gen_agent/tools.py | 125 | except Exception as e: | STRING |
| MEDIUM | …/agent_framework_comparison/code_based_agent/router.py | 75 | print("Error: Invalid JSON, treating arguments as string") | CODE |
| LOW | examples/agent_framework_comparison/db/database.py | 23 | except Exception as e: | CODE |
| MEDIUM | examples/agent_framework_comparison/db/database.py | 24 | print(f"Error handling nested type: {e}") | CODE |
| MEDIUM | examples/agent_framework_comparison/db/database.py | 11 | def handle_nested_types(y): | CODE |
| MEDIUM⚡ | scripts/update_helm.py | 47 | print("Error: could not find appVersion in Chart.yaml", file=sys.stderr) | CODE |
| MEDIUM⚡ | scripts/update_helm.py | 54 | print("Error: could not find chart version in Chart.yaml", file=sys.stderr) | CODE |
| LOW⚡ | scripts/docker/devops/scripts/test_ldap_tls.py | 175 | except Exception as e: | CODE |
| 345 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | evals/pxi/conftest.py | 53 | CODE | |
| LOW | evals/pxi/gate.py | 90 | CODE | |
| LOW | evals/pxi/gate.py | 124 | CODE | |
| LOW | evals/pxi/evaluators/tools.py | 408 | CODE | |
| LOW | evals/pxi/harness/run_experiment.py | 90 | CODE | |
| LOW | evals/pxi/harness/agent_task.py | 294 | CODE | |
| LOW | evals/pxi/harness/reporting.py | 141 | CODE | |
| LOW | tests/unit/conftest.py | 54 | CODE | |
| LOW | tests/unit/graphql.py | 98 | CODE | |
| LOW | tests/unit/server/api/test_wasm_backend.py | 136 | CODE | |
| LOW | tests/unit/server/api/test_wasm_backend.py | 200 | CODE | |
| LOW | tests/unit/server/api/test_wasm_backend.py | 270 | CODE | |
| LOW | tests/unit/server/api/test_wasm_backend.py | 340 | CODE | |
| LOW | tests/unit/server/api/test_wasm_backend.py | 414 | CODE | |
| LOW | tests/unit/server/api/test_wasm_backend.py | 453 | CODE | |
| LOW | tests/unit/server/api/test_wasm_backend.py | 481 | CODE | |
| LOW | tests/unit/server/api/test_wasm_backend.py | 719 | CODE | |
| LOW | tests/unit/server/api/test_subscriptions.py | 1932 | CODE | |
| LOW | tests/unit/server/api/types/test_Span.py | 356 | CODE | |
| LOW | tests/unit/server/api/types/test_Project.py | 1998 | CODE | |
| LOW | tests/unit/server/api/types/test_Project.py | 2827 | CODE | |
| LOW | tests/unit/server/api/dataloaders/conftest.py | 12 | CODE | |
| LOW | tests/unit/server/api/helpers/test_evaluators.py | 2502 | CODE | |
| LOW | tests/unit/server/api/helpers/test_evaluators.py | 3008 | CODE | |
| LOW | tests/unit/server/api/helpers/test_evaluators.py | 3229 | CODE | |
| LOW | …t/server/cost_tracking/test_cost_details_calculator.py | 811 | CODE | |
| LOW | tests/unit/db/test_helpers.py | 65 | CODE | |
| LOW | tests/integration/conftest.py | 159 | CODE | |
| LOW | tests/integration/conftest.py | 162 | CODE | |
| LOW | tests/integration/_mock_ldap_server.py | 283 | CODE | |
| LOW | tests/integration/_mock_ldap_server.py | 498 | CODE | |
| LOW | tests/integration/_mock_llm_server.py | 344 | CODE | |
| LOW | tests/integration/_mock_llm_server.py | 460 | CODE | |
| LOW | tests/integration/_mock_llm_server.py | 208 | CODE | |
| LOW | tests/integration/_mock_llm_server.py | 602 | CODE | |
| LOW | tests/integration/_helpers.py | 690 | CODE | |
| LOW | tests/integration/_helpers.py | 2389 | CODE | |
| LOW | tests/integration/experiments/conftest.py | 431 | CODE | |
| LOW | tests/integration/experiments/test_playground.py | 161 | CODE | |
| LOW | tests/integration/experiments/test_playground.py | 330 | CODE | |
| LOW | tests/integration/experiments/test_playground.py | 266 | CODE | |
| LOW | tests/integration/experiments/test_playground.py | 375 | CODE | |
| LOW | …/integration/project_sessions/test_project_sessions.py | 34 | CODE | |
| LOW | tests/integration/client/test_datasets.py | 889 | CODE | |
| LOW | tests/integration/client/test_annotations.py | 2957 | CODE | |
| LOW | tests/integration/client/test_users.py | 101 | CODE | |
| LOW | tests/integration/client/test_experiments.py | 277 | CODE | |
| LOW | tests/integration/client/test_experiments.py | 482 | CODE | |
| LOW | tests/integration/client/test_experiments.py | 2953 | CODE | |
| LOW | tests/integration/client/test_experiments.py | 3087 | CODE | |
| LOW | tests/integration/client/test_spans.py | 745 | CODE | |
| LOW | tests/integration/db_migrations/__init__.py | 87 | CODE | |
| LOW | .agents/skills/phoenix-frontend/scripts/scale-svg.py | 39 | CODE | |
| LOW | .agents/skills/phoenix-frontend/scripts/scale-svg.py | 84 | CODE | |
| LOW | api_reference/source/conf.py | 27 | CODE | |
| LOW | examples/agents/tau_bench_openai_agents/agent.py | 83 | CODE | |
| LOW | examples/agents/traject_bench_langgraph/agent.py | 150 | CODE | |
| LOW | examples/agents/tau_bench_langgraph/agent.py | 124 | CODE | |
| LOW | examples/computer_use_agent/utils.py | 94 | CODE | |
| LOW | examples/computer_use_agent/tools/edit.py | 41 | CODE | |
| 262 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | …ts/project/integrationSnippets/environmentVariables.ts | 17 | const apiKeyValue = apiKey || "<your-api-key>"; | CODE |
| HIGH | …/src/components/experiment/RunExperimentCodeDialog.tsx | 30 | `os.environ["PHOENIX_API_KEY"] = "<your-api-key>"`; | CODE |
| HIGH | js/examples/apps/mastra-agent/README.md | 44 | 1. Be sure to include: PHOENIX_API_KEY=your-api-key | CODE |
| HIGH | js/packages/phoenix-mcp/README.md | 57 | "your-api-key" | CODE |
| HIGH | js/packages/phoenix-cli/README.md | 43 | export PHOENIX_API_KEY=your-api-key # if authentication is enabled | CODE |
| HIGH | js/packages/phoenix-config/README.md | 60 | PHOENIX_API_KEY=your-api-key | CODE |
| HIGH | js/packages/phoenix-config/src/env.ts | 50 | * process.env[ENV_PHOENIX_API_KEY] = "your-api-key"; | COMMENT |
| HIGH | js/packages/phoenix-otel/README.md | 93 | export PHOENIX_API_KEY="your-api-key" | CODE |
| HIGH | js/packages/phoenix-otel/examples/register_example.ts | 6 | apiKey: "your-api-key", | CODE |
| HIGH | js/packages/phoenix-otel/src/register.ts | 51 | * apiKey: "your-api-key", | COMMENT |
| HIGH | js/packages/phoenix-otel/src/register.ts | 443 | * headers: { 'Authorization': 'Bearer your-api-key' } | COMMENT |
| HIGH | js/packages/phoenix-otel/src/register.ts | 553 | * apiKey: 'your-api-key', | COMMENT |
| HIGH | .agents/skills/phoenix-cli/SKILL.md | 60 | export PHOENIX_API_KEY=your-api-key # if auth is enabled | CODE |
| HIGH | …/skills/phoenix-tracing/references/setup-typescript.md | 41 | export PHOENIX_API_KEY="your-api-key" | CODE |
| HIGH | …ents/skills/phoenix-tracing/references/setup-python.md | 34 | export PHOENIX_API_KEY="your-api-key" # Required for Phoenix Cloud | CODE |
| HIGH | packages/phoenix-evals/src/phoenix/evals/llm/wrapper.py | 167 | api_key="your-api-key", | STRING |
| HIGH | packages/phoenix-evals/src/phoenix/evals/llm/wrapper.py | 176 | api_key="your-api-key", | STRING |
| HIGH⚡ | packages/phoenix-client/README.md | 55 | export PHOENIX_API_KEY="your-api-key" | CODE |
| HIGH⚡ | packages/phoenix-client/README.md | 60 | export PHOENIX_API_KEY="your-api-key" | CODE |
| HIGH⚡ | packages/phoenix-client/README.md | 63 | export PHOENIX_CLIENT_HEADERS="Authorization=Bearer your-api-key,custom-header=value" | CODE |
| HIGH | packages/phoenix-client/README.md | 76 | PHOENIX_API_KEY=your-api-key | CODE |
| HIGH⚡ | packages/phoenix-client/README.md | 109 | api_key="your-api-key" | CODE |
| HIGH⚡ | packages/phoenix-client/README.md | 115 | headers={"Authorization": "Bearer your-api-key"} | CODE |
| HIGH⚡ | packages/phoenix-client/README.md | 123 | api_key="your-api-key" | CODE |
| HIGH⚡ | packages/phoenix-client/docs/source/index.md | 31 | export PHOENIX_API_KEY="your-api-key" | CODE |
| HIGH⚡ | packages/phoenix-client/docs/source/index.md | 36 | export PHOENIX_API_KEY="your-api-key" | CODE |
| HIGH⚡ | packages/phoenix-client/docs/source/index.md | 39 | export PHOENIX_CLIENT_HEADERS="Authorization=Bearer your-api-key,custom-header=value" | CODE |
| HIGH⚡ | packages/phoenix-client/docs/source/index.md | 57 | api_key="your-api-key" | CODE |
| HIGH⚡ | packages/phoenix-client/docs/source/index.md | 63 | headers={"Authorization": "Bearer your-api-key"} | CODE |
| HIGH⚡ | packages/phoenix-client/docs/source/index.md | 71 | api_key="your-api-key" | CODE |
| HIGH | packages/phoenix-otel/README.md | 90 | export PHOENIX_API_KEY="your-api-key" | CODE |
| HIGH | packages/phoenix-otel/README.md | 95 | tracer_provider = register(api_key="your-api-key") | CODE |
| HIGH | packages/phoenix-otel/README.md | 136 | api_key="your-api-key", # Authentication | CODE |
| HIGH | packages/phoenix-otel/README.md | 179 | | `PHOENIX_API_KEY` | Authentication key | `your-api-key` | | CODE |
| HIGH | packages/phoenix-otel/README.md | 194 | PHOENIX_API_KEY=your-api-key | CODE |
| HIGH | packages/phoenix-otel/docs/source/index.md | 97 | # export PHOENIX_API_KEY=your-api-key | COMMENT |
| HIGH⚡ | packages/phoenix-otel/docs/source/index.md | 261 | export PHOENIX_API_KEY="your-api-key" | CODE |
| HIGH⚡ | packages/phoenix-otel/docs/source/index.md | 267 | export PHOENIX_API_KEY="your-api-key" | CODE |
| HIGH⚡ | packages/phoenix-otel/docs/source/index.md | 268 | export PHOENIX_CLIENT_HEADERS="Authorization=Bearer your-api-key,custom-header=value" | CODE |
| HIGH | packages/phoenix-otel/src/phoenix/otel/otel.py | 127 | api_key="your-api-key" | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | tests/unit/server/sandbox/test_modal_backend.py | 0 | a classified sdk exception inside run_code must propagate, not wrap. without the narrow re-raise the manager has no sign | STRING |
| HIGH | tests/unit/server/sandbox/test_vercel_backend.py | 0 | a classified sdk exception inside run_code must propagate, not wrap. without the narrow re-raise the manager has no sign | STRING |
| HIGH | tests/unit/server/sandbox/test_e2b_backend.py | 0 | a classified sdk exception inside run_code must propagate, not wrap. without the narrow re-raise the manager has no sign | STRING |
| HIGH | tests/unit/server/api/routers/v1/test_traces.py | 0 | contains a project with a single trace and a single span. | STRING |
| HIGH | tests/unit/server/api/routers/v1/test_spans.py | 0 | contains a project with a single trace and a single span. | STRING |
| HIGH | tests/unit/server/api/types/test_SpanAnnotation.py | 0 | contains a project with a single trace and a single span. | STRING |
| HIGH | tests/unit/server/api/types/test_Span.py | 0 | contains a project with a single trace and a single span. | STRING |
| HIGH | tests/unit/server/api/types/test_TraceAnnotation.py | 0 | contains a project with a single trace and a single span. | STRING |
| HIGH | tests/integration/dataset_labels/conftest.py | 0 | combine all environment variable configurations for testing. | STRING |
| HIGH | tests/integration/auth/conftest.py | 0 | combine all environment variable configurations for testing. | STRING |
| HIGH | tests/integration/experiments/conftest.py | 0 | combine all environment variable configurations for testing. | STRING |
| HIGH | tests/integration/project_sessions/conftest.py | 0 | combine all environment variable configurations for testing. | STRING |
| HIGH | tests/integration/server/test_db_disk_usage_monitor.py | 0 | combine all environment variable configurations for testing. | STRING |
| HIGH | tests/integration/client/conftest.py | 0 | combine all environment variable configurations for testing. | STRING |
| HIGH | tests/integration/annotation_configs/conftest.py | 0 | combine all environment variable configurations for testing. | STRING |
| HIGH | examples/agents/tau_bench_openai_agents/run.py | 0 | return a timestamped default output path under examples/results. | STRING |
| HIGH | examples/agents/traject_bench_langgraph/run.py | 0 | return a timestamped default output path under examples/results. | STRING |
| HIGH | examples/agents/tau_bench_langgraph/run.py | 0 | return a timestamped default output path under examples/results. | STRING |
| HIGH | …amples/agents/tau_bench_openai_agents/phoenix_setup.py | 0 | resolve otlp traces endpoint, preferring phoenix_collector_endpoint env var. appends `/v1/traces` if the env var points | STRING |
| HIGH | …amples/agents/traject_bench_langgraph/phoenix_setup.py | 0 | resolve otlp traces endpoint, preferring phoenix_collector_endpoint env var. appends `/v1/traces` if the env var points | STRING |
| HIGH | examples/agents/tau_bench_langgraph/phoenix_setup.py | 0 | resolve otlp traces endpoint, preferring phoenix_collector_endpoint env var. appends `/v1/traces` if the env var points | STRING |
| HIGH | …amples/agents/tau_bench_openai_agents/phoenix_setup.py | 0 | return the phoenix ui base url (without the otlp `/v1/traces` suffix). uses phoenix_collector_endpoint if set, otherwise | STRING |
| HIGH | …amples/agents/traject_bench_langgraph/phoenix_setup.py | 0 | return the phoenix ui base url (without the otlp `/v1/traces` suffix). uses phoenix_collector_endpoint if set, otherwise | STRING |
| HIGH | examples/agents/tau_bench_langgraph/phoenix_setup.py | 0 | return the phoenix ui base url (without the otlp `/v1/traces` suffix). uses phoenix_collector_endpoint if set, otherwise | STRING |
| HIGH | …t_calculations/generate_spans_for_cost_calculations.py | 0 | generate n random numbers that sum to total using dirichlet distribution. args: total (int): the total sum that the gene | STRING |
| HIGH | …_for_large_session/generate_spans_for_large_session.py | 0 | generate n random numbers that sum to total using dirichlet distribution. args: total (int): the total sum that the gene | STRING |
| HIGH | scripts/generate_spans/generate_spans_deeply_nested.py | 0 | generate n random numbers that sum to total using dirichlet distribution. args: total (int): the total sum that the gene | STRING |
| HIGH | …ripts/generate_spans/generate_spans_for_time_series.py | 0 | generate n random numbers that sum to total using dirichlet distribution. args: total (int): the total sum that the gene | STRING |
| HIGH | …t_calculations/generate_spans_for_cost_calculations.py | 0 | generate a synthetic llm span with random token counts and model information. creates a span with the following attribut | STRING |
| HIGH | …_for_large_session/generate_spans_for_large_session.py | 0 | generate a synthetic llm span with random token counts and model information. creates a span with the following attribut | STRING |
| HIGH | scripts/generate_spans/generate_spans_deeply_nested.py | 0 | generate a synthetic llm span with random token counts and model information. creates a span with the following attribut | STRING |
| HIGH | …ripts/generate_spans/generate_spans_for_time_series.py | 0 | generate a synthetic llm span with random token counts and model information. creates a span with the following attribut | STRING |
| HIGH | …_llama_index_with_recursive_character_text_splitter.py | 0 | loads documents from a gitbook url. args: docs_url (str): url to gitbook docs. returns: list[langchaindocument]: list of | STRING |
| HIGH | scripts/data/build_langchain_vector_store.py | 0 | loads documents from a gitbook url. args: docs_url (str): url to gitbook docs. returns: list[langchaindocument]: list of | STRING |
| HIGH | scripts/data/fetch_arize_documentation.py | 0 | loads documents from a gitbook url. args: docs_url (str): url to gitbook docs. returns: list[langchaindocument]: list of | STRING |
| HIGH | …kages/phoenix-evals/src/phoenix/evals/rate_limiters.py | 0 | an adaptive rate-limiter that adjusts the rate based on the number of rate limit errors. this rate limiter does not need | STRING |
| HIGH | …oenix-client/src/phoenix/client/utils/rate_limiters.py | 0 | an adaptive rate-limiter that adjusts the rate based on the number of rate limit errors. this rate limiter does not need | STRING |
| HIGH | src/phoenix/server/rate_limiters.py | 0 | an adaptive rate-limiter that adjusts the rate based on the number of rate limit errors. this rate limiter does not need | STRING |
| HIGH | …kages/phoenix-evals/src/phoenix/evals/rate_limiters.py | 0 | lazily initialize async primitives to ensure they are created in the correct event loop. | STRING |
| HIGH | …oenix-client/src/phoenix/client/utils/rate_limiters.py | 0 | lazily initialize async primitives to ensure they are created in the correct event loop. | STRING |
| HIGH | src/phoenix/server/rate_limiters.py | 0 | lazily initialize async primitives to ensure they are created in the correct event loop. | STRING |
| HIGH | …client/src/phoenix/client/resources/traces/__init__.py | 0 | log multiple session annotations from a pandas dataframe asynchronously. requires phoenix server >= 12.0.0. this method | STRING |
| HIGH | …-client/src/phoenix/client/resources/spans/__init__.py | 0 | log multiple session annotations from a pandas dataframe asynchronously. requires phoenix server >= 12.0.0. this method | STRING |
| HIGH | …ient/src/phoenix/client/resources/sessions/__init__.py | 0 | log multiple session annotations from a pandas dataframe asynchronously. requires phoenix server >= 12.0.0. this method | STRING |
| HIGH | src/phoenix/server/api/types/Trace.py | 0 | retrieves and summarizes annotations associated with this span. this method aggregates annotation data by name and label | STRING |
| HIGH | src/phoenix/server/api/types/ProjectSession.py | 0 | retrieves and summarizes annotations associated with this span. this method aggregates annotation data by name and label | STRING |
| HIGH | src/phoenix/server/api/types/Span.py | 0 | retrieves and summarizes annotations associated with this span. this method aggregates annotation data by name and label | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | js/examples/apps/tracing-tutorial/support-agent.ts | 244 | // Step 1: Classify the query | COMMENT |
| LOW | js/examples/apps/tracing-tutorial/support-agent.ts | 292 | // Step 2: Route based on classification | COMMENT |
| LOW | js/examples/apps/tracing-tutorial/evaluate-traces.ts | 142 | // Step 1: Fetch spans from Phoenix | COMMENT |
| LOW | js/examples/apps/tracing-tutorial/evaluate-traces.ts | 162 | // Step 2: Filter spans by type | COMMENT |
| LOW | js/examples/apps/tracing-tutorial/evaluate-traces.ts | 192 | // Step 3: Evaluate tool spans (simple code-based check) | COMMENT |
| LOW | js/examples/apps/tracing-tutorial/evaluate-traces.ts | 264 | // Step 4: Log annotations to Phoenix | COMMENT |
| LOW | js/examples/apps/tracing-tutorial/evaluate-traces.ts | 280 | // Step 5: Summary | COMMENT |
| LOW | js/examples/apps/tracing-tutorial/evaluate-traces.ts | 337 | // Step 1: Fetch spans from Phoenix | COMMENT |
| LOW | js/examples/apps/tracing-tutorial/evaluate-traces.ts | 359 | // Step 2: Group spans by session ID | COMMENT |
| LOW | js/examples/apps/tracing-tutorial/evaluate-traces.ts | 395 | // Step 3: Evaluate each session | COMMENT |
| LOW | js/examples/apps/tracing-tutorial/evaluate-traces.ts | 486 | // Step 4: Log session annotations to Phoenix | COMMENT |
| LOW | js/examples/apps/tracing-tutorial/evaluate-traces.ts | 504 | // Step 5: Summary | COMMENT |
| LOW | …/mastra-agent/src/experiments/configure-experiments.ts | 10 | // Step 1: define the task to run (we call the agent with the question) | COMMENT |
| LOW | …/mastra-agent/src/experiments/configure-experiments.ts | 21 | // Step 2: define the dataset of questions to ask the agent | COMMENT |
| LOW | …/mastra-agent/src/experiments/configure-experiments.ts | 45 | // Step 3: Define the evaluators | COMMENT |
| LOW | …s/apps/mastra-agent/src/experiments/run-experiments.ts | 7 | //Step 4: Run the experiment | COMMENT |
| LOW | …s/phoenix-client/examples/create_dataset_from_spans.ts | 117 | // Step 1: Fetch spans from the project | COMMENT |
| LOW | …s/phoenix-client/examples/create_dataset_from_spans.ts | 138 | // Step 2: Transform spans into dataset examples with span ID links | COMMENT |
| LOW | …s/phoenix-client/examples/create_dataset_from_spans.ts | 172 | // Step 3: Create the dataset with span associations | COMMENT |
| LOW | …/phoenix-client/examples/experiment_trace_evaluator.ts | 36 | // Step 1: Define the agent's tools | COMMENT |
| LOW | …/phoenix-client/examples/experiment_trace_evaluator.ts | 60 | // Step 2: Define the agent | COMMENT |
| LOW | …/phoenix-client/examples/experiment_trace_evaluator.ts | 80 | // Step 3: Build the evaluator | COMMENT |
| LOW | …/phoenix-client/examples/experiment_trace_evaluator.ts | 126 | // Step 4: Run the experiment and evaluate | COMMENT |
| LOW | …ernal_docs/specs/ldap-authentication/migration-plan.md | 89 | # Step 1: Update CHECK constraint to allow 'LDAP' | COMMENT |
| LOW | …ernal_docs/specs/ldap-authentication/migration-plan.md | 97 | # Step 2: Migrate LDAP users | COMMENT |
| LOW | …_docs/specs/ldap-authentication/future-improvements.md | 387 | # Step 1: Get tokenGroups (binary SIDs) | COMMENT |
| LOW | …_docs/specs/ldap-authentication/future-improvements.md | 400 | # Step 2: Resolve SIDs to DNs | COMMENT |
| LOW⚡ | …ernal_docs/specs/ldap-authentication/optional-email.md | 181 | # Step 1: Lookup by unique_id first (always, when configured) | COMMENT |
| LOW⚡ | …ernal_docs/specs/ldap-authentication/optional-email.md | 186 | # Step 2: Fallback to email lookup (only for real emails) | COMMENT |
| LOW⚡ | …ernal_docs/specs/ldap-authentication/optional-email.md | 193 | # Step 3: Return existing user (update email if changed) | COMMENT |
| LOW⚡ | …ernal_docs/specs/ldap-authentication/optional-email.md | 199 | # Step 4: Create new user | COMMENT |
| LOW | tests/unit/_helpers.py | 235 | # Step 1: Get ALL revisions up to and including this dataset version | COMMENT |
| LOW⚡ | tests/unit/_helpers.py | 248 | # Step 2: Find the latest revision for each example (simple Python logic) | COMMENT |
| LOW⚡ | tests/unit/_helpers.py | 256 | # Step 3: Filter out DELETE revisions | COMMENT |
| LOW⚡ | tests/unit/_helpers.py | 263 | # Step 4: Check if experiment has assigned splits | COMMENT |
| LOW⚡ | tests/unit/_helpers.py | 272 | # Step 5: Filter by splits if any are assigned | COMMENT |
| LOW⚡ | tests/integration/_helpers.py | 1559 | # Step 2: Validate PKCE (if required) | COMMENT |
| LOW | tests/integration/_helpers.py | 1538 | # Step 1: Validate client authentication (if required) | COMMENT |
| LOW | tests/integration/_helpers.py | 1595 | # Step 3: Determine authentication mode and validate | COMMENT |
| LOW⚡ | tests/integration/auth/test_ldap.py | 503 | # Step 1: Create LDAP user via login | COMMENT |
| LOW⚡ | tests/integration/auth/test_ldap.py | 509 | # Step 2: Query GraphQL to verify authMethod is 'LDAP' | COMMENT |
| LOW | tests/integration/auth/test_ldap.py | 664 | # Step 1: Admin pre-creates LDAP user (only needs email) | COMMENT |
| LOW⚡ | tests/integration/auth/test_ldap.py | 698 | # Step 2: Add user to LDAP with matching email | STRING |
| LOW⚡ | tests/integration/auth/test_ldap.py | 707 | # Step 3: User logs in via LDAP (email lookup finds pre-created user) | STRING |
| LOW⚡ | tests/integration/auth/test_ldap.py | 713 | # Step 4: Verify username remains stable (not synced from LDAP) | STRING |
| LOW⚡ | tests/integration/auth/test_ldap.py | 721 | # Step 5: Verify subsequent logins work | STRING |
| LOW | tests/integration/client/test_experiments.py | 2621 | # Step 1: Run experiment WITHOUT evaluators (task execution only) | COMMENT |
| LOW | tests/integration/client/test_experiments.py | 2640 | # Step 2: Add evaluations to the completed experiment | COMMENT |
| LOW⚡ | …/skills/phoenix-server/references/database-patterns.md | 35 | # Step 1: Add column as nullable | COMMENT |
| LOW⚡ | …/skills/phoenix-server/references/database-patterns.md | 39 | # Step 2: Backfill | COMMENT |
| LOW⚡ | …/skills/phoenix-server/references/database-patterns.md | 42 | # Step 3: Add constraints | COMMENT |
| LOW | .agents/skills/phoenix-typescript-package-docs/SKILL.md | 172 | ### Step 1: Determine the affected package and modules | COMMENT |
| LOW | .agents/skills/phoenix-typescript-package-docs/SKILL.md | 182 | ### Step 2: Update canonical Mintlify docs | COMMENT |
| LOW | .agents/skills/phoenix-typescript-package-docs/SKILL.md | 195 | ### Step 3: Sync generated package docs | COMMENT |
| LOW | .agents/skills/phoenix-typescript-package-docs/SKILL.md | 221 | ### Step 4: Verify the npm artifact | COMMENT |
| LOW | .agents/skills/phoenix-typescript-package-docs/SKILL.md | 236 | ### Step 5: Check for nav and path regressions | COMMENT |
| LOW | …x-evals/references/experiments-synthetic-typescript.md | 26 | // Step 1: Create tuples | COMMENT |
| LOW | …x-evals/references/experiments-synthetic-typescript.md | 33 | // Step 2: Convert to natural query | COMMENT |
| LOW | …oenix-evals/references/experiments-synthetic-python.md | 23 | # Step 1: Create tuples | COMMENT |
| LOW | …oenix-evals/references/experiments-synthetic-python.md | 29 | # Step 2: Convert to natural query | COMMENT |
| 75 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | app/stories/Dialog.stories.tsx | 72 | Lorem ipsum dolor sit amet consectetur adipisicing elit. Quisquam, | CODE |
| LOW | app/stories/Dialog.stories.tsx | 72 | Lorem ipsum dolor sit amet consectetur adipisicing elit. Quisquam, | CODE |
| LOW | app/stories/Dialog.stories.tsx | 109 | Lorem ipsum dolor sit amet consectetur adipisicing elit. | CODE |
| LOW | app/stories/Dialog.stories.tsx | 109 | Lorem ipsum dolor sit amet consectetur adipisicing elit. | CODE |
| LOW | app/stories/Dialog.stories.tsx | 140 | Lorem ipsum dolor sit amet consectetur adipisicing elit. | CODE |
| LOW | app/stories/Dialog.stories.tsx | 140 | Lorem ipsum dolor sit amet consectetur adipisicing elit. | CODE |
| LOW | app/stories/TableColumnOrdering.stories.tsx | 38 | name: "John Doe", | CODE |
| LOW | app/stories/Table.stories.tsx | 70 | name: "John Doe", | CODE |
| LOW | app/stories/Table.stories.tsx | 719 | for (const name of ["John Doe", "David Kim"]) { | CODE |
| LOW | app/stories/ExpandableContent.stories.tsx | 78 | Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed do eiusmod tempor incididunt ut labore | CODE |
| LOW | app/stories/ExpandableContent.stories.tsx | 78 | Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed do eiusmod tempor incididunt ut labore | CODE |
| LOW | app/stories/ExpandableContent.stories.tsx | 91 | name: "John Doe", | CODE |
| LOW | app/stories/Modal.stories.tsx | 44 | Lorem ipsum dolor sit amet consectetur adipisicing elit. | CODE |
| LOW | app/stories/Modal.stories.tsx | 44 | Lorem ipsum dolor sit amet consectetur adipisicing elit. | CODE |
| LOW | app/stories/Modal.stories.tsx | 74 | Lorem ipsum dolor sit amet consectetur adipisicing elit. | CODE |
| LOW | app/stories/Modal.stories.tsx | 74 | Lorem ipsum dolor sit amet consectetur adipisicing elit. | CODE |
| LOW | app/stories/Modal.stories.tsx | 104 | Lorem ipsum dolor sit amet consectetur adipisicing elit. | CODE |
| LOW | app/stories/Modal.stories.tsx | 104 | Lorem ipsum dolor sit amet consectetur adipisicing elit. | CODE |
| LOW | …ks/evals-benchmarks/src/tool_response_handling.eval.ts | 133 | address: "123 Main St", | CODE |
| LOW | …es/phoenix-evals/test/utils/objectMappingUtils.test.ts | 10 | name: "John Doe", | CODE |
| LOW | …es/phoenix-evals/test/utils/objectMappingUtils.test.ts | 14 | street: "123 Main St", | CODE |
| LOW | …es/phoenix-evals/test/utils/objectMappingUtils.test.ts | 46 | userName: "John Doe", // Added mapped field | CODE |
| LOW | …es/phoenix-evals/test/utils/objectMappingUtils.test.ts | 88 | extractedName: "John Doe", | CODE |
| LOW | …es/phoenix-evals/test/utils/objectMappingUtils.test.ts | 105 | extractedStreet: "123 Main St", | CODE |
| LOW | …es/phoenix-evals/test/utils/objectMappingUtils.test.ts | 202 | extractedName: "John Doe", | CODE |
| LOW | …es/phoenix-evals/test/utils/objectMappingUtils.test.ts | 203 | extractedStreet: "123 Main St", | CODE |
| LOW | …es/phoenix-evals/test/utils/objectMappingUtils.test.ts | 315 | fullName: "John Doe", | CODE |
| LOW | …packages/phoenix-evals/src/utils/objectMappingUtils.ts | 117 | * // fullName: "John Doe", | COMMENT |
| LOW⚡ | …al_docs/vignettes/sqlalchemy/polymorphic_user/users.py | 25 | local_user = LocalUser(email="user@example.com", password="secret") | STRING |
| LOW | …al_docs/vignettes/sqlalchemy/polymorphic_user/users.py | 181 | user = LocalUser(email="user@example.com", password="secret") | STRING |
| LOW | …l_docs/vignettes/sqlalchemy/polymorphic_user/README.md | 33 | - During object creation (e.g., `LocalUser(email="user@example.com", password="secret")`): | CODE |
| LOW | …l_docs/vignettes/sqlalchemy/polymorphic_user/README.md | 93 | local_user = LocalUser(email="user@example.com", password="secret") | CODE |
| LOW | …rnal_docs/vignettes/json-jsonb-demo/src/performance.ts | 61 | name: 'John Doe', | CODE |
| LOW | …rnal_docs/vignettes/json-jsonb-demo/src/performance.ts | 64 | street: '123 Main St', | CODE |
| LOW | …rnal_docs/vignettes/json-jsonb-demo/src/performance.ts | 135 | WHERE data_json->'user'->>'name' = 'John Doe' | CODE |
| LOW | …rnal_docs/vignettes/json-jsonb-demo/src/performance.ts | 148 | WHERE data_jsonb->'user'->>'name' = 'John Doe' | CODE |
| LOW | …rnal_docs/vignettes/json-jsonb-demo/src/performance.ts | 179 | WHERE data_jsonb->'user'->>'name' = 'John Doe' | CODE |
| LOW | …l_docs/vignettes/json-jsonb-demo/src/query-examples.ts | 31 | name: 'John Doe', | CODE |
| LOW | …rnal_docs/specs/ldap-authentication/database-schema.md | 12 | username = 'John Doe' ← Display name (synced from LDAP) | CODE |
| LOW | tests/unit/test_config.py | 200 | "user@example.com": "User=With=Equals", | CODE |
| LOW⚡ | tests/unit/server/test_ldap.py | 822 | mock_attr.values = ["user@example.com"] | CODE |
| LOW⚡ | tests/unit/server/test_ldap.py | 826 | assert result == "user@example.com" | CODE |
| LOW | tests/unit/server/test_ldap.py | 172 | mock_display_name.values = ["John Doe"] | CODE |
| LOW | tests/unit/server/test_ldap.py | 188 | assert result.display_name == "John Doe" | CODE |
| LOW | tests/unit/server/test_ldap.py | 274 | mock_display_name.values = ["John Doe"] | CODE |
| LOW | tests/unit/server/test_ldap.py | 324 | mock_display_name.values = ["John Doe"] | CODE |
| LOW | tests/unit/server/test_ldap.py | 405 | mock_display_name.values = ["John Doe"] | CODE |
| LOW⚡ | tests/unit/server/test_rate_limiters.py | 541 | limiter.record_failure("user@example.com") | CODE |
| LOW⚡ | tests/unit/server/test_rate_limiters.py | 542 | limiter.record_failure("user@example.com") | CODE |
| LOW⚡ | tests/unit/server/test_rate_limiters.py | 543 | limiter.check("user@example.com") # 2 failures < 3 max, should not raise | CODE |
| LOW⚡ | tests/unit/server/test_rate_limiters.py | 555 | limiter.record_failure("user@example.com") | CODE |
| LOW⚡ | tests/unit/server/test_rate_limiters.py | 556 | limiter.record_failure("user@example.com") | CODE |
| LOW⚡ | tests/unit/server/test_rate_limiters.py | 557 | limiter.record_failure("user@example.com") | CODE |
| LOW⚡ | tests/unit/server/test_rate_limiters.py | 559 | limiter.check("user@example.com") | CODE |
| LOW⚡ | tests/unit/server/test_rate_limiters.py | 568 | limiter.record_failure("user@example.com") | CODE |
| LOW⚡ | tests/unit/server/test_rate_limiters.py | 569 | limiter.record_failure("user@example.com") | CODE |
| LOW⚡ | tests/unit/server/test_rate_limiters.py | 570 | limiter.record_success("user@example.com") | CODE |
| LOW⚡ | tests/unit/server/test_rate_limiters.py | 572 | limiter.record_failure("user@example.com") | CODE |
| LOW⚡ | tests/unit/server/test_rate_limiters.py | 573 | limiter.record_failure("user@example.com") | CODE |
| LOW⚡ | tests/unit/server/test_rate_limiters.py | 574 | limiter.check("user@example.com") # should not raise | CODE |
| 145 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | evals/pxi/evaluators/__init__.py | 31 | __all__ = [ | CODE |
| LOW⚡ | …ents/capabilities/tools/internal/test_call_subagent.py | 50 | def set_subagent_final_tool_output(chunk: ToolOutputAvailableChunk) -> None: | CODE |
| LOW⚡ | …ents/capabilities/tools/internal/test_call_subagent.py | 91 | def set_subagent_final_tool_output(chunk: ToolOutputAvailableChunk) -> None: | CODE |
| LOW | tests/unit/server/api/routers/test_oauth2.py | 29 | async def _setup_role_ids(self, asgi_app: ASGIApp, db: DbSessionFactory) -> None: | CODE |
| LOW | tests/unit/server/api/routers/test_oauth2.py | 161 | async def _setup_role_ids(self, asgi_app: ASGIApp, db: DbSessionFactory) -> None: | CODE |
| LOW | tests/unit/db/test_facilitator.py | 299 | def _update_manifest(self, manifest_path: Path, models: list[dict[str, Any]]) -> None: | CODE |
| LOW | tests/integration/_mock_ldap_server.py | 40 | logger = logging.getLogger(__name__) | CODE |
| LOW⚡ | tests/integration/_mock_llm_server.py | 145 | logger = logging.getLogger(__name__) | CODE |
| LOW | tests/integration/_helpers.py | 1442 | def _setup_routes(self) -> None: | CODE |
| LOW | tests/integration/_helpers.py | 1878 | def set_role(self, role: Optional[str], num_logins: int) -> None: | CODE |
| LOW | tests/integration/_helpers.py | 1896 | def set_email(self, email: str, num_logins: int) -> None: | CODE |
| LOW | tests/integration/_helpers.py | 1920 | def set_picture(self, picture: str, num_logins: int) -> None: | CODE |
| LOW | examples/agents/tau_bench_openai_agents/tools.py | 58 | def set_data(data: dict[str, Any]) -> None: | CODE |
| LOW | examples/agents/tau_bench_langgraph/tools.py | 58 | def set_data(data: dict[str, Any]) -> None: | CODE |
| LOW | scripts/update_helm.py | 30 | def update_helm_values(new_version: str) -> None: | CODE |
| LOW⚡ | scripts/update_helm.py | 41 | def update_helm_chart(new_version: str) -> None: | CODE |
| LOW | scripts/docker/devops/scripts/test_ldap_tls.py | 45 | logger = logging.getLogger(__name__) | CODE |
| LOW | scripts/docker/devops/scripts/ldap_mitm_proxy.py | 44 | logger = logging.getLogger(__name__) | CODE |
| LOW⚡ | scripts/prompts/compile_python_prompts.py | 62 | __all__ = [ | CODE |
| LOW | scripts/data/build_llama_index_with_markdown_reader.py | 17 | logger = logging.getLogger(__name__) | CODE |
| LOW | packages/phoenix-evals/src/phoenix/evals/tracing.py | 25 | logger = logging.getLogger(__name__) | CODE |
| LOW | packages/phoenix-evals/src/phoenix/evals/__init__.py | 35 | __all__ = [ | CODE |
| LOW | packages/phoenix-evals/src/phoenix/evals/executors.py | 30 | logger = logging.getLogger(__name__) | CODE |
| LOW⚡ | packages/phoenix-evals/src/phoenix/evals/executors.py | 129 | def _update_concurrency_target(self) -> None: | CODE |
| LOW | packages/phoenix-evals/src/phoenix/evals/utils.py | 509 | __all__ = [ | CODE |
| LOW | packages/phoenix-evals/src/phoenix/evals/evaluators.py | 1736 | __all__ = [ | CODE |
| LOW | …es/phoenix-evals/src/phoenix/evals/metrics/__init__.py | 14 | __all__ = [ | CODE |
| LOW | …ckages/phoenix-evals/src/phoenix/evals/llm/__init__.py | 13 | __all__ = [ | CODE |
| LOW | packages/phoenix-evals/src/phoenix/evals/llm/types.py | 8 | __all__ = [ | CODE |
| LOW | …ages/phoenix-evals/src/phoenix/evals/llm/registries.py | 9 | logger = logging.getLogger(__name__) | CODE |
| LOW | …oenix-evals/src/phoenix/evals/llm/adapters/__init__.py | 11 | __all__ = [ | CODE |
| LOW | …ls/src/phoenix/evals/llm/adapters/langchain/adapter.py | 21 | logger = logging.getLogger(__name__) | CODE |
| LOW | …s/src/phoenix/evals/llm/adapters/langchain/__init__.py | 5 | __all__ = [ | CODE |
| LOW⚡ | …evals/src/phoenix/evals/llm/adapters/google/adapter.py | 17 | logger = logging.getLogger(__name__) | CODE |
| LOW | …vals/src/phoenix/evals/llm/adapters/google/__init__.py | 3 | __all__ = ["GoogleGenAIAdapter"] | CODE |
| LOW | …ls/src/phoenix/evals/llm/adapters/anthropic/adapter.py | 16 | logger = logging.getLogger(__name__) | CODE |
| LOW | …s/src/phoenix/evals/llm/adapters/anthropic/__init__.py | 3 | __all__ = ["AnthropicAdapter"] | CODE |
| LOW | …vals/src/phoenix/evals/llm/adapters/litellm/adapter.py | 27 | logger = logging.getLogger(__name__) | CODE |
| LOW | …als/src/phoenix/evals/llm/adapters/litellm/__init__.py | 5 | __all__ = [ | CODE |
| LOW | …evals/src/phoenix/evals/llm/adapters/openai/adapter.py | 21 | logger = logging.getLogger(__name__) | CODE |
| LOW | …vals/src/phoenix/evals/llm/adapters/openai/__init__.py | 3 | __all__ = [ | CODE |
| LOW | …phoenix-evals/src/phoenix/evals/templating/__init__.py | 31 | __all__ = [ | CODE |
| LOW | packages/phoenix-client/src/phoenix/client/__init__.py | 7 | __all__ = [ | CODE |
| LOW | …es/phoenix-client/src/phoenix/client/types/__init__.py | 3 | __all__ = [ | CODE |
| LOW | …es/phoenix-client/src/phoenix/client/pytest/tracing.py | 26 | logger = logging.getLogger(__name__) | CODE |
| LOW | …es/phoenix-client/src/phoenix/client/pytest/tracing.py | 158 | def _set_io(span: Any, input_value: Any, output_value: Any) -> None: | CODE |
| LOW | …es/phoenix-client/src/phoenix/client/pytest/session.py | 24 | logger = logging.getLogger(__name__) | CODE |
| LOW | …s/phoenix-client/src/phoenix/client/pytest/__init__.py | 23 | __all__ = ["log_output", "log_evaluation", "evaluate"] | CODE |
| LOW | …es/phoenix-client/src/phoenix/client/pytest/context.py | 13 | logger = logging.getLogger(__name__) | CODE |
| LOW | …es/phoenix-client/src/phoenix/client/pytest/context.py | 28 | def set_output(self, value: Any) -> None: | CODE |
| LOW | …ges/phoenix-client/src/phoenix/client/pytest/plugin.py | 37 | logger = logging.getLogger(__name__) | CODE |
| LOW | …enix-client/src/phoenix/client/experiments/__init__.py | 1027 | __all__ = [ | CODE |
| LOW | …-client/src/phoenix/client/resources/spans/__init__.py | 40 | logger = logging.getLogger(__name__) | CODE |
| LOW | …t/src/phoenix/client/resources/experiments/__init__.py | 62 | logger = logging.getLogger(__name__) | CODE |
| LOW | …ient/src/phoenix/client/resources/projects/__init__.py | 15 | logger = logging.getLogger(__name__) | CODE |
| LOW | …ient/src/phoenix/client/resources/datasets/__init__.py | 40 | logger = logging.getLogger(__name__) | CODE |
| LOW | …lient/src/phoenix/client/resources/prompts/__init__.py | 17 | logger = logging.getLogger(__name__) | CODE |
| LOW | …ages/phoenix-client/src/phoenix/client/utils/config.py | 27 | logger = logging.getLogger(__name__) | CODE |
| LOW | …s/phoenix-client/src/phoenix/client/utils/executors.py | 30 | logger = logging.getLogger(__name__) | CODE |
| LOW⚡ | …s/phoenix-client/src/phoenix/client/utils/executors.py | 140 | def _update_concurrency_target(self) -> None: | CODE |
| 100 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH⚡ | tests/unit/server/test_oauth2.py | 1341 | email_attribute_path="contains(email_aliases[*], 'primary') && email || alternate_email", | CODE |
| HIGH⚡ | tests/unit/server/test_oauth2.py | 1350 | "contains(email_aliases[*], 'primary') && email || alternate_email" | CODE |
| HIGH⚡ | tests/unit/server/test_oauth2.py | 632 | """Test that edge case claim values are handled gracefully (null, empty, unicode, nested objects).""" | STRING |
| HIGH | tests/unit/server/api/routers/v1/test_datasets.py | 1019 | jsonl_content = b'{"a": null, "b": "x"}\n{"b": "y"}\n' | CODE |
| HIGH | tests/unit/server/api/routers/v1/test_datasets.py | 2859 | jsonl_content = b'{"a": null, "b": 2}\n' | CODE |
| HIGH | tests/unit/server/api/routers/v1/test_datasets.py | 2990 | b'"metadata": {"difficulty": null}}\n' | CODE |
| HIGH | tests/unit/server/api/routers/v1/test_datasets.py | 1424 | """Test JSON upload with various split formats: string, list, null, and mixed.""" | STRING |
| HIGH | tests/unit/server/api/types/test_Evaluator.py | 347 | """When dataset evaluator description is null, it should return base evaluator's description.""" | STRING |
| HIGH | tests/unit/server/api/types/test_Evaluator.py | 368 | """When both dataset evaluator and base evaluator descriptions are null, it returns null.""" | STRING |
| HIGH | tests/unit/server/api/types/test_Dataset.py | 34 | query ($exampleId: ID!, $datasetVersionId: ID = null) { | CODE |
| HIGH | tests/unit/server/api/types/test_Dataset.py | 175 | query ($datasetId: ID!, $datasetVersionId: ID = null) { | CODE |
| HIGH | tests/unit/server/api/types/test_Dataset.py | 281 | query ($datasetId: ID!, $datasetVersionId: ID = null, $revisionDatasetVersionId: ID = null, $filter: String = null | CODE |
| HIGH | tests/unit/server/api/types/test_Dataset.py | 734 | query ($datasetId: ID!, $datasetVersionId: ID = null, $includeEphemeral: Boolean = false) { | CODE |
| HIGH | tests/unit/server/api/types/test_Project.py | 1110 | query ($projectId: ID!, $after: String = null, $before: String = null, $filterCondition: String = null, $first: In | CODE |
| HIGH | …mutations/test_project_session_annotation_mutations.py | 51 | - Setting values to null (score/label/explanation to null) | STRING |
| HIGH | …mutations/test_project_session_annotation_mutations.py | 622 | # B5. Update with score=null, label=null, explanation="" should fail validation | STRING |
| HIGH | …mutations/test_project_session_annotation_mutations.py | 645 | # Fields not provided in the input are reset to their default values (null) | STRING |
| HIGH | …sts/unit/server/api/mutations/test_secret_mutations.py | 323 | # Test 12: Delete existing secret (by passing value: null) | STRING |
| HIGH | tests/integration/client/test_spans.py | 1385 | 'the capital of France?"}, "format": "MUSTACHE"}, "prompt_name": null}' | CODE |
| HIGH | tests/integration/client/test_spans.py | 1392 | '"prompt_name": null}' | CODE |
| HIGH | scripts/ci/test_helm.py | 2693 | # (--set attrEmail=null would be interpreted as YAML null, becoming empty string) | COMMENT |
| HIGH | …/phoenix-evals/tests/phoenix/evals/llm/test_prompts.py | 181 | ("Mixed: {None} and {null}", TemplateFormat.F_STRING), | CODE |
| HIGH | …ents/capabilities/tools/external/write_prompt_tools.py | 59 | "A function tool to create or update. Omit `id` (or pass null) to create a new " | CODE |
| HIGH | src/phoenix/server/sandbox/daytona_backend.py | 149 | f"${{r.stderr || r.stdout}}`);\n" | CODE |
| HIGH | src/phoenix/server/api/routers/v1/experiments.py | 412 | # dropped — otherwise `{"name": null, "description": "x"}` would 200 and ignore the name. | COMMENT |
| HIGH | src/phoenix/server/api/routers/v1/secrets.py | 46 | """Values, when non-null, must be non-empty strings (after trimming).""" | STRING |
| HIGH | src/phoenix/server/api/types/Span.py | 209 | description="the parent span ID. If null, it is a root span", | CODE |
| HIGH | src/phoenix/server/api/mutations/secret_mutations.py | 50 | {"key": "OPENAI_API_KEY", "value": null} | STRING |
| HIGH | src/phoenix/server/api/mutations/secret_mutations.py | 55 | description="The value of the secret. If null, the secret will be deleted." | CODE |
| HIGH⚡ | src/phoenix/server/api/mutations/secret_mutations.py | 82 | determines the final action (upsert with that value, or delete if value is null). | STRING |
| HIGH⚡ | src/phoenix/server/api/mutations/secret_mutations.py | 82 | determines the final action (upsert with that value, or delete if value is null). | STRING |
| HIGH⚡ | src/phoenix/server/api/mutations/secret_mutations.py | 89 | "or delete if value is null)." | CODE |
| HIGH⚡ | src/phoenix/server/api/mutations/secret_mutations.py | 89 | "or delete if value is null)." | CODE |
| HIGH | src/phoenix/db/facilitator.py | 371 | applies to all projects without a specific policy (i.e. foreign key is null). | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | js/examples/apps/tracing-tutorial/support-agent.ts | 244 | // Step 1: Classify the query | COMMENT |
| LOW | js/examples/apps/tracing-tutorial/support-agent.ts | 292 | // Step 2: Route based on classification | COMMENT |
| LOW | js/examples/apps/tracing-tutorial/evaluate-traces.ts | 142 | // Step 1: Fetch spans from Phoenix | COMMENT |
| LOW | js/examples/apps/tracing-tutorial/evaluate-traces.ts | 162 | // Step 2: Filter spans by type | COMMENT |
| LOW | js/examples/apps/tracing-tutorial/evaluate-traces.ts | 192 | // Step 3: Evaluate tool spans (simple code-based check) | COMMENT |
| LOW | js/examples/apps/tracing-tutorial/evaluate-traces.ts | 264 | // Step 4: Log annotations to Phoenix | COMMENT |
| LOW | js/examples/apps/tracing-tutorial/evaluate-traces.ts | 280 | // Step 5: Summary | COMMENT |
| LOW | js/examples/apps/tracing-tutorial/evaluate-traces.ts | 337 | // Step 1: Fetch spans from Phoenix | COMMENT |
| LOW | js/examples/apps/tracing-tutorial/evaluate-traces.ts | 359 | // Step 2: Group spans by session ID | COMMENT |
| LOW | js/examples/apps/tracing-tutorial/evaluate-traces.ts | 395 | // Step 3: Evaluate each session | COMMENT |
| LOW | js/examples/apps/tracing-tutorial/evaluate-traces.ts | 486 | // Step 4: Log session annotations to Phoenix | COMMENT |
| LOW | js/examples/apps/tracing-tutorial/evaluate-traces.ts | 504 | // Step 5: Summary | COMMENT |
| LOW | …/mastra-agent/src/experiments/configure-experiments.ts | 10 | // Step 1: define the task to run (we call the agent with the question) | COMMENT |
| LOW | …/mastra-agent/src/experiments/configure-experiments.ts | 21 | // Step 2: define the dataset of questions to ask the agent | COMMENT |
| LOW | …/mastra-agent/src/experiments/configure-experiments.ts | 45 | // Step 3: Define the evaluators | COMMENT |
| LOW | …s/apps/mastra-agent/src/experiments/run-experiments.ts | 7 | //Step 4: Run the experiment | COMMENT |
| LOW | …s/phoenix-client/examples/create_dataset_from_spans.ts | 117 | // Step 1: Fetch spans from the project | COMMENT |
| LOW | …s/phoenix-client/examples/create_dataset_from_spans.ts | 138 | // Step 2: Transform spans into dataset examples with span ID links | COMMENT |
| LOW | …s/phoenix-client/examples/create_dataset_from_spans.ts | 172 | // Step 3: Create the dataset with span associations | COMMENT |
| LOW | …/phoenix-client/examples/experiment_trace_evaluator.ts | 36 | // Step 1: Define the agent's tools | COMMENT |
| LOW | …/phoenix-client/examples/experiment_trace_evaluator.ts | 60 | // Step 2: Define the agent | COMMENT |
| LOW | …/phoenix-client/examples/experiment_trace_evaluator.ts | 80 | // Step 3: Build the evaluator | COMMENT |
| LOW | …/phoenix-client/examples/experiment_trace_evaluator.ts | 126 | // Step 4: Run the experiment and evaluate | COMMENT |
| LOW | tests/unit/_helpers.py | 235 | # Step 1: Get ALL revisions up to and including this dataset version | COMMENT |
| LOW⚡ | tests/unit/_helpers.py | 248 | # Step 2: Find the latest revision for each example (simple Python logic) | COMMENT |
| LOW⚡ | tests/unit/_helpers.py | 256 | # Step 3: Filter out DELETE revisions | COMMENT |
| LOW⚡ | tests/unit/_helpers.py | 263 | # Step 4: Check if experiment has assigned splits | COMMENT |
| LOW⚡ | tests/unit/_helpers.py | 272 | # Step 5: Filter by splits if any are assigned | COMMENT |
| LOW⚡ | tests/integration/_helpers.py | 1559 | # Step 2: Validate PKCE (if required) | COMMENT |
| LOW | tests/integration/_helpers.py | 1538 | # Step 1: Validate client authentication (if required) | COMMENT |
| LOW | tests/integration/_helpers.py | 1595 | # Step 3: Determine authentication mode and validate | COMMENT |
| LOW⚡ | tests/integration/auth/test_ldap.py | 503 | # Step 1: Create LDAP user via login | COMMENT |
| LOW⚡ | tests/integration/auth/test_ldap.py | 509 | # Step 2: Query GraphQL to verify authMethod is 'LDAP' | COMMENT |
| LOW | tests/integration/auth/test_ldap.py | 664 | # Step 1: Admin pre-creates LDAP user (only needs email) | COMMENT |
| LOW⚡ | tests/integration/auth/test_ldap.py | 698 | # Step 2: Add user to LDAP with matching email | STRING |
| LOW⚡ | tests/integration/auth/test_ldap.py | 707 | # Step 3: User logs in via LDAP (email lookup finds pre-created user) | STRING |
| LOW⚡ | tests/integration/auth/test_ldap.py | 713 | # Step 4: Verify username remains stable (not synced from LDAP) | STRING |
| LOW⚡ | tests/integration/auth/test_ldap.py | 721 | # Step 5: Verify subsequent logins work | STRING |
| LOW | tests/integration/client/test_experiments.py | 2621 | # Step 1: Run experiment WITHOUT evaluators (task execution only) | COMMENT |
| LOW | tests/integration/client/test_experiments.py | 2640 | # Step 2: Add evaluations to the completed experiment | COMMENT |
| LOW⚡ | …ts/uv/type_check/get_phoenix_modules_with_ty_errors.py | 53 | # Step 1: Import phoenix to trace import order | COMMENT |
| LOW⚡ | …ts/uv/type_check/get_phoenix_modules_with_ty_errors.py | 59 | # Step 2: Run ty check | COMMENT |
| LOW⚡ | …ts/uv/type_check/get_phoenix_modules_with_ty_errors.py | 68 | # Step 3: Parse ty output to find files with errors | COMMENT |
| LOW⚡ | …ts/uv/type_check/get_phoenix_modules_with_ty_errors.py | 87 | # Step 4: Order files by import order | COMMENT |
| LOW | …ts/uv/type_check/get_phoenix_modules_with_ty_errors.py | 104 | # Step 5: Write to file | COMMENT |
| LOW⚡ | packages/phoenix-evals/src/phoenix/evals/utils.py | 450 | # Step 1: Identify all span_id sources | COMMENT |
| LOW⚡ | packages/phoenix-evals/src/phoenix/evals/utils.py | 454 | # Step 2: Create a working copy and preserve all span info | COMMENT |
| LOW⚡ | packages/phoenix-evals/src/phoenix/evals/utils.py | 468 | # Step 3: Reset index to avoid conflicts during concatenation | COMMENT |
| LOW⚡ | packages/phoenix-evals/src/phoenix/evals/utils.py | 471 | # Step 4: Validate we have span_id information | COMMENT |
| LOW⚡ | packages/phoenix-evals/src/phoenix/evals/utils.py | 475 | # Step 5: Process each score name | COMMENT |
| LOW⚡ | packages/phoenix-evals/src/phoenix/evals/utils.py | 488 | # Step 6: Concatenate with new sequential indices | COMMENT |
| LOW⚡ | …ent/examples/experiments/experiment_trace_evaluator.py | 47 | # Step 1: Define the agent's tools | COMMENT |
| LOW⚡ | …ent/examples/experiments/experiment_trace_evaluator.py | 81 | # Step 2: Define the agent | COMMENT |
| LOW⚡ | …ent/examples/experiments/experiment_trace_evaluator.py | 104 | # Step 3: Build the evaluator | COMMENT |
| LOW⚡ | …ent/examples/experiments/experiment_trace_evaluator.py | 155 | # Step 4: Run the experiment and evaluate | COMMENT |
| LOW⚡ | …examples/datasets/create_dataset_from_spans_example.py | 38 | # Step 1: Set up Tracing with phoenix-otel | COMMENT |
| LOW⚡ | …examples/datasets/create_dataset_from_spans_example.py | 53 | # Step 2: Simulate LLM Calls using OpenInference Decorators | COMMENT |
| LOW⚡ | …examples/datasets/create_dataset_from_spans_example.py | 138 | # Step 3: Query Spans from Phoenix using get_spans_dataframe | COMMENT |
| LOW⚡ | …examples/datasets/create_dataset_from_spans_example.py | 218 | # Step 4: Create Dataset with Span Associations | COMMENT |
| LOW⚡ | …examples/datasets/create_dataset_from_spans_example.py | 301 | # Step 1: Log LLM spans using OpenInference decorators/context managers | COMMENT |
| 18 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | README.md | 130 | | | [Guardrails](https://arize.com/docs/phoenix/tracing/integrations-tracing/guardrails) | `openinference-instrumentatio | CODE |
| MEDIUM | sitemap.xml | 1060 | <loc>https://arize.com/docs/phoenix/integrations/python/guardrails-ai</loc> | CODE |
| MEDIUM | sitemap.xml | 1064 | <loc>https://arize.com/docs/phoenix/integrations/python/guardrails-ai/guardrails-ai-tracing</loc> | CODE |
| MEDIUM | sitemap.xml | 1896 | <loc>https://arize.com/docs/phoenix/cookbook/tracing/agentic-rag-tracing</loc> | CODE |
| MEDIUM | sitemap.xml | 1988 | <loc>https://arize.com/docs/phoenix/cookbook/guardrails/jailbreak-and-prompt-injection-defense</loc> | CODE |
| MEDIUM | sitemap.xml | 1992 | <loc>https://arize.com/docs/phoenix/cookbook/guardrails/designing-realtime-guardrails</loc> | CODE |
| MEDIUM | sitemap.xml | 2640 | <loc>https://arize.com/docs/phoenix/release-notes/2024/07-18-2024-guardrails-ai-integrations</loc> | CODE |
| MEDIUM | evals/pxi/evaluators/tools.py | 293 | # scaffold keys an earlier write left behind. | COMMENT |
| MEDIUM | internal_docs/specs/online-evals.md | 203 | [feedback use case](#evaluate-on-user-feedback)). The third is event-driven, not | CODE |
| MEDIUM | tests/unit/trace/dsl/test_filter.py | 350 | # shapes that bypass the structural guardrails ``SpanFilter`` enforces. | COMMENT |
| MEDIUM | tests/unit/pxi/evals/test_datasets.py | 321 | # The preservation case must assert the scaffold keys survive a | STRING |
| MEDIUM⚡ | tests/unit/pxi/evals/test_evaluator_matchers.py | 246 | # The preservation case: scaffold keys must survive a whole-metadata | COMMENT |
| MEDIUM | tests/unit/pxi/evals/test_evaluator_matchers.py | 280 | # Clobbering the scaffold (dropping ``hypothesis``) must fail the check. | COMMENT |
| MEDIUM | …ents/skills/phoenix-evals/references/evaluators-rag.md | 1 | # Evaluators: RAG Systems | COMMENT |
| MEDIUM | …ents/skills/phoenix-evals/references/evaluators-rag.md | 85 | ## RAG Failure Taxonomy | COMMENT |
| MEDIUM | …phoenix-evals/references/experiments-running-python.md | 35 | # With context (RAG) | COMMENT |
| MEDIUM | …lls/phoenix-evals/references/common-mistakes-python.md | 118 | # RIGHT for RAG evaluation — fetch child spans for retriever/LLM metrics | COMMENT |
| MEDIUM | …kills/phoenix-tracing/references/annotations-python.md | 85 | ## RAG Pipeline Example | COMMENT |
| MEDIUM | …s/phoenix-tracing/references/annotations-typescript.md | 126 | ## RAG Pipeline Example | COMMENT |
| MEDIUM | docs/PROMPT.md | 95 | | `guardrails` | Guardrails AI | https://arize.com/docs/phoenix/integrations/python/guardrails-ai | | CODE |
| MEDIUM | docs/sitemap.xml | 134 | <loc>https://arize.com/docs/phoenix/cookbook/guardrails/jailbreak-and-prompt-injection-defense</loc> | CODE |
| MEDIUM | docs/sitemap.xml | 199 | <loc>https://arize.com/docs/phoenix/cookbook/tracing/agentic-rag-tracing</loc> | CODE |
| MEDIUM | docs/sitemap.xml | 787 | <loc>https://arize.com/docs/phoenix/integrations/python/guardrails-ai</loc> | CODE |
| MEDIUM | docs/sitemap.xml | 792 | <loc>https://arize.com/docs/phoenix/integrations/python/guardrails-ai/guardrails-ai-tracing</loc> | CODE |
| MEDIUM | docs/sitemap.xml | 1617 | <loc>https://arize.com/docs/phoenix/release-notes/2024/07-18-2024-guardrails-ai-integrations</loc> | CODE |
| MEDIUM | docs/phoenix/llms.txt | 266 | - [Guardrails AI Tracing](https://arize.com/docs/phoenix/integrations/python/guardrails-ai/guardrails-ai-tracing): Auto- | CODE |
| MEDIUM | docs/phoenix/llms.txt | 289 | - [OpenAI Agents SDK (TypeScript)](https://arize.com/docs/phoenix/integrations/typescript/openai-agents): Trace @openai/ | CODE |
| MEDIUM⚡ | docs/phoenix/llms.txt | 349 | - [Agent Spec](https://arize.com/docs/phoenix/integrations/python/agentspec): Open Agent Spec (Agent Spec) is a portable | CODE |
| MEDIUM⚡ | docs/phoenix/llms.txt | 357 | - [Guardrails AI](https://arize.com/docs/phoenix/integrations/python/guardrails-ai): Guardrails is an open-source Python | CODE |
| MEDIUM⚡ | docs/phoenix/llms.txt | 365 | - [Portkey](https://arize.com/docs/phoenix/integrations/python/portkey): Portkey is an AI Gateway and observability plat | CODE |
| MEDIUM⚡ | docs/phoenix/llms.txt | 709 | - [Designing Realtime Guardrails: Input, Output, and the Cost of Blocking](https://arize.com/docs/phoenix/cookbook/guard | CODE |
| MEDIUM⚡ | docs/phoenix/llms.txt | 714 | - [Jailbreak and Prompt Injection Defense](https://arize.com/docs/phoenix/cookbook/guardrails/jailbreak-and-prompt-injec | CODE |
| MEDIUM⚡ | docs/phoenix/llms.txt | 724 | - [Agentic RAG Tracing](https://arize.com/docs/phoenix/cookbook/tracing/agentic-rag-tracing): Build and trace an agentic | CODE |
| MEDIUM | docs/phoenix/llms.txt | 741 | - [07.18.2024: Guardrails AI integrations](https://arize.com/docs/phoenix/release-notes/2024/07-18-2024-guardrails-ai-in | CODE |
| MEDIUM | docs/phoenix/sitemap.xml | 1060 | <loc>https://arize.com/docs/phoenix/integrations/python/guardrails-ai</loc> | CODE |
| MEDIUM | docs/phoenix/sitemap.xml | 1064 | <loc>https://arize.com/docs/phoenix/integrations/python/guardrails-ai/guardrails-ai-tracing</loc> | CODE |
| MEDIUM | docs/phoenix/sitemap.xml | 1896 | <loc>https://arize.com/docs/phoenix/cookbook/tracing/agentic-rag-tracing</loc> | CODE |
| MEDIUM | docs/phoenix/sitemap.xml | 1988 | <loc>https://arize.com/docs/phoenix/cookbook/guardrails/jailbreak-and-prompt-injection-defense</loc> | CODE |
| MEDIUM | docs/phoenix/sitemap.xml | 1992 | <loc>https://arize.com/docs/phoenix/cookbook/guardrails/designing-realtime-guardrails</loc> | CODE |
| MEDIUM | docs/phoenix/sitemap.xml | 2640 | <loc>https://arize.com/docs/phoenix/release-notes/2024/07-18-2024-guardrails-ai-integrations</loc> | CODE |
| MEDIUM | examples/rag_agent/README.md | 1 | # Agentic RAG Examples | COMMENT |
| MEDIUM | examples/rag_agent/app.py | 110 | gr.Markdown("## Chat with RAG Agent 🔥") | STRING |
| MEDIUM | examples/rag_agent/app.py | 143 | gr.Markdown("### Chat with RAG Agent 💬") | STRING |
| MEDIUM | scripts/docker/devops/LDAP-TLS-TESTING.md | 47 | │ └── test_ldap_tls.py # Test orchestration (all tests here) | CODE |
| MEDIUM⚡ | packages/phoenix-evals/src/phoenix/evals/utils.py | 441 | # Log only hallucination annotations | STRING |
| MEDIUM | …evals/src/phoenix/evals/llm/adapters/openai/adapter.py | 489 | # documented ``name`` field used to label few-shot exemplars or | COMMENT |
| MEDIUM | packages/phoenix-client/docs/source/index.md | 252 | #### RAG Retrieval Evaluation | STRING |
| MEDIUM | packages/phoenix-client/docs/source/index.md | 289 | #### RAG Q&A Evaluation | STRING |
| MEDIUM | packages/phoenix-client/docs/source/index.md | 321 | #### Time-Filtered RAG Spans | STRING |
| MEDIUM | tutorials/ai_evals_course/hw3_phoenix/README.md | 227 | │ ├── develop_judge.py # Develop LLM judge with few-shot examples | CODE |
| MEDIUM | tutorials/ai_evals_course/hw3_phoenix/README.md | 234 | │ ├── train_set.csv # Training examples for few-shot (~23) | CODE |
| MEDIUM | …s/ai_evals_course/hw3_phoenix/scripts/develop_judge.py | 106 | # Add few-shot examples | STRING |
| MEDIUM | src/phoenix/server/agents/prompts/templating.py | 14 | autoescape=False, # escaping `<` to `<` would corrupt the XML scaffolding | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …s/apps/cli-agent-starter-kit/scripts/ensure-phoenix.sh | 30 | # Check if docker is running | COMMENT |
| LOW | …s/apps/cli-agent-starter-kit/scripts/ensure-phoenix.sh | 44 | # Check if Phoenix container is running | COMMENT |
| LOW | …s/apps/cli-agent-starter-kit/scripts/ensure-phoenix.sh | 69 | # Check if Phoenix container exists but is stopped | COMMENT |
| LOW | tests/unit/server/api/conftest.py | 1183 | # Assign examples to splits | COMMENT |
| LOW | tests/unit/db/test_helpers.py | 1122 | # Assign examples to splits | COMMENT |
| LOW | tests/unit/db/test_helpers.py | 1357 | # Assign splits to experiments | COMMENT |
| LOW | tests/integration/_mock_llm_server.py | 2121 | # Check if SSE format is requested (alt=sse query param) | COMMENT |
| LOW | tests/integration/_helpers.py | 2465 | # Check if we have a complete part | STRING |
| LOW | tests/integration/client/test_prompts.py | 1531 | # Assign labels to prompts using GraphQL mutation | STRING |
| LOW | tests/integration/client/test_prompts.py | 1542 | # Assign label1 to prompt1 | STRING |
| LOW | tests/integration/client/test_prompts.py | 1558 | # Assign label2 to prompt3 | STRING |
| LOW | tests/integration/client/test_prompts.py | 1688 | # Assign labels to prompts | STRING |
| LOW | tests/integration/client/test_annotations.py | 813 | # Check if we have all expected annotations | STRING |
| LOW | tests/integration/client/test_annotations.py | 2064 | # Check if we have all expected annotations | STRING |
| LOW | tests/integration/client/test_annotations.py | 2793 | # Check if we have all expected annotations | STRING |
| LOW | tests/integration/client/test_spans.py | 1406 | # Set index to span_id so dataframe_to_spans uses the right span IDs | COMMENT |
| LOW | tests/integration/db_migrations/__init__.py | 122 | # Check if table exists | COMMENT |
| LOW | tests/integration/db_migrations/__init__.py | 194 | # Check if table exists | COMMENT |
| LOW | api_reference/source/conf.py | 40 | # Check if we are still in the automodule block | COMMENT |
| LOW | api_reference/source/conf.py | 45 | # Check if we are out of the automodule block | COMMENT |
| LOW⚡ | examples/computer_use_agent/tools/edit.py | 83 | # Check if its an absolute path | COMMENT |
| LOW⚡ | examples/computer_use_agent/tools/edit.py | 89 | # Check if path exists | COMMENT |
| LOW⚡ | examples/computer_use_agent/tools/edit.py | 96 | # Check if the path points to a directory | COMMENT |
| LOW | examples/computer_use_agent/tools/edit.py | 146 | # Read the file content | COMMENT |
| LOW | examples/computer_use_agent/tools/edit.py | 151 | # Check if old_str is unique in the file | COMMENT |
| LOW | examples/computer_use_agent/image/xvfb_startup.sh | 30 | # Check if Xvfb is already running | COMMENT |
| LOW | scripts/docker/devops/dev.sh | 69 | # Add profile to the list | COMMENT |
| LOW | scripts/docker/devops/dev.sh | 562 | # Check if Phoenix container is running | COMMENT |
| LOW | scripts/docker/devops/scripts/test_ldap_tls.py | 604 | # Check if any Phase 2 application tests failed - we can't claim security | COMMENT |
| LOW⚡ | scripts/generate_data_via_plpgsql/generate_spans.py | 127 | # Check if the command was successful | COMMENT |
| LOW⚡ | scripts/generate_data_via_plpgsql/generate_spans.py | 133 | # Print the output if requested | COMMENT |
| LOW⚡ | scripts/generate_data_via_plpgsql/generate_spans.py | 213 | print_output=True, # Print output for report_spans_table_sizes.sql | CODE |
| LOW | …/generate_data_via_plpgsql/report_spans_table_sizes.py | 103 | # Check if the command was successful | COMMENT |
| LOW⚡ | …generate_data_via_plpgsql/generate_span_annotations.py | 239 | # Check if the command was successful | COMMENT |
| LOW⚡ | …generate_data_via_plpgsql/generate_span_annotations.py | 245 | # Print the output if requested | COMMENT |
| LOW | scripts/ci/test_helm.py | 2099 | # Check if the error contains the expected substring | COMMENT |
| LOW | scripts/ddl/generate_ddl_postgresql.py | 1012 | # Check if identifier needs quotes to preserve case/special chars | COMMENT |
| LOW | scripts/ddl/generate_ddl_postgresql.py | 1157 | # Check if line ends with semicolon (end of statement) | COMMENT |
| LOW | scripts/uv/type_check/type_check.py | 57 | # Check if we have fewer errors than expected (good news!) | COMMENT |
| LOW | …ripts/generate_spans/generate_spans_for_time_series.py | 249 | # Set pandas to display all rows without truncation | STRING |
| LOW⚡ | packages/phoenix-evals/src/phoenix/evals/tracing.py | 144 | # Check if function has a trace_id parameter | COMMENT |
| LOW⚡ | packages/phoenix-evals/src/phoenix/evals/tracing.py | 241 | # Check if function has a trace_id parameter | COMMENT |
| LOW | …vals/src/phoenix/evals/llm/adapters/litellm/adapter.py | 580 | # Check if schema has properties and required fields | COMMENT |
| LOW | …kages/phoenix-client/src/phoenix/client/types/spans.py | 36 | # Check if the key is a legacy name that should be replaced. | COMMENT |
| LOW | …t/src/phoenix/client/resources/experiments/__init__.py | 1241 | # Check if response is HTML (endpoint doesn't exist on old server) | COMMENT |
| LOW | …t/src/phoenix/client/resources/experiments/__init__.py | 1431 | # Check if response is HTML (endpoint doesn't exist on old server) | COMMENT |
| LOW | …t/src/phoenix/client/resources/experiments/__init__.py | 1543 | # Check if response is HTML (endpoint doesn't exist on old server) | COMMENT |
| LOW | …t/src/phoenix/client/resources/experiments/__init__.py | 3091 | # Check if response is HTML (endpoint doesn't exist on old server) | COMMENT |
| LOW | …t/src/phoenix/client/resources/experiments/__init__.py | 3285 | # Check if response is HTML (endpoint doesn't exist on old server) | COMMENT |
| LOW | …t/src/phoenix/client/resources/experiments/__init__.py | 3399 | # Check if response is HTML (endpoint doesn't exist on old server) | COMMENT |
| LOW | …ient/src/phoenix/client/resources/datasets/__init__.py | 2271 | # Check if there's at least one data row | COMMENT |
| LOW | …lient/src/phoenix/client/resources/prompts/__init__.py | 297 | # Check if a specific tag exists | STRING |
| LOW | …lient/src/phoenix/client/resources/prompts/__init__.py | 586 | # Check if a specific tag exists | STRING |
| LOW | …nix-client/src/phoenix/client/helpers/atif/_convert.py | 609 | # Output message | COMMENT |
| LOW | …ix-client/src/phoenix/client/helpers/spans/__init__.py | 356 | # Check if events is not null/nan using pandas-safe method | COMMENT |
| LOW | src/phoenix/config.py | 1003 | # Check if TLS is enabled | COMMENT |
| LOW | src/phoenix/trace/trace_dataset.py | 94 | # Check if the dataframe has any document columns | COMMENT |
| LOW | src/phoenix/trace/trace_dataset.py | 330 | # Append the evaluations to the list of evaluations | COMMENT |
| LOW | src/phoenix/trace/span_json_decoder.py | 41 | # Check if the dict can be interpreted as a Span | COMMENT |
| LOW | src/phoenix/server/ldap.py | 1039 | # Check if results were truncated by server's size limit | COMMENT |
| 16 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | evals/pxi/pytest.ini | 6 | # doctest-collect the eval harness and this suite's conftest, which it must not. | COMMENT |
| MEDIUM | evals/pxi/thresholds.yaml | 1 | # Per-(evaluator, split) pass-rate gate policy for the PXI eval harness. | COMMENT |
| MEDIUM | evals/pxi/datasets/set_spans_filter.yaml | 707 | # We do NOT require any specific tool be called (the eval harness may | COMMENT |
| MEDIUM | evals/pxi/datasets/product_knowledge.yaml | 642 | # deferred tool ends the single-shot harness run before any text is | COMMENT |
| MEDIUM | evals/pxi/datasets/experiments_skill_trigger.yaml | 261 | # Given a primed experiment id and a compare question under the harness's | COMMENT |
| MEDIUM | evals/pxi/datasets/experiments_skill_trigger.yaml | 264 | # to the known harness toolset on purpose; the skill itself stays | COMMENT |
| MEDIUM | app/tests/code-evaluators.spec.ts | 310 | // is robust to that prefix. | COMMENT |
| MEDIUM⚡ | tests/unit/test_auth.py | 99 | # Test comprehensive sanitization | COMMENT |
| MEDIUM | tests/unit/server/api/routers/test_oauth2.py | 36 | """Single comprehensive test covering all sign-in scenarios.""" | STRING |
| MEDIUM | tests/unit/server/api/routers/v1/test_experiments.py | 1092 | # Create specific scenarios for comprehensive testing | COMMENT |
| MEDIUM | tests/unit/server/api/types/test_Project.py | 276 | # Full time range for comprehensive testing | COMMENT |
| MEDIUM | tests/unit/server/api/types/test_Project.py | 116 | """Create comprehensive test data for cost and token testing. | STRING |
| MEDIUM | tests/unit/db/test_models.py | 76 | # Create comprehensive test data that covers various Unicode and special character scenarios | COMMENT |
| MEDIUM | tests/unit/db/test_models.py | 252 | # Single comprehensive payload testing ALL serialization scenarios: | COMMENT |
| MEDIUM | tests/unit/db/test_models.py | 277 | # NaN/Inf values (consolidated comprehensive coverage from test_json_sanitization) | COMMENT |
| MEDIUM | tests/unit/db/test_models.py | 396 | # Create all metadata entities with the comprehensive payload | COMMENT |
| MEDIUM | tests/unit/db/test_models.py | 225 | """Test comprehensive orjson serialization of all special object types across JSON columns. | STRING |
| MEDIUM | tests/unit/db/test_helpers.py | 391 | # Create dataset versions for comprehensive evolution testing | COMMENT |
| MEDIUM | tests/unit/db/test_helpers.py | 433 | # Create comprehensive split structure | COMMENT |
| MEDIUM | tests/unit/db/test_helpers.py | 466 | # Create overlapping split assignments for comprehensive testing | COMMENT |
| MEDIUM | tests/unit/db/test_helpers.py | 507 | # Create comprehensive revision patterns with direct object references | COMMENT |
| MEDIUM | tests/unit/db/test_helpers.py | 642 | # Return comprehensive mapping for all test scenarios | COMMENT |
| MEDIUM | tests/unit/db/test_helpers.py | 1000 | # Collect IDs for comprehensive duplicate analysis | COMMENT |
| MEDIUM | tests/unit/db/test_helpers.py | 1020 | # Expected examples based on our comprehensive fixture: | COMMENT |
| MEDIUM⚡ | tests/integration/server/test_db_disk_usage_monitor.py | 348 | # This ensures the blocking behavior is comprehensive and not bypass-able. | COMMENT |
| MEDIUM⚡ | tests/integration/client/test_annotations.py | 2298 | # Ensure we have different sessions for a more robust test | STRING |
| MEDIUM | tests/integration/client/test_experiments.py | 2079 | # Create first dataset for comprehensive mixed scenarios (examples 0-4) | STRING |
| MEDIUM | tests/integration/client/test_spans.py | 1287 | # Create comprehensive test DataFrame with nested attributes | COMMENT |
| MEDIUM | tests/integration/client/test_spans.py | 1297 | # Create comprehensive test data with nested attributes and complex types | COMMENT |
| MEDIUM | …/skills/pxi-eval-dataset/scripts/annotate_via_codex.sh | 77 | # --skip-git-repo-check: harness may be invoked from non-git working dirs. | COMMENT |
| MEDIUM | scripts/docker/devops/scripts/test_ldap_tls.py | 507 | """Run comprehensive security test suite.""" | STRING |
| MEDIUM | scripts/docker/devops/scripts/ldap_mitm_proxy.py | 580 | """Print comprehensive security summary.""" | STRING |
| LOW⚡ | packages/phoenix-evals/src/phoenix/evals/utils.py | 464 | # Index name matches existing column - just add the index values | COMMENT |
| MEDIUM⚡ | …_evals_course/hw3_phoenix/results/dev_predictions.json | 1 | {"llm_as_judge_label":{"0":"FAIL","1":"FAIL","2":"FAIL","3":"FAIL","4":"PASS","5":"PASS","6":"PASS","7":"PASS","8":"PASS | CODE |
| MEDIUM⚡ | …_evals_course/hw3_phoenix/results/dev_predictions.json | 1 | {"llm_as_judge_label":{"0":"FAIL","1":"FAIL","2":"FAIL","3":"FAIL","4":"PASS","5":"PASS","6":"PASS","7":"PASS","8":"PASS | CODE |
| MEDIUM⚡ | …evals_course/hw3_phoenix/results/test_predictions.json | 1 | {"ground_truth_label":{"0":"FAIL","1":"PASS","2":"PASS","3":"PASS","4":"PASS","5":"PASS","6":"PASS","7":"PASS","8":"FAIL | CODE |
| MEDIUM | …ls/mcp/tracing_between_mcp_client_and_server/server.py | 216 | """Generates a comprehensive investment report based on client profile and market analysis.""" | STRING |
| LOW | src/phoenix/server/api/routers/ldap.py | 161 | # Null email mode: email column is nullable, so just use None | COMMENT |
| LOW | …x/server/api/dataloaders/document_retrieval_metrics.py | 45 | # it simple and just use one approach for all backends. | COMMENT |
| MEDIUM | src/phoenix/server/api/helpers/message_helpers.py | 24 | # This makes it easier to leverage openinference.instrumentation helpers | COMMENT |
| MEDIUM | src/phoenix/db/models.py | 659 | # Enforce best practices for naming constraints | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | render.yaml | 1 | # Render Blueprint for Arize Phoenix | COMMENT |
| LOW | .readthedocs.yaml | 21 | # builder: "dirhtml" | COMMENT |
| LOW | evals/pxi/thresholds.yaml | 1 | # Per-(evaluator, split) pass-rate gate policy for the PXI eval harness. | COMMENT |
| LOW | evals/pxi/thresholds.yaml | 21 | # regression, not flake. gate.py compares with STRICT less-than, so for a cell | COMMENT |
| LOW | evals/pxi/thresholds.yaml | 101 | # This dataset (n=6) proved nondeterministic: correct_tools_called and | COMMENT |
| LOW | evals/pxi/evaluators/tools.py | 261 | COMMENT | |
| LOW | evals/pxi/evaluators/tools.py | 281 | # - ``non_empty: true`` -- the key must be present and contain non-whitespace text. | COMMENT |
| LOW | app/src/components/core/tabs/Tabs.tsx | 141 | // dissolves together. | COMMENT |
| LOW | app/src/components/agent/agentFabPositioning.ts | 1 | import type { AgentFabPlacement } from "@phoenix/store/agentStore"; | COMMENT |
| LOW | app/src/components/agent/SkillHighlightOverlay.tsx | 41 | mark { | COMMENT |
| LOW | js/packages/phoenix-config/README.md | 81 | re-discover it. | COMMENT |
| LOW | js/packages/phoenix-client/src/testing/runner.ts | 141 | ); | COMMENT |
| LOW | …ternal_docs/specs/ldap-authentication/configuration.md | 121 | COMMENT | |
| LOW | …ternal_docs/specs/ldap-authentication/configuration.md | 141 | # - starttls: Start with plaintext, upgrade to TLS (port 389) | COMMENT |
| LOW | …ternal_docs/specs/ldap-authentication/configuration.md | 161 | COMMENT | |
| LOW | …ternal_docs/specs/ldap-authentication/configuration.md | 181 | COMMENT | |
| LOW | …ternal_docs/specs/ldap-authentication/configuration.md | 201 | # - OpenLDAP: "entryUUID" | COMMENT |
| LOW | …ternal_docs/specs/ldap-authentication/configuration.md | 221 | # When set: Reads the specified attribute from the user's LDAP entry | COMMENT |
| LOW | …ternal_docs/specs/ldap-authentication/configuration.md | 241 | COMMENT | |
| LOW | …ternal_docs/specs/ldap-authentication/configuration.md | 261 | # - "MEMBER" → MEMBER role (Phoenix) | COMMENT |
| LOW | …ternal_docs/specs/ldap-authentication/configuration.md | 281 | # ============================================================================== | COMMENT |
| LOW | tests/unit/test_tracers.py | 701 | assert fetched_span.cumulative_llm_token_count_prompt == 0 | COMMENT |
| LOW | tests/unit/test_tracers.py | 721 | # | COMMENT |
| LOW | tests/integration/server/test_db_disk_usage_monitor.py | 161 | # an alert was properly sent to the configured admin email address. | COMMENT |
| LOW | tests/integration/server/test_db_disk_usage_monitor.py | 241 | # ======================================================================== | COMMENT |
| LOW | tests/integration/server/test_db_disk_usage_monitor.py | 341 | _gql(_app, access_token, query=query) | COMMENT |
| LOW | …/skills/pxi-eval-dataset/scripts/annotate_via_codex.sh | 1 | #!/usr/bin/env bash | COMMENT |
| LOW | scripts/capabilities_poc.py | 41 | print(part.content) | COMMENT |
| LOW | scripts/docker/devops/overrides/ldap-test.yml | 1 | # Docker Compose override for LDAP testing | COMMENT |
| LOW | scripts/docker/devops/overrides/ldap-test.yml | 21 | # 6. POSIX mode (GROUP_SEARCH_FILTER) - phoenix-posix → ldap-posix | COMMENT |
| LOW | scripts/docker/devops/overrides/ldap-test.yml | 401 | COMMENT | |
| LOW | …son-canonicalization-schema/verify-canonicalization.py | 1 | ############################################################################## | COMMENT |
| LOW | …ipts/ci/json-canonicalization-schema/verify-numbers.py | 1 | ############################################################################## | COMMENT |
| LOW | …pts/ci/json-canonicalization-schema/testdata/numgen.js | 1 | // | COMMENT |
| LOW | …pts/ci/json-canonicalization-schema/testdata/numgen.go | 1 | // | COMMENT |
| LOW | scripts/perf/postgres/postgres_explain_analyze.py | 1 | # /// script | COMMENT |
| LOW | packages/phoenix-evals/src/phoenix/evals/llm/prompts.py | 581 | # """Initialize an image URL content part template. | COMMENT |
| LOW | …ls/src/phoenix/evals/llm/adapters/anthropic/adapter.py | 321 | if not prompt: | COMMENT |
| LOW | packages/phoenix-client/examples/pytest/eval-ci.yml | 41 | # OPTIONAL example: post the printed summary back to the PR as a single, | COMMENT |
| LOW | packages/phoenix-client/examples/pytest/eval-ci.yml | 61 | # PHOENIX_API_KEY: ${{ secrets.PHOENIX_API_KEY }} | COMMENT |
| LOW | packages/phoenix-client/examples/pytest/eval-ci.yml | 81 | # } else { | COMMENT |
| LOW | …ent/examples/experiments/experiment_trace_evaluator.py | 101 | COMMENT | |
| LOW | packages/phoenix-client/scripts/codegen/transform.py | 1 | import ast | COMMENT |
| LOW | packages/phoenix-client/scripts/codegen/transform.py | 21 | # Supported transformations: | COMMENT |
| LOW | …nix-client/src/phoenix/client/helpers/atif/_convert.py | 1 | # pyright: reportUnknownMemberType=false, reportUnknownVariableType=false, reportUnknownArgumentType=false | COMMENT |
| LOW | .github/workflows/claude-code-review.yml | 21 | # Same-repo PRs only: fork-origin pull_request runs get neither repository | COMMENT |
| LOW | .github/workflows/claude.yml | 101 | COMMENT | |
| LOW | helm/values.yaml | 21 | nodeSelector: {} | COMMENT |
| LOW | helm/values.yaml | 61 | enabled: false | COMMENT |
| LOW | helm/values.yaml | 161 | workingDir: "/data" | COMMENT |
| LOW | helm/values.yaml | 181 | COMMENT | |
| LOW | helm/values.yaml | 201 | # version: stable | COMMENT |
| LOW | helm/values.yaml | 241 | # 3. Leave database.url empty (Phoenix will automatically use SQLite in the persistent volume) | COMMENT |
| LOW | helm/values.yaml | 281 | port: 5432 | COMMENT |
| LOW | helm/values.yaml | 301 | useAzureManagedIdentity: false | COMMENT |
| LOW | helm/values.yaml | 381 | secret: | COMMENT |
| LOW | helm/values.yaml | 421 | # Server connection settings | COMMENT |
| LOW | helm/values.yaml | 461 | # -- List of base DNs for user searches (required when enabled). Searched in order. | COMMENT |
| LOW | helm/values.yaml | 481 | COMMENT | |
| LOW | helm/values.yaml | 501 | COMMENT | |
| 28 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| CRITICAL | …als/test/template/createTemplateVariablesProxy.test.ts | 192 | expect(proxied.level1.level2.level3.level4.toString()).toBe( | CODE |
| CRITICAL | …_docs/specs/ldap-authentication/future-improvements.md | 142 | from ldap3.core.exceptions import LDAPSocketOpenError, LDAPSocketReceiveError | CODE |
| CRITICAL | …ment-runner-background-process/appendix-error-types.md | 219 | from azure.core.exceptions import HttpResponseError | CODE |
| CRITICAL | tests/unit/vcr.py | 31 | module_name = self._request.node.module.__name__.split(".")[-1] | CODE |
| CRITICAL | tests/unit/server/sandbox/test_modal_backend.py | 406 | modal_mock.Image.debian_slim.return_value.pip_install.assert_called_once_with(packages) | CODE |
| CRITICAL | tests/integration/_mock_ldap_server.py | 22 | from ldap3.core.exceptions import LDAPInvalidDnError | CODE |
| CRITICAL | src/phoenix/config.py | 28 | from ldap3.core.exceptions import LDAPInvalidDnError | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | …c/components/evaluators/ContainsEvaluatorCodeBlock.tsx | 15 | return False | CODE |
| HIGH | app/src/pages/prompt/promptCodeSnippets.tsx | 102 | print(completion.choices[0].message) | CODE |
| HIGH | app/src/pages/prompt/promptCodeSnippets.tsx | 132 | print(response.choices[0].message.content) | CODE |
| HIGH | app/src/pages/prompt/promptCodeSnippets.tsx | 181 | print(completion.content) | CODE |
| HIGH | …/prompt/__tests__/promptCodeSnippets.anthropic.test.ts | 274 | print(completion.content)" | CODE |
| HIGH | …/prompt/__tests__/promptCodeSnippets.anthropic.test.ts | 392 | print(completion.content)" | CODE |
| HIGH | …/prompt/__tests__/promptCodeSnippets.anthropic.test.ts | 436 | print(completion.content)" | CODE |
| HIGH | …ges/prompt/__tests__/promptCodeSnippets.openai.test.ts | 295 | print(completion.choices[0].message)" | CODE |
| HIGH | …ges/prompt/__tests__/promptCodeSnippets.openai.test.ts | 402 | print(completion.choices[0].message)" | CODE |
| HIGH | …ges/prompt/__tests__/promptCodeSnippets.openai.test.ts | 439 | print(completion.choices[0].message)" | CODE |
| HIGH | …ges/prompt/__tests__/promptCodeSnippets.openai.test.ts | 487 | print(completion.choices[0].message)" | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/unit/_helpers.py | 96 | CODE | |
| LOW | tests/unit/server/api/test_code_evaluator_runner.py | 96 | CODE | |
| LOW | examples/agent_framework_comparison/li_workflow/main.py | 16 | CODE | |
| LOW | …generate_data_via_plpgsql/generate_span_annotations.py | 162 | CODE | |
| LOW | scripts/ci/test_helm.py | 747 | CODE | |
| LOW | scripts/ci/test_helm.py | 1099 | CODE | |
| LOW | …kages/phoenix-evals/src/phoenix/evals/rate_limiters.py | 180 | CODE | |
| LOW | packages/phoenix-evals/src/phoenix/evals/executors.py | 198 | CODE | |
| LOW | …enix-client/src/phoenix/client/experiments/__init__.py | 17 | CODE | |
| LOW | …enix-client/src/phoenix/client/experiments/__init__.py | 207 | CODE | |
| LOW | …client/src/phoenix/client/resources/traces/__init__.py | 339 | CODE | |
| LOW | …client/src/phoenix/client/resources/traces/__init__.py | 735 | CODE | |
| LOW | …-client/src/phoenix/client/resources/spans/__init__.py | 171 | CODE | |
| LOW | …-client/src/phoenix/client/resources/spans/__init__.py | 250 | CODE | |
| LOW | …-client/src/phoenix/client/resources/spans/__init__.py | 476 | CODE | |
| LOW | …-client/src/phoenix/client/resources/spans/__init__.py | 626 | CODE | |
| LOW | …-client/src/phoenix/client/resources/spans/__init__.py | 1447 | CODE | |
| LOW | …-client/src/phoenix/client/resources/spans/__init__.py | 1754 | CODE | |
| LOW | …-client/src/phoenix/client/resources/spans/__init__.py | 1903 | CODE | |
| LOW | …t/src/phoenix/client/resources/experiments/__init__.py | 633 | CODE | |
| LOW | …t/src/phoenix/client/resources/experiments/__init__.py | 2134 | CODE | |
| LOW | …t/src/phoenix/client/resources/experiments/__init__.py | 2476 | CODE | |
| LOW | …t/src/phoenix/client/resources/experiments/__init__.py | 3968 | CODE | |
| LOW | …ient/src/phoenix/client/resources/datasets/__init__.py | 785 | CODE | |
| LOW | …ient/src/phoenix/client/resources/datasets/__init__.py | 953 | CODE | |
| LOW | …ient/src/phoenix/client/resources/datasets/__init__.py | 1151 | CODE | |
| LOW | …ient/src/phoenix/client/resources/datasets/__init__.py | 1254 | CODE | |
| LOW | …ient/src/phoenix/client/resources/datasets/__init__.py | 1711 | CODE | |
| LOW | …ient/src/phoenix/client/resources/datasets/__init__.py | 1852 | CODE | |
| LOW | …ient/src/phoenix/client/resources/datasets/__init__.py | 2032 | CODE | |
| LOW | …ient/src/phoenix/client/resources/datasets/__init__.py | 2133 | CODE | |
| LOW | …oenix-client/src/phoenix/client/utils/rate_limiters.py | 165 | CODE | |
| LOW | …s/phoenix-client/src/phoenix/client/utils/executors.py | 209 | CODE | |
| LOW | packages/phoenix-otel/src/phoenix/otel/otel.py | 65 | CODE | |
| LOW | src/phoenix/server/oauth2.py | 64 | CODE | |
| LOW | src/phoenix/server/rate_limiters.py | 193 | CODE | |
| LOW | src/phoenix/server/app.py | 570 | CODE | |
| LOW | src/phoenix/server/app.py | 832 | CODE | |
| LOW | src/phoenix/server/agents/server_agents.py | 55 | CODE | |
| LOW | src/phoenix/server/agents/agent_factory.py | 65 | CODE | |
| LOW | src/phoenix/server/api/context.py | 117 | CODE | |
| LOW | src/phoenix/server/api/routers/auth.py | 212 | CODE | |
| LOW | src/phoenix/server/api/routers/auth.py | 394 | CODE | |
| LOW | src/phoenix/server/api/routers/oauth2.py | 290 | CODE | |
| LOW | src/phoenix/server/api/routers/oauth2.py | 875 | CODE | |
| LOW | src/phoenix/server/api/routers/v1/spans.py | 727 | CODE | |
| LOW | src/phoenix/server/api/routers/v1/spans.py | 904 | CODE | |
| LOW | src/phoenix/server/api/routers/v1/traces.py | 122 | CODE | |
| LOW | src/phoenix/server/api/types/Dataset.py | 221 | CODE | |
| LOW | src/phoenix/server/api/types/Project.py | 472 | CODE | |
| LOW | src/phoenix/db/models.py | 1111 | CODE | |
| LOW | src/phoenix/db/models.py | 1122 | CODE | |
| LOW | src/phoenix/db/models.py | 1132 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | js/examples/apps/tracing-tutorial/support-agent.ts | 305 | system: `You are a helpful customer support agent. When customers ask about order status, use the lookupOr | CODE |
| MEDIUM | js/benchmarks/evals-benchmarks/src/refusal.eval.ts | 32 | "I'm sorry, but I'm unable to help with that request. Please contact a plumber for assistance.", | CODE |
| MEDIUM | js/benchmarks/evals-benchmarks/src/refusal.eval.ts | 106 | "I don't have access to internal company communications or meeting transcripts, so I'm unable to answer that.", | CODE |
| MEDIUM | js/benchmarks/evals-benchmarks/src/conciseness.eval.ts | 106 | "It's worth noting that the speed of light in a vacuum is approximately 299,792 kilometers per second. It's impo | CODE |
| MEDIUM | js/benchmarks/evals-benchmarks/src/conciseness.eval.ts | 141 | "As an AI language model, I can tell you that pi (π) is approximately 3.14159. This is a mathematical constant r | CODE |
| MEDIUM⚡ | …al_docs/vignettes/sqlalchemy/polymorphic_user/users.py | 287 | # This is for demonstration purposes only | COMMENT |
| LOW | examples/llamaindex-workflows-research-agent/report.py | 29 | - Don't forget to add a reference list at the end of the report in {report_format} format and full | STRING |
| MEDIUM | …ix-client/src/phoenix/client/helpers/spans/__init__.py | 287 | # Now you can use these spans with other APIs | STRING |
| MEDIUM | …nfigs/CONCISENESS_CLASSIFICATION_EVALUATOR_CONFIG.yaml | 25 | - Hedging language or excessive qualifiers (e.g., "It's worth noting that...", "It's important to understand that. | CODE |
| MEDIUM | …r_configs/REFUSAL_CLASSIFICATION_EVALUATOR_CONFIG.yaml | 14 | - Explicit refusals (e.g., "I can't help with that", "I'm unable to answer") | CODE |
| MEDIUM⚡ | …_evals_course/hw3_phoenix/results/dev_predictions.json | 1 | {"llm_as_judge_label":{"0":"FAIL","1":"FAIL","2":"FAIL","3":"FAIL","4":"PASS","5":"PASS","6":"PASS","7":"PASS","8":"PASS | CODE |
| MEDIUM⚡ | …_evals_course/hw3_phoenix/results/dev_predictions.json | 1 | {"llm_as_judge_label":{"0":"FAIL","1":"FAIL","2":"FAIL","3":"FAIL","4":"PASS","5":"PASS","6":"PASS","7":"PASS","8":"PASS | CODE |
| MEDIUM⚡ | …_evals_course/hw3_phoenix/results/dev_predictions.json | 1 | {"llm_as_judge_label":{"0":"FAIL","1":"FAIL","2":"FAIL","3":"FAIL","4":"PASS","5":"PASS","6":"PASS","7":"PASS","8":"PASS | CODE |
| MEDIUM⚡ | …evals_course/hw3_phoenix/results/test_predictions.json | 1 | {"ground_truth_label":{"0":"FAIL","1":"PASS","2":"PASS","3":"PASS","4":"PASS","5":"PASS","6":"PASS","7":"PASS","8":"FAIL | CODE |
| MEDIUM⚡ | …evals_course/hw3_phoenix/results/test_predictions.json | 1 | {"ground_truth_label":{"0":"FAIL","1":"PASS","2":"PASS","3":"PASS","4":"PASS","5":"PASS","6":"PASS","7":"PASS","8":"FAIL | CODE |
| LOW⚡ | …evals_course/hw3_phoenix/results/test_predictions.json | 1 | {"ground_truth_label":{"0":"FAIL","1":"PASS","2":"PASS","3":"PASS","4":"PASS","5":"PASS","6":"PASS","7":"PASS","8":"FAIL | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | evals/pxi/harness/agent_task.py | 54 | CODE | |
| MEDIUM | …agents/pydantic_ai/test_openinference_model_wrapper.py | 101 | CODE | |
| MEDIUM | …agents/pydantic_ai/test_openinference_agent_wrapper.py | 110 | CODE | |
| MEDIUM | …agents/pydantic_ai/test_openinference_agent_wrapper.py | 154 | CODE | |
| MEDIUM | tests/unit/server/api/test_subscriptions.py | 184 | CODE | |
| MEDIUM | src/phoenix/server/daemons/experiment_runner.py | 240 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | js/benchmarks/evals-benchmarks/src/correctness.eval.ts | 208 | "Airplanes fly due to the interaction of four forces: lift (generated by air flowing over curved wings, creating | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …/components/project/integrationSnippets/phoenixOtel.ts | 15 | def my_function(input: str) -> str: | CODE |
| LOW | tests/integration/_helpers.py | 659 | def handle_request(self, request: httpx.Request) -> httpx.Response: | CODE |
| LOW | examples/agents/traject_bench_langgraph/agent.py | 150 | def run_task(task: TrajectTask, model: str = "gpt-4o") -> TaskResult: | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …/skills/pxi-eval-dataset/scripts/annotate_via_codex.sh | 15 | # Usage: | COMMENT |