:cloud: :rocket: :bar_chart: :chart_with_upwards_trend: Evaluating state of the art in AI
This report summarizes scanner evidence for Cloud-CV/EvalAI, a Python project with 2,045 GitHub stars. SynthScan examined 138,685 lines of code across 690 source files, recording 2541 pattern matches distributed across 21 syntactic categories. The legacy adjusted pattern signal is 18.9 (Elevated pattern signal); it is not an authorship estimate.
Use the findings to ask whether maintainers can explain intent, verify dependencies, simplify unnecessary scaffolding, and focus review effort. Risk Payload v1 adds ownership, supply-chain, over-engineering, and review-effort vectors when available. These are prompts for human verification, not probabilities, security findings, or measured maintenance costs.
Use each vector to guide manual verification of the underlying findings.
Can maintainers explain intent, assumptions, and trade-offs? 2240 finding(s).
Are imports declared and package/API names verified? 118 finding(s).
Are wrappers, guards, and repeated structures necessary? 2198 finding(s).
Which changes need focused human review? 2541 finding(s).
This chart tracks the legacy adjusted pattern score across scan runs. Changes can reflect source changes, scan coverage, or detector/scoring versions; they do not measure AI adoption or prove that a team refactored code. Compare runs only when their scoring versions are compatible. The dashed line tracks the raw pattern-hit count.
Groups findings by configured severity weight. Severity is not a confidence estimate, security rating, or proof that a package is missing. Verify imports against the project's manifests and inspect each finding in its local design context.
This horizontal bar chart shows where configured pattern scores accumulate across directories. Treat the bars as locations to inspect, not as evidence of authorship or code quality. Use the file-level findings to decide whether a directory needs a focused ownership, dependency, complexity, or review check.
The scanner identified 2541 distinct pattern matches across 21 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a detector match to inspect in context. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the configured finding weight (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — legacy comment-context matches receive a ×1.5 score weight. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier under the legacy scoring model.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …d/tests/controllers-test/featuredChallengeCtrl.test.js | 230 | function leaderboardWithSubmissionAt(date) { | CODE |
| LOW | frontend/src/js/directives/directives.js | 164 | function dashboardFooterController($scope) { | CODE |
| LOW | tests/unit/settings/test_storage_domain.py | 75 | def test_blank_values_fall_back_instead_of_building_a_hostless_url(self): | CODE |
| LOW | tests/unit/settings/test_storage_domain.py | 96 | def test_environment_variable_repoints_storage_at_a_cdn(self): | CODE |
| LOW | tests/unit/settings/test_storage_domain.py | 101 | def test_static_and_media_urls_both_follow_the_configured_domain(self): | CODE |
| LOW | tests/unit/settings/test_storage_domain.py | 112 | def test_staging_inherits_the_override(self): | CODE |
| LOW⚡ | tests/unit/participants/test_utils.py | 300 | def test_get_participant_team_member_count(self): | CODE |
| LOW⚡ | tests/unit/participants/test_utils.py | 305 | def test_get_effective_max_team_members_returns_none_without_limits(self): | CODE |
| LOW⚡ | tests/unit/participants/test_utils.py | 314 | def test_get_effective_max_team_members_returns_strictest_limit(self): | CODE |
| LOW⚡ | tests/unit/participants/test_utils.py | 350 | def test_team_can_add_member_when_below_limit(self): | CODE |
| LOW⚡ | tests/unit/participants/test_utils.py | 359 | def test_team_can_add_member_when_at_limit(self): | CODE |
| LOW⚡ | tests/unit/participants/test_utils.py | 368 | def test_team_exceeds_challenge_max_members(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 24 | def test_field_error_for_wrong_related_name(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 62 | def test_returns_team_id_when_user_participated_in_challenge(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 77 | def test_returns_none_when_user_team_not_in_challenge(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 90 | def test_returns_none_when_user_has_no_teams(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 98 | def test_returns_correct_team_when_user_has_multiple_teams(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 122 | def test_returns_none_for_different_user(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 187 | def test_returns_true_when_participated_in_active_require_complete_profile_challenge( | CODE |
| LOW | tests/unit/participants/test_utils.py | 204 | def test_returns_false_when_challenge_has_ended(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 220 | def test_returns_true_when_one_active_and_one_ended(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 244 | def test_returns_false_when_participated_in_normal_challenge(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 259 | def test_returns_false_when_not_participated(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 331 | def test_get_team_capacity_blocking_challenge_titles(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 380 | def test_team_does_not_exceed_challenge_when_limit_unset(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 390 | def test_host_only_team_exempt_from_own_challenge_limit(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 427 | def test_mixed_team_not_exempt_from_organizer_challenge(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 450 | def test_team_with_denied_host_not_exempt_from_organizer_challenge(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 473 | def test_host_challenge_exempt_but_other_challenge_still_limits(self): | CODE |
| LOW⚡ | tests/unit/participants/test_serializers.py | 30 | def test_get_team_members_empty_team(self): | CODE |
| LOW⚡ | tests/unit/participants/test_serializers.py | 36 | def test_get_team_members_email_ids_empty_team(self): | CODE |
| LOW⚡ | tests/unit/participants/test_serializers.py | 42 | def test_get_team_members_with_participants(self): | CODE |
| LOW | tests/unit/participants/test_serializers.py | 57 | def test_get_team_members_email_ids_with_participants(self): | CODE |
| LOW | tests/unit/participants/test_serializers.py | 74 | def test_validate_email_user_does_not_exist(self): | CODE |
| LOW | tests/unit/participants/test_serializers.py | 87 | def test_validate_email_self_invite(self): | CODE |
| LOW | tests/unit/participants/test_serializers.py | 133 | def test_get_members_returns_all_team_members(self): | CODE |
| LOW | tests/unit/participants/test_serializers.py | 143 | def test_get_members_with_prefetched_data(self): | CODE |
| LOW | tests/unit/participants/test_serializers.py | 170 | def test_get_members_with_prefetch_avoids_n_plus_one_queries(self): | CODE |
| LOW | tests/unit/participants/test_serializers.py | 200 | def test_serializer_includes_all_expected_fields(self): | CODE |
| LOW | tests/unit/participants/test_serializers.py | 239 | def test_get_profile_returns_user_profile(self): | CODE |
| LOW | tests/unit/participants/test_serializers.py | 255 | def test_get_profile_with_prefetched_profile(self): | CODE |
| LOW | tests/unit/participants/test_serializers.py | 271 | def test_member_fields_are_correct(self): | CODE |
| LOW | tests/unit/participants/test_serializers.py | 282 | def test_get_profile_creates_profile_when_missing(self): | CODE |
| LOW | tests/unit/participants/test_apps.py | 5 | def test_participants_config_direct_import(): | CODE |
| LOW | tests/unit/participants/test_views.py | 431 | # def test_particular_participant_team_delete(self): | COMMENT |
| LOW⚡ | tests/unit/participants/test_views.py | 473 | def test_invite_participant_to_team_with_all_data(self): | CODE |
| LOW⚡ | tests/unit/participants/test_views.py | 479 | def test_invite_participant_to_team_with_no_data(self): | CODE |
| LOW⚡ | tests/unit/participants/test_views.py | 484 | def test_invite_blocked_when_team_at_max_members_for_participated_challenge( | CODE |
| LOW⚡ | tests/unit/participants/test_views.py | 680 | def test_invite_user_which_does_not_exist_to_team(self): | CODE |
| LOW⚡ | tests/unit/participants/test_views.py | 687 | def test_particular_participant_team_for_invite_does_not_exist(self): | CODE |
| LOW⚡ | tests/unit/participants/test_views.py | 697 | def test_invite_when_team_participants_emails_are_banned(self): | CODE |
| LOW⚡ | tests/unit/participants/test_views.py | 1367 | def test_get_teams_and_corresponding_challenges_for_a_participant_team_with_no_challenges( # noqa: E501 | CODE |
| LOW⚡ | tests/unit/participants/test_views.py | 1810 | def test_remove_participant_team_success(self): | CODE |
| LOW⚡ | tests/unit/participants/test_views.py | 1815 | def test_remove_participant_team_when_user_is_not_authorized(self): | CODE |
| LOW⚡ | tests/unit/participants/test_views.py | 1824 | def test_remove_participant_team_when_not_participated(self): | CODE |
| LOW⚡ | tests/unit/participants/test_views.py | 1832 | def test_remove_participant_team_when_team_has_submissions(self): | CODE |
| LOW | tests/unit/participants/test_views.py | 143 | def test_no_n_plus_one_queries_for_participant_teams(self): | CODE |
| LOW | tests/unit/participants/test_views.py | 257 | def test_create_participant_team_with_all_data(self): | CODE |
| LOW | tests/unit/participants/test_views.py | 261 | def test_create_participant_team_with_team_name_same_as_with_existing_team( | CODE |
| LOW | tests/unit/participants/test_views.py | 278 | def test_create_participant_team_with_no_data(self): | CODE |
| 1911 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | tests/unit/challenges/test_serializers.py | 186 | except OSError: | CODE |
| HIGH | tests/unit/challenges/test_views.py | 3901 | except OSError: | CODE |
| HIGH | tests/unit/challenges/test_views.py | 4863 | except OSError: | CODE |
| HIGH | tests/unit/challenges/test_views.py | 5669 | except OSError: | CODE |
| HIGH | tests/unit/challenges/test_views.py | 6052 | except OSError: | CODE |
| HIGH | tests/unit/challenges/test_models.py | 52 | except OSError: | CODE |
| HIGH | tests/unit/challenges/test_models.py | 96 | except OSError: | CODE |
| HIGH | tests/unit/jobs/test_admin.py | 75 | except OSError: | CODE |
| HIGH | tests/unit/jobs/test_admin_filters.py | 63 | except OSError: | CODE |
| HIGH | tests/unit/jobs/test_admin_filters.py | 243 | except OSError: | CODE |
| HIGH | tests/unit/jobs/test_admin_filters.py | 579 | except OSError: | CODE |
| HIGH | tests/unit/jobs/test_views.py | 119 | except OSError: | CODE |
| HIGH | tests/unit/jobs/test_models.py | 43 | except OSError: | CODE |
| HIGH | tests/unit/jobs/test_urls.py | 80 | except OSError: | CODE |
| HIGH | tests/unit/worker/test_submission_worker.py | 1169 | except UnboundLocalError: | CODE |
| HIGH | tests/unit/base/test_utils.py | 83 | except OSError: | CODE |
| HIGH | tests/unit/analytics/test_views.py | 137 | except OSError: | CODE |
| HIGH | tests/integration/worker/test_submission_worker.py | 81 | except OSError: | CODE |
| HIGH | tests/integration/worker/test_submission_worker.py | 130 | except OSError: | CODE |
| HIGH | scripts/monitoring/monitor_containers.py | 43 | except Exception: | CODE |
| HIGH | apps/challenges/aws_utils.py | 785 | except ClientError: | CODE |
| HIGH | apps/challenges/aws_utils.py | 797 | except ClientError: | CODE |
| HIGH | apps/challenges/aws_utils.py | 963 | except ClientError: | CODE |
| HIGH | apps/challenges/aws_utils.py | 1659 | except ClientError: | CODE |
| HIGH | apps/challenges/views.py | 5745 | except ChallengeEvaluationCluster.DoesNotExist: | CODE |
| HIGH | apps/accounts/views.py | 170 | except TokenError: | CODE |
| HIGH | apps/jobs/views.py | 539 | except KeyError: | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/unit/worker/test_code_upload_submission_worker.py | 274 | except Exception as e: | CODE |
| LOW | docs/superpowers/plans/2026-06-06-scout.md | 2166 | except Exception: | CODE |
| LOW | scripts/seed.py | 814 | except Exception as exc: | CODE |
| LOW | scripts/tools/list_ongoing_challenges_s3_usage.py | 97 | except Exception: | CODE |
| LOW | …ripts/tools/cleanup_data/orphaned_challenge_configs.py | 279 | except Exception as e: | CODE |
| LOW | …ripts/tools/cleanup_data/orphaned_challenge_configs.py | 362 | except Exception: | CODE |
| LOW | …ripts/tools/cleanup_data/orphaned_challenge_configs.py | 396 | except Exception as e: | CODE |
| LOW | …ripts/tools/cleanup_data/orphaned_challenge_configs.py | 427 | except Exception as e: | CODE |
| LOW | scripts/tools/cleanup_data/s3_orphans.py | 370 | except Exception as e: | CODE |
| LOW | scripts/tools/cleanup_data/s3_orphans.py | 491 | except Exception as e: | CODE |
| LOW | scripts/tools/cleanup_data/s3_orphans.py | 512 | except Exception as e: | CODE |
| LOW | scripts/tools/cleanup_data/s3.py | 224 | except Exception: | CODE |
| LOW | scripts/tools/cleanup_data/s3.py | 304 | except Exception as e: | CODE |
| LOW | scripts/lambda/auto_scale_eks_nodes_lambda.py | 365 | except Exception as err: | CODE |
| LOW | scripts/lambda/challenge_cleanup_lambda.py | 224 | except Exception as e: # noqa: BLE001 - any failure here must fail safe | CODE |
| LOW | scripts/lambda/challenge_cleanup_lambda.py | 389 | except Exception as e: # noqa: BLE001 - surface any sync failure | CODE |
| MEDIUM | scripts/workers/code_upload_submission_worker.py | 162 | def create_configmap(core_v1_api_instance, config_map): | CODE |
| LOW⚡ | scripts/workers/code_upload_submission_worker.py | 646 | except Exception as e: | CODE |
| LOW⚡ | scripts/workers/code_upload_submission_worker.py | 651 | except Exception as e: | CODE |
| LOW | scripts/workers/code_upload_submission_worker.py | 176 | except Exception as e: | CODE |
| LOW | scripts/workers/code_upload_submission_worker.py | 528 | except Exception as e: | CODE |
| LOW | scripts/workers/code_upload_submission_worker.py | 772 | except Exception as e: | CODE |
| LOW | scripts/workers/code_upload_submission_worker.py | 914 | except Exception as e: | CODE |
| LOW | scripts/workers/code_upload_submission_worker.py | 948 | except Exception as e: | CODE |
| MEDIUM | scripts/workers/remote_submission_worker.py | 258 | def process_submission_callback(body): | CODE |
| LOW⚡ | scripts/workers/remote_submission_worker.py | 112 | except Exception as e: | CODE |
| LOW⚡ | scripts/workers/remote_submission_worker.py | 129 | except Exception as e: | CODE |
| LOW⚡ | scripts/workers/remote_submission_worker.py | 250 | except Exception: | CODE |
| LOW⚡ | scripts/workers/remote_submission_worker.py | 262 | except Exception as e: | CODE |
| LOW | scripts/workers/remote_submission_worker.py | 147 | except Exception as e: | CODE |
| LOW | scripts/workers/remote_submission_worker.py | 190 | except Exception: | CODE |
| LOW | scripts/workers/remote_submission_worker.py | 542 | except Exception: | CODE |
| MEDIUM | scripts/workers/submission_worker.py | 1172 | def process_submission_callback(body): | CODE |
| LOW⚡ | scripts/workers/submission_worker.py | 218 | except Exception as e: | CODE |
| LOW⚡ | scripts/workers/submission_worker.py | 236 | except Exception as e: | CODE |
| LOW⚡ | scripts/workers/submission_worker.py | 271 | except Exception as e: | CODE |
| LOW⚡ | scripts/workers/submission_worker.py | 282 | except Exception as e: | CODE |
| LOW⚡ | scripts/workers/submission_worker.py | 932 | except Exception: | CODE |
| LOW⚡ | scripts/workers/submission_worker.py | 945 | except Exception: | CODE |
| LOW⚡ | scripts/workers/submission_worker.py | 1182 | except Exception as e: | CODE |
| LOW | scripts/workers/submission_worker.py | 178 | except Exception as e: | CODE |
| LOW | scripts/workers/submission_worker.py | 257 | except Exception as e: | CODE |
| LOW | scripts/workers/submission_worker.py | 676 | except Exception: | CODE |
| LOW | scripts/workers/submission_worker.py | 829 | except Exception: | CODE |
| LOW | scripts/workers/submission_worker.py | 859 | except Exception: | CODE |
| LOW | scripts/workers/submission_worker.py | 983 | except Exception: | CODE |
| LOW | scripts/workers/submission_worker.py | 1031 | except Exception: | CODE |
| LOW | scripts/workers/submission_worker.py | 1076 | except Exception: | CODE |
| LOW | scripts/monitoring/monitor_containers.py | 43 | except Exception: | CODE |
| LOW | scripts/monitoring/monitor_containers.py | 109 | except Exception: | CODE |
| LOW | scripts/monitoring/monitor_containers.py | 143 | except Exception: | CODE |
| MEDIUM | scripts/monitoring/monitor_containers.py | 105 | def get_docker_client(): | CODE |
| LOW | scripts/monitoring/auto_scale_ec2_workers.py | 92 | except Exception as e: # noqa: F841 | CODE |
| LOW | scripts/monitoring/auto_scale_ec2_workers.py | 121 | except Exception as e: | CODE |
| MEDIUM | scripts/monitoring/auto_scale_ec2_workers.py | 84 | def start_or_stop_workers(challenge, evalai_interface): | CODE |
| LOW | scripts/monitoring/auto_cancel_submissions.py | 59 | except Exception as e: | CODE |
| LOW | scripts/monitoring/auto_scale_eks_nodes.py | 185 | except Exception as e: # noqa: F841 | CODE |
| LOW | scripts/monitoring/auto_scale_eks_nodes.py | 284 | except Exception as e: | CODE |
| MEDIUM | scripts/monitoring/auto_scale_eks_nodes.py | 171 | def scale_up_or_down_workers( | CODE |
| LOW | scripts/monitoring/auto_scale_workers.py | 133 | except Exception: # noqa: F841 | CODE |
| 51 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM⚡ | tests/unit/participants/test_views.py | 1371 | # Create a second team for the user that has no challenges | COMMENT |
| MEDIUM | tests/unit/participants/test_views.py | 163 | # Create a new user for each team | COMMENT |
| MEDIUM | tests/unit/participants/test_views.py | 176 | # Create the team | COMMENT |
| MEDIUM | tests/unit/challenges/test_serializers.py | 931 | # Create an existing ChallengeConfiguration | COMMENT |
| MEDIUM | tests/unit/challenges/test_serializers.py | 954 | # Create an existing ChallengeConfiguration | COMMENT |
| MEDIUM | tests/unit/challenges/test_admin_filters.py | 11 | # Create a user | COMMENT |
| MEDIUM | tests/unit/challenges/test_admin_filters.py | 16 | # Create a challenge host team | COMMENT |
| MEDIUM | tests/unit/challenges/test_challenge_config_utils.py | 44 | # Create a zip file in memory with no YAML files | COMMENT |
| MEDIUM | tests/unit/challenges/test_challenge_config_utils.py | 68 | # Create a mock exception with problem and problem_mark attributes | COMMENT |
| MEDIUM | tests/unit/challenges/test_challenge_config_utils.py | 89 | # Create a mock exception without problem and problem_mark attributes | COMMENT |
| MEDIUM | tests/unit/challenges/test_challenge_config_utils.py | 217 | # Create a challenge host team | COMMENT |
| MEDIUM⚡ | tests/unit/challenges/test_views.py | 7172 | # Create a annotation using multipart upload | COMMENT |
| MEDIUM | tests/unit/challenges/test_views.py | 6704 | # Create a user who unstarred (is_starred=False) | COMMENT |
| MEDIUM | tests/unit/challenges/test_views.py | 7558 | # Create a different user who is not a challenge host | COMMENT |
| MEDIUM | tests/unit/challenges/test_views.py | 7664 | # Create a finished submission for the additional phase to satisfy the submission check # noqa: C0301 | COMMENT |
| MEDIUM | tests/unit/challenges/test_views.py | 7717 | # Create a challenge phase without finished submissions | COMMENT |
| MEDIUM | tests/unit/challenges/test_aws_utils.py | 1105 | # Creating a mock challenge object | COMMENT |
| MEDIUM | tests/unit/challenges/test_aws_utils.py | 1145 | # Creating a mock challenge object | COMMENT |
| MEDIUM | tests/unit/challenges/test_aws_utils.py | 1175 | # Creating a mock challenge object | COMMENT |
| MEDIUM | tests/unit/challenges/test_aws_utils.py | 1200 | # Creating a mock challenge object | COMMENT |
| MEDIUM⚡ | tests/unit/hosts/test_views.py | 555 | # Create a new user and make a request using this user which doesn't | COMMENT |
| MEDIUM⚡ | tests/unit/jobs/test_views.py | 2898 | # Create a submission using multipart upload | COMMENT |
| MEDIUM⚡ | tests/unit/worker/test_code_upload_submission_worker.py | 234 | # Define the mock config map | COMMENT |
| MEDIUM⚡ | tests/unit/worker/test_code_upload_submission_worker.py | 255 | # Define the mock config map | COMMENT |
| MEDIUM | tests/unit/worker/test_code_upload_submission_worker.py | 344 | ) # Create a mock for V1JobSpec or provide a mock spec if you have one | CODE |
| MEDIUM | docs/source/conf.py | 6 | # This file is execfile()d with the current directory set to its | COMMENT |
| MEDIUM | scripts/seed.py | 710 | # Create a leaderboard object for each challenge | COMMENT |
| MEDIUM | scripts/workers/code_upload_submission_worker.py | 324 | # Create the specification of deployment. activeDeadlineSeconds is a hard | COMMENT |
| MEDIUM | scripts/workers/code_upload_submission_worker.py | 448 | # Create the specification of deployment. activeDeadlineSeconds is a hard | COMMENT |
| MEDIUM | apps/base/utils.py | 55 | # Create a new paginator instance for each request to avoid shared state | COMMENT |
| MEDIUM | apps/base/utils.py | 69 | # Create a new paginator instance for each request to avoid shared state | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 26 | ### Step 1: Use template | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 33 | ### Step 2: Generate github token | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 39 | ### Step 3: Setup host configuration | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 48 | ### Step 4: Setup automated update push | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 56 | ### Step 5: Update challenge details | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 60 | ### Step 6: Push changes to the challenge | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 68 | ### Step 7: Verify challenge | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 76 | ### Step 1: Setup challenge configuration | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 80 | ### Step 2: Edit challenge configuration | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 86 | ### Step 3: Edit evaluation script | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 92 | ### Step 4: Edit challenge HTML templates | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 98 | ### Step 5: Upload configuration on EvalAI | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 108 | ### Step 1: Set up the challenge | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 112 | ### Step 2: Edit challenge configuration | COMMENT |
| LOW | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 124 | ### Step 3: Edit remote evaluation script | COMMENT |
| LOW | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 129 | ### Step 4: Set up remote evaluation worker | COMMENT |
| LOW | …-for-challenge-hosts/hosting-guide/approval-process.md | 11 | ### Step 1: Approve challenge using django admin | COMMENT |
| LOW | …-for-challenge-hosts/hosting-guide/approval-process.md | 26 | ### Step 2: Reload submission worker | COMMENT |
| LOW⚡ | …2-for-challenge-hosts/hosting-guide/getting-started.md | 11 | ## Step 1: Create a repository from EvalAI-Starters | COMMENT |
| LOW⚡ | …2-for-challenge-hosts/hosting-guide/getting-started.md | 19 | ## Step 2: Configure GitHub secrets | COMMENT |
| LOW⚡ | …2-for-challenge-hosts/hosting-guide/getting-started.md | 24 | ## Step 3: Collect EvalAI credentials | COMMENT |
| LOW⚡ | …2-for-challenge-hosts/hosting-guide/getting-started.md | 34 | ## Step 4: Use the `challenge` branch | COMMENT |
| LOW⚡ | …2-for-challenge-hosts/hosting-guide/getting-started.md | 42 | ## Step 5: Configure the challenge | COMMENT |
| LOW⚡ | …2-for-challenge-hosts/hosting-guide/getting-started.md | 50 | ## Step 6: Push and verify the build | COMMENT |
| LOW | …2-for-challenge-hosts/hosting-guide/getting-started.md | 61 | ## Step 7: Approval | COMMENT |
| LOW⚡ | …03-for-participants/visibility/baseline-submissions.md | 20 | ### Step 1: Prepare Your Baseline Submission | COMMENT |
| LOW⚡ | …03-for-participants/visibility/baseline-submissions.md | 25 | ### Step 2: Submit the Baseline | COMMENT |
| LOW⚡ | …03-for-participants/visibility/baseline-submissions.md | 32 | ### Step 3: Open the My Submissions Tab | COMMENT |
| LOW⚡ | …03-for-participants/visibility/baseline-submissions.md | 36 | ### Step 4: Locate the Submission | COMMENT |
| LOW⚡ | …03-for-participants/visibility/baseline-submissions.md | 40 | ### Step 5: Mark the Submission as Baseline | COMMENT |
| LOW⚡ | …03-for-participants/visibility/baseline-submissions.md | 44 | ### Step 6: Confirm Baseline Display | COMMENT |
| LOW⚡ | …or-participants/participation-guide/getting-started.md | 5 | ## Step 1: Visit EvalAI | COMMENT |
| LOW⚡ | …or-participants/participation-guide/getting-started.md | 11 | ## Step 2: Sign up or log in | COMMENT |
| LOW⚡ | …or-participants/participation-guide/getting-started.md | 21 | ## Step 3: Choose a challenge | COMMENT |
| LOW⚡ | …or-participants/participation-guide/getting-started.md | 27 | ## Step 4: Read the challenge page | COMMENT |
| LOW⚡ | …or-participants/participation-guide/getting-started.md | 33 | ## Step 5: Create or select a participant team | COMMENT |
| LOW⚡ | …or-participants/participation-guide/getting-started.md | 39 | ## Step 6: Participate | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 3 | # Step 1: Updating package repository | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 7 | # Step 2: Cloning EvalAI repository at /home/ubuntu | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 13 | # Step 3: Installing awscli | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 17 | # Step 4: Installing Docker Compose plugin (v2) | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 22 | # Step 5: Adding user to docker group | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 28 | # Step 6: Configuring AWS credentials | COMMENT |
| LOW | scripts/deployment/deploy_ec2_worker.sh | 39 | # Step 7: Copying Docker environment file | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 57 | # Step 8: Pulling worker Docker image | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 67 | # Step 9: Running worker Docker container | COMMENT |
| LOW | scripts/deployment/deploy_ec2_worker.sh | 85 | # Step 10: Setting up crontab | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| CRITICAL | tests/unit/participants/test_utils.py | 5 | from django.core.exceptions import FieldError | CODE |
| CRITICAL⚡ | tests/unit/challenges/test_aws_utils.py | 9165 | mock_cluster.objects.filter.return_value.update.assert_called_once_with( | CODE |
| CRITICAL | tests/unit/challenges/test_aws_utils.py | 3903 | mock_evaluation_cluster.objects.filter.return_value.update.assert_called_once_with( | CODE |
| CRITICAL | tests/unit/challenges/test_aws_utils.py | 9187 | mock_cluster.objects.filter.return_value.update.assert_called_once_with( | CODE |
| CRITICAL⚡ | tests/unit/accounts/test_adapter.py | 7 | from django.core.exceptions import ValidationError | CODE |
| CRITICAL | apps/challenges/serializers.py | 9 | from django.core.exceptions import ValidationError as DjangoValidationError | CODE |
| CRITICAL | apps/challenges/utils.py | 16 | from django.core.exceptions import ValidationError | CODE |
| CRITICAL | apps/accounts/adapter.py | 8 | from django.core.exceptions import ValidationError | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | settings/prod.py | 129 | # Set traces_sample_rate to 1.0 to capture 100% of transactions for | COMMENT |
| LOW | settings/prod.py | 132 | # Set profiles_sample_rate to 1.0 to profile 100% of sampled transactions. | COMMENT |
| LOW | tests/unit/challenges/test_utils.py | 470 | # Check if send_email was called for each email | COMMENT |
| LOW | tests/unit/challenges/test_views.py | 7116 | # Check if all fields for cli exists | COMMENT |
| LOW | tests/unit/challenges/test_aws_utils.py | 3743 | # Check if the first argument of logger.exception contains the correct | COMMENT |
| LOW | tests/unit/challenges/test_aws_utils.py | 4042 | # Check if the actual error message contains the expected message | COMMENT |
| LOW | tests/unit/challenges/test_aws_utils.py | 4543 | # Check if start_ec2_instance was called since the EC2 instance already | COMMENT |
| LOW | tests/unit/challenges/test_aws_utils.py | 4563 | # Check if create_ec2_instance was called since the EC2 instance | COMMENT |
| LOW⚡ | tests/unit/worker/test_remote_submission_worker.py | 404 | # Check if logger.info was called | COMMENT |
| LOW | tests/unit/worker/test_remote_submission_worker.py | 354 | # Check if logger.info was called | COMMENT |
| LOW | tests/unit/worker/test_remote_submission_worker.py | 379 | # Check if logger.info was called | COMMENT |
| LOW⚡ | tests/unit/worker/test_code_upload_submission_worker.py | 174 | # Check if V1ObjectMeta was called with the right parameters | COMMENT |
| LOW⚡ | tests/unit/worker/test_code_upload_submission_worker.py | 177 | # Check if V1ConfigMap was called with the right parameters | COMMENT |
| LOW⚡ | tests/unit/worker/test_code_upload_submission_worker.py | 185 | # Check if the correct ConfigMap object is returned | COMMENT |
| LOW⚡ | tests/unit/worker/test_code_upload_submission_worker.py | 245 | # Check if the CoreV1Api method was called with the right parameters | COMMENT |
| LOW | scripts/tools/cleanup_code.sh | 14 | # Check if file is staged | COMMENT |
| LOW | scripts/tools/cleanup_data/set_challenge_end_date.py | 61 | # Set time to 23:59:59 UTC | COMMENT |
| LOW | scripts/tools/cleanup_data/s3_orphans.py | 347 | # Check if referenced (check both with and without prefix) | COMMENT |
| LOW⚡ | scripts/workers/submission_worker.py | 925 | # Check if the challenge_phase_split exists for the | COMMENT |
| LOW⚡ | scripts/workers/submission_worker.py | 941 | # Check if the dataset_split exists for the codename in the | COMMENT |
| LOW | scripts/workers/submission_worker.py | 986 | # Set submission_output to None to handle case when evaluation script throws exception | COMMENT |
| LOW | apps/participants/views.py | 256 | # Check if team participants emails are banned | COMMENT |
| LOW | apps/participants/views.py | 274 | # Check if invited user is banned | COMMENT |
| LOW | apps/participants/views.py | 285 | # Check if user is in allowed list. | COMMENT |
| LOW | apps/participants/views.py | 298 | # Check if user is in blocked list. | COMMENT |
| LOW | apps/challenges/aws_utils.py | 1355 | ): # Verify if the challenge is new (i.e, service not yet created.). | CODE |
| LOW | apps/challenges/utils.py | 852 | # Check if the sponsor exist in database. but not in the YAML file. | COMMENT |
| LOW | apps/challenges/views.py | 603 | # Check if user is banned | COMMENT |
| LOW | apps/challenges/views.py | 616 | # Check if user is in allowed list. | COMMENT |
| LOW | apps/challenges/views.py | 637 | # Check if user is in blocked list. | COMMENT |
| LOW | apps/challenges/views.py | 647 | # Check if challenge requires complete profile | COMMENT |
| LOW | apps/challenges/views.py | 1339 | # Check if user is a challenge host or staff | COMMENT |
| LOW | apps/challenges/views.py | 4237 | # Check if the challenge phase exists or not | COMMENT |
| LOW | apps/challenges/views.py | 4321 | # Check if the challenge phase exists or not | COMMENT |
| LOW | apps/challenges/views.py | 5360 | # Check if the user is a host of this challenge | COMMENT |
| LOW | apps/hosts/views.py | 284 | # Check if the user requesting this API is part of host team | COMMENT |
| LOW | apps/jobs/utils.py | 380 | # Check if challenge phase leaderboard is public for participant user or | COMMENT |
| LOW | apps/jobs/views.py | 2932 | # Check if the challenge phase exists or not | COMMENT |
| LOW | apps/jobs/views.py | 2945 | # Check if challenge phase is active | COMMENT |
| LOW | apps/jobs/views.py | 2956 | # Check if user is a challenge host or a participant | COMMENT |
| LOW | apps/jobs/views.py | 2959 | # Check if challenge phase is public and accepting solutions | COMMENT |
| LOW | apps/jobs/views.py | 3127 | # Check if the challenge phase exists or not | COMMENT |
| LOW | apps/jobs/views.py | 3140 | # Check if challenge phase is active | COMMENT |
| LOW | apps/jobs/views.py | 3151 | # Check if user is a challenge host or a participant | COMMENT |
| LOW | apps/jobs/views.py | 3153 | # Check if challenge phase is public and accepting solutions | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | docs/superpowers/plans/2026-06-06-scout.md | 0 | one row per webhook delivery — audit trail of raw yutori payloads. | STRING |
| HIGH | …superpowers/specs/2026-06-06-yutori-scouting-design.md | 0 | one row per webhook delivery — audit trail of raw yutori payloads. | STRING |
| HIGH | apps/scout/models.py | 0 | one row per webhook delivery — audit trail of raw yutori payloads. | STRING |
| HIGH | docs/superpowers/plans/2026-06-06-scout.md | 0 | a deduped benchmark/challenge discovered by the scout. | STRING |
| HIGH | …superpowers/specs/2026-06-06-yutori-scouting-design.md | 0 | a deduped benchmark/challenge discovered by the scout. | STRING |
| HIGH | apps/scout/models.py | 0 | a deduped benchmark/challenge discovered by the scout. | STRING |
| HIGH | …ripts/tools/cleanup_data/orphaned_challenge_configs.py | 0 | simple logger that writes to both console and file. | STRING |
| HIGH | scripts/tools/cleanup_data/s3_orphans.py | 0 | simple logger that writes to both console and file. | STRING |
| HIGH | scripts/tools/cleanup_data/s3.py | 0 | simple logger that writes to both console and file. | STRING |
| HIGH | …ripts/tools/cleanup_data/orphaned_challenge_configs.py | 0 | get boto3 s3 client and bucket name from django-storages settings. | STRING |
| HIGH | scripts/tools/cleanup_data/s3_orphans.py | 0 | get boto3 s3 client and bucket name from django-storages settings. | STRING |
| HIGH | scripts/tools/cleanup_data/s3.py | 0 | get boto3 s3 client and bucket name from django-storages settings. | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | frontend/tests/controllers-test/challengeCtrl.test.js | 3285 | userKey = 'dummy-token'; | CODE |
| LOW | frontend/tests/controllers-test/challengeCtrl.test.js | 3522 | var userKey = "dummy-token"; | CODE |
| LOW | frontend/tests/controllers-test/challengeCtrl.test.js | 3564 | expect(opts.headers.Authorization).toBe("Token dummy-token"); | CODE |
| LOW | frontend/tests/controllers-test/challengeCtrl.test.js | 4187 | userKey = 'dummy-token'; | CODE |
| LOW | tests/unit/participants/test_views.py | 709 | leaderboard_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit", | CODE |
| LOW | tests/unit/participants/test_views.py | 709 | leaderboard_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit", | CODE |
| LOW | tests/unit/participants/test_views.py | 745 | leaderboard_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit", | CODE |
| LOW | tests/unit/participants/test_views.py | 745 | leaderboard_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit", | CODE |
| LOW | tests/unit/participants/test_views.py | 778 | leaderboard_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit", | CODE |
| LOW | tests/unit/participants/test_views.py | 778 | leaderboard_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit", | CODE |
| LOW | tests/unit/participants/test_views.py | 813 | leaderboard_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit", | CODE |
| LOW | tests/unit/participants/test_views.py | 813 | leaderboard_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit", | CODE |
| LOW | tests/unit/participants/test_views.py | 1199 | leaderboard_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit", | CODE |
| LOW | tests/unit/participants/test_views.py | 1199 | leaderboard_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit", | CODE |
| LOW | tests/unit/participants/test_views.py | 1789 | leaderboard_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit", | CODE |
| LOW | tests/unit/participants/test_views.py | 1789 | leaderboard_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit", | CODE |
| LOW | tests/unit/challenges/test_admin.py | 377 | email="admin@example.com", | CODE |
| LOW | tests/unit/challenges/test_utils.py | 646 | profile.address_street = "123 Main St" | CODE |
| LOW⚡ | tests/unit/challenges/test_views.py | 547 | "leaderboard_description": "Lorem ipsum dolor sit amet, consectetur adipiscing elit.", # noqa: C0301 | CODE |
| LOW⚡ | tests/unit/challenges/test_views.py | 547 | "leaderboard_description": "Lorem ipsum dolor sit amet, consectetur adipiscing elit.", # noqa: C0301 | CODE |
| LOW⚡ | tests/unit/challenges/test_views.py | 7182 | "AWS_ACCESS_KEY_ID": "dummy-key", | CODE |
| LOW | tests/unit/challenges/test_views.py | 1512 | profile.address_street = "123 Main St" | CODE |
| LOW | tests/unit/challenges/test_views.py | 7147 | "AWS_ACCESS_KEY_ID": "dummy-key", | CODE |
| LOW⚡ | tests/unit/accounts/test_adapter.py | 96 | self.adapter.clean_email("admin@example.com") | CODE |
| LOW⚡ | tests/unit/accounts/test_throttles.py | 46 | self.assertIn("user@example.com", cache_key) | CODE |
| LOW⚡ | tests/unit/accounts/test_models.py | 41 | self.profile.address_street = "123 Main St" | CODE |
| LOW⚡ | tests/unit/accounts/test_models.py | 63 | self.profile.address_street = "123 Main St" | CODE |
| LOW⚡ | tests/unit/accounts/test_models.py | 76 | self.profile.address_street = "123 Main St" | CODE |
| LOW⚡ | tests/unit/accounts/test_models.py | 89 | self.profile.address_street = "123 Main St" | CODE |
| LOW | tests/unit/accounts/test_models.py | 102 | self.assertEqual(self.profile.get_full_name(), "John Doe") | CODE |
| LOW | tests/unit/jobs/test_serializer.py | 120 | email="test@test.com", | CODE |
| LOW⚡ | tests/unit/jobs/test_views.py | 2906 | "AWS_ACCESS_KEY_ID": "dummy-key", | CODE |
| LOW | tests/unit/jobs/test_views.py | 2869 | "AWS_ACCESS_KEY_ID": "dummy-key", | CODE |
| LOW | tests/unit/jobs/test_views.py | 2981 | "AWS_ACCESS_KEY_ID": "dummy-key", | CODE |
| LOW | tests/unit/jobs/test_views.py | 3017 | "AWS_ACCESS_KEY_ID": "dummy-key", | CODE |
| LOW | tests/unit/jobs/test_views.py | 3048 | "AWS_ACCESS_KEY_ID": "dummy-key", | CODE |
| LOW | tests/unit/jobs/test_views.py | 3113 | "AWS_ACCESS_KEY_ID": "dummy-key", | CODE |
| LOW | tests/unit/jobs/test_views.py | 3189 | "AWS_ACCESS_KEY_ID": "dummy-key", | CODE |
| LOW | tests/unit/jobs/test_views.py | 3256 | "AWS_ACCESS_KEY_ID": "dummy-key", | CODE |
| LOW | tests/unit/worker/test_code_upload_submission_worker.py | 915 | "aws_eks_bearer_token": "fake-token" | CODE |
| LOW | tests/unit/worker/test_code_upload_submission_worker.py | 930 | MockConfiguration().api_key["authorization"] = "fake-token" | CODE |
| LOW | tests/unit/worker/test_code_upload_submission_worker.py | 945 | "aws_eks_bearer_token": "fake-token" | CODE |
| LOW | tests/unit/worker/test_code_upload_submission_worker.py | 960 | MockConfiguration().api_key["authorization"] = "fake-token" | CODE |
| LOW | tests/unit/worker/test_code_upload_submission_worker.py | 977 | "aws_eks_bearer_token": "fake-token" | CODE |
| LOW | tests/unit/worker/test_code_upload_submission_worker.py | 992 | MockConfiguration().api_key["authorization"] = "fake-token" | CODE |
| LOW | tests/unit/worker/test_worker_utils.py | 11 | self.AUTH_TOKEN = "dummy_token" | CODE |
| LOW | tests/unit/worker/test_worker_utils.py | 23 | self.assertEqual(headers, {"Authorization": "Bearer dummy_token"}) | CODE |
| LOW⚡ | tests/unit/monitoring/test_evalai_interface.py | 11 | self.AUTH_TOKEN = "dummy_token" | CODE |
| LOW⚡ | tests/unit/monitoring/test_evalai_interface.py | 17 | self.assertEqual(headers, {"Authorization": "Bearer dummy_token"}) | CODE |
| LOW⚡ | tests/unit/monitoring/test_evalai_interface.py | 24 | "Authorization": "Bearer dummy_token", | CODE |
| LOW | examples/example2/test_zip_file/evaluation_details.html | 2 | <p><span style="font-weight: 400;">Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lo | CODE |
| LOW | …xample2/test_zip_file/challenge_phase_description.html | 2 | <p><span style="font-weight: 400;">Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lo | CODE |
| LOW | …ples/example2/test_zip_file/submission_guidelines.html | 2 | <p><span style="font-weight: 400;">Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lo | CODE |
| LOW | examples/example2/test_zip_file/description.html | 2 | <p><span style="font-weight: 400;">Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lo | CODE |
| LOW | …mples/example2/test_zip_file/terms_and_conditions.html | 2 | <p><span style="font-weight: 400;">Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lo | CODE |
| LOW | examples/example4/test_zip_file/evaluation_details.html | 2 | <p><span style="font-weight: 400;">Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lo | CODE |
| LOW | …xample4/test_zip_file/challenge_phase_description.html | 2 | <p><span style="font-weight: 400;">Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lo | CODE |
| LOW | …ples/example4/test_zip_file/submission_guidelines.html | 2 | <p><span style="font-weight: 400;">Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lo | CODE |
| LOW | examples/example4/test_zip_file/description.html | 2 | <p><span style="font-weight: 400;">Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lo | CODE |
| LOW | …mples/example4/test_zip_file/terms_and_conditions.html | 2 | <p><span style="font-weight: 400;">Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lo | CODE |
| 7 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH⚡ | tests/unit/participants/test_views.py | 1370 | """Participant team with no challenges returns (null, team) entry.""" | STRING |
| HIGH | tests/unit/worker/test_code_upload_submission_worker.py | 381 | "apt update && apt install -y curl && curl_command", | CODE |
| HIGH | tests/unit/worker/test_code_upload_submission_worker.py | 709 | "apt update && apt install -y curl && sh /evalai_scripts/monitor_submission.sh", | CODE |
| HIGH | scripts/seed.py | 106 | "To force re-seed, run: python manage.py flush && python manage.py seed" | CODE |
| HIGH⚡ | scripts/tools/list_ongoing_challenges_s3_usage.py | 133 | and (end_date >= now or end_date is null). | STRING |
| HIGH⚡ | scripts/tools/list_ongoing_challenges_s3_usage.py | 136 | "Fetching present/ongoing challenges (approved_by_admin, start_date <= now, end_date >= now or null)..." | CODE |
| HIGH | scripts/workers/code_upload_submission_worker.py | 237 | "apt update && apt install -y curl && {}".format(curl_request), | CODE |
| HIGH | scripts/workers/code_upload_submission_worker.py | 423 | "apt update && apt install -y curl && sh /evalai_scripts/monitor_submission.sh", | CODE |
| HIGH | apps/jobs/views.py | 813 | "next": null, | STRING |
| HIGH | apps/jobs/views.py | 814 | "previous": null, | STRING |
| HIGH | apps/jobs/views.py | 827 | "error": null, | STRING |
| HIGH | apps/jobs/views.py | 837 | "submission__submission_metadata": null, | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/unit/challenges/test_views.py | 5281 | CODE | |
| LOW | tests/unit/challenges/test_views.py | 6160 | CODE | |
| LOW | …ripts/tools/cleanup_data/orphaned_challenge_configs.py | 237 | CODE | |
| LOW | …ripts/tools/cleanup_data/orphaned_challenge_configs.py | 292 | CODE | |
| LOW | scripts/tools/cleanup_data/set_challenge_end_date.py | 106 | CODE | |
| LOW | scripts/tools/cleanup_data/s3_orphans.py | 299 | CODE | |
| LOW | scripts/tools/cleanup_data/s3_orphans.py | 386 | CODE | |
| LOW | scripts/tools/cleanup_data/s3_orphans.py | 426 | CODE | |
| LOW | scripts/tools/cleanup_data/s3.py | 234 | CODE | |
| LOW | scripts/workers/code_upload_submission_worker.py | 659 | CODE | |
| LOW | scripts/workers/code_upload_submission_worker.py | 816 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 343 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 592 | CODE | |
| LOW | scripts/workers/submission_worker.py | 171 | CODE | |
| LOW | scripts/workers/submission_worker.py | 244 | CODE | |
| LOW | scripts/workers/submission_worker.py | 275 | CODE | |
| LOW | scripts/workers/submission_worker.py | 444 | CODE | |
| LOW | scripts/workers/submission_worker.py | 762 | CODE | |
| LOW | scripts/workers/submission_worker.py | 1207 | CODE | |
| LOW | apps/participants/views.py | 143 | CODE | |
| LOW | apps/participants/views.py | 201 | CODE | |
| LOW | apps/participants/views.py | 594 | CODE | |
| LOW | apps/challenges/aws_utils.py | 1504 | CODE | |
| LOW | apps/challenges/aws_utils.py | 1578 | CODE | |
| LOW | apps/challenges/aws_utils.py | 1718 | CODE | |
| LOW | apps/challenges/aws_utils.py | 2169 | CODE | |
| LOW | apps/challenges/aws_utils.py | 2536 | CODE | |
| LOW | apps/challenges/aws_utils.py | 3783 | CODE | |
| LOW | apps/challenges/utils.py | 46 | CODE | |
| LOW | apps/challenges/utils.py | 816 | CODE | |
| LOW | apps/challenges/permissions.py | 14 | CODE | |
| LOW | apps/challenges/challenge_config_utils.py | 664 | CODE | |
| LOW | apps/challenges/challenge_config_utils.py | 741 | CODE | |
| LOW | apps/challenges/challenge_config_utils.py | 818 | CODE | |
| LOW | apps/challenges/challenge_config_utils.py | 990 | CODE | |
| LOW | apps/challenges/challenge_config_utils.py | 1162 | CODE | |
| LOW | apps/challenges/views.py | 242 | CODE | |
| LOW | apps/challenges/views.py | 399 | CODE | |
| LOW | apps/challenges/views.py | 1232 | CODE | |
| LOW | apps/challenges/views.py | 1358 | CODE | |
| LOW | apps/challenges/views.py | 2521 | CODE | |
| LOW | apps/challenges/views.py | 3368 | CODE | |
| LOW | apps/challenges/views.py | 3450 | CODE | |
| LOW | apps/challenges/views.py | 3907 | CODE | |
| LOW | apps/challenges/views.py | 4306 | CODE | |
| LOW | apps/challenges/views.py | 4393 | CODE | |
| LOW | apps/challenges/views.py | 5249 | CODE | |
| LOW | apps/scout/views.py | 18 | CODE | |
| LOW | apps/accounts/serializers.py | 129 | CODE | |
| LOW | …accounts/management/commands/merge_duplicate_emails.py | 135 | CODE | |
| LOW | apps/jobs/s3_retention.py | 207 | CODE | |
| LOW | apps/jobs/views.py | 149 | CODE | |
| LOW | apps/jobs/views.py | 456 | CODE | |
| LOW | apps/jobs/views.py | 1063 | CODE | |
| LOW | apps/jobs/views.py | 1273 | CODE | |
| LOW | apps/jobs/views.py | 1754 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scripts/workers/worker_utils.py | 54 | CODE | |
| LOW | scripts/workers/worker_utils.py | 61 | CODE | |
| LOW | scripts/workers/worker_utils.py | 67 | CODE | |
| LOW | scripts/workers/worker_utils.py | 75 | CODE | |
| LOW | scripts/workers/worker_utils.py | 81 | CODE | |
| LOW | scripts/workers/worker_utils.py | 89 | CODE | |
| LOW | scripts/workers/worker_utils.py | 95 | CODE | |
| LOW | scripts/workers/worker_utils.py | 101 | CODE | |
| LOW | scripts/workers/worker_utils.py | 107 | CODE | |
| LOW | scripts/workers/worker_utils.py | 113 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 413 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 422 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 429 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 436 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 443 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 452 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 459 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 466 | CODE | |
| LOW | scripts/monitoring/monitor_containers.py | 87 | CODE | |
| LOW | scripts/monitoring/monitor_containers.py | 102 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 63 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 70 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 76 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 84 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 90 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 98 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 104 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 110 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 116 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 122 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 128 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 134 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 143 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 149 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 156 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 163 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 170 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 177 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 185 | CODE | |
| LOW | scripts/monitoring/auto_scale_eks_nodes.py | 97 | CODE | |
| LOW | scripts/monitoring/auto_scale_eks_nodes.py | 114 | CODE | |
| LOW | scripts/monitoring/auto_scale_workers.py | 33 | CODE | |
| LOW | scripts/monitoring/auto_scale_workers.py | 45 | CODE | |
| LOW | apps/challenges/utils.py | 513 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | .coderabbit.yaml | 1 | # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json | COMMENT |
| LOW | settings/prod.py | 81 | # Amazon S3 Configurations | COMMENT |
| LOW | docker/prod/django/container-start.sh | 1 | #!/bin/sh | COMMENT |
| LOW | frontend/src/css/modules/error_404.scss | 101 | circle, | COMMENT |
| LOW | tests/unit/participants/test_views.py | 221 | # Capture queries during the request | COMMENT |
| LOW | .claude/hooks/docker-proxy-ca.sh | 1 | #!/usr/bin/env bash | COMMENT |
| LOW | requirements/uv-build-constraints.txt | 1 | # Build-time constraints for uv's isolated PEP 517 source builds (UV_BUILD_CONSTRAINT). | COMMENT |
| LOW | requirements/worker_py3_8.txt | 1 | # Default-provided worker stack (Python 3.8). These packages are baked into every | COMMENT |
| LOW | requirements/worker_py3_7.txt | 1 | # Default-provided worker stack (Python 3.7). These packages are baked into every | COMMENT |
| LOW | docs/source/conf.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | docs/source/conf.py | 81 | # for a list of supported languages. | COMMENT |
| LOW | docs/source/conf.py | 101 | # The reST default role (used for this markup: `text`) to use for all | COMMENT |
| LOW | docs/source/conf.py | 141 | # further. For a list of options available for each theme, see the | COMMENT |
| LOW | docs/source/conf.py | 161 | # html_logo = None | COMMENT |
| LOW | docs/source/conf.py | 181 | # bottom, using the given strftime format. | COMMENT |
| LOW | docs/source/conf.py | 201 | # | COMMENT |
| LOW | docs/source/conf.py | 221 | # | COMMENT |
| LOW | docs/source/conf.py | 241 | # 'ja' uses this config value. | COMMENT |
| LOW | docs/source/conf.py | 261 | # | COMMENT |
| LOW | docs/source/conf.py | 281 | ) | COMMENT |
| LOW | docs/source/conf.py | 301 | COMMENT | |
| LOW | docs/source/conf.py | 341 | "Miscellaneous", | COMMENT |
| LOW | scripts/lambda/deploy_challenge_cleanup_lambda.sh | 1 | #!/usr/bin/env bash | COMMENT |
| LOW | scripts/lambda/deploy_challenge_cleanup_lambda.sh | 21 | # LAMBDA_AUTH_TOKEN=xxx \ | COMMENT |
| LOW | scripts/lambda/deploy_auto_scale_eks_nodes_lambda.sh | 1 | #!/usr/bin/env bash | COMMENT |
| LOW | …rkers/code_upload_worker_utils/install_dependencies.sh | 61 | # Install cilium | COMMENT |
| LOW | …rkers/code_upload_worker_utils/install_dependencies.sh | 101 | # from silently attempting an unsupported jump. Upgrade Cilium manually, | COMMENT |
| LOW | …rs/code_upload_worker_utils/wait_for_cilium_rollout.sh | 1 | #!/bin/bash | COMMENT |
| LOW | apps/jobs/sender.py | 61 | # FIFO grouping and Slack both need the submission; load once. | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scripts/workers/worker_utils.py | 5 | logger = logging.getLogger(__name__) | CODE |
| LOW | scripts/workers/code_upload_submission_worker.py | 35 | logger = logging.getLogger(__name__) | CODE |
| LOW | scripts/workers/remote_submission_worker.py | 28 | logger = logging.getLogger(__name__) | CODE |
| LOW | scripts/workers/submission_worker.py | 60 | logger = logging.getLogger(__name__) | CODE |
| LOW | scripts/monitoring/evalai_interface.py | 5 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/web/views.py | 22 | logger = logging.getLogger(__name__) | CODE |
| LOW⚡ | apps/challenges/challenge_notification_util.py | 5 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/challenges/aws_utils.py | 49 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/challenges/utils.py | 33 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/challenges/challenge_config_utils.py | 38 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/challenges/views.py | 179 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/scout/tasks.py | 10 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/scout/views.py | 13 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/accounts/tasks.py | 10 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/accounts/adapter.py | 10 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/accounts/bounce_handler.py | 11 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/accounts/views.py | 34 | logger = logging.getLogger(__name__) | CODE |
| LOW | …accounts/management/commands/merge_duplicate_emails.py | 18 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/jobs/sender.py | 12 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/jobs/tasks.py | 20 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/jobs/models.py | 19 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/jobs/admin.py | 18 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/jobs/utils.py | 29 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/jobs/s3_retention.py | 10 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/jobs/views.py | 98 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/base/models.py | 8 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/base/utils.py | 26 | logger = logging.getLogger(__name__) | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | manage.py | 14 | CODE | |
| LOW | settings/test.py | 3 | CODE | |
| LOW | settings/dev.py | 4 | CODE | |
| LOW | settings/prod.py | 7 | CODE | |
| LOW | settings/staging.py | 1 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 1 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 1 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 1 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 1 | CODE | |
| LOW | scripts/workers/submission_worker.py | 1 | CODE | |
| LOW | scripts/workers/submission_worker.py | 1 | CODE | |
| LOW | scripts/workers/submission_worker.py | 1 | CODE | |
| LOW | scripts/workers/submission_worker.py | 1 | CODE | |
| LOW | apps/participants/models.py | 1 | CODE | |
| LOW | apps/participants/apps.py | 1 | CODE | |
| LOW | apps/web/models.py | 1 | CODE | |
| LOW | apps/web/apps.py | 1 | CODE | |
| LOW | apps/challenges/models.py | 1 | CODE | |
| LOW | apps/challenges/apps.py | 1 | CODE | |
| LOW | apps/scout/models.py | 1 | CODE | |
| LOW | apps/scout/apps.py | 1 | CODE | |
| LOW | apps/accounts/models.py | 1 | CODE | |
| LOW | apps/accounts/apps.py | 1 | CODE | |
| LOW | apps/hosts/models.py | 1 | CODE | |
| LOW | apps/hosts/apps.py | 1 | CODE | |
| LOW | apps/jobs/sender.py | 1 | CODE | |
| LOW | apps/jobs/models.py | 1 | CODE | |
| LOW | apps/jobs/apps.py | 1 | CODE | |
| LOW | apps/base/models.py | 1 | CODE | |
| LOW | apps/base/apps.py | 1 | CODE | |
| LOW | apps/analytics/apps.py | 1 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 3 | # Step 1: Updating package repository | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 7 | # Step 2: Cloning EvalAI repository at /home/ubuntu | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 13 | # Step 3: Installing awscli | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 17 | # Step 4: Installing Docker Compose plugin (v2) | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 22 | # Step 5: Adding user to docker group | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 28 | # Step 6: Configuring AWS credentials | COMMENT |
| LOW | scripts/deployment/deploy_ec2_worker.sh | 39 | # Step 7: Copying Docker environment file | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 57 | # Step 8: Pulling worker Docker image | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 67 | # Step 9: Running worker Docker container | COMMENT |
| LOW | scripts/deployment/deploy_ec2_worker.sh | 85 | # Step 10: Setting up crontab | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | scripts/tools/cleanup_data/s3_orphans.py | 88 | # ============================================================================= | COMMENT |
| MEDIUM | scripts/tools/cleanup_data/s3_orphans.py | 90 | # ============================================================================= | COMMENT |
| MEDIUM | scripts/tools/cleanup_data/s3_orphans.py | 107 | # ============================================================================= | COMMENT |
| MEDIUM | scripts/tools/cleanup_data/s3.py | 54 | # ============================================================================= | COMMENT |
| MEDIUM | scripts/tools/cleanup_data/s3.py | 56 | # ============================================================================= | COMMENT |
| MEDIUM | scripts/tools/cleanup_data/s3.py | 76 | # ============================================================================= | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | apps/challenges/views.py | 3328 | Endpoint to generate AWS Credentails for CLI Args: - challenge: Challenge model - participant_t | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scripts/lambda/deploy_challenge_cleanup_lambda.sh | 17 | # Usage: | COMMENT |
| LOW | scripts/lambda/deploy_auto_scale_eks_nodes_lambda.sh | 10 | # Usage: | COMMENT |
| LOW | scripts/monitoring/auto_cancel_submissions.py | 63 | # Example usage: | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/unit/hosts/test_views.py | 251 | # TODO: Add test back with API | COMMENT |
| LOW | scripts/workers/code_upload_submission_worker.py | 11 | # TODO: Add exception in all the commands | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | .github/workflows/ci-cd.yml | 350 | # dropped event-driven invocation left a nodegroup running indefinitely | COMMENT |