:cloud: :rocket: :bar_chart: :chart_with_upwards_trend: Evaluating state of the art in AI
This report presents the forensic synthetic code analysis of Cloud-CV/EvalAI, a Python project with 2,030 GitHub stars. SynthScan v2.0 examined 128,931 lines of code across 674 source files, recording 2237 pattern matches distributed across 19 syntactic categories. The overall adjusted score of 17.6 places this repository in the Moderate AI signal band.
The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).
This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.
Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.
This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.
The scanner identified 2237 distinct pattern matches across 19 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …d/tests/controllers-test/featuredChallengeCtrl.test.js | 230 | function leaderboardWithSubmissionAt(date) { | CODE |
| LOW | frontend/src/js/directives/directives.js | 164 | function dashboardFooterController($scope) { | CODE |
| LOW⚡ | tests/unit/participants/test_utils.py | 300 | def test_get_participant_team_member_count(self): | CODE |
| LOW⚡ | tests/unit/participants/test_utils.py | 305 | def test_get_effective_max_team_members_returns_none_without_limits(self): | CODE |
| LOW⚡ | tests/unit/participants/test_utils.py | 314 | def test_get_effective_max_team_members_returns_strictest_limit(self): | CODE |
| LOW⚡ | tests/unit/participants/test_utils.py | 350 | def test_team_can_add_member_when_below_limit(self): | CODE |
| LOW⚡ | tests/unit/participants/test_utils.py | 359 | def test_team_can_add_member_when_at_limit(self): | CODE |
| LOW⚡ | tests/unit/participants/test_utils.py | 368 | def test_team_exceeds_challenge_max_members(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 24 | def test_field_error_for_wrong_related_name(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 62 | def test_returns_team_id_when_user_participated_in_challenge(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 77 | def test_returns_none_when_user_team_not_in_challenge(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 90 | def test_returns_none_when_user_has_no_teams(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 98 | def test_returns_correct_team_when_user_has_multiple_teams(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 122 | def test_returns_none_for_different_user(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 187 | def test_returns_true_when_participated_in_active_require_complete_profile_challenge( | CODE |
| LOW | tests/unit/participants/test_utils.py | 204 | def test_returns_false_when_challenge_has_ended(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 220 | def test_returns_true_when_one_active_and_one_ended(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 244 | def test_returns_false_when_participated_in_normal_challenge(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 259 | def test_returns_false_when_not_participated(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 331 | def test_get_team_capacity_blocking_challenge_titles(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 380 | def test_team_does_not_exceed_challenge_when_limit_unset(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 390 | def test_host_only_team_exempt_from_own_challenge_limit(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 427 | def test_mixed_team_not_exempt_from_organizer_challenge(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 450 | def test_team_with_denied_host_not_exempt_from_organizer_challenge(self): | CODE |
| LOW | tests/unit/participants/test_utils.py | 473 | def test_host_challenge_exempt_but_other_challenge_still_limits(self): | CODE |
| LOW⚡ | tests/unit/participants/test_serializers.py | 30 | def test_get_team_members_empty_team(self): | CODE |
| LOW⚡ | tests/unit/participants/test_serializers.py | 36 | def test_get_team_members_email_ids_empty_team(self): | CODE |
| LOW⚡ | tests/unit/participants/test_serializers.py | 42 | def test_get_team_members_with_participants(self): | CODE |
| LOW | tests/unit/participants/test_serializers.py | 57 | def test_get_team_members_email_ids_with_participants(self): | CODE |
| LOW | tests/unit/participants/test_serializers.py | 74 | def test_validate_email_user_does_not_exist(self): | CODE |
| LOW | tests/unit/participants/test_serializers.py | 87 | def test_validate_email_self_invite(self): | CODE |
| LOW | tests/unit/participants/test_serializers.py | 133 | def test_get_members_returns_all_team_members(self): | CODE |
| LOW | tests/unit/participants/test_serializers.py | 143 | def test_get_members_with_prefetched_data(self): | CODE |
| LOW | tests/unit/participants/test_serializers.py | 170 | def test_get_members_with_prefetch_avoids_n_plus_one_queries(self): | CODE |
| LOW | tests/unit/participants/test_serializers.py | 200 | def test_serializer_includes_all_expected_fields(self): | CODE |
| LOW | tests/unit/participants/test_serializers.py | 239 | def test_get_profile_returns_user_profile(self): | CODE |
| LOW | tests/unit/participants/test_serializers.py | 255 | def test_get_profile_with_prefetched_profile(self): | CODE |
| LOW | tests/unit/participants/test_serializers.py | 271 | def test_member_fields_are_correct(self): | CODE |
| LOW | tests/unit/participants/test_serializers.py | 282 | def test_get_profile_creates_profile_when_missing(self): | CODE |
| LOW | tests/unit/participants/test_apps.py | 5 | def test_participants_config_direct_import(): | CODE |
| LOW | tests/unit/participants/test_views.py | 433 | # def test_particular_participant_team_delete(self): | COMMENT |
| LOW⚡ | tests/unit/participants/test_views.py | 475 | def test_invite_participant_to_team_with_all_data(self): | CODE |
| LOW⚡ | tests/unit/participants/test_views.py | 481 | def test_invite_participant_to_team_with_no_data(self): | CODE |
| LOW⚡ | tests/unit/participants/test_views.py | 486 | def test_invite_blocked_when_team_at_max_members_for_participated_challenge( | CODE |
| LOW⚡ | tests/unit/participants/test_views.py | 682 | def test_invite_user_which_does_not_exist_to_team(self): | CODE |
| LOW⚡ | tests/unit/participants/test_views.py | 689 | def test_particular_participant_team_for_invite_does_not_exist(self): | CODE |
| LOW⚡ | tests/unit/participants/test_views.py | 699 | def test_invite_when_team_participants_emails_are_banned(self): | CODE |
| LOW⚡ | tests/unit/participants/test_views.py | 1368 | def test_get_teams_and_corresponding_challenges_for_a_participant_team_with_no_challenges( # noqa: E501 | CODE |
| LOW⚡ | tests/unit/participants/test_views.py | 1810 | def test_remove_participant_team_success(self): | CODE |
| LOW⚡ | tests/unit/participants/test_views.py | 1815 | def test_remove_participant_team_when_user_is_not_authorized(self): | CODE |
| LOW⚡ | tests/unit/participants/test_views.py | 1824 | def test_remove_participant_team_when_not_participated(self): | CODE |
| LOW⚡ | tests/unit/participants/test_views.py | 1832 | def test_remove_participant_team_when_team_has_submissions(self): | CODE |
| LOW | tests/unit/participants/test_views.py | 144 | def test_no_n_plus_one_queries_for_participant_teams(self): | CODE |
| LOW | tests/unit/participants/test_views.py | 259 | def test_create_participant_team_with_all_data(self): | CODE |
| LOW | tests/unit/participants/test_views.py | 263 | def test_create_participant_team_with_team_name_same_as_with_existing_team( | CODE |
| LOW | tests/unit/participants/test_views.py | 280 | def test_create_participant_team_with_no_data(self): | CODE |
| LOW | tests/unit/participants/test_views.py | 315 | def test_get_particular_participant_team(self): | CODE |
| LOW | tests/unit/participants/test_views.py | 368 | def test_particular_participant_team_does_not_exist(self): | CODE |
| LOW | tests/unit/participants/test_views.py | 393 | def test_particular_participant_team_partial_update(self): | CODE |
| LOW | tests/unit/participants/test_views.py | 407 | def test_particular_participant_team_update(self): | CODE |
| 1659 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/unit/worker/test_code_upload_submission_worker.py | 214 | except Exception as e: | CODE |
| LOW | docs/superpowers/plans/2026-06-06-scout.md | 2166 | except Exception: | CODE |
| LOW | scripts/seed.py | 777 | except Exception as exc: | CODE |
| LOW | scripts/tools/list_ongoing_challenges_s3_usage.py | 97 | except Exception: | CODE |
| LOW | …ripts/tools/cleanup_data/orphaned_challenge_configs.py | 279 | except Exception as e: | CODE |
| LOW | …ripts/tools/cleanup_data/orphaned_challenge_configs.py | 362 | except Exception: | CODE |
| LOW | …ripts/tools/cleanup_data/orphaned_challenge_configs.py | 396 | except Exception as e: | CODE |
| LOW | …ripts/tools/cleanup_data/orphaned_challenge_configs.py | 427 | except Exception as e: | CODE |
| LOW | scripts/tools/cleanup_data/s3_orphans.py | 370 | except Exception as e: | CODE |
| LOW | scripts/tools/cleanup_data/s3_orphans.py | 491 | except Exception as e: | CODE |
| LOW | scripts/tools/cleanup_data/s3_orphans.py | 512 | except Exception as e: | CODE |
| LOW | scripts/tools/cleanup_data/s3.py | 224 | except Exception: | CODE |
| LOW | scripts/tools/cleanup_data/s3.py | 304 | except Exception as e: | CODE |
| LOW | scripts/workers/code_upload_submission_worker.py | 121 | except Exception as e: | CODE |
| LOW | scripts/workers/code_upload_submission_worker.py | 452 | except Exception as e: | CODE |
| LOW⚡ | scripts/workers/code_upload_submission_worker.py | 570 | except Exception as e: | CODE |
| LOW⚡ | scripts/workers/code_upload_submission_worker.py | 575 | except Exception as e: | CODE |
| LOW | scripts/workers/code_upload_submission_worker.py | 663 | except Exception as e: | CODE |
| LOW | scripts/workers/code_upload_submission_worker.py | 795 | except Exception as e: | CODE |
| MEDIUM | scripts/workers/code_upload_submission_worker.py | 107 | def create_configmap(core_v1_api_instance, config_map): | CODE |
| MEDIUM | scripts/workers/remote_submission_worker.py | 258 | def process_submission_callback(body): | CODE |
| LOW⚡ | scripts/workers/remote_submission_worker.py | 112 | except Exception as e: | CODE |
| LOW⚡ | scripts/workers/remote_submission_worker.py | 129 | except Exception as e: | CODE |
| LOW⚡ | scripts/workers/remote_submission_worker.py | 250 | except Exception: | CODE |
| LOW⚡ | scripts/workers/remote_submission_worker.py | 262 | except Exception as e: | CODE |
| LOW | scripts/workers/remote_submission_worker.py | 147 | except Exception as e: | CODE |
| LOW | scripts/workers/remote_submission_worker.py | 190 | except Exception: | CODE |
| LOW | scripts/workers/remote_submission_worker.py | 539 | except Exception: | CODE |
| MEDIUM | scripts/workers/submission_worker.py | 846 | def process_submission_callback(body): | CODE |
| LOW⚡ | scripts/workers/submission_worker.py | 197 | except Exception as e: | CODE |
| LOW⚡ | scripts/workers/submission_worker.py | 215 | except Exception as e: | CODE |
| LOW⚡ | scripts/workers/submission_worker.py | 250 | except Exception as e: | CODE |
| LOW⚡ | scripts/workers/submission_worker.py | 261 | except Exception as e: | CODE |
| LOW⚡ | scripts/workers/submission_worker.py | 644 | except Exception: | CODE |
| LOW⚡ | scripts/workers/submission_worker.py | 657 | except Exception: | CODE |
| LOW | scripts/workers/submission_worker.py | 157 | except Exception as e: | CODE |
| LOW | scripts/workers/submission_worker.py | 236 | except Exception as e: | CODE |
| LOW | scripts/workers/submission_worker.py | 359 | except Exception as e: | CODE |
| LOW | scripts/workers/submission_worker.py | 397 | except Exception: | CODE |
| LOW | scripts/workers/submission_worker.py | 550 | except Exception: | CODE |
| LOW | scripts/workers/submission_worker.py | 692 | except Exception: | CODE |
| LOW | scripts/workers/submission_worker.py | 856 | except Exception as e: | CODE |
| LOW | scripts/monitoring/monitor_containers.py | 43 | except Exception: | CODE |
| LOW | scripts/monitoring/monitor_containers.py | 109 | except Exception: | CODE |
| LOW | scripts/monitoring/monitor_containers.py | 143 | except Exception: | CODE |
| MEDIUM | scripts/monitoring/monitor_containers.py | 105 | def get_docker_client(): | CODE |
| LOW | scripts/monitoring/auto_scale_ec2_workers.py | 92 | except Exception as e: # noqa: F841 | CODE |
| LOW | scripts/monitoring/auto_scale_ec2_workers.py | 121 | except Exception as e: | CODE |
| MEDIUM | scripts/monitoring/auto_scale_ec2_workers.py | 84 | def start_or_stop_workers(challenge, evalai_interface): | CODE |
| LOW | scripts/monitoring/auto_cancel_submissions.py | 59 | except Exception as e: | CODE |
| LOW | scripts/monitoring/auto_scale_eks_nodes.py | 189 | except Exception as e: # noqa: F841 | CODE |
| LOW | scripts/monitoring/auto_scale_eks_nodes.py | 286 | except Exception as e: | CODE |
| MEDIUM | scripts/monitoring/auto_scale_eks_nodes.py | 175 | def scale_up_or_down_workers( | CODE |
| LOW | scripts/monitoring/auto_scale_workers.py | 64 | except Exception: # noqa: F841 | CODE |
| LOW⚡ | scripts/monitoring/auto_scale_workers.py | 96 | except Exception as e: | CODE |
| LOW⚡ | scripts/monitoring/auto_scale_workers.py | 101 | except Exception as e: | CODE |
| LOW | scripts/monitoring/auto_scale_workers.py | 116 | except Exception as e: | CODE |
| MEDIUM | scripts/monitoring/auto_scale_workers.py | 61 | def scale_up_or_down_workers(challenge, challenge_metrics): | CODE |
| LOW | scripts/migration/set_phase_slug_as_unique.py | 25 | except Exception as e: | CODE |
| LOW | scripts/migration/set_monthly_submission_limit.py | 11 | except Exception as e: | CODE |
| 44 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM⚡ | tests/unit/participants/test_views.py | 1372 | # Create a second team for the user that has no challenges | COMMENT |
| MEDIUM | tests/unit/participants/test_views.py | 164 | # Create a new user for each team | COMMENT |
| MEDIUM | tests/unit/participants/test_views.py | 177 | # Create the team | COMMENT |
| MEDIUM | tests/unit/challenges/test_serializers.py | 888 | # Create an existing ChallengeConfiguration | COMMENT |
| MEDIUM | tests/unit/challenges/test_serializers.py | 911 | # Create an existing ChallengeConfiguration | COMMENT |
| MEDIUM | tests/unit/challenges/test_admin_filters.py | 11 | # Create a user | COMMENT |
| MEDIUM | tests/unit/challenges/test_admin_filters.py | 16 | # Create a challenge host team | COMMENT |
| MEDIUM | tests/unit/challenges/test_challenge_config_utils.py | 44 | # Create a zip file in memory with no YAML files | COMMENT |
| MEDIUM | tests/unit/challenges/test_challenge_config_utils.py | 68 | # Create a mock exception with problem and problem_mark attributes | COMMENT |
| MEDIUM | tests/unit/challenges/test_challenge_config_utils.py | 89 | # Create a mock exception without problem and problem_mark attributes | COMMENT |
| MEDIUM | tests/unit/challenges/test_challenge_config_utils.py | 217 | # Create a challenge host team | COMMENT |
| MEDIUM⚡ | tests/unit/challenges/test_views.py | 7067 | # Create a annotation using multipart upload | COMMENT |
| MEDIUM | tests/unit/challenges/test_views.py | 6598 | # Create a user who unstarred (is_starred=False) | COMMENT |
| MEDIUM | tests/unit/challenges/test_views.py | 7454 | # Create a different user who is not a challenge host | COMMENT |
| MEDIUM | tests/unit/challenges/test_views.py | 7560 | # Create a finished submission for the additional phase to satisfy the submission check # noqa: C0301 | COMMENT |
| MEDIUM | tests/unit/challenges/test_views.py | 7613 | # Create a challenge phase without finished submissions | COMMENT |
| MEDIUM | tests/unit/challenges/test_aws_utils.py | 784 | # Creating a mock challenge object | COMMENT |
| MEDIUM | tests/unit/challenges/test_aws_utils.py | 824 | # Creating a mock challenge object | COMMENT |
| MEDIUM | tests/unit/challenges/test_aws_utils.py | 854 | # Creating a mock challenge object | COMMENT |
| MEDIUM | tests/unit/challenges/test_aws_utils.py | 879 | # Creating a mock challenge object | COMMENT |
| MEDIUM⚡ | tests/unit/hosts/test_views.py | 557 | # Create a new user and make a request using this user which doesn't | COMMENT |
| MEDIUM⚡ | tests/unit/jobs/test_views.py | 2898 | # Create a submission using multipart upload | COMMENT |
| MEDIUM⚡ | tests/unit/worker/test_code_upload_submission_worker.py | 174 | # Define the mock config map | COMMENT |
| MEDIUM⚡ | tests/unit/worker/test_code_upload_submission_worker.py | 195 | # Define the mock config map | COMMENT |
| MEDIUM | tests/unit/worker/test_code_upload_submission_worker.py | 284 | ) # Create a mock for V1JobSpec or provide a mock spec if you have one | CODE |
| MEDIUM | docs/source/conf.py | 6 | # This file is execfile()d with the current directory set to its | COMMENT |
| MEDIUM | scripts/seed.py | 673 | # Create a leaderboard object for each challenge | COMMENT |
| MEDIUM | scripts/workers/code_upload_submission_worker.py | 269 | # Create the specification of deployment | COMMENT |
| MEDIUM | scripts/workers/code_upload_submission_worker.py | 385 | # Create the specification of deployment | COMMENT |
| MEDIUM | apps/base/utils.py | 55 | # Create a new paginator instance for each request to avoid shared state | COMMENT |
| MEDIUM | apps/base/utils.py | 69 | # Create a new paginator instance for each request to avoid shared state | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 26 | ### Step 1: Use template | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 33 | ### Step 2: Generate github token | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 39 | ### Step 3: Setup host configuration | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 48 | ### Step 4: Setup automated update push | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 56 | ### Step 5: Update challenge details | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 60 | ### Step 6: Push changes to the challenge | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 68 | ### Step 7: Verify challenge | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 76 | ### Step 1: Setup challenge configuration | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 80 | ### Step 2: Edit challenge configuration | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 86 | ### Step 3: Edit evaluation script | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 92 | ### Step 4: Edit challenge HTML templates | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 98 | ### Step 5: Upload configuration on EvalAI | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 108 | ### Step 1: Set up the challenge | COMMENT |
| LOW⚡ | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 112 | ### Step 2: Edit challenge configuration | COMMENT |
| LOW | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 124 | ### Step 3: Edit remote evaluation script | COMMENT |
| LOW | …02-for-challenge-hosts/hosting-guide/host-challenge.md | 129 | ### Step 4: Set up remote evaluation worker | COMMENT |
| LOW | …-for-challenge-hosts/hosting-guide/approval-process.md | 11 | ### Step 1: Approve challenge using django admin | COMMENT |
| LOW | …-for-challenge-hosts/hosting-guide/approval-process.md | 26 | ### Step 2: Reload submission worker | COMMENT |
| LOW⚡ | …2-for-challenge-hosts/hosting-guide/getting-started.md | 11 | ## Step 1: Create a repository from EvalAI-Starters | COMMENT |
| LOW⚡ | …2-for-challenge-hosts/hosting-guide/getting-started.md | 19 | ## Step 2: Configure GitHub secrets | COMMENT |
| LOW⚡ | …2-for-challenge-hosts/hosting-guide/getting-started.md | 24 | ## Step 3: Collect EvalAI credentials | COMMENT |
| LOW⚡ | …2-for-challenge-hosts/hosting-guide/getting-started.md | 34 | ## Step 4: Use the `challenge` branch | COMMENT |
| LOW⚡ | …2-for-challenge-hosts/hosting-guide/getting-started.md | 42 | ## Step 5: Configure the challenge | COMMENT |
| LOW⚡ | …2-for-challenge-hosts/hosting-guide/getting-started.md | 50 | ## Step 6: Push and verify the build | COMMENT |
| LOW | …2-for-challenge-hosts/hosting-guide/getting-started.md | 61 | ## Step 7: Approval | COMMENT |
| LOW⚡ | …03-for-participants/visibility/baseline-submissions.md | 20 | ### Step 1: Prepare Your Baseline Submission | COMMENT |
| LOW⚡ | …03-for-participants/visibility/baseline-submissions.md | 25 | ### Step 2: Submit the Baseline | COMMENT |
| LOW⚡ | …03-for-participants/visibility/baseline-submissions.md | 32 | ### Step 3: Open the My Submissions Tab | COMMENT |
| LOW⚡ | …03-for-participants/visibility/baseline-submissions.md | 36 | ### Step 4: Locate the Submission | COMMENT |
| LOW⚡ | …03-for-participants/visibility/baseline-submissions.md | 40 | ### Step 5: Mark the Submission as Baseline | COMMENT |
| LOW⚡ | …03-for-participants/visibility/baseline-submissions.md | 44 | ### Step 6: Confirm Baseline Display | COMMENT |
| LOW⚡ | …or-participants/participation-guide/getting-started.md | 5 | ## Step 1: Visit EvalAI | COMMENT |
| LOW⚡ | …or-participants/participation-guide/getting-started.md | 11 | ## Step 2: Sign up or log in | COMMENT |
| LOW⚡ | …or-participants/participation-guide/getting-started.md | 21 | ## Step 3: Choose a challenge | COMMENT |
| LOW⚡ | …or-participants/participation-guide/getting-started.md | 27 | ## Step 4: Read the challenge page | COMMENT |
| LOW⚡ | …or-participants/participation-guide/getting-started.md | 33 | ## Step 5: Create or select a participant team | COMMENT |
| LOW⚡ | …or-participants/participation-guide/getting-started.md | 39 | ## Step 6: Participate | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 3 | # Step 1: Updating package repository | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 7 | # Step 2: Cloning EvalAI repository at /home/ubuntu | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 13 | # Step 3: Installing awscli | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 17 | # Step 4: Installing Docker Compose plugin (v2) | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 22 | # Step 5: Adding user to docker group | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 28 | # Step 6: Configuring AWS credentials | COMMENT |
| LOW | scripts/deployment/deploy_ec2_worker.sh | 39 | # Step 7: Copying Docker environment file | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 57 | # Step 8: Pulling worker Docker image | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 67 | # Step 9: Running worker Docker container | COMMENT |
| LOW | scripts/deployment/deploy_ec2_worker.sh | 85 | # Step 10: Setting up crontab | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | settings/prod.py | 104 | # Set traces_sample_rate to 1.0 to capture 100% of transactions for | COMMENT |
| LOW | settings/prod.py | 107 | # Set profiles_sample_rate to 1.0 to profile 100% of sampled transactions. | COMMENT |
| LOW | tests/unit/challenges/test_utils.py | 450 | # Check if send_email was called for each email | COMMENT |
| LOW | tests/unit/challenges/test_views.py | 7011 | # Check if all fields for cli exists | COMMENT |
| LOW | tests/unit/challenges/test_aws_utils.py | 3193 | # Check if the first argument of logger.exception contains the correct | COMMENT |
| LOW | tests/unit/challenges/test_aws_utils.py | 3403 | # Check if the actual error message contains the expected message | COMMENT |
| LOW | tests/unit/challenges/test_aws_utils.py | 3647 | # Check if start_ec2_instance was called since the EC2 instance already | COMMENT |
| LOW | tests/unit/challenges/test_aws_utils.py | 3667 | # Check if create_ec2_instance was called since the EC2 instance | COMMENT |
| LOW⚡ | tests/unit/worker/test_remote_submission_worker.py | 404 | # Check if logger.info was called | COMMENT |
| LOW | tests/unit/worker/test_remote_submission_worker.py | 354 | # Check if logger.info was called | COMMENT |
| LOW | tests/unit/worker/test_remote_submission_worker.py | 379 | # Check if logger.info was called | COMMENT |
| LOW⚡ | tests/unit/worker/test_code_upload_submission_worker.py | 114 | # Check if V1ObjectMeta was called with the right parameters | COMMENT |
| LOW⚡ | tests/unit/worker/test_code_upload_submission_worker.py | 117 | # Check if V1ConfigMap was called with the right parameters | COMMENT |
| LOW⚡ | tests/unit/worker/test_code_upload_submission_worker.py | 125 | # Check if the correct ConfigMap object is returned | COMMENT |
| LOW⚡ | tests/unit/worker/test_code_upload_submission_worker.py | 185 | # Check if the CoreV1Api method was called with the right parameters | COMMENT |
| LOW | scripts/tools/cleanup_code.sh | 14 | # Check if file is staged | COMMENT |
| LOW | scripts/tools/cleanup_data/set_challenge_end_date.py | 61 | # Set time to 23:59:59 UTC | COMMENT |
| LOW | scripts/tools/cleanup_data/s3_orphans.py | 347 | # Check if referenced (check both with and without prefix) | COMMENT |
| LOW⚡ | scripts/workers/submission_worker.py | 637 | # Check if the challenge_phase_split exists for the | COMMENT |
| LOW⚡ | scripts/workers/submission_worker.py | 653 | # Check if the dataset_split exists for the codename in the | COMMENT |
| LOW | scripts/workers/submission_worker.py | 695 | # Set submission_output to None to handle case when evaluation script throws exception | COMMENT |
| LOW | scripts/workers/submission_worker.py | 894 | # Check if the queue exists. If no, then create one | COMMENT |
| LOW | apps/participants/views.py | 257 | # Check if team participants emails are banned | COMMENT |
| LOW | apps/participants/views.py | 275 | # Check if invited user is banned | COMMENT |
| LOW | apps/participants/views.py | 286 | # Check if user is in allowed list. | COMMENT |
| LOW | apps/participants/views.py | 299 | # Check if user is in blocked list. | COMMENT |
| LOW | apps/challenges/aws_utils.py | 1248 | ): # Verify if the challenge is new (i.e, service not yet created.). | CODE |
| LOW | apps/challenges/utils.py | 852 | # Check if the sponsor exist in database. but not in the YAML file. | COMMENT |
| LOW | apps/challenges/views.py | 583 | # Check if user is banned | COMMENT |
| LOW | apps/challenges/views.py | 596 | # Check if user is in allowed list. | COMMENT |
| LOW | apps/challenges/views.py | 617 | # Check if user is in blocked list. | COMMENT |
| LOW | apps/challenges/views.py | 627 | # Check if challenge requires complete profile | COMMENT |
| LOW | apps/challenges/views.py | 1305 | # Check if user is a challenge host or staff | COMMENT |
| LOW | apps/challenges/views.py | 4203 | # Check if the challenge phase exists or not | COMMENT |
| LOW | apps/challenges/views.py | 4287 | # Check if the challenge phase exists or not | COMMENT |
| LOW | apps/challenges/views.py | 5322 | # Check if the user is a host of this challenge | COMMENT |
| LOW | apps/hosts/views.py | 285 | # Check if the user requesting this API is part of host team | COMMENT |
| LOW | apps/jobs/sender.py | 56 | # Check if the queue exists. If not, then create one. | COMMENT |
| LOW | apps/jobs/utils.py | 384 | # Check if challenge phase leaderboard is public for participant user or | COMMENT |
| LOW | apps/jobs/views.py | 2930 | # Check if the challenge phase exists or not | COMMENT |
| LOW | apps/jobs/views.py | 2943 | # Check if challenge phase is active | COMMENT |
| LOW | apps/jobs/views.py | 2954 | # Check if user is a challenge host or a participant | COMMENT |
| LOW | apps/jobs/views.py | 2957 | # Check if challenge phase is public and accepting solutions | COMMENT |
| LOW | apps/jobs/views.py | 3123 | # Check if the challenge phase exists or not | COMMENT |
| LOW | apps/jobs/views.py | 3136 | # Check if challenge phase is active | COMMENT |
| LOW | apps/jobs/views.py | 3147 | # Check if user is a challenge host or a participant | COMMENT |
| LOW | apps/jobs/views.py | 3149 | # Check if challenge phase is public and accepting solutions | COMMENT |
| LOW | apps/base/utils.py | 378 | # Check if the queue exists. If no, then create one | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | docs/superpowers/plans/2026-06-06-scout.md | 0 | one row per webhook delivery — audit trail of raw yutori payloads. | STRING |
| HIGH | …superpowers/specs/2026-06-06-yutori-scouting-design.md | 0 | one row per webhook delivery — audit trail of raw yutori payloads. | STRING |
| HIGH | apps/scout/models.py | 0 | one row per webhook delivery — audit trail of raw yutori payloads. | STRING |
| HIGH | docs/superpowers/plans/2026-06-06-scout.md | 0 | a deduped benchmark/challenge discovered by the scout. | STRING |
| HIGH | …superpowers/specs/2026-06-06-yutori-scouting-design.md | 0 | a deduped benchmark/challenge discovered by the scout. | STRING |
| HIGH | apps/scout/models.py | 0 | a deduped benchmark/challenge discovered by the scout. | STRING |
| HIGH | …ripts/tools/cleanup_data/orphaned_challenge_configs.py | 0 | simple logger that writes to both console and file. | STRING |
| HIGH | scripts/tools/cleanup_data/s3_orphans.py | 0 | simple logger that writes to both console and file. | STRING |
| HIGH | scripts/tools/cleanup_data/s3.py | 0 | simple logger that writes to both console and file. | STRING |
| HIGH | …ripts/tools/cleanup_data/orphaned_challenge_configs.py | 0 | get boto3 s3 client and bucket name from django-storages settings. | STRING |
| HIGH | scripts/tools/cleanup_data/s3_orphans.py | 0 | get boto3 s3 client and bucket name from django-storages settings. | STRING |
| HIGH | scripts/tools/cleanup_data/s3.py | 0 | get boto3 s3 client and bucket name from django-storages settings. | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | frontend/tests/controllers-test/challengeCtrl.test.js | 3140 | userKey = 'dummy-token'; | CODE |
| LOW | frontend/tests/controllers-test/challengeCtrl.test.js | 3377 | var userKey = "dummy-token"; | CODE |
| LOW | frontend/tests/controllers-test/challengeCtrl.test.js | 3419 | expect(opts.headers.Authorization).toBe("Token dummy-token"); | CODE |
| LOW | frontend/tests/controllers-test/challengeCtrl.test.js | 4042 | userKey = 'dummy-token'; | CODE |
| LOW | tests/unit/participants/test_views.py | 711 | leaderboard_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit", | CODE |
| LOW | tests/unit/participants/test_views.py | 711 | leaderboard_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit", | CODE |
| LOW | tests/unit/participants/test_views.py | 747 | leaderboard_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit", | CODE |
| LOW | tests/unit/participants/test_views.py | 747 | leaderboard_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit", | CODE |
| LOW | tests/unit/participants/test_views.py | 780 | leaderboard_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit", | CODE |
| LOW | tests/unit/participants/test_views.py | 780 | leaderboard_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit", | CODE |
| LOW | tests/unit/participants/test_views.py | 815 | leaderboard_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit", | CODE |
| LOW | tests/unit/participants/test_views.py | 815 | leaderboard_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit", | CODE |
| LOW | tests/unit/participants/test_views.py | 1201 | leaderboard_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit", | CODE |
| LOW | tests/unit/participants/test_views.py | 1201 | leaderboard_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit", | CODE |
| LOW | tests/unit/participants/test_views.py | 1789 | leaderboard_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit", | CODE |
| LOW | tests/unit/participants/test_views.py | 1789 | leaderboard_description="Lorem ipsum dolor sit amet, consectetur adipiscing elit", | CODE |
| LOW | tests/unit/challenges/test_admin.py | 285 | email="admin@example.com", | CODE |
| LOW | tests/unit/challenges/test_utils.py | 628 | profile.address_street = "123 Main St" | CODE |
| LOW⚡ | tests/unit/challenges/test_views.py | 545 | "leaderboard_description": "Lorem ipsum dolor sit amet, consectetur adipiscing elit.", # noqa: C0301 | CODE |
| LOW⚡ | tests/unit/challenges/test_views.py | 545 | "leaderboard_description": "Lorem ipsum dolor sit amet, consectetur adipiscing elit.", # noqa: C0301 | CODE |
| LOW⚡ | tests/unit/challenges/test_views.py | 7077 | "AWS_ACCESS_KEY_ID": "dummy-key", | CODE |
| LOW | tests/unit/challenges/test_views.py | 1471 | profile.address_street = "123 Main St" | CODE |
| LOW | tests/unit/challenges/test_views.py | 7042 | "AWS_ACCESS_KEY_ID": "dummy-key", | CODE |
| LOW⚡ | tests/unit/accounts/test_adapter.py | 96 | self.adapter.clean_email("admin@example.com") | CODE |
| LOW⚡ | tests/unit/accounts/test_throttles.py | 46 | self.assertIn("user@example.com", cache_key) | CODE |
| LOW⚡ | tests/unit/accounts/test_models.py | 41 | self.profile.address_street = "123 Main St" | CODE |
| LOW⚡ | tests/unit/accounts/test_models.py | 63 | self.profile.address_street = "123 Main St" | CODE |
| LOW⚡ | tests/unit/accounts/test_models.py | 76 | self.profile.address_street = "123 Main St" | CODE |
| LOW⚡ | tests/unit/accounts/test_models.py | 89 | self.profile.address_street = "123 Main St" | CODE |
| LOW | tests/unit/accounts/test_models.py | 102 | self.assertEqual(self.profile.get_full_name(), "John Doe") | CODE |
| LOW | tests/unit/jobs/test_serializer.py | 120 | email="test@test.com", | CODE |
| LOW⚡ | tests/unit/jobs/test_views.py | 2906 | "AWS_ACCESS_KEY_ID": "dummy-key", | CODE |
| LOW | tests/unit/jobs/test_views.py | 2869 | "AWS_ACCESS_KEY_ID": "dummy-key", | CODE |
| LOW | tests/unit/jobs/test_views.py | 2981 | "AWS_ACCESS_KEY_ID": "dummy-key", | CODE |
| LOW | tests/unit/jobs/test_views.py | 3017 | "AWS_ACCESS_KEY_ID": "dummy-key", | CODE |
| LOW | tests/unit/jobs/test_views.py | 3048 | "AWS_ACCESS_KEY_ID": "dummy-key", | CODE |
| LOW | tests/unit/jobs/test_views.py | 3113 | "AWS_ACCESS_KEY_ID": "dummy-key", | CODE |
| LOW | tests/unit/jobs/test_views.py | 3189 | "AWS_ACCESS_KEY_ID": "dummy-key", | CODE |
| LOW | tests/unit/jobs/test_views.py | 3256 | "AWS_ACCESS_KEY_ID": "dummy-key", | CODE |
| LOW | tests/unit/worker/test_code_upload_submission_worker.py | 835 | "aws_eks_bearer_token": "fake-token" | CODE |
| LOW | tests/unit/worker/test_code_upload_submission_worker.py | 850 | MockConfiguration().api_key["authorization"] = "fake-token" | CODE |
| LOW | tests/unit/worker/test_code_upload_submission_worker.py | 865 | "aws_eks_bearer_token": "fake-token" | CODE |
| LOW | tests/unit/worker/test_code_upload_submission_worker.py | 880 | MockConfiguration().api_key["authorization"] = "fake-token" | CODE |
| LOW | tests/unit/worker/test_code_upload_submission_worker.py | 897 | "aws_eks_bearer_token": "fake-token" | CODE |
| LOW | tests/unit/worker/test_code_upload_submission_worker.py | 912 | MockConfiguration().api_key["authorization"] = "fake-token" | CODE |
| LOW | tests/unit/worker/test_worker_utils.py | 11 | self.AUTH_TOKEN = "dummy_token" | CODE |
| LOW | tests/unit/worker/test_worker_utils.py | 23 | self.assertEqual(headers, {"Authorization": "Bearer dummy_token"}) | CODE |
| LOW⚡ | tests/unit/monitoring/test_evalai_interface.py | 11 | self.AUTH_TOKEN = "dummy_token" | CODE |
| LOW⚡ | tests/unit/monitoring/test_evalai_interface.py | 17 | self.assertEqual(headers, {"Authorization": "Bearer dummy_token"}) | CODE |
| LOW⚡ | tests/unit/monitoring/test_evalai_interface.py | 24 | "Authorization": "Bearer dummy_token", | CODE |
| LOW | examples/example2/test_zip_file/evaluation_details.html | 2 | <p><span style="font-weight: 400;">Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lo | CODE |
| LOW | …xample2/test_zip_file/challenge_phase_description.html | 2 | <p><span style="font-weight: 400;">Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lo | CODE |
| LOW | …ples/example2/test_zip_file/submission_guidelines.html | 2 | <p><span style="font-weight: 400;">Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lo | CODE |
| LOW | examples/example2/test_zip_file/description.html | 2 | <p><span style="font-weight: 400;">Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lo | CODE |
| LOW | …mples/example2/test_zip_file/terms_and_conditions.html | 2 | <p><span style="font-weight: 400;">Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lo | CODE |
| LOW | examples/example4/test_zip_file/evaluation_details.html | 2 | <p><span style="font-weight: 400;">Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lo | CODE |
| LOW | …xample4/test_zip_file/challenge_phase_description.html | 2 | <p><span style="font-weight: 400;">Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lo | CODE |
| LOW | …ples/example4/test_zip_file/submission_guidelines.html | 2 | <p><span style="font-weight: 400;">Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lo | CODE |
| LOW | examples/example4/test_zip_file/description.html | 2 | <p><span style="font-weight: 400;">Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lo | CODE |
| LOW | …mples/example4/test_zip_file/terms_and_conditions.html | 2 | <p><span style="font-weight: 400;">Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lo | CODE |
| 7 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| CRITICAL | tests/unit/participants/test_utils.py | 5 | from django.core.exceptions import FieldError | CODE |
| CRITICAL⚡ | tests/unit/accounts/test_adapter.py | 7 | from django.core.exceptions import ValidationError | CODE |
| CRITICAL | apps/challenges/serializers.py | 9 | from django.core.exceptions import ValidationError as DjangoValidationError | CODE |
| CRITICAL | apps/challenges/utils.py | 16 | from django.core.exceptions import ValidationError | CODE |
| CRITICAL | apps/accounts/adapter.py | 8 | from django.core.exceptions import ValidationError | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH⚡ | tests/unit/participants/test_views.py | 1371 | """Participant team with no challenges returns (null, team) entry.""" | STRING |
| HIGH | tests/unit/worker/test_code_upload_submission_worker.py | 321 | "apt update && apt install -y curl && curl_command", | CODE |
| HIGH | tests/unit/worker/test_code_upload_submission_worker.py | 631 | "apt update && apt install -y curl && sh /evalai_scripts/monitor_submission.sh", | CODE |
| HIGH | scripts/seed.py | 69 | "To force re-seed, run: python manage.py flush && python manage.py seed" | CODE |
| HIGH⚡ | scripts/tools/list_ongoing_challenges_s3_usage.py | 133 | and (end_date >= now or end_date is null). | STRING |
| HIGH⚡ | scripts/tools/list_ongoing_challenges_s3_usage.py | 136 | "Fetching present/ongoing challenges (approved_by_admin, start_date <= now, end_date >= now or null)..." | CODE |
| HIGH | scripts/workers/code_upload_submission_worker.py | 182 | "apt update && apt install -y curl && {}".format(curl_request), | CODE |
| HIGH | scripts/workers/code_upload_submission_worker.py | 360 | "apt update && apt install -y curl && sh /evalai_scripts/monitor_submission.sh", | CODE |
| HIGH | apps/jobs/views.py | 812 | "next": null, | STRING |
| HIGH | apps/jobs/views.py | 813 | "previous": null, | STRING |
| HIGH | apps/jobs/views.py | 826 | "error": null, | STRING |
| HIGH | apps/jobs/views.py | 836 | "submission__submission_metadata": null, | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/unit/challenges/test_views.py | 5173 | CODE | |
| LOW | tests/unit/challenges/test_views.py | 6054 | CODE | |
| LOW | …ripts/tools/cleanup_data/orphaned_challenge_configs.py | 237 | CODE | |
| LOW | …ripts/tools/cleanup_data/orphaned_challenge_configs.py | 292 | CODE | |
| LOW | scripts/tools/cleanup_data/set_challenge_end_date.py | 106 | CODE | |
| LOW | scripts/tools/cleanup_data/s3_orphans.py | 299 | CODE | |
| LOW | scripts/tools/cleanup_data/s3_orphans.py | 386 | CODE | |
| LOW | scripts/tools/cleanup_data/s3_orphans.py | 426 | CODE | |
| LOW | scripts/tools/cleanup_data/s3.py | 234 | CODE | |
| LOW | scripts/workers/code_upload_submission_worker.py | 583 | CODE | |
| LOW | scripts/workers/code_upload_submission_worker.py | 707 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 343 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 589 | CODE | |
| LOW | scripts/workers/submission_worker.py | 150 | CODE | |
| LOW | scripts/workers/submission_worker.py | 223 | CODE | |
| LOW | scripts/workers/submission_worker.py | 254 | CODE | |
| LOW | scripts/workers/submission_worker.py | 483 | CODE | |
| LOW | scripts/workers/submission_worker.py | 932 | CODE | |
| LOW | scripts/monitoring/auto_stop_workers.py | 72 | CODE | |
| LOW | apps/participants/views.py | 143 | CODE | |
| LOW | apps/participants/views.py | 202 | CODE | |
| LOW | apps/participants/views.py | 596 | CODE | |
| LOW | apps/challenges/aws_utils.py | 1366 | CODE | |
| LOW | apps/challenges/aws_utils.py | 1492 | CODE | |
| LOW | apps/challenges/aws_utils.py | 2277 | CODE | |
| LOW | apps/challenges/aws_utils.py | 2794 | CODE | |
| LOW | apps/challenges/utils.py | 46 | CODE | |
| LOW | apps/challenges/utils.py | 816 | CODE | |
| LOW | apps/challenges/permissions.py | 14 | CODE | |
| LOW | apps/challenges/challenge_config_utils.py | 664 | CODE | |
| LOW | apps/challenges/challenge_config_utils.py | 741 | CODE | |
| LOW | apps/challenges/challenge_config_utils.py | 818 | CODE | |
| LOW | apps/challenges/challenge_config_utils.py | 990 | CODE | |
| LOW | apps/challenges/challenge_config_utils.py | 1162 | CODE | |
| LOW | apps/challenges/views.py | 241 | CODE | |
| LOW | apps/challenges/views.py | 379 | CODE | |
| LOW | apps/challenges/views.py | 1212 | CODE | |
| LOW | apps/challenges/views.py | 1324 | CODE | |
| LOW | apps/challenges/views.py | 2485 | CODE | |
| LOW | apps/challenges/views.py | 3333 | CODE | |
| LOW | apps/challenges/views.py | 3415 | CODE | |
| LOW | apps/challenges/views.py | 3873 | CODE | |
| LOW | apps/challenges/views.py | 4272 | CODE | |
| LOW | apps/challenges/views.py | 4359 | CODE | |
| LOW | apps/challenges/views.py | 5212 | CODE | |
| LOW | apps/scout/views.py | 18 | CODE | |
| LOW | apps/accounts/serializers.py | 129 | CODE | |
| LOW | …accounts/management/commands/merge_duplicate_emails.py | 135 | CODE | |
| LOW | apps/jobs/s3_retention.py | 207 | CODE | |
| LOW | apps/jobs/views.py | 149 | CODE | |
| LOW | apps/jobs/views.py | 455 | CODE | |
| LOW | apps/jobs/views.py | 1057 | CODE | |
| LOW | apps/jobs/views.py | 1267 | CODE | |
| LOW | apps/jobs/views.py | 1748 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scripts/workers/worker_utils.py | 54 | CODE | |
| LOW | scripts/workers/worker_utils.py | 61 | CODE | |
| LOW | scripts/workers/worker_utils.py | 67 | CODE | |
| LOW | scripts/workers/worker_utils.py | 75 | CODE | |
| LOW | scripts/workers/worker_utils.py | 81 | CODE | |
| LOW | scripts/workers/worker_utils.py | 89 | CODE | |
| LOW | scripts/workers/worker_utils.py | 95 | CODE | |
| LOW | scripts/workers/worker_utils.py | 101 | CODE | |
| LOW | scripts/workers/worker_utils.py | 107 | CODE | |
| LOW | scripts/workers/worker_utils.py | 113 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 410 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 419 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 426 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 433 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 440 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 449 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 456 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 463 | CODE | |
| LOW | scripts/monitoring/monitor_containers.py | 87 | CODE | |
| LOW | scripts/monitoring/monitor_containers.py | 102 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 63 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 70 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 76 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 84 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 90 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 98 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 104 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 110 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 116 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 122 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 128 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 134 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 143 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 149 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 156 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 163 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 170 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 177 | CODE | |
| LOW | scripts/monitoring/evalai_interface.py | 185 | CODE | |
| LOW | scripts/monitoring/auto_stop_workers.py | 28 | CODE | |
| LOW | scripts/monitoring/auto_stop_workers.py | 36 | CODE | |
| LOW | scripts/monitoring/auto_stop_workers.py | 48 | CODE | |
| LOW | scripts/monitoring/auto_scale_eks_nodes.py | 101 | CODE | |
| LOW | scripts/monitoring/auto_scale_eks_nodes.py | 118 | CODE | |
| LOW | apps/challenges/utils.py | 513 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scripts/workers/worker_utils.py | 5 | logger = logging.getLogger(__name__) | CODE |
| LOW | scripts/workers/code_upload_submission_worker.py | 35 | logger = logging.getLogger(__name__) | CODE |
| LOW | scripts/workers/remote_submission_worker.py | 28 | logger = logging.getLogger(__name__) | CODE |
| LOW | scripts/workers/submission_worker.py | 65 | logger = logging.getLogger(__name__) | CODE |
| LOW | scripts/monitoring/evalai_interface.py | 5 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/web/views.py | 22 | logger = logging.getLogger(__name__) | CODE |
| LOW⚡ | apps/challenges/challenge_notification_util.py | 5 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/challenges/aws_utils.py | 46 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/challenges/utils.py | 33 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/challenges/challenge_config_utils.py | 38 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/challenges/views.py | 178 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/scout/tasks.py | 10 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/scout/views.py | 13 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/accounts/tasks.py | 10 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/accounts/adapter.py | 10 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/accounts/bounce_handler.py | 11 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/accounts/views.py | 34 | logger = logging.getLogger(__name__) | CODE |
| LOW | …accounts/management/commands/merge_duplicate_emails.py | 18 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/jobs/sender.py | 17 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/jobs/tasks.py | 20 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/jobs/models.py | 19 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/jobs/admin.py | 18 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/jobs/utils.py | 29 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/jobs/s3_retention.py | 10 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/jobs/views.py | 98 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/base/models.py | 8 | logger = logging.getLogger(__name__) | CODE |
| LOW | apps/base/utils.py | 26 | logger = logging.getLogger(__name__) | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | manage.py | 14 | CODE | |
| LOW | settings/test.py | 1 | CODE | |
| LOW | settings/dev.py | 3 | CODE | |
| LOW | settings/prod.py | 7 | CODE | |
| LOW | settings/staging.py | 1 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 1 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 1 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 1 | CODE | |
| LOW | scripts/workers/remote_submission_worker.py | 1 | CODE | |
| LOW | scripts/workers/submission_worker.py | 1 | CODE | |
| LOW | scripts/workers/submission_worker.py | 1 | CODE | |
| LOW | scripts/workers/submission_worker.py | 1 | CODE | |
| LOW | scripts/workers/submission_worker.py | 1 | CODE | |
| LOW | apps/participants/models.py | 1 | CODE | |
| LOW | apps/participants/apps.py | 1 | CODE | |
| LOW | apps/web/models.py | 1 | CODE | |
| LOW | apps/web/apps.py | 1 | CODE | |
| LOW | apps/challenges/models.py | 1 | CODE | |
| LOW | apps/challenges/apps.py | 1 | CODE | |
| LOW | apps/scout/models.py | 1 | CODE | |
| LOW | apps/scout/apps.py | 1 | CODE | |
| LOW | apps/accounts/models.py | 1 | CODE | |
| LOW | apps/accounts/apps.py | 1 | CODE | |
| LOW | apps/hosts/models.py | 1 | CODE | |
| LOW | apps/hosts/apps.py | 1 | CODE | |
| LOW | apps/jobs/sender.py | 1 | CODE | |
| LOW | apps/jobs/models.py | 1 | CODE | |
| LOW | apps/jobs/apps.py | 1 | CODE | |
| LOW | apps/base/models.py | 1 | CODE | |
| LOW | apps/base/apps.py | 1 | CODE | |
| LOW | apps/analytics/apps.py | 1 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 3 | # Step 1: Updating package repository | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 7 | # Step 2: Cloning EvalAI repository at /home/ubuntu | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 13 | # Step 3: Installing awscli | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 17 | # Step 4: Installing Docker Compose plugin (v2) | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 22 | # Step 5: Adding user to docker group | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 28 | # Step 6: Configuring AWS credentials | COMMENT |
| LOW | scripts/deployment/deploy_ec2_worker.sh | 39 | # Step 7: Copying Docker environment file | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 57 | # Step 8: Pulling worker Docker image | COMMENT |
| LOW⚡ | scripts/deployment/deploy_ec2_worker.sh | 67 | # Step 9: Running worker Docker container | COMMENT |
| LOW | scripts/deployment/deploy_ec2_worker.sh | 85 | # Step 10: Setting up crontab | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | scripts/tools/cleanup_data/s3_orphans.py | 88 | # ============================================================================= | COMMENT |
| MEDIUM | scripts/tools/cleanup_data/s3_orphans.py | 90 | # ============================================================================= | COMMENT |
| MEDIUM | scripts/tools/cleanup_data/s3_orphans.py | 107 | # ============================================================================= | COMMENT |
| MEDIUM | scripts/tools/cleanup_data/s3.py | 54 | # ============================================================================= | COMMENT |
| MEDIUM | scripts/tools/cleanup_data/s3.py | 56 | # ============================================================================= | COMMENT |
| MEDIUM | scripts/tools/cleanup_data/s3.py | 76 | # ============================================================================= | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | .coderabbit.yaml | 1 | # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json | COMMENT |
| LOW | frontend/src/css/modules/error_404.scss | 101 | circle, | COMMENT |
| LOW | docs/source/conf.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | docs/source/conf.py | 81 | # for a list of supported languages. | COMMENT |
| LOW | docs/source/conf.py | 101 | # The reST default role (used for this markup: `text`) to use for all | COMMENT |
| LOW | docs/source/conf.py | 141 | # further. For a list of options available for each theme, see the | COMMENT |
| LOW | docs/source/conf.py | 161 | # html_logo = None | COMMENT |
| LOW | docs/source/conf.py | 181 | # bottom, using the given strftime format. | COMMENT |
| LOW | docs/source/conf.py | 201 | # | COMMENT |
| LOW | docs/source/conf.py | 221 | # | COMMENT |
| LOW | docs/source/conf.py | 241 | # 'ja' uses this config value. | COMMENT |
| LOW | docs/source/conf.py | 261 | # | COMMENT |
| LOW | docs/source/conf.py | 281 | ) | COMMENT |
| LOW | docs/source/conf.py | 301 | COMMENT | |
| LOW | docs/source/conf.py | 341 | "Miscellaneous", | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | apps/challenges/views.py | 3293 | Endpoint to generate AWS Credentails for CLI Args: - challenge: Challenge model - participant_t | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/unit/hosts/test_views.py | 251 | # TODO: Add test back with API | COMMENT |
| LOW | scripts/workers/code_upload_submission_worker.py | 11 | # TODO: Add exception in all the commands | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scripts/monitoring/auto_cancel_submissions.py | 63 | # Example usage: | COMMENT |