AI Slop Code RIsk Monitor

Unstructured-IO/unstructured

Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.

8.9 Moderate pattern signal View on GitHub

Maintenance and Governance Review

This report summarizes scanner evidence for Unstructured-IO/unstructured, a HTML project with 15,557 GitHub stars. SynthScan examined 476,874 lines of code across 1048 source files, recording 3592 pattern matches distributed across 20 syntactic categories. The legacy adjusted pattern signal is 8.9 (Moderate pattern signal); it is not an authorship estimate.

Use the findings to ask whether maintainers can explain intent, verify dependencies, simplify unnecessary scaffolding, and focus review effort. Risk Payload v1 adds ownership, supply-chain, over-engineering, and review-effort vectors when available. These are prompts for human verification, not probabilities, security findings, or measured maintenance costs.

8.9
Adjusted Signal
8.9
Raw Pattern Score
100%
Time Factor
2026-10-10
Last Push
15.6K
Stars
HTML
Language
476.9K
Lines of Code
1.0K
Files
3.6K
Pattern Hits
2026-10-12
Scan Date
0.04
HC Hit Rate

Risk Review · v3

Use each vector to guide manual verification of the underlying findings.

Cognitive Ownership Risk 22.1/100 · LOW

Can maintainers explain intent, assumptions, and trade-offs? 2616 finding(s).

Hallucinated Dependency & Supply Chain Risk 5.8/100 · LOW

Are imports declared and package/API names verified? 591 finding(s).

Agent Over-Engineering Ratio 29.9/100 · MEDIUM

Are wrappers, guards, and repeated structures necessary? 2781 finding(s).

PR Review Velocity Impact 37.3/100 · MEDIUM

Which changes need focused human review? 3592 finding(s).

What These Metrics Mean

Adjusted Signal
Legacy pattern score normalized per 1,000 scanned lines and multiplied by the repository-level time factor. It is not an estimate of AI authorship.
Raw Pattern Score
Sum of severity- and context-weighted pattern findings before applying the time factor.
Time Factor
A coarse multiplier based on repository last-push metadata. It does not identify when individual code was written.
Pattern Hits
Count of configured pattern findings. Review each location and surrounding code before acting.
HC Hit Rate
High- and critical-severity findings per scanned file. Severity is a configured detector weight, not proof of a vulnerability or authorship.
Lines / Files
Scan coverage denominators used by the legacy normalized score.

Pattern-Signal History

This chart tracks the legacy adjusted pattern score across scan runs. Changes can reflect source changes, scan coverage, or detector/scoring versions; they do not measure AI adoption or prove that a team refactored code. Compare runs only when their scoring versions are compatible. The dashed line tracks the raw pattern-hit count.

Severity Breakdown

Groups findings by configured severity weight. Severity is not a confidence estimate, security rating, or proof that a package is missing. Verify imports against the project's manifests and inspect each finding in its local design context.

CRITICAL 0HIGH 40MEDIUM 330LOW 3222

Directory Score Breakdown

This horizontal bar chart shows where configured pattern scores accumulate across directories. Treat the bars as locations to inspect, not as evidence of authorship or code quality. Use the file-level findings to decide whether a directory needs a focused ownership, dependency, complexity, or review check.

Pattern Findings

The scanner identified 3592 distinct pattern matches across 20 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a detector match to inspect in context. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the configured finding weight (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — legacy comment-context matches receive a ×1.5 score weight. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier under the legacy scoring model.

Hyper-Verbose Identifiers2375 hits · 2078 pts
SeverityFileLineSnippetContext
LOW⚡test_unstructured/test_safe_http.py88 def test_allows_public_addresses(self, ip: str):CODE
LOW⚡test_unstructured/test_safe_http.py91 def test_fails_closed_on_unparseable_input(self):CODE
LOW⚡test_unstructured/test_safe_http.py102 def test_returns_lowercase_for_ascii_input(self):CODE
LOW⚡test_unstructured/test_safe_http.py111 def test_idn_collapses_to_punycode(self):CODE
LOW⚡test_unstructured/test_safe_http.py132 def test_rejects_non_http_scheme(self, url: str):CODE
LOW⚡test_unstructured/test_safe_http.py136 def test_rejects_missing_hostname(self):CODE
LOW⚡test_unstructured/test_safe_http.py145 def test_rejects_out_of_range_port(self, url: str):CODE
LOW⚡test_unstructured/test_safe_http.py177 def test_rejects_blocked_ip_literals(self, url: str):CODE
LOW⚡test_unstructured/test_safe_http.py182 def test_rejects_hostname_resolving_to_non_routable(self, mock_getaddrinfo):CODE
LOW⚡test_unstructured/test_safe_http.py188 def test_allows_public_hostname(self, mock_getaddrinfo):CODE
LOW⚡test_unstructured/test_safe_http.py193 def test_rejects_when_any_resolved_ip_is_blocked(self, mock_getaddrinfo):CODE
LOW⚡test_unstructured/test_safe_http.py201 def test_skips_validation_with_allow_private(self):CODE
LOW⚡test_unstructured/test_safe_http.py204 def test_skips_validation_with_env_var(self, monkeypatch):CODE
LOW⚡test_unstructured/test_safe_http.py216 def test_rejects_blocked_resolved_address_at_connect(self, mock_getaddrinfo):CODE
LOW⚡test_unstructured/test_safe_http.py223 def test_connects_to_validated_public_address(self, mock_getaddrinfo, MockSocket):CODE
LOW⚡test_unstructured/test_safe_http.py245 def test_returns_response_for_public_url(self, MockSession, mock_getaddrinfo):CODE
LOW⚡test_unstructured/test_safe_http.py375 def test_rejects_proxies_kwarg(self, mock_getaddrinfo):CODE
LOW⚡test_unstructured/test_safe_http.py389 def test_mounts_safe_adapter_and_disables_trust_env(self, MockSession, mock_getaddrinfo):CODE
LOW⚡test_unstructured/test_safe_http.py420 def test_proxy_manager_is_refused(self):CODE
LOW⚡test_unstructured/test_safe_http.py432 def test_same_origin_not_cross(self):CODE
LOW⚡test_unstructured/test_safe_http.py435 def test_different_host_is_cross(self):CODE
LOW⚡test_unstructured/test_safe_http.py438 def test_http_to_https_upgrade_is_not_cross(self):CODE
LOW⚡test_unstructured/test_safe_http.py441 def test_downgrade_on_same_nondefault_port_is_cross(self):CODE
LOW⚡test_unstructured/test_safe_http.py444 def test_explicit_default_port_same_origin_not_cross(self):CODE
LOW⚡test_unstructured/test_safe_http.py447 def test_unparseable_port_fails_safe(self):CODE
LOW⚡test_unstructured/test_safe_http.py452 def test_removes_credential_headers(self):CODE
LOWtest_unstructured/test_safe_http.py74 def test_blocks_sixtofour_embedded(self):CODE
LOWtest_unstructured/test_safe_http.py162 def test_rejects_blocked_hostname(self, hostname: str):CODE
LOWtest_unstructured/test_safe_http.py260 def test_applies_default_timeout(self, MockSession, mock_getaddrinfo):CODE
LOWtest_unstructured/test_safe_http.py272 def test_respects_explicit_timeout(self, MockSession, mock_getaddrinfo):CODE
LOWtest_unstructured/test_safe_http.py284 def test_forces_allow_redirects_false(self, MockSession, mock_getaddrinfo):CODE
LOWtest_unstructured/test_safe_http.py296 def test_validates_redirect_targets(self, MockSession, mock_getaddrinfo):CODE
LOWtest_unstructured/test_safe_http.py312 def test_follows_valid_redirect_chain(self, MockSession, mock_getaddrinfo):CODE
LOWtest_unstructured/test_safe_http.py329 def test_enforces_max_redirects(self, MockSession, mock_getaddrinfo):CODE
LOWtest_unstructured/test_safe_http.py342 def test_returns_response_on_missing_location_header(self, MockSession, mock_getaddrinfo):CODE
LOWtest_unstructured/test_safe_http.py353 def test_rejects_non_routable_url_without_fetch(self):CODE
LOWtest_unstructured/test_safe_http.py359 def test_session_closed_on_validation_error(self, MockSession, mock_getaddrinfo):CODE
LOWtest_unstructured/test_safe_http.py403 def test_allow_private_skips_adapter_and_permits_proxies(self, MockSession):CODE
LOWtest_unstructured/test_safe_http.py464 def test_removes_auth_and_cookies_kwargs(self):CODE
LOWtest_unstructured/test_safe_http.py480 def test_strips_credentials_on_cross_origin_redirect(self, MockSession, mock_getaddrinfo):CODE
LOW⚡test_unstructured/test_utils.py46def test_requires_dependencies_decorator():CODE
LOW⚡test_unstructured/test_utils.py54def test_requires_dependencies_decorator_multiple():CODE
LOW⚡test_unstructured/test_utils.py63def test_requires_dependencies_decorator_import_error():CODE
LOW⚡test_unstructured/test_utils.py72def test_requires_dependencies_decorator_import_error_multiple():CODE
LOW⚡test_unstructured/test_utils.py82def test_requires_dependencies_decorator_in_class():CODE
LOW⚡test_unstructured/test_utils.py315def test_only_returns_singleton_iterable():CODE
LOW⚡test_unstructured/test_utils.py321def test_only_raises_on_non_singleton_iterable():CODE
LOW⚡test_unstructured/test_utils.py327def test_calculate_shared_ngram_percentage_returns_null_vals_for_empty_str():CODE
LOWtest_unstructured/test_utils.py97def test_first_raises_if_empty(iterator):CODE
LOWtest_unstructured/test_utils.py114def test_only_raises_if_empty(iterator):CODE
LOWtest_unstructured/test_utils.py265def test_catch_overlapping_and_nested_bboxes(CODE
LOWtest_unstructured/test_utils.py291def test_catch_overlapping_and_nested_bboxes_non_overlapping_case():CODE
LOWtest_unstructured/test_utils.py339 def it_groups_elements_by_parent_id_with_orphans_in_none_group(self):CODE
LOWtest_unstructured/test_utils.py357 def it_assigns_orphans_to_previous_element_group_when_assign_orphans_is_true(self):CODE
LOWtest_unstructured/test_utils.py374 def it_keeps_first_orphan_in_none_group_when_assign_orphans_is_true(self):CODE
LOW⚡test_unstructured/test_runtime_telemetry.py164def test_caller_process_control_exception_is_not_swallowed(monkeypatch):CODE
LOW⚡test_unstructured/test_runtime_telemetry.py172def test_delivery_failure_is_contained_and_releases_slot(runtime_session):CODE
LOW⚡test_unstructured/test_runtime_telemetry.py182def test_delivery_disables_environment_redirects_retries_and_body_download(CODE
LOW⚡test_unstructured/test_runtime_telemetry.py466def test_structured_remote_or_serialized_table_does_not_claim_local_extraction(captured_events):CODE
LOW⚡test_unstructured/test_runtime_telemetry.py475def test_structured_native_table_reports_local_extraction(captured_events):CODE
2315 more matches not shown…
Decorative Section Separators296 hits · 1062 pts
SeverityFileLineSnippetContext
MEDIUM…ingest/src/local-single-file-chunk-no-orig-elements.sh3# ------------------------------------------------------------------------------------------------COMMENT
MEDIUM…ingest/src/local-single-file-chunk-no-orig-elements.sh7# ------------------------------------------------------------------------------------------------COMMENT
MEDIUM⚡test_unstructured/test_safe_http.py96# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡test_unstructured/test_safe_http.py98# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡test_unstructured/test_safe_http.py116# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡test_unstructured/test_safe_http.py118# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡test_unstructured/test_safe_http.py209# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡test_unstructured/test_safe_http.py211# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡test_unstructured/test_safe_http.py237# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡test_unstructured/test_safe_http.py239# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡test_unstructured/test_safe_http.py381# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡test_unstructured/test_safe_http.py383# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡test_unstructured/test_safe_http.py426# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡test_unstructured/test_safe_http.py428# ---------------------------------------------------------------------------COMMENT
MEDIUMtest_unstructured/test_safe_http.py22# ---------------------------------------------------------------------------COMMENT
MEDIUMtest_unstructured/test_safe_http.py24# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡test_unstructured/unit_utils.py111# ------------------------------------------------------------------------------------------------COMMENT
MEDIUM⚡test_unstructured/unit_utils.py113# ------------------------------------------------------------------------------------------------COMMENT
MEDIUM⚡test_unstructured/unit_utils.py116# ------------------------------------------------------------------------------------------------COMMENT
MEDIUMtest_unstructured/chunking/test_basic.py224# ------------------------------------------------------------------------------------------------COMMENT
MEDIUMtest_unstructured/chunking/test_basic.py226# ------------------------------------------------------------------------------------------------COMMENT
MEDIUM⚡test_unstructured/chunking/test_title.py30# ================================================================================================COMMENT
MEDIUM⚡test_unstructured/chunking/test_title.py32# ================================================================================================COMMENT
MEDIUM⚡test_unstructured/chunking/test_title.py35# ================================================================================================COMMENT
MEDIUM⚡test_unstructured/chunking/test_title.py546# ================================================================================================COMMENT
MEDIUM⚡test_unstructured/chunking/test_title.py548# ================================================================================================COMMENT
MEDIUM⚡test_unstructured/chunking/test_title.py551# ================================================================================================COMMENT
MEDIUM⚡test_unstructured/chunking/test_title.py705# ================================================================================================COMMENT
MEDIUM⚡test_unstructured/chunking/test_title.py707# ================================================================================================COMMENT
MEDIUM⚡test_unstructured/chunking/test_base.py55# ================================================================================================COMMENT
MEDIUM⚡test_unstructured/chunking/test_base.py57# ================================================================================================COMMENT
MEDIUM⚡test_unstructured/chunking/test_base.py329# ================================================================================================COMMENT
MEDIUM⚡test_unstructured/chunking/test_base.py331# ================================================================================================COMMENT
MEDIUM⚡test_unstructured/chunking/test_base.py398# ================================================================================================COMMENT
MEDIUM⚡test_unstructured/chunking/test_base.py400# ================================================================================================COMMENT
MEDIUM⚡test_unstructured/chunking/test_base.py546# ================================================================================================COMMENT
MEDIUM⚡test_unstructured/chunking/test_base.py548# ================================================================================================COMMENT
MEDIUM⚡test_unstructured/chunking/test_base.py1017# ================================================================================================COMMENT
MEDIUM⚡test_unstructured/chunking/test_base.py1019# ================================================================================================COMMENT
MEDIUM⚡test_unstructured/chunking/test_base.py3318# ================================================================================================COMMENT
MEDIUM⚡test_unstructured/chunking/test_base.py3320# ================================================================================================COMMENT
MEDIUM⚡test_unstructured/chunking/test_base.py5671# ================================================================================================COMMENT
MEDIUM⚡test_unstructured/chunking/test_base.py5673# ================================================================================================COMMENT
MEDIUMtest_unstructured/chunking/test_base.py760# ================================================================================================COMMENT
MEDIUMtest_unstructured/chunking/test_base.py762# ================================================================================================COMMENT
MEDIUMtest_unstructured/chunking/test_base.py5944# ================================================================================================COMMENT
MEDIUMtest_unstructured/chunking/test_base.py5946# ================================================================================================COMMENT
MEDIUM⚡test_unstructured/partition/test_ndjson.py305# ------------------------------------------------------------------------------------------------COMMENT
MEDIUM⚡test_unstructured/partition/test_xml.py175# ------------------------------------------------------------------------------------------------COMMENT
MEDIUM⚡test_unstructured/partition/test_docx.py116# ------------------------------------------------------------------------------------------------COMMENT
MEDIUM⚡test_unstructured/partition/test_docx.py334# ------------------------------------------------------------------------------------------------COMMENT
MEDIUM⚡test_unstructured/partition/test_docx.py890# ------------------------------------------------------------------------------------------------COMMENT
MEDIUM⚡test_unstructured/partition/test_docx.py1495# ================================================================================================COMMENT
MEDIUM⚡test_unstructured/partition/test_docx.py1497# ================================================================================================COMMENT
MEDIUM⚡test_unstructured/partition/test_docx.py1500# ================================================================================================COMMENT
MEDIUMtest_unstructured/partition/test_docx.py1256# ------------------------------------------------------------------------------------------------COMMENT
MEDIUMtest_unstructured/partition/test_docx.py1933 # ┌───────┐COMMENT
MEDIUMtest_unstructured/partition/test_docx.py1963 # ┌───────┬───┬───┐COMMENT
MEDIUMtest_unstructured/partition/test_docx.py1971 # └───────┘COMMENT
MEDIUM⚡test_unstructured/partition/test_email.py460# ================================================================================================COMMENT
236 more matches not shown…
Fake / Example Data432 hits · 411 pts
SeverityFileLineSnippetContext
LOWREADME.md135elements = partition(filename="example-docs/eml/fake-email.eml")CODE
LOW…t-with-permissions/SitePages/This-is-a-title.aspx.json49 "text": "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam ex tellus, sodales non nulla et, sodales cCODE
LOW…t-with-permissions/SitePages/This-is-a-title.aspx.json49 "text": "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam ex tellus, sodales non nulla et, sodales cCODE
LOW⚡…airtable-diff/appJ43QmP8I17zu88/tblbj2vBlL2dN2xqq.json20 "text_as_html": "<table border=\"1\" class=\"dataframe\">\n <tbody>\n <tr>\n <td></td>\n <td>AssigneCODE
LOW⚡…airtable-diff/appJ43QmP8I17zu88/tblbj2vBlL2dN2xqq.json20 "text_as_html": "<table border=\"1\" class=\"dataframe\">\n <tbody>\n <tr>\n <td></td>\n <td>AssigneCODE
LOW⚡…airtable-diff/appJ43QmP8I17zu88/tblbj2vBlL2dN2xqq.json22 "text": "\n\n\n\nAssignee\nCheckboxes\nDates\nName\nNotes 1\nStatus\n\n\n0.0\n{'id': 'usrNdSxRQjglQU4eR', 'email': 'CODE
LOW⚡…airtable-diff/appJ43QmP8I17zu88/tblbj2vBlL2dN2xqq.json22 "text": "\n\n\n\nAssignee\nCheckboxes\nDates\nName\nNotes 1\nStatus\n\n\n0.0\n{'id': 'usrNdSxRQjglQU4eR', 'email': 'CODE
LOW⚡…airtable-diff/appJ43QmP8I17zu88/tblfu7DzEcCWNKwP4.json20 "text_as_html": "<table border=\"1\" class=\"dataframe\">\n <tbody>\n <tr>\n <td></td>\n <td>AssigneCODE
LOW⚡…airtable-diff/appJ43QmP8I17zu88/tblfu7DzEcCWNKwP4.json20 "text_as_html": "<table border=\"1\" class=\"dataframe\">\n <tbody>\n <tr>\n <td></td>\n <td>AssigneCODE
LOW⚡…airtable-diff/appJ43QmP8I17zu88/tblfu7DzEcCWNKwP4.json22 "text": "\n\n\n\nAssignee\nCheckboxes\nDates\nName\nNotes 1\nStatus\n\n\n0.0\n{'id': 'usrNdSxRQjglQU4eR', 'email': 'CODE
LOW⚡…airtable-diff/appJ43QmP8I17zu88/tblfu7DzEcCWNKwP4.json22 "text": "\n\n\n\nAssignee\nCheckboxes\nDates\nName\nNotes 1\nStatus\n\n\n0.0\n{'id': 'usrNdSxRQjglQU4eR', 'email': 'CODE
LOW⚡…airtable-diff/app5YQxSfp220fWtm/tblxdPc7L2meGIZLE.json21 "text_as_html": "<table border=\"1\" class=\"dataframe\">\n <tbody>\n <tr>\n <td></td>\n <td>AssigneCODE
LOW⚡…airtable-diff/app5YQxSfp220fWtm/tblxdPc7L2meGIZLE.json21 "text_as_html": "<table border=\"1\" class=\"dataframe\">\n <tbody>\n <tr>\n <td></td>\n <td>AssigneCODE
LOW⚡…airtable-diff/app5YQxSfp220fWtm/tblxdPc7L2meGIZLE.json23 "text": "\n\n\n\nAssignee\nCheckboxes\nDates\nName\nNotes 1\nStatus\n\n\n0.0\n\n\n2023-08-26T14:00:00.000Z\nName3\n#CODE
LOW⚡…airtable-diff/app5YQxSfp220fWtm/tblxdPc7L2meGIZLE.json23 "text": "\n\n\n\nAssignee\nCheckboxes\nDates\nName\nNotes 1\nStatus\n\n\n0.0\n\n\n2023-08-26T14:00:00.000Z\nName3\n#CODE
LOW⚡…airtable-diff/app5YQxSfp220fWtm/tblBoUk54tWXGqYai.json20 "text_as_html": "<table border=\"1\" class=\"dataframe\">\n <tbody>\n <tr>\n <td></td>\n <td>AssigneCODE
LOW⚡…airtable-diff/app5YQxSfp220fWtm/tblBoUk54tWXGqYai.json20 "text_as_html": "<table border=\"1\" class=\"dataframe\">\n <tbody>\n <tr>\n <td></td>\n <td>AssigneCODE
LOW⚡…airtable-diff/app5YQxSfp220fWtm/tblBoUk54tWXGqYai.json22 "text": "\n\n\n\nAssignee\nCheckboxes\nDates\nName\nNotes 1\nStatus\n\n\n0.0\n{'id': 'usrNdSxRQjglQU4eR', 'email': 'CODE
LOW⚡…airtable-diff/app5YQxSfp220fWtm/tblBoUk54tWXGqYai.json22 "text": "\n\n\n\nAssignee\nCheckboxes\nDates\nName\nNotes 1\nStatus\n\n\n0.0\n{'id': 'usrNdSxRQjglQU4eR', 'email': 'CODE
LOW…expected-structured-output/google-drive/fake.docx.json5 "text": "Lorem ipsum dolor sit amet.",CODE
LOW…expected-structured-output/google-drive/fake.docx.json5 "text": "Lorem ipsum dolor sit amet.",CODE
LOW…d-structured-output/google-drive/nested/fake.docx.json5 "text": "Lorem ipsum dolor sit amet.",CODE
LOW…d-structured-output/google-drive/nested/fake.docx.json5 "text": "Lorem ipsum dolor sit amet.",CODE
LOW…gest/expected-structured-output/jira-diff/1/10000.json219 "text": "Unstructured Devops My comment 1 Unstructured Devops My attachment image lorem ipsum:",CODE
LOW…gest/expected-structured-output/jira-diff/1/10000.json329 "text": "Lorem ipsum dolor sit amet, consectetur adipiscing elit. [Nam quid possumus facere melius?|http://loripsum.CODE
LOW…gest/expected-structured-output/jira-diff/1/10000.json329 "text": "Lorem ipsum dolor sit amet, consectetur adipiscing elit. [Nam quid possumus facere melius?|http://loripsum.CODE
LOW…-output/Sharepoint/SitePages/This-is-a-title.aspx.json49 "text": "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam ex tellus, sodales non nulla et, sodales cCODE
LOW…-output/Sharepoint/SitePages/This-is-a-title.aspx.json49 "text": "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam ex tellus, sodales non nulla et, sodales cCODE
LOW…int-with-permissions/SitePages/This-is-a-title.aspx.md3Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam ex tellus, sodales non nulla et, sodales consequat turpCODE
LOW…int-with-permissions/SitePages/This-is-a-title.aspx.md3Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam ex tellus, sodales non nulla et, sodales consequat turpCODE
LOW⚡…n/airtable-diff/appJ43QmP8I17zu88/tblbj2vBlL2dN2xqq.md18 <td># Nec lapathi suavitatem acupenseri Galloni Laelius anteponebat, sed suavitatem ipsam neglegebat;\nLorem ipsumCODE
LOW⚡…n/airtable-diff/appJ43QmP8I17zu88/tblbj2vBlL2dN2xqq.md18 <td># Nec lapathi suavitatem acupenseri Galloni Laelius anteponebat, sed suavitatem ipsam neglegebat;\nLorem ipsumCODE
LOW⚡…n/airtable-diff/appJ43QmP8I17zu88/tblbj2vBlL2dN2xqq.md27 <td># Quis est enim, in quo sit cupiditas, quin recte cupidus dici possit?\nLorem ipsum dolor sit amet, consectetuCODE
LOW⚡…n/airtable-diff/appJ43QmP8I17zu88/tblbj2vBlL2dN2xqq.md27 <td># Quis est enim, in quo sit cupiditas, quin recte cupidus dici possit?\nLorem ipsum dolor sit amet, consectetuCODE
LOW⚡…n/airtable-diff/appJ43QmP8I17zu88/tblbj2vBlL2dN2xqq.md36 <td># Nescio quo modo praetervolavit oratio.\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Quid ad utiCODE
LOW⚡…n/airtable-diff/appJ43QmP8I17zu88/tblbj2vBlL2dN2xqq.md36 <td># Nescio quo modo praetervolavit oratio.\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Quid ad utiCODE
LOW⚡…n/airtable-diff/appJ43QmP8I17zu88/tblfu7DzEcCWNKwP4.md18 <td># Nec lapathi suavitatem acupenseri Galloni Laelius anteponebat, sed suavitatem ipsam neglegebat;\nLorem ipsumCODE
LOW⚡…n/airtable-diff/appJ43QmP8I17zu88/tblfu7DzEcCWNKwP4.md18 <td># Nec lapathi suavitatem acupenseri Galloni Laelius anteponebat, sed suavitatem ipsam neglegebat;\nLorem ipsumCODE
LOW⚡…n/airtable-diff/appJ43QmP8I17zu88/tblfu7DzEcCWNKwP4.md27 <td># Nescio quo modo praetervolavit oratio.\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Quid ad utiCODE
LOW⚡…n/airtable-diff/appJ43QmP8I17zu88/tblfu7DzEcCWNKwP4.md27 <td># Nescio quo modo praetervolavit oratio.\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Quid ad utiCODE
LOW⚡…n/airtable-diff/appJ43QmP8I17zu88/tblfu7DzEcCWNKwP4.md36 <td># Quis est enim, in quo sit cupiditas, quin recte cupidus dici possit?\nLorem ipsum dolor sit amet, consectetuCODE
LOW⚡…n/airtable-diff/appJ43QmP8I17zu88/tblfu7DzEcCWNKwP4.md36 <td># Quis est enim, in quo sit cupiditas, quin recte cupidus dici possit?\nLorem ipsum dolor sit amet, consectetuCODE
LOW⚡…n/airtable-diff/app5YQxSfp220fWtm/tblBoUk54tWXGqYai.md18 <td># Nec lapathi suavitatem acupenseri Galloni Laelius anteponebat, sed suavitatem ipsam neglegebat;\nLorem ipsumCODE
LOW⚡…n/airtable-diff/app5YQxSfp220fWtm/tblBoUk54tWXGqYai.md18 <td># Nec lapathi suavitatem acupenseri Galloni Laelius anteponebat, sed suavitatem ipsam neglegebat;\nLorem ipsumCODE
LOW⚡…n/airtable-diff/app5YQxSfp220fWtm/tblBoUk54tWXGqYai.md27 <td># Nescio quo modo praetervolavit oratio.\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Quid ad utiCODE
LOW⚡…n/airtable-diff/app5YQxSfp220fWtm/tblBoUk54tWXGqYai.md27 <td># Nescio quo modo praetervolavit oratio.\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Quid ad utiCODE
LOW⚡…n/airtable-diff/app5YQxSfp220fWtm/tblBoUk54tWXGqYai.md36 <td># Quis est enim, in quo sit cupiditas, quin recte cupidus dici possit?\nLorem ipsum dolor sit amet, consectetuCODE
LOW⚡…n/airtable-diff/app5YQxSfp220fWtm/tblBoUk54tWXGqYai.md36 <td># Quis est enim, in quo sit cupiditas, quin recte cupidus dici possit?\nLorem ipsum dolor sit amet, consectetuCODE
LOW⚡…n/airtable-diff/app5YQxSfp220fWtm/tblxdPc7L2meGIZLE.md18 <td># Quis est enim, in quo sit cupiditas, quin recte cupidus dici possit?\nLorem ipsum dolor sit amet, consectetuCODE
LOW⚡…n/airtable-diff/app5YQxSfp220fWtm/tblxdPc7L2meGIZLE.md18 <td># Quis est enim, in quo sit cupiditas, quin recte cupidus dici possit?\nLorem ipsum dolor sit amet, consectetuCODE
LOW⚡…n/airtable-diff/app5YQxSfp220fWtm/tblxdPc7L2meGIZLE.md27 <td># Nescio quo modo praetervolavit oratio.\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Quid ad utiCODE
LOW⚡…n/airtable-diff/app5YQxSfp220fWtm/tblxdPc7L2meGIZLE.md27 <td># Nescio quo modo praetervolavit oratio.\nLorem ipsum dolor sit amet, consectetur adipiscing elit. Quid ad utiCODE
LOW⚡…n/airtable-diff/app5YQxSfp220fWtm/tblxdPc7L2meGIZLE.md36 <td># Nec lapathi suavitatem acupenseri Galloni Laelius anteponebat, sed suavitatem ipsam neglegebat;\nLorem ipsumCODE
LOW⚡…n/airtable-diff/app5YQxSfp220fWtm/tblxdPc7L2meGIZLE.md36 <td># Nec lapathi suavitatem acupenseri Galloni Laelius anteponebat, sed suavitatem ipsam neglegebat;\nLorem ipsumCODE
LOW…d-structured-output-markdown/google-drive/fake.docx.md1Lorem ipsum dolor sit amet.CODE
LOW…d-structured-output-markdown/google-drive/fake.docx.md1Lorem ipsum dolor sit amet.CODE
LOW…tured-output-markdown/google-drive/nested/fake.docx.md1Lorem ipsum dolor sit amet.CODE
LOW…tured-output-markdown/google-drive/nested/fake.docx.md1Lorem ipsum dolor sit amet.CODE
LOW⚡…pected-structured-output-markdown/jira-diff/1/10000.md10Unstructured Devops My comment 1 Unstructured Devops My attachment image lorem ipsum:CODE
LOW⚡…pected-structured-output-markdown/jira-diff/1/10000.md15Lorem ipsum dolor sit amet, consectetur adipiscing elit. [Nam quid possumus facere melius?|http://loripsum.net/] Ita relCODE
372 more matches not shown…
Unused Imports155 hits · 135 pts
SeverityFileLineSnippetContext
LOWtest_unstructured/test_safe_http.py3CODE
LOWtest_unstructured/test_utils.py1CODE
LOWtest_unstructured/test_utils.py49CODE
LOWtest_unstructured/test_utils.py57CODE
LOWtest_unstructured/test_utils.py58CODE
LOWtest_unstructured/test_utils.py66CODE
LOWtest_unstructured/test_utils.py75CODE
LOWtest_unstructured/test_utils.py76CODE
LOWtest_unstructured/test_utils.py86CODE
LOWtest_unstructured/test_runtime_telemetry.py3CODE
LOWtest_unstructured/test_cli_doctor.py3CODE
LOWtest_unstructured/test_telemetry.py8CODE
LOWtest_unstructured/unit_utils.py3CODE
LOWtest_unstructured/unit_utils.py10CODE
LOWtest_unstructured/unit_utils.py10CODE
LOWtest_unstructured/unit_utils.py10CODE
LOWtest_unstructured/unit_utils.py21CODE
LOWtest_unstructured/unit_utils.py21CODE
LOWtest_unstructured/unit_utils.py21CODE
LOWtest_unstructured/metrics/test_element_type.py1CODE
LOWtest_unstructured/staging/test_base.py14CODE
LOWtest_unstructured/chunking/test_dispatch.py5CODE
LOWtest_unstructured/chunking/test_basic.py7CODE
LOWtest_unstructured/chunking/test_table_isolation.py10CODE
LOWtest_unstructured/chunking/test_title.py5CODE
LOWtest_unstructured/chunking/test_base.py5CODE
LOWtest_unstructured/chunking/test_lazy.py9CODE
LOWtest_unstructured/common/test_html_table.py5CODE
LOWtest_unstructured/documents/test_elements.py5CODE
LOWtest_unstructured/partition/test_ndjson.py3CODE
LOWtest_unstructured/partition/test_xml.py3CODE
LOWtest_unstructured/partition/test_docx.py5CODE
LOWtest_unstructured/partition/test_email.py5CODE
LOWtest_unstructured/partition/test_odt.py5CODE
LOWtest_unstructured/partition/test_tsv.py3CODE
LOWtest_unstructured/partition/test_md.py1CODE
LOWtest_unstructured/partition/test_audio.py5CODE
LOWtest_unstructured/partition/test_org.py1CODE
LOWtest_unstructured/partition/test_ppt.py3CODE
LOWtest_unstructured/partition/test_xlsx.py5CODE
LOWtest_unstructured/partition/test_msg.py3CODE
LOWtest_unstructured/partition/test_text.py3CODE
LOWtest_unstructured/partition/test_rst.py1CODE
LOWtest_unstructured/partition/test_rtf.py1CODE
LOWtest_unstructured/partition/test_auto.py3CODE
LOWtest_unstructured/partition/test_json.py3CODE
LOWtest_unstructured/partition/test_pptx.py5CODE
LOWtest_unstructured/partition/test_doc.py5CODE
LOWtest_unstructured/partition/test_csv.py3CODE
LOWtest_unstructured/partition/test_epub.py1CODE
LOW…tured/partition/utils/ocr_models/test_ocr_interface.py5CODE
LOW…t_unstructured/partition/html/test_xss_sanitization.py12CODE
LOWtest_unstructured/partition/html/test_partition.py5CODE
LOWtest_unstructured/partition/common/test_lang.py5CODE
LOWtest_unstructured/partition/common/test_metadata.py5CODE
LOW…nstructured/partition/common/test_json_partitioning.py3CODE
LOWtest_unstructured/partition/pdf_image/test_image.py1CODE
LOWtest_unstructured/file_utils/test_model.py3CODE
LOWtest_unstructured/file_utils/test_filetype.py5CODE
LOWscripts/performance/benchmark_partition.py23CODE
95 more matches not shown…
Cross-File Repetition22 hits · 110 pts
SeverityFileLineSnippetContext
HIGHtest_unstructured/partition/test_docx.py0all default arguments for `htmlpartitioneroptions`. individual argument values can be changed to suit each test. makes cSTRING
HIGHtest_unstructured/partition/test_xlsx.py0all default arguments for `htmlpartitioneroptions`. individual argument values can be changed to suit each test. makes cSTRING
HIGHtest_unstructured/partition/test_pptx.py0all default arguments for `htmlpartitioneroptions`. individual argument values can be changed to suit each test. makes cSTRING
HIGHtest_unstructured/partition/test_msg.py0all default arguments for `htmlpartitioneroptions`. individual argument values can be changed to suit each test. makes cSTRING
HIGHtest_unstructured/partition/html/test_partition.py0all default arguments for `htmlpartitioneroptions`. individual argument values can be changed to suit each test. makes cSTRING
HIGHunstructured/partition/docx.py0encapsulates partitioning option validation, computation, and application of defaults.STRING
HIGHunstructured/partition/email.py0encapsulates partitioning option validation, computation, and application of defaults.STRING
HIGHunstructured/partition/xlsx.py0encapsulates partitioning option validation, computation, and application of defaults.STRING
HIGHunstructured/partition/msg.py0encapsulates partitioning option validation, computation, and application of defaults.STRING
HIGHunstructured/partition/pptx.py0encapsulates partitioning option validation, computation, and application of defaults.STRING
HIGHunstructured/partition/html/partition.py0encapsulates partitioning option validation, computation, and application of defaults.STRING
HIGHunstructured/partition/docx.py0the best last-modified date available, none if no sources are available.STRING
HIGHunstructured/partition/csv.py0the best last-modified date available, none if no sources are available.STRING
HIGHunstructured/partition/xlsx.py0the best last-modified date available, none if no sources are available.STRING
HIGHunstructured/partition/pptx.py0the best last-modified date available, none if no sources are available.STRING
HIGHunstructured/partition/html/partition.py0the best last-modified date available, none if no sources are available.STRING
HIGHunstructured/partition/docx.py0the best available file-path for this document or `none` if unavailable.STRING
HIGHunstructured/partition/xlsx.py0the best available file-path for this document or `none` if unavailable.STRING
HIGHunstructured/partition/pptx.py0the best available file-path for this document or `none` if unavailable.STRING
HIGHunstructured/partition/org.py0partitions an rst document. the document is first converted to html and then partitioned using partition_html. parameterSTRING
HIGHunstructured/partition/rtf.py0partitions an rst document. the document is first converted to html and then partitioned using partition_html. parameterSTRING
HIGHunstructured/partition/rst.py0partitions an rst document. the document is first converted to html and then partitioned using partition_html. parameterSTRING
Deep Nesting88 hits · 72 pts
SeverityFileLineSnippetContext
LOWtest_unstructured/test_usage_notice.py219CODE
LOWtest_unstructured/chunking/test_base.py5009CODE
LOWtest_unstructured/partition/test_audio.py190CODE
LOWtest_unstructured/partition/test_auto.py1527CODE
LOWtest_unstructured/partition/pdf_image/test_pdf.py291CODE
LOWtest_unstructured/partition/pdf_image/test_pdf.py974CODE
LOWtest_unstructured/partition/pdf_image/test_pdf.py1357CODE
LOWtest_unstructured/partition/pdf_image/test_pdf.py2181CODE
LOWtest_unstructured/file_utils/test_encoding.py15CODE
LOWunstructured/telemetry.py437CODE
LOWunstructured/utils.py367CODE
LOWunstructured/safe_http.py190CODE
LOWunstructured/metrics/text_extraction.py123CODE
LOWunstructured/metrics/evaluate.py510CODE
LOWunstructured/metrics/evaluate.py599CODE
LOWunstructured/metrics/evaluate.py282CODE
LOWunstructured/metrics/table/table_alignment.py59CODE
LOWunstructured/metrics/table/table_extraction.py15CODE
LOWunstructured/staging/weaviate.py89CODE
LOWunstructured/staging/base.py197CODE
LOWunstructured/staging/base.py360CODE
LOWunstructured/staging/base.py578CODE
LOWunstructured/staging/base.py732CODE
LOWunstructured/chunking/dispatch.py234CODE
LOWunstructured/chunking/base.py766CODE
LOWunstructured/chunking/base.py919CODE
LOWunstructured/chunking/base.py1315CODE
LOWunstructured/chunking/base.py1574CODE
LOWunstructured/chunking/base.py1714CODE
LOWunstructured/chunking/base.py2473CODE
LOWunstructured/chunking/base.py2516CODE
LOWunstructured/chunking/base.py2742CODE
LOWunstructured/chunking/base.py928CODE
LOWunstructured/chunking/base.py1950CODE
LOWunstructured/common/html_table.py107CODE
LOWunstructured/nlp/tokenize.py55CODE
LOWunstructured/documents/elements.py376CODE
LOWunstructured/partition/md.py50CODE
LOWunstructured/partition/docx.py517CODE
LOWunstructured/partition/docx.py887CODE
LOWunstructured/partition/docx.py902CODE
LOWunstructured/partition/docx.py959CODE
LOWunstructured/partition/docx.py566CODE
LOWunstructured/partition/docx.py964CODE
LOWunstructured/partition/pdf.py272CODE
LOWunstructured/partition/pdf.py506CODE
LOWunstructured/partition/pdf.py644CODE
LOWunstructured/partition/pdf.py724CODE
LOWunstructured/partition/pdf.py988CODE
LOWunstructured/partition/pdf.py1625CODE
LOWunstructured/partition/api.py235CODE
LOWunstructured/partition/csv.py376CODE
LOWunstructured/partition/csv.py408CODE
LOWunstructured/partition/csv.py216CODE
LOWunstructured/partition/text.py113CODE
LOWunstructured/partition/xlsx.py53CODE
LOWunstructured/partition/xlsx.py504CODE
LOWunstructured/partition/xlsx.py548CODE
LOWunstructured/partition/pptx.py143CODE
LOWunstructured/partition/pptx.py219CODE
28 more matches not shown…
Over-Commented Block69 hits · 68 pts
SeverityFileLineSnippetContext
LOWtest_unstructured_ingest/json-to-text.sh1#!/usr/bin/env bashCOMMENT
LOW…ed-output-markdown/s3/2023-Jan-economic-outlook.pdf.md61The balance of risks to the global outlook remains tilted to the downside, with scope for lower growth and higher inflatCOMMENT
LOW…-structured-output-markdown/s3/Silent-Giant-(1).pdf.md61However, markets fail to give due credit to electricity generators, such as nuclear energy, that are able to meet these COMMENT
LOW…-structured-output-markdown/s3/Silent-Giant-(1).pdf.md81# iCOMMENT
LOW…ed-output-markdown/s3/recalibrating-risk-report.pdf.md418COMMENT
LOW…uctured-output-markdown/confluence-diff/MFS/1605928.md1# \uD83D\uDDD3 DateCOMMENT
LOW…tructured-output-markdown/azure/spring-weather.html.md1<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"COMMENT
LOW…tructured-output-markdown/azure/spring-weather.html.md21# <div class="center-content">COMMENT
LOW…tructured-output-markdown/azure/spring-weather.html.md41# </div><!-- end of <div class="center"> -->COMMENT
LOW…nedrive/utic-test-ingest-fixtures/tests-example.xls.md1<table><tr><td>MC</td><td>What is 2+2?</td><td>4</td><td>correct</td><td>3</td><td>incorrect</td><td/><td/><td/></tr><trCOMMENT
LOW…ctured-output-markdown/opensearch/movies-4-57554198.md1# AmericanCOMMENT
LOW…ctured-output-markdown/opensearch/movies-1-57554198.md1# AmericanCOMMENT
LOW…ctured-output-markdown/opensearch/movies-8-57554198.md1# AmericanCOMMENT
LOW…ctured-output-markdown/opensearch/movies-2-57554198.md1# AmericanCOMMENT
LOW…ctured-output-markdown/opensearch/movies-7-57554198.md1# AmericanCOMMENT
LOW…ctured-output-markdown/opensearch/movies-9-57554198.md1# AmericanCOMMENT
LOW…ctured-output-markdown/opensearch/movies-5-57554198.md1# AmericanCOMMENT
LOW…ctured-output-markdown/opensearch/movies-6-57554198.md1# AmericanCOMMENT
LOW…ctured-output-markdown/opensearch/movies-3-57554198.md1# AmericanCOMMENT
LOW…ed-output-markdown/mongodb/659daefa21dd8c9054b084b7.md11901COMMENT
LOW…ed-output-markdown/mongodb/659daefa21dd8c9054b084b6.md11901COMMENT
LOW…ed-output-markdown/mongodb/659daefa21dd8c9054b084b9.md11908COMMENT
LOW…ed-output-markdown/mongodb/659daefa21dd8c9054b084b8.md11908COMMENT
LOW…uctured-output-markdown/hubspot/products/2362691415.md1Example product description. This is a text containing relevant information pertaining to ProductA. The text represents COMMENT
LOW…uctured-output-markdown/hubspot/products/2362691417.md1Example product description. This is a text containing relevant information pertaining to ProductC. The text represents COMMENT
LOW…uctured-output-markdown/hubspot/products/2362691416.md1Example product description. This is a text containing relevant information pertaining to ProductB. The text represents COMMENT
LOW…rkdown/local-single-file/UDHR_first_article_all.txt.md81Chin, Tedim Mihingte khempeuh in thupitak leh thuneihna tawh suakta tak leh akibang in kipiang ciat ahi hi. Asia leh aphCOMMENT
LOW…rkdown/local-single-file/UDHR_first_article_all.txt.md101# 人个顶个生而自由,在尊严和权利上般儿般儿大。他们趁理性和良心,并应以兄弟关系的精神相对待。COMMENT
LOW…pected-structured-output-markdown/jira-diff/1/10002.md1# IssueID_IssueKey:10002 JCTP1-3COMMENT
LOW…pected-structured-output-markdown/jira-diff/1/10013.md1# IssueID_IssueKey:10013 JCTP1-4COMMENT
LOW…pected-structured-output-markdown/jira-diff/1/10000.md1# IssueID_IssueKey:10000 JCTP1-1COMMENT
LOW…pected-structured-output-markdown/jira-diff/1/10000.md21# test.atlassian.net/rest/api/2/attachment/10000COMMENT
LOW…pected-structured-output-markdown/jira-diff/1/10001.md1# IssueID_IssueKey:10001 JCTP1-2COMMENT
LOW…ed-structured-output-markdown/jira-diff/JCTP3/10014.md1# IssueID_IssueKey:10014 JCTP3-1COMMENT
LOW…ed-structured-output-markdown/jira-diff/JCTP2/10009.md1# IssueID_IssueKey:10009 JCTP2-7COMMENT
LOW…ed-structured-output-markdown/jira-diff/JCTP2/10006.md1# IssueID_IssueKey:10006 JCTP2-4COMMENT
LOW…ed-structured-output-markdown/jira-diff/JCTP2/10012.md1# IssueID_IssueKey:10012 JCTP2-10COMMENT
LOW…ed-structured-output-markdown/jira-diff/JCTP2/10010.md1# IssueID_IssueKey:10010 JCTP2-8COMMENT
LOW…ed-structured-output-markdown/jira-diff/JCTP2/10015.md1# IssueID_IssueKey:10015 JCTP2-11COMMENT
LOW…markdown/salesforce/Campaign/701Hu000001eX9FIAU.xml.md1# attributes.type: CampaignCOMMENT
LOW…markdown/salesforce/Campaign/701Hu000001eX9FIAU.xml.md21# NumberOfResponses: 0COMMENT
LOW…markdown/salesforce/Campaign/701Hu000001eX9GIAU.xml.md1# attributes.type: CampaignCOMMENT
LOW…markdown/salesforce/Campaign/701Hu000001eX9GIAU.xml.md21# NumberOfResponses: 0COMMENT
LOW…markdown/salesforce/Campaign/701Hu000001eX9EIAU.xml.md1# attributes.type: CampaignCOMMENT
LOW…markdown/salesforce/Campaign/701Hu000001eX9EIAU.xml.md21# NumberOfResponses: 0COMMENT
LOW…markdown/salesforce/Campaign/701Hu000001eX9HIAU.xml.md1# attributes.type: CampaignCOMMENT
LOW…markdown/salesforce/Campaign/701Hu000001eX9HIAU.xml.md21# NumberOfResponses: 0COMMENT
LOW…red-output-markdown/elasticsearch/movies-4-57554198.md1# AmericanCOMMENT
LOW…red-output-markdown/elasticsearch/movies-1-57554198.md1# AmericanCOMMENT
LOW…red-output-markdown/elasticsearch/movies-8-57554198.md1# AmericanCOMMENT
LOW…red-output-markdown/elasticsearch/movies-2-57554198.md1# AmericanCOMMENT
LOW…red-output-markdown/elasticsearch/movies-7-57554198.md1# AmericanCOMMENT
LOW…red-output-markdown/elasticsearch/movies-9-57554198.md1# AmericanCOMMENT
LOW…red-output-markdown/elasticsearch/movies-5-57554198.md1# AmericanCOMMENT
LOW…red-output-markdown/elasticsearch/movies-6-57554198.md1# AmericanCOMMENT
LOW…red-output-markdown/elasticsearch/movies-3-57554198.md1# AmericanCOMMENT
LOWtest_unstructured_ingest/src/confluence-large.sh61# We are expecting two directories: one for the space, and one is the output directory itselfCOMMENT
LOW…ctured/testfiles/staging/UDHR_first_article_all.txt.md81Chin, Matu Thlangboeih he rhimomna, vanpitna, yalpona hamhmoel ka tawn thlang la cuun la ng’om u. Thlanghing he athae-thCOMMENT
LOW…ctured/testfiles/staging/UDHR_first_article_all.txt.md101# Chinese, Mandarin (Tianjin)COMMENT
LOWtest_unstructured/partition/test_docx.py1921 # ┌───┐COMMENT
9 more matches not shown…
Self-Referential Comments19 hits · 57 pts
SeverityFileLineSnippetContext
MEDIUMtest_unstructured_ingest/structured-json-to-html.sh3# Define the input and output top directoriesCOMMENT
MEDIUMtest_unstructured_ingest/structured-json-to-markdown.sh3# Define the input and output top directoriesCOMMENT
MEDIUMtest_unstructured/partition/utils/test_xycut.py114 # Define the expected image with the square drawnCOMMENT
MEDIUMtest_unstructured/partition/html/test_partition.py1587 # Create a file-like object with empty contentCOMMENT
MEDIUMtest_unstructured/partition/html/test_partition.py1591 # Create a temporary file with the given contentCOMMENT
MEDIUMtest_unstructured/partition/pdf_image/test_pdf.py1254 # Create an exception that will be raised directly after OCR is called to stop executionCOMMENT
MEDIUM…nstructured/partition/pdf_image/test_merge_elements.py30 # Create a PageLayout for the inferred layoutCOMMENT
MEDIUMexample-docs/logger.py7# Create a custom logging levelCOMMENT
MEDIUMexample-docs/logger.py12# Create a custom log method for the "DETAIL" levelCOMMENT
MEDIUM…rs/destination_connector/create-opensearch-instance.sh7# Create the Opensearch clusterCOMMENT
MEDIUM…elpers/source_connector/create-and-check-opensearch.sh7# Create the Opensearch clusterCOMMENT
MEDIUM…destination_connector/create-elasticsearch-instance.sh8# Create the Elasticsearch clusterCOMMENT
MEDIUM…t-helpers/source_connector/create-fill-and-check-es.sh8# Create the Elasticsearch clusterCOMMENT
MEDIUMscripts/kafka-test-helpers/create-kafka-instance.sh7# Create the Weaviate instanceCOMMENT
MEDIUMscripts/sql-test-helpers/create-sql-instance.sh9# Create the SQL instanceCOMMENT
MEDIUM…ipts/weaviate-test-helpers/create-weaviate-instance.sh7# Create the Weaviate instanceCOMMENT
MEDIUMscripts/performance/benchmark-local.sh4# This file is separated out to allow us to easily execute this part of the test script inside a Docker container.COMMENT
MEDIUMunstructured/logger.py6# Create a custom logging levelCOMMENT
MEDIUMunstructured/logger.py11# Create a custom log method for the "DETAIL" levelCOMMENT
Empty Error Handlers10 hits · 50 pts
SeverityFileLineSnippetContext
HIGHscripts/performance/compare_benchmark.py84except OSError:CODE
HIGHunstructured/telemetry.py249except Exception:CODE
HIGHunstructured/telemetry.py551except BaseException:CODE
HIGHunstructured/safe_http.py165except ValueError:CODE
HIGHunstructured/nlp/tokenize.py124except OSError:CODE
HIGHunstructured/nlp/tokenize.py135except OSError:CODE
HIGHunstructured/partition/html/parser.py358except (ValueError, TypeError):CODE
HIGHunstructured/partition/common/common.py251except (psutil.NoSuchProcess, psutil.AccessDenied, psutil.ZombieProcess):CODE
HIGHunstructured/partition/pdf_image/pdfminer_processing.py1052except Exception:CODE
HIGHunstructured/file_utils/filetype.py344except (json.JSONDecodeError, RecursionError):CODE
Redundant / Tautological Comments26 hits · 42 pts
SeverityFileLineSnippetContext
LOWtest_unstructured_ingest/test-ingest-src.sh61 # Check if the test is in tests_to_ignoreCOMMENT
LOW…structured_ingest/python/test-produce-kafka-message.py47 # Read the file in binary mode and encode content in base64COMMENT
LOW…structured_ingest/python/test-ingest-astradb-output.py38 # Print the resultsCOMMENT
LOWtest_unstructured/partition/utils/test_xycut.py132 # Check if the resulting image contains the expected shapes and labelsCOMMENT
LOWtest_unstructured/partition/utils/test_xycut.py178 # Check if vis_points was called with the correct argumentsCOMMENT
LOW⚡test_unstructured/partition/pdf_image/test_ocr.py352 # Check if the final layout contains the original layout elementsCOMMENT
LOW⚡test_unstructured/partition/pdf_image/test_ocr.py356 # Check if the final layout contains the OCR-derived elementsCOMMENT
LOW⚡test_unstructured/partition/pdf_image/test_ocr.py359 # Check if the OCR-derived elements that are subregions of layout elements are removedCOMMENT
LOWtest_unstructured/partition/pdf_image/test_ocr.py381 # Check if the out layout's text attribute is updated with aggregated OCR textCOMMENT
LOWtest_unstructured/partition/pdf_image/test_ocr.py384 # Check if the final layout contains both original elements and OCR-derived elementsCOMMENT
LOWtest_unstructured/partition/pdf_image/test_ocr.py530 # Check if the out layout's text attribute is updated with aggregated OCR textCOMMENT
LOWtest_unstructured/partition/pdf_image/test_ocr.py533 # Check if the final layout contains both original elements and OCR-derived elementsCOMMENT
LOW…structured/partition/pdf_image/test_pdf_image_utils.py406 # Check if images for both layouts were savedCOMMENT
LOW…structured/partition/pdf_image/test_pdf_image_utils.py427 # Check if only the inferred layout image was saved if extracted layout is NoneCOMMENT
LOWscripts/check-new-release-version.sh22# Check if the current version is a non-dev version and not matching the main versionCOMMENT
LOWscripts/user/u-tables-inspect.sh33# Check if the directory exists, if not create itCOMMENT
LOWscripts/user/u-tables-inspect.sh65 # Open the file in a new browser windowCOMMENT
LOWscripts/user/process-pdf-parallel-through-api.sh31# Check if UNST_API_KEY is setCOMMENT
LOWscripts/user/process-pdf-parallel-through-api.sh49# Check if PDF parts directory existsCOMMENT
LOWscripts/user/unstructured-get-json.sh198# Check if vlm-provider or vlm-model are provided without --vlmCOMMENT
LOWunstructured/metrics/utils.py238 # Check if the file existsCOMMENT
LOWunstructured/staging/base.py182 # Check if we hit the size limit or if data is actually incompleteCOMMENT
LOWunstructured/embed/voyageai.py113 # Check if adding this text would exceed limitsCOMMENT
LOW⚡unstructured/partition/docx.py1033 # Check if category depth can be determined from style ilvlCOMMENT
LOWunstructured/partition/pdf_image/pdfminer_processing.py444 # Check if text is low_fidelity:COMMENT
LOWunstructured/partition/pdf_image/pdfminer_utils.py447 # Check if this is a duplicate of the last LTCharCOMMENT
Excessive Try-Catch Wrapping36 hits · 41 pts
SeverityFileLineSnippetContext
LOW⚡test_unstructured/partition/pdf_image/test_pdf.py1159 except Exception as e:CODE
LOWscripts/performance/quick_partition_bench.py41 except Exception as exc: # noqa: BLE001CODE
LOWunstructured/telemetry.py167 except Exception:CODE
LOWunstructured/telemetry.py175 except Exception:CODE
LOWunstructured/telemetry.py192 except Exception:CODE
LOW⚡unstructured/telemetry.py249 except Exception:CODE
LOWunstructured/telemetry.py322 except Exception:CODE
LOWunstructured/telemetry.py529 except Exception:CODE
LOWunstructured/_usage_notice.py61 except Exception:CODE
LOWunstructured/doctor.py54 except Exception as exc: # noqa: BLE001 — surface libmagic load/binary issuesCODE
MEDIUMunstructured/doctor.py47def _libmagic_status() -> tuple[Status, str]:CODE
LOWunstructured/metrics/evaluate.py197 except Exception as e:CODE
LOW⚡unstructured/metrics/table/table_extraction.py163 except Exception as e:CODE
MEDIUM⚡unstructured/metrics/table/table_extraction.py164 print(f"Error converting ground truth data: {e}")CODE
LOW⚡unstructured/metrics/table/table_extraction.py254 except Exception as e:CODE
MEDIUM⚡unstructured/metrics/table/table_extraction.py255 print(f"Error converting Unstructured table data: {e}")CODE
LOWunstructured/partition/docx.py546 except Exception as e:CODE
LOWunstructured/partition/docx.py608 except Exception as e:CODE
LOWunstructured/partition/docx.py919 except Exception as e:CODE
LOWunstructured/partition/docx.py1133 except Exception:CODE
LOWunstructured/partition/docx.py1150 except Exception:CODE
LOWunstructured/partition/docx.py1159 except Exception:CODE
LOWunstructured/partition/pdf.py345 except Exception as e:CODE
LOWunstructured/partition/pdf.py831 except Exception:CODE
LOWunstructured/partition/pdf.py874 except Exception:CODE
LOWunstructured/partition/pdf.py904 except Exception:CODE
LOWunstructured/partition/pdf.py949 except Exception as e:CODE
LOW…ition/utils/speech_to_text/speech_to_text_interface.py82 except Exception as e:CODE
LOW…ructured/partition/utils/speech_to_text/whisper_stt.py48 except Exception as exc:CODE
LOWunstructured/partition/common/json_partitioning.py69 except Exception as e:CODE
LOWunstructured/partition/pdf_image/pdf_image_utils.py396 except Exception as e:CODE
LOWunstructured/partition/pdf_image/pdfminer_processing.py1052 except Exception:CODE
LOWunstructured/partition/pdf_image/pdfminer_processing.py1074 except Exception:CODE
LOWunstructured/partition/pdf_image/ocr.py203 except Exception as e:CODE
LOW…red/partition/pdf_image/analysis/bbox_visualisation.py682 except Exception as ex: # noqa: E722CODE
LOW…red/partition/pdf_image/analysis/bbox_visualisation.py704 except Exception as ex: # noqa: E722CODE
Cross-Language Confusion4 hits · 28 pts
SeverityFileLineSnippetContext
HIGH⚡test_unstructured/staging/test_base.py152 assert df.type.equals(expected_df.type) is True # type: ignoreCODE
HIGH⚡test_unstructured/staging/test_base.py153 assert df.text.equals(expected_df.text) is True # type: ignoreCODE
HIGHtest_unstructured/partition/test_csv.py556 assert frame.equals(expected_frame)CODE
HIGHunstructured/partition/common/json_partitioning.py102 # -- scalar (str / number / bool / null) --COMMENT
Docstring Block Structure4 hits · 20 pts
SeverityFileLineSnippetContext
HIGHunstructured/metrics/text_extraction.py76 Calculates edit distance using Levenshtein distance between two strings. Args: output (str): The targeSTRING
HIGHunstructured/partition/html/transformations.py410 Parses the given HTML code and converts it into an Element object. Args: html_code (str): The HTML codSTRING
HIGHunstructured/partition/pdf_image/pdf_image_utils.py35 Write an image to a specified file path, supporting both PIL Image and numpy ndarray formats. Parameters: STRING
HIGHunstructured/file_utils/filetype.py76Determine file-type of specified file using libmagic and/or fallback methods. One of `file_path` or `file` must be STRING
AI Structural Patterns21 hits · 18 pts
SeverityFileLineSnippetContext
LOWunstructured/staging/datasaur.py24CODE
LOWunstructured/embed/vertexai.py64CODE
LOWunstructured/chunking/basic.py24CODE
LOWunstructured/chunking/basic.py112CODE
LOWunstructured/chunking/title.py23CODE
LOWunstructured/chunking/title.py119CODE
LOWunstructured/documents/elements.py247CODE
LOWunstructured/partition/pdf.py146CODE
LOWunstructured/partition/pdf.py272CODE
LOWunstructured/partition/pdf.py724CODE
LOWunstructured/partition/pdf.py988CODE
LOWunstructured/partition/pdf.py1356CODE
LOWunstructured/partition/pdf.py1625CODE
LOWunstructured/partition/api.py26CODE
LOWunstructured/partition/image.py19CODE
LOWunstructured/partition/auto.py29CODE
LOWunstructured/partition/html/partition.py30CODE
LOWunstructured/partition/common/common.py157CODE
LOWunstructured/partition/pdf_image/ocr.py36CODE
LOWunstructured/partition/pdf_image/ocr.py106CODE
LOWunstructured/partition/pdf_image/analysis/tools.py57CODE
Modern Structural Boilerplate16 hits · 16 pts
SeverityFileLineSnippetContext
LOWscripts/performance/benchmark_partition.py35logger = logging.getLogger(__name__)CODE
LOWscripts/performance/benchmark_partition.py86def _set_github_output(key: str, value: str) -> None:CODE
LOWscripts/performance/compare_benchmark.py51logger = logging.getLogger(__name__)CODE
LOWscripts/convert/elements_json_to_format.py10logger = logging.getLogger(__name__)CODE
LOWscripts/convert/rendered_html_from_elements.py24logger = logging.getLogger(__name__)CODE
LOWunstructured/telemetry.py216def set_partition_document_type(document_type: Any) -> None:CODE
LOWunstructured/telemetry.py238def set_partition_document_type_from_mime(mime_type: str | None) -> None:CODE
LOW⚡unstructured/telemetry.py253def set_partition_strategy_used(strategy: Any) -> None:CODE
LOW⚡unstructured/safe_http.py23logger = logging.getLogger(__name__)CODE
LOWunstructured/chunking/__init__.py15__all__ = [CODE
LOWunstructured/nlp/tokenize.py19logger = logging.getLogger(__name__)CODE
LOWunstructured/partition/utils/config.py61 def _setup_tmpdir(self, tmpdir: str) -> None:CODE
LOWunstructured/partition/utils/speech_to_text/__init__.py8__all__ = ["SpeechToTextAgent", "TranscriptionSegment"]CODE
LOWunstructured/partition/html/convert.py11logger = logging.getLogger(__name__)CODE
LOWunstructured/partition/html/convert.py80 def set_children(self, children: list["ElementHtml"]) -> None:STRING
LOWunstructured/partition/html/__init__.py3__all__ = ["partition_html"]CODE
AI Slop Vocabulary6 hits · 14 pts
SeverityFileLineSnippetContext
MEDIUMunstructured/chunking/base.py717 # -- efficient and definitely more robust than hoping two different computations of combinedCOMMENT
MEDIUMunstructured/common/html_table.py174 # -- root is always a `<table>` element so far but let's be robust --COMMENT
LOWunstructured/partition/pptx.py154 # -- functions like `._iter_shape_elements()` where the "just return when done"COMMENT
MEDIUMunstructured/partition/pptx.py439 # -- any documents, however there's no reason not to make the mechanism robust againstCOMMENT
LOWunstructured/partition/pdf_image/pdfminer_processing.py89 # TODO: refactor to just use np array as inputCOMMENT
LOWunstructured/partition/pdf_image/ocr.py396 # results. Can we just use ocr bounding boxes (gonna be many but at least we saveCOMMENT
Dead Code5 hits · 10 pts
SeverityFileLineSnippetContext
MEDIUM…/airtable-test-helpers/create_scale_test_components.py47CODE
MEDIUMunstructured/partition/docx.py1354CODE
MEDIUMunstructured/partition/pptx.py519CODE
MEDIUMunstructured/partition/html/parser.py629CODE
MEDIUMunstructured/partition/html/parser.py951CODE
Modern AI Meta-Vocabulary3 hits · 9 pts
SeverityFileLineSnippetContext
MEDIUMunstructured/chunking/base.py2193 # -- Mixed expiries or gaps need a per-row blank scaffold. RetainedCOMMENT
MEDIUMunstructured/chunking/base.py2263 # -- An infeasible blank scaffold must not force one-character textCOMMENT
MEDIUMunstructured/chunking/base.py2275 # -- The blank scaffold leaves less than half the normal textCOMMENT
Example Usage Blocks4 hits · 6 pts
SeverityFileLineSnippetContext
LOWscripts/performance/benchmark.sh3# Usage:COMMENT
LOWscripts/performance/profile.sh8# Usage:COMMENT
LOWscripts/performance/profile.sh23# Usage example:COMMENT
LOWunstructured/metrics/object_detection.py705 # Example usageCOMMENT
Slop Phrases1 hit · 3 pts
SeverityFileLineSnippetContext
MEDIUM…ctured/partition/pdf_image/test_pdfminer_processing.py579# Now you can use it in testsCOMMENT