nltk/nltk

23.5

Adjusted Score

23.5

Raw Score

100%

Time Factor

2026-08-28

Last Push

14.7K

Stars

Python

Language

181.8K

Lines of Code

585

Files

3.2K

Pattern Hits

2026-08-29

Scan Date

0.15

HC Hit Rate

What These Metrics Mean

Adjusted Score: Primary synthetic code indicator. Raw score normalised per 1,000 lines of code and multiplied by the temporal discount factor. This is the definitive comparative metric — use it to rank repositories by AI authorship density.
Raw Score: The unmodified sum of all severity-weighted, context-multiplied pattern match scores before temporal discounting. Reflects the absolute signal strength independent of when the repository was last active.
Time Factor: The temporal discount multiplier (0–100%) applied to the raw score. Repositories last updated before ChatGPT's launch (Nov 2022) receive a 5% factor. Full signal is only assigned to repositories active in the post-adoption era (Jan 2024+).
Pattern Hits: Total count of individual pattern matches across all files and categories. A high hit count with a low score may indicate a very large codebase with isolated AI snippets; a low count with a high score indicates dense, concentrated AI signatures.
HC Hit Rate: High+Critical pattern hits per file, averaged across the repository. This orthogonal signal catches repositories where a few files are densely packed with high-severity AI tells — a strong indicator even when the normalised score appears moderate due to codebase size.
Lines of Code / Files: Total lines and files analysed. The scanner examines 94 file extensions. These denominators are used to normalise the score, enabling fair comparison between repositories of vastly different sizes.

Score History

This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.

Severity Breakdown

Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.

CRITICAL 0HIGH 90MEDIUM 309LOW 2845

Directory Score Breakdown

This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.

Pattern Findings

The scanner identified 3244 distinct pattern matches across 17 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.

Hyper-Verbose Identifiers1434 hits · 1477 pts

Severity	File	Line	Snippet	Context
LOW	nltk/internals.py	927	def _reject_cwd_relative_bare_match(filename, matches, kind):	CODE
LOW	nltk/tgrep.py	298	def _tgrep_node_literal_value(node):	CODE
LOW	nltk/tgrep.py	387	def _tgrep_nltk_tree_pos_action(_s, _l, tokens):	CODE
LOW	nltk/tgrep.py	634	def _tgrep_conjunction_action(_s, _l, tokens, join_char="&"):	CODE
LOW	nltk/tgrep.py	669	def _tgrep_segmented_pattern_action(_s, _l, tokens):	CODE
LOW	nltk/tgrep.py	711	def _tgrep_node_label_use_action(_s, _l, tokens):	CODE
LOW	nltk/tgrep.py	730	def _tgrep_node_label_pred_use_action(_s, _l, tokens):	CODE
LOW	nltk/tgrep.py	759	def _tgrep_bind_node_label_action(_s, _l, tokens):	CODE
LOW	nltk/tgrep.py	798	def _tgrep_rel_disjunction_action(_s, _l, tokens):	CODE
LOW⚡	nltk/downloader.py	1522	def _simple_interactive_download(self, args):	CODE
LOW	nltk/downloader.py	1552	def _simple_interactive_update(self):	CODE
LOW	nltk/downloader.py	1603	def _simple_interactive_config(self):	CODE
LOW	nltk/util.py	393	def unweighted_minimum_spanning_digraph(tree, children=iter, shapes=None, attr=None):	CODE
LOW	nltk/util.py	561	def acyclic_branches_depth_first(	CODE
LOW	nltk/util.py	698	def unweighted_minimum_spanning_dict(tree, children=iter):	CODE
LOW	nltk/util.py	743	def unweighted_minimum_spanning_tree(tree, children=iter):	CODE
LOW	nltk/pathsec.py	315	def _reject_colliding_members(members):	CODE
LOW	nltk/pathsec.py	509	def _reject_aliased_or_special(text, context, check_links=False):	CODE
LOW	nltk/pathsec.py	1015	def _resolve_and_validate_host(host, port):	CODE
LOW	nltk/pathsec.py	1129	def _reject_unpinnable_proxied_fetch(url_str):	CODE
LOW	nltk/grammar.py	725	def is_flexible_chomsky_normal_form(self):	CODE
LOW	nltk/grammar.py	1524	def _read_dependency_production(s):	CODE
LOW	nltk/picklesec.py	39	def _reject_extension_opcodes(pickle_source: Any) -> None:	CODE
LOW	nltk/featstruct.py	1707	def _apply_forwards_to_bindings(forward, bindings):	CODE
LOW	nltk/data.py	65	def _assert_no_encoded_bypass(name, error_label=None):	CODE
LOW	nltk/data.py	103	def _assert_no_normalized_bypass(name, error_label=None):	CODE
LOW	nltk/data.py	127	def _reject_unsafe_no_protocol(resource_url):	CODE
LOW	nltk/data.py	238	def _staging_dir_is_still_safe(path):	CODE
LOW	nltk/data.py	363	def _nltk_wrap_secure_fileobj(cls, filename, fileobj):	CODE
LOW	nltk/data.py	859	def _check_decompression_bomb(info):	CODE
LOW	nltk/data.py	901	def _reject_decompression_total(total, compress_size, name, kind):	CODE
LOW	nltk/tree/tree.py	433	def treeposition_spanning_leaves(self, start, end):	CODE
LOW	nltk/metrics/paice.py	26	def get_words_from_dictionary(lemmas):	CODE
LOW	nltk/metrics/paice.py	265	def _get_truncation_coordinates(self, cutlength=0):	CODE
LOW	nltk/metrics/agreement.py	231	def _chance_corrected_agreement(self, observed, expected):	CODE
LOW	nltk/metrics/distance.py	41	def _check_distance_input_len(s1, s2, func_name):	CODE
LOW	nltk/app/concordance_app.py	321	def search_enter_keypress_handler(self, *event):	CODE
LOW	nltk/app/concordance_app.py	428	def handle_error_loading_corpus(self, event):	CODE
LOW	nltk/app/chunkparser_app.py	769	def _adaptively_modify_eval_chunk(self, t):	CODE
LOW	nltk/app/chunkparser_app.py	1215	def _syntax_highlight_grammar(self, grammar):	CODE
LOW	nltk/app/rdparser_app.py	986	def _animate_match_backtrack_frame(self, frame, widget, dy):	CODE
LOW	nltk/app/chartparser_app.py	2517	def bottom_up_leftcorner_strategy(self, *e):	CODE
LOW	nltk/app/collocations_app.py	259	def handle_error_loading_corpus(self, event):	CODE
LOW	nltk/app/wordnet_app.py	210	def get_unique_counter_from_url(sp):	CODE
LOW	nltk/app/wordnet_app.py	930	def get_static_welcome_message():	STRING
LOW	nltk/classify/util.py	171	def binary_names_demo_features(name):	CODE
LOW⚡	nltk/classify/maxent.py	1129	def calculate_empirical_fcount(train_toks, encoding):	CODE
LOW⚡	nltk/classify/maxent.py	1139	def calculate_estimated_fcount(classifier, train_toks, encoding):	CODE
LOW	nltk/classify/maxent.py	211	def most_informative_features(self, n=10):	CODE
LOW	nltk/classify/maxent.py	225	def show_most_informative_features(self, n=10, show="all"):	CODE
LOW	nltk/classify/maxent.py	1033	def train_maxent_classifier_with_gis(	CODE
LOW	nltk/classify/maxent.py	1157	def train_maxent_classifier_with_iis(	CODE
LOW	nltk/classify/maxent.py	1410	def train_maxent_classifier_with_megam(	STRING
LOW	nltk/classify/naivebayes.py	124	def show_most_informative_features(self, n=10):	CODE
LOW	nltk/classify/naivebayes.py	154	def most_informative_features(self, n=100):	CODE
LOW	nltk/test/conftest.py	10	def _authorize_pytest_basetemp(tmp_path_factory):	CODE
LOW	nltk/test/test_filestring_sandbox.py	21	def test_rejects_parent_traversal(tmp_path):	CODE
LOW	nltk/test/test_filestring_sandbox.py	36	def test_rejects_symlink_escape(tmp_path):	CODE
LOW	nltk/test/test_filestring_sandbox.py	56	def test_preserves_file_like_objects():	CODE
LOW⚡	nltk/test/unit/test_quadratic_dos.py	56	def test_correctness_preserved(self):	CODE
1374 more matches not shown…

Decorative Section Separators183 hits · 707 pts

Severity	File	Line	Snippet	Context
MEDIUM	nltk/langnames.py	131	# =======================================================================	COMMENT
MEDIUM	nltk/langnames.py	257	# ======================================================================	COMMENT
MEDIUM	nltk/downloader.py	70	# ----------------------------------------------------------------------	COMMENT
MEDIUM	nltk/downloader.py	997	# ---------------------------------------------------	COMMENT
MEDIUM	nltk/downloader.py	2744	# -----------------------------------------------------------	COMMENT
MEDIUM	nltk/util.py	479	# ==========================================================================	COMMENT
MEDIUM	nltk/util.py	695	# ==========================================================================	COMMENT
MEDIUM	nltk/tabdata.py	60	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	nltk/tabdata.py	62	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	nltk/tabdata.py	98	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	nltk/tabdata.py	100	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	nltk/data.py	519	# ----------------------------------------------------------------------	STRING
MEDIUM	nltk/data.py	521	# ----------------------------------------------------------------------	STRING
MEDIUM	nltk/chunk/named_entity.py	322	# ======================================================================================	COMMENT
MEDIUM	nltk/chunk/named_entity.py	401	# ======================================================================================	COMMENT
MEDIUM⚡	nltk/test/unit/test_quadratic_dos.py	50	# ==========================================================================	COMMENT
MEDIUM⚡	nltk/test/unit/test_quadratic_dos.py	52	# ==========================================================================	COMMENT
MEDIUM⚡	nltk/test/unit/test_quadratic_dos.py	162	# ==========================================================================	COMMENT
MEDIUM⚡	nltk/test/unit/test_quadratic_dos.py	164	# ==========================================================================	COMMENT
MEDIUM⚡	nltk/test/unit/test_quadratic_dos.py	382	# ==========================================================================	COMMENT
MEDIUM⚡	nltk/test/unit/test_quadratic_dos.py	384	# ==========================================================================	COMMENT
MEDIUM⚡	nltk/test/unit/test_pathsec_sweep_dataset_loading.py	127	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_pathsec_sweep_dataset_loading.py	129	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_pathsec_sweep_dataset_loading.py	147	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_pathsec_sweep_dataset_loading.py	149	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_pathsec_sweep_dataset_loading.py	163	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_pathsec_sweep_dataset_loading.py	165	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_pathsec_sweep_dataset_loading.py	190	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_pathsec_sweep_dataset_loading.py	192	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_pathsec_sweep_dataset_loading.py	205	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_pathsec_sweep_dataset_loading.py	207	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_pathsec_sweep_dataset_loading.py	242	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_pathsec_sweep_dataset_loading.py	244	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_pathsec_sweep_dataset_loading.py	267	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_pathsec_sweep_dataset_loading.py	269	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_pathsec_sweep_dataset_loading.py	278	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_pathsec_sweep_dataset_loading.py	280	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_pathsec_sweep_dataset_loading.py	307	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_pathsec_sweep_dataset_loading.py	309	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_corpus_reader_hardening.py	40	# ----------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_corpus_reader_hardening.py	42	# ----------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_corpus_reader_hardening.py	105	# ----------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_corpus_reader_hardening.py	107	# ----------------------------------------------------------------------------	COMMENT
MEDIUM	nltk/test/unit/test_corpus_reader_hardening.py	125	# ----------------------------------------------------------------------------	COMMENT
MEDIUM	nltk/test/unit/test_corpus_reader_hardening.py	127	# ----------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_aline.py	87	# ---------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_aline.py	89	# ---------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_tokenize_stanford_pathsec.py	95	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_tokenize_stanford_pathsec.py	97	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_tokenize_stanford_pathsec.py	192	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_tokenize_stanford_pathsec.py	194	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_tokenize_stanford_pathsec.py	230	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_tokenize_stanford_pathsec.py	232	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_corpus_reader_pathsec.py	312	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	nltk/test/unit/test_corpus_reader_pathsec.py	318	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	nltk/test/unit/test_corpus_reader_pathsec.py	146	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	nltk/test/unit/test_corpus_reader_pathsec.py	148	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	nltk/test/unit/test_corpus_reader_pathsec.py	523	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	nltk/test/unit/test_corpus_reader_pathsec.py	528	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	…k/test/unit/test_module_import_and_functional_smoke.py	78	# ---------------------------------------------------------------------------	COMMENT
123 more matches not shown…

Unused Imports656 hits · 582 pts

Severity	File	Line	Context
LOW	tools/find_deprecated.py	30	CODE
LOW	nltk/internals.py	22	CODE
LOW	nltk/downloader.py	173	CODE
LOW	nltk/toolbox.py	13	CODE
LOW	nltk/util.py	31	CODE
LOW	nltk/util.py	32	CODE
LOW	nltk/util.py	32	CODE
LOW	nltk/__init__.py	23	CODE
LOW	nltk/__init__.py	111	CODE
LOW	nltk/__init__.py	141	CODE
LOW	nltk/__init__.py	142	CODE
LOW	nltk/__init__.py	142	CODE
LOW	nltk/__init__.py	143	CODE
LOW	nltk/__init__.py	144	CODE
LOW	nltk/__init__.py	145	CODE
LOW	nltk/__init__.py	146	CODE
LOW	nltk/__init__.py	147	CODE
LOW	nltk/__init__.py	148	CODE
LOW	nltk/__init__.py	154	CODE
LOW	nltk/__init__.py	155	CODE
LOW	nltk/__init__.py	156	CODE
LOW	nltk/__init__.py	157	CODE
LOW	nltk/__init__.py	158	CODE
LOW	nltk/__init__.py	159	CODE
LOW	nltk/__init__.py	160	CODE
LOW	nltk/__init__.py	161	CODE
LOW	nltk/__init__.py	162	CODE
LOW	nltk/__init__.py	163	CODE
LOW	nltk/__init__.py	164	CODE
LOW	nltk/__init__.py	187	CODE
LOW	nltk/__init__.py	187	CODE
LOW	nltk/__init__.py	207	CODE
LOW	nltk/__init__.py	207	CODE
LOW	nltk/__init__.py	207	CODE
LOW	nltk/__init__.py	207	CODE
LOW	nltk/__init__.py	208	CODE
LOW	nltk/__init__.py	208	CODE
LOW	nltk/__init__.py	208	CODE
LOW	nltk/__init__.py	208	CODE
LOW	nltk/__init__.py	208	CODE
LOW	nltk/__init__.py	208	CODE
LOW	nltk/__init__.py	209	CODE
LOW	nltk/__init__.py	209	CODE
LOW	nltk/__init__.py	209	CODE
LOW	nltk/__init__.py	209	CODE
LOW	nltk/__init__.py	209	CODE
LOW	nltk/__init__.py	209	CODE
LOW	nltk/__init__.py	210	CODE
LOW	nltk/__init__.py	210	CODE
LOW	nltk/__init__.py	210	CODE
LOW	nltk/__init__.py	210	CODE
LOW	nltk/__init__.py	210	CODE
LOW	nltk/__init__.py	210	CODE
LOW	nltk/__init__.py	115	CODE
LOW	nltk/__init__.py	181	CODE
LOW	nltk/__init__.py	185	CODE
LOW	nltk/__init__.py	194	CODE
LOW	nltk/grammar.py	1589	CODE
LOW	nltk/book.py	9	CODE
LOW	nltk/book.py	18	CODE
596 more matches not shown…

Deep Nesting386 hits · 356 pts

Severity	File	Line	Context
LOW	tools/find_deprecated.py	116	CODE
LOW	tools/find_deprecated.py	151	CODE
LOW	tools/check_no_unsandboxed_open.py	54	CODE
LOW	nltk/internals.py	225	CODE
LOW	nltk/internals.py	792	CODE
LOW	nltk/internals.py	1062	CODE
LOW	nltk/internals.py	1474	CODE
LOW	nltk/tgrep.py	322	CODE
LOW	nltk/tgrep.py	403	CODE
LOW	nltk/downloader.py	2680	CODE
LOW	nltk/downloader.py	2947	CODE
LOW	nltk/downloader.py	579	CODE
LOW	nltk/downloader.py	738	CODE
LOW	nltk/downloader.py	1053	CODE
LOW	nltk/downloader.py	1168	CODE
LOW	nltk/downloader.py	1270	CODE
LOW	nltk/downloader.py	1481	CODE
LOW	nltk/downloader.py	1522	CODE
LOW	nltk/downloader.py	1552	CODE
LOW	nltk/downloader.py	1603	CODE
LOW	nltk/downloader.py	2016	CODE
LOW	nltk/downloader.py	2112	CODE
LOW	nltk/downloader.py	2180	CODE
LOW	nltk/downloader.py	2271	CODE
LOW	nltk/downloader.py	2505	CODE
LOW	nltk/toolbox.py	281	CODE
LOW	nltk/toolbox.py	468	CODE
LOW	nltk/util.py	229	CODE
LOW	nltk/util.py	308	CODE
LOW	nltk/util.py	350	CODE
LOW	nltk/util.py	439	CODE
LOW	nltk/util.py	482	CODE
LOW	nltk/util.py	561	CODE
LOW	nltk/util.py	647	CODE
LOW	nltk/util.py	698	CODE
LOW	nltk/util.py	1231	CODE
LOW	nltk/pathsec.py	149	CODE
LOW	nltk/pathsec.py	760	CODE
LOW	nltk/pathsec.py	890	CODE
LOW	nltk/pathsec.py	935	CODE
LOW	nltk/pathsec.py	1170	CODE
LOW	nltk/grammar.py	1374	CODE
LOW	nltk/grammar.py	1439	CODE
LOW	nltk/grammar.py	563	CODE
LOW	nltk/grammar.py	854	CODE
LOW	nltk/cli.py	42	CODE
LOW	nltk/collections.py	63	CODE
LOW	nltk/collections.py	396	CODE
LOW	nltk/probability.py	693	CODE
LOW	nltk/featstruct.py	1169	CODE
LOW	nltk/featstruct.py	1254	CODE
LOW	nltk/featstruct.py	1586	CODE
LOW	nltk/featstruct.py	2641	CODE
LOW	nltk/featstruct.py	154	CODE
LOW	nltk/featstruct.py	642	CODE
LOW	nltk/featstruct.py	680	CODE
LOW	nltk/featstruct.py	698	CODE
LOW	nltk/featstruct.py	777	CODE
LOW	nltk/featstruct.py	821	CODE
LOW	nltk/featstruct.py	957	CODE
326 more matches not shown…

Cross-File Repetition62 hits · 310 pts

Severity	File	Line	Snippet	Context
HIGH	nltk/grammar.py	0	s -> np vp [1.0] np -> det n [0.5] \| np pp [0.25] \| 'john' [0.1] \| 'i' [0.15] det -> 'the' [0.8] \| 'my' [0.2] n -> 'man'	STRING
HIGH	nltk/parse/pchart.py	0	s -> np vp [1.0] np -> det n [0.5] \| np pp [0.25] \| 'john' [0.1] \| 'i' [0.15] det -> 'the' [0.8] \| 'my' [0.2] n -> 'man'	STRING
HIGH	nltk/parse/viterbi.py	0	s -> np vp [1.0] np -> det n [0.5] \| np pp [0.25] \| 'john' [0.1] \| 'i' [0.15] det -> 'the' [0.8] \| 'my' [0.2] n -> 'man'	STRING
HIGH	nltk/grammar.py	0	s -> np vp [1.0] vp -> v np [.59] vp -> v [.40] vp -> vp pp [.01] np -> det n [.41] np -> name [.28] np -> np pp [.31] p	STRING
HIGH	nltk/parse/pchart.py	0	s -> np vp [1.0] vp -> v np [.59] vp -> v [.40] vp -> vp pp [.01] np -> det n [.41] np -> name [.28] np -> np pp [.31] p	STRING
HIGH	nltk/parse/viterbi.py	0	s -> np vp [1.0] vp -> v np [.59] vp -> v [.40] vp -> vp pp [.01] np -> det n [.41] np -> name [.28] np -> np pp [.31] p	STRING
HIGH	nltk/app/chunkparser_app.py	0	enter the tkinter mainloop. this function must be called if this window is created from a non-interactive program (e.g.	STRING
HIGH	nltk/app/rdparser_app.py	0	enter the tkinter mainloop. this function must be called if this window is created from a non-interactive program (e.g.	STRING
HIGH	nltk/app/srparser_app.py	0	enter the tkinter mainloop. this function must be called if this window is created from a non-interactive program (e.g.	STRING
HIGH	nltk/app/chartparser_app.py	0	enter the tkinter mainloop. this function must be called if this window is created from a non-interactive program (e.g.	STRING
HIGH	nltk/sem/drt_glue_demo.py	0	enter the tkinter mainloop. this function must be called if this window is created from a non-interactive program (e.g.	STRING
HIGH	nltk/draw/tree.py	0	enter the tkinter mainloop. this function must be called if this window is created from a non-interactive program (e.g.	STRING
HIGH	nltk/draw/util.py	0	enter the tkinter mainloop. this function must be called if this window is created from a non-interactive program (e.g.	STRING
HIGH	nltk/test/unit/test_pathsec_sweep_transitionparser.py	0	economic jj 2 att news nn 3 sbj has vbd 0 root little jj 5 att effect nn 3 obj on in 5 att financial jj 8 att markets nn	STRING
HIGH	nltk/test/unit/test_pickle_load_warnings.py	0	economic jj 2 att news nn 3 sbj has vbd 0 root little jj 5 att effect nn 3 obj on in 5 att financial jj 8 att markets nn	STRING
HIGH	nltk/test/unit/test_pickle_allowlist_security.py	0	economic jj 2 att news nn 3 sbj has vbd 0 root little jj 5 att effect nn 3 obj on in 5 att financial jj 8 att markets nn	STRING
HIGH	nltk/test/unit/test_reviews_security.py	0	run ``target(result_q, *args)`` in a separate process with a deadline. returns ``(finished, status, payload)``. if the w	STRING
HIGH	nltk/test/unit/test_chunk_redos_security.py	0	run ``target(result_q, *args)`` in a separate process with a deadline. returns ``(finished, status, payload)``. if the w	STRING
HIGH	nltk/test/unit/test_texttiling_security.py	0	run ``target(result_q, *args)`` in a separate process with a deadline. returns ``(finished, status, payload)``. if the w	STRING
HIGH	nltk/test/unit/test_inference_security.py	0	run ``target(result_q, *args)`` in a separate process with a deadline. returns ``(finished, status, payload)``. if the w	STRING
HIGH	nltk/translate/ibm2.py	0	probability of target sentence and an alignment given the source sentence	STRING
HIGH	nltk/translate/ibm3.py	0	probability of target sentence and an alignment given the source sentence	STRING
HIGH	nltk/translate/ibm4.py	0	probability of target sentence and an alignment given the source sentence	STRING
HIGH	nltk/translate/ibm5.py	0	probability of target sentence and an alignment given the source sentence	STRING
HIGH	nltk/translate/ibm1.py	0	probability of target sentence and an alignment given the source sentence	STRING
HIGH	nltk/translate/ibm2.py	0	data object to store counts of various parameters during training. includes counts for vacancies.	STRING
HIGH	nltk/translate/ibm3.py	0	data object to store counts of various parameters during training. includes counts for vacancies.	STRING
HIGH	nltk/translate/ibm4.py	0	data object to store counts of various parameters during training. includes counts for vacancies.	STRING
HIGH	nltk/translate/ibm5.py	0	data object to store counts of various parameters during training. includes counts for vacancies.	STRING
HIGH	nltk/sem/glue.py	0	pick an alphabetic character as identifier for an entity in the model. :param value: where to index into the list of cha	STRING
HIGH	nltk/sem/lfg.py	0	pick an alphabetic character as identifier for an entity in the model. :param value: where to index into the list of cha	STRING
HIGH	nltk/inference/mace.py	0	pick an alphabetic character as identifier for an entity in the model. :param value: where to index into the list of cha	STRING
HIGH	nltk/corpus/reader/reviews.py	0	:param root: the root directory for the corpus. :param fileids: a list or regexp specifying the fileids in the corpus. :	STRING
HIGH	nltk/corpus/reader/categorized_sents.py	0	:param root: the root directory for the corpus. :param fileids: a list or regexp specifying the fileids in the corpus. :	STRING
HIGH	nltk/corpus/reader/comparative_sents.py	0	:param root: the root directory for the corpus. :param fileids: a list or regexp specifying the fileids in the corpus. :	STRING
HIGH	nltk/corpus/reader/pros_cons.py	0	:param root: the root directory for the corpus. :param fileids: a list or regexp specifying the fileids in the corpus. :	STRING
HIGH	nltk/corpus/reader/reviews.py	0	return all words and punctuation symbols in the corpus or in the specified files/categories. :param fileids: a list or r	STRING
HIGH	nltk/corpus/reader/categorized_sents.py	0	return all words and punctuation symbols in the corpus or in the specified files/categories. :param fileids: a list or r	STRING
HIGH	nltk/corpus/reader/comparative_sents.py	0	return all words and punctuation symbols in the corpus or in the specified files/categories. :param fileids: a list or r	STRING
HIGH	nltk/corpus/reader/pros_cons.py	0	return all words and punctuation symbols in the corpus or in the specified files/categories. :param fileids: a list or r	STRING
HIGH	nltk/corpus/reader/aligned.py	0	:return: the given file(s) as a list of words and punctuation symbols. :rtype: list(str)	STRING
HIGH	nltk/corpus/reader/plaintext.py	0	:return: the given file(s) as a list of words and punctuation symbols. :rtype: list(str)	STRING
HIGH	nltk/corpus/reader/chunked.py	0	:return: the given file(s) as a list of words and punctuation symbols. :rtype: list(str)	STRING
HIGH	nltk/corpus/reader/tagged.py	0	:return: the given file(s) as a list of words and punctuation symbols. :rtype: list(str)	STRING
HIGH	nltk/corpus/reader/semcor.py	0	:return: the given file(s) as a list of words and punctuation symbols. :rtype: list(str)	STRING
HIGH	nltk/corpus/reader/aligned.py	0	:return: the given file(s) as a list of sentences or utterances, each encoded as a list of word strings. :rtype: list(li	STRING
HIGH	nltk/corpus/reader/plaintext.py	0	:return: the given file(s) as a list of sentences or utterances, each encoded as a list of word strings. :rtype: list(li	STRING
HIGH	nltk/corpus/reader/chunked.py	0	:return: the given file(s) as a list of sentences or utterances, each encoded as a list of word strings. :rtype: list(li	STRING
HIGH	nltk/corpus/reader/tagged.py	0	:return: the given file(s) as a list of sentences or utterances, each encoded as a list of word strings. :rtype: list(li	STRING
HIGH	nltk/corpus/reader/plaintext.py	0	:return: the given file(s) as a list of chapters, each encoded as a list of sentences, which are in turn encoded as list	STRING
HIGH	nltk/corpus/reader/chunked.py	0	:return: the given file(s) as a list of chapters, each encoded as a list of sentences, which are in turn encoded as list	STRING
HIGH	nltk/corpus/reader/tagged.py	0	:return: the given file(s) as a list of chapters, each encoded as a list of sentences, which are in turn encoded as list	STRING
HIGH	nltk/corpus/reader/plaintext.py	0	initialize the corpus reader. categorization arguments (``cat_pattern``, ``cat_map``, and ``cat_file``) are passed to th	STRING
HIGH	nltk/corpus/reader/markdown.py	0	initialize the corpus reader. categorization arguments (``cat_pattern``, ``cat_map``, and ``cat_file``) are passed to th	STRING
HIGH	nltk/corpus/reader/tagged.py	0	initialize the corpus reader. categorization arguments (``cat_pattern``, ``cat_map``, and ``cat_file``) are passed to th	STRING
HIGH	nltk/parse/recursivedescent.py	0	set the level of tracing output that should be generated when parsing a text. :type trace: int :param trace: the trace l	STRING
HIGH	nltk/parse/pchart.py	0	set the level of tracing output that should be generated when parsing a text. :type trace: int :param trace: the trace l	STRING
HIGH	nltk/parse/shiftreduce.py	0	set the level of tracing output that should be generated when parsing a text. :type trace: int :param trace: the trace l	STRING
HIGH	nltk/parse/viterbi.py	0	set the level of tracing output that should be generated when parsing a text. :type trace: int :param trace: the trace l	STRING
HIGH	nltk/inference/tableau.py	0	:param goal: input expression to prove :type goal: sem.expression :param assumptions: input expressions to use as assump	STRING
2 more matches not shown…

Excessive Try-Catch Wrapping166 hits · 191 pts

Severity	File	Line	Snippet	Context
LOW	tools/global_replace.py	38	except Exception:	CODE
MEDIUM	tools/global_replace.py	19	def update(file, pattern, replacement):	CODE
LOW	tools/huggingface/push_stopwords.py	199	except Exception:	CODE
LOW	nltk/internals.py	249	except Exception:	CODE
LOW	nltk/internals.py	260	except Exception:	CODE
LOW	nltk/internals.py	274	except Exception:	CODE
MEDIUM⚡	nltk/downloader.py	1515	print("Error reading from server: %s" % e)	CODE
MEDIUM⚡	nltk/downloader.py	1517	print("Error connecting to server: %s" % e.reason)	CODE
MEDIUM	nltk/downloader.py	1631	print(f"Error reading <{new_url!r}>:\n {e}")	CODE
MEDIUM	nltk/downloader.py	766	def _within(child, root):	CODE
LOW	nltk/downloader.py	1382	except Exception:	CODE
LOW	nltk/downloader.py	1630	except Exception as e:	CODE
LOW	nltk/downloader.py	2361	except Exception:	CODE
LOW	nltk/downloader.py	2712	except Exception as e:	CODE
LOW	nltk/downloader.py	2765	except Exception as e:	CODE
LOW	nltk/downloader.py	2785	except Exception as e:	CODE
LOW	nltk/downloader.py	2788	except Exception as e:	CODE
LOW	nltk/downloader.py	2969	except Exception as e:	CODE
LOW	nltk/downloader.py	2973	except Exception as e:	CODE
LOW	nltk/pathsec.py	278	except Exception:	CODE
LOW	nltk/pathsec.py	1002	except Exception:	CODE
LOW	nltk/picklesec.py	62	except Exception:	CODE
LOW	nltk/featstruct.py	2738	except Exception:	STRING
LOW	nltk/data.py	980	except Exception as e:	CODE
LOW	nltk/metrics/distance.py	306	except Exception:	CODE
MEDIUM	nltk/twitter/twitterclient.py	100	print(f"Error (stream will continue): {e}")	CODE
MEDIUM	nltk/twitter/twitterclient.py	117	print(f"Error (stream will continue): {e}")	CODE
LOW	nltk/app/concordance_app.py	403	except Exception:	CODE
LOW	nltk/app/concordance_app.py	654	except Exception as e:	CODE
MEDIUM	nltk/app/concordance_app.py	647	def run(self):	CODE
LOW	nltk/app/chunkparser_app.py	1463	except Exception:	CODE
MEDIUM	nltk/app/chunkparser_app.py	1333	def _chunkparse(self, words):	CODE
LOW	nltk/app/rdparser_app.py	711	except Exception:	CODE
LOW	nltk/app/rdparser_app.py	725	except Exception:	CODE
LOW	nltk/app/srparser_app.py	656	except Exception:	CODE
LOW	nltk/app/srparser_app.py	671	except Exception:	CODE
MEDIUM	nltk/app/chartparser_app.py	1816	print("Error creating Tree View")	CODE
LOW	nltk/app/chartparser_app.py	188	except Exception:	CODE
LOW	nltk/app/chartparser_app.py	532	except Exception:	CODE
LOW	nltk/app/chartparser_app.py	622	except Exception:	CODE
LOW	nltk/app/chartparser_app.py	800	except Exception as e:	CODE
LOW	nltk/app/chartparser_app.py	811	except Exception as e:	CODE
LOW	nltk/app/chartparser_app.py	1815	except Exception:	CODE
LOW	nltk/app/chartparser_app.py	2241	except Exception:	CODE
LOW	nltk/app/chartparser_app.py	2283	except Exception as e:	CODE
LOW	nltk/app/chartparser_app.py	2297	except Exception as e:	CODE
LOW	nltk/app/chartparser_app.py	2316	except Exception as e:	CODE
LOW	nltk/app/chartparser_app.py	2338	except Exception as e:	CODE
LOW	nltk/app/collocations_app.py	421	except Exception as e:	CODE
MEDIUM	nltk/app/collocations_app.py	406	def run(self):	CODE
LOW	nltk/app/nemo_app.py	95	except Exception:	STRING
LOW	nltk/app/nemo_app.py	115	except Exception:	STRING
LOW	nltk/app/wordnet_app.py	742	except Exception as e:	STRING
LOW	nltk/classify/weka.py	96	except Exception:	CODE
MEDIUM	nltk/classify/weka.py	89	def _check_weka_version(jar):	CODE
MEDIUM	nltk/test/unit/test_corpus_reader_pathsec.py	372	def _under_attacker_root(p):	CODE
LOW	nltk/test/unit/test_corpus_reader_pathsec.py	375	except Exception:	CODE
LOW	nltk/test/unit/test_corpus_reader_pathsec.py	405	except Exception:	CODE
LOW	nltk/test/unit/test_corpus_reader_pathsec.py	440	except Exception:	CODE
LOW	nltk/test/unit/test_corpus_reader_pathsec.py	454	except Exception:	CODE
106 more matches not shown…

Self-Referential Comments74 hits · 186 pts

Severity	File	Line	Snippet	Context
MEDIUM	tools/find_deprecated.py	55	# Define a regexp to search for deprecated definitions.	COMMENT
MEDIUM	nltk/downloader.py	1083	# Define a helper function for displaying output:	COMMENT
MEDIUM	nltk/downloader.py	1749	# Create the main window.	COMMENT
MEDIUM	nltk/downloader.py	1796	# Create the top-level frame structures	COMMENT
MEDIUM	nltk/downloader.py	1819	# Create the tabs	COMMENT
MEDIUM	nltk/downloader.py	1828	# Create the table.	COMMENT
MEDIUM	nltk/downloader.py	1934	# Create a menu to control which columns of the table are	COMMENT
MEDIUM	nltk/downloader.py	1949	# Create a sort menu	COMMENT
MEDIUM	nltk/downloader.py	2449	# Create a new data server object for the download operation,	COMMENT
MEDIUM	nltk/toolbox.py	332	"""This class is the base class for settings files."""	STRING
MEDIUM	nltk/lazyimport.py	1	# This module is from mx/DateTime/LazyModule.py and is	COMMENT
MEDIUM	nltk/text.py	217	# Create the pretty lines with the query_word in the middle.	COMMENT
MEDIUM	nltk/text.py	224	# Create the ConcordanceLine	COMMENT
MEDIUM	nltk/text.py	623	# Create the model when using it the first time.	COMMENT
MEDIUM	nltk/probability.py	1176	# Create a heldout probability distribution for each pair of	COMMENT
MEDIUM	nltk/probability.py	1325	# This method is problematic because the situation ``N(c+1) == 0``	COMMENT
MEDIUM	nltk/featstruct.py	2262	# Create the new feature structure	COMMENT
MEDIUM	nltk/tree/transforms.py	128	# This method is 7x faster which helps when parsing 40,000 sentences.	STRING
MEDIUM	nltk/app/rdparser_app.py	113	# Create the basic frames.	COMMENT
MEDIUM	nltk/app/srparser_app.py	122	# Create the basic frames.	COMMENT
MEDIUM	nltk/app/chartparser_app.py	449	# Create a widget for it.	COMMENT
MEDIUM	nltk/app/chartparser_app.py	1054	# Create the chart canvas.	COMMENT
MEDIUM	nltk/app/chartparser_app.py	1059	# Create the sentence canvas.	COMMENT
MEDIUM	nltk/app/chartparser_app.py	1070	# Create the tree canvas.	COMMENT
MEDIUM	nltk/app/chartparser_app.py	1789	# Create the root window.	COMMENT
MEDIUM⚡	nltk/classify/util.py	243	# Create a list of male names to be used as positive-labeled examples for training	COMMENT
MEDIUM⚡	nltk/classify/util.py	246	# Create a list of male and female names to be used as unlabeled examples	COMMENT
MEDIUM⚡	nltk/classify/util.py	249	# Create a test set with correctly-labeled male and female names	COMMENT
MEDIUM	nltk/classify/__init__.py	49	>>> # Define a feature detector function.	STRING
MEDIUM	nltk/classify/positivenaivebayes.py	142	# Create the P(label) distribution.	COMMENT
MEDIUM	nltk/classify/positivenaivebayes.py	147	# Create the P(fval\|label, fname) distribution.	COMMENT
MEDIUM	nltk/classify/naivebayes.py	235	# Create the P(label) distribution	COMMENT
MEDIUM	nltk/classify/naivebayes.py	238	# Create the P(fval\|label, fname) distribution	COMMENT
MEDIUM	nltk/test/unit/test_corpus_reader.py	24	# Create a symlink inside corpus_root that points outside corpus_root	COMMENT
MEDIUM	nltk/sem/drt.py	84	"""This method is intended to be overridden for logics that	STRING
MEDIUM	nltk/sem/util.py	140	# Initialize a variable assignment with parameter ``dom``	COMMENT
MEDIUM⚡	nltk/sem/logic.py	1958	"""This class represents implications"""	STRING
MEDIUM⚡	nltk/sem/logic.py	1965	"""This class represents biconditionals"""	STRING
MEDIUM⚡	nltk/sem/logic.py	1972	"""This class represents equality expressions like "(x = y)"."""	STRING
MEDIUM	nltk/sem/logic.py	323	"""This method is intended to be overridden for logics that	STRING
MEDIUM	nltk/sem/logic.py	1487	"""This class represents a variable to be used as a predicate or entity"""	STRING
MEDIUM	nltk/sem/logic.py	1559	"""This class represents variables that take the form of a single lowercase	STRING
MEDIUM	nltk/sem/logic.py	1589	"""This class represents variables that take the form of a single uppercase	STRING
MEDIUM	nltk/sem/logic.py	1604	"""This class represents variables that take the form of a single lowercase	STRING
MEDIUM	nltk/sem/logic.py	1611	"""This class represents variables that do not take the form of a single	STRING
MEDIUM	nltk/sem/logic.py	1932	"""This class represents conjunctions"""	STRING
MEDIUM	nltk/sem/logic.py	1945	"""This class represents disjunctions"""	STRING
MEDIUM	nltk/sem/drt_glue_demo.py	55	# Create the basic frames.	COMMENT
MEDIUM	nltk/corpus/europarl_raw.py	13	# Create a new corpus reader instance for each European language	COMMENT
MEDIUM	nltk/parse/pchart.py	508	# Define a list of parsers. We'll use all parsers.	COMMENT
MEDIUM	nltk/parse/chart.py	639	# Create the index.	COMMENT
MEDIUM	nltk/parse/nonprojectivedependencyparser.py	517	# Create a new node v_n+1 with address = len(nodes) + 1	COMMENT
MEDIUM	nltk/parse/featurechart.py	203	# Create the index.	COMMENT
MEDIUM	nltk/parse/featurechart.py	287	# Create a copy of the bindings.	STRING
MEDIUM	nltk/parse/featurechart.py	300	# Create a copy of the bindings.	STRING
MEDIUM	nltk/parse/earleychart.py	103	# Create the index.	COMMENT
MEDIUM	nltk/parse/earleychart.py	153	# Create the index.	COMMENT
MEDIUM	nltk/draw/cfg.py	185	# Create the top-level window.	STRING
MEDIUM	nltk/draw/cfg.py	248	# Create the basic Text widget & scrollbar.	STRING
MEDIUM	nltk/draw/cfg.py	582	# Create the basic frames	STRING
14 more matches not shown…

Cross-Language Confusion28 hits · 160 pts

Severity	File	Line	Snippet	Context
HIGH	nltk/featstruct.py	537	Return True if ``self`` subsumes ``other``. I.e., return true	STRING
HIGH	nltk/featstruct.py	1489	contain the unified value, the value of ``fstruct2`` is undefined,	STRING
HIGH	nltk/cluster/util.py	157	queue.push((priority, node))	CODE
HIGH	nltk/app/chartparser_app.py	1237	edgelen = max(edge.length(), 1)	CODE
HIGH⚡	nltk/classify/maxent.py	1130	fcount = numpy.zeros(encoding.length(), "d")	CODE
HIGH⚡	nltk/classify/maxent.py	1140	fcount = numpy.zeros(encoding.length(), "d")	CODE
HIGH	nltk/classify/maxent.py	123	assert encoding.length() == len(weights)	CODE
HIGH	nltk/classify/maxent.py	135	assert self._encoding.length() == len(new_weights)	CODE
HIGH	nltk/classify/maxent.py	244	self._encoding.length(),	CODE
HIGH	nltk/classify/maxent.py	1351	deltas = numpy.ones(encoding.length(), "d")	STRING
HIGH	nltk/classify/maxent.py	1356	A = numpy.zeros((len(nfmap), encoding.length()), "d")	STRING
HIGH	nltk/classify/maxent.py	1496	weights = parse_megam_weights(stdout, encoding.length(), explicit)	STRING
HIGH	nltk/classify/tadm.py	115	for i in range(encoding.length()):	CODE
HIGH⚡	nltk/test/unit/translate/test_stack_decoder.py	241	stack.push(_Hypothesis(0.2))	CODE
HIGH⚡	nltk/test/unit/translate/test_stack_decoder.py	242	stack.push(poor_hypothesis)	CODE
HIGH⚡	nltk/test/unit/translate/test_stack_decoder.py	243	stack.push(_Hypothesis(0.1))	CODE
HIGH⚡	nltk/test/unit/translate/test_stack_decoder.py	244	stack.push(_Hypothesis(0.3))	CODE
HIGH⚡	nltk/test/unit/translate/test_stack_decoder.py	256	stack.push(poor_hypothesis)	CODE
HIGH⚡	nltk/test/unit/translate/test_stack_decoder.py	257	stack.push(worse_hypothesis)	CODE
HIGH⚡	nltk/test/unit/translate/test_stack_decoder.py	258	stack.push(_Hypothesis(0.9)) # greatly superior hypothesis	CODE
HIGH⚡	nltk/test/unit/translate/test_stack_decoder.py	270	stack.push(_Hypothesis(0.9)) # greatly superior hypothesis	CODE
HIGH⚡	nltk/test/unit/translate/test_stack_decoder.py	271	stack.push(poor_hypothesis)	CODE
HIGH⚡	nltk/test/unit/translate/test_stack_decoder.py	282	stack.push(_Hypothesis(0.0))	CODE
HIGH⚡	nltk/test/unit/translate/test_stack_decoder.py	283	stack.push(best_hypothesis)	CODE
HIGH⚡	nltk/test/unit/translate/test_stack_decoder.py	284	stack.push(_Hypothesis(0.5))	CODE
HIGH	nltk/parse/pchart.py	422	queue.sort(key=lambda edge: edge.length())	CODE
HIGH	nltk/parse/chart.py	602	:param length: Only generate edges ``e`` where ``e.length()==length``	STRING
HIGH	nltk/parse/chart.py	888	edges = sorted((e.length(), e.start(), e) for e in self)	CODE

Over-Commented Block115 hits · 107 pts

Severity	File	Line	Snippet	Context
LOW	tools/global_replace.py	1	#!/usr/bin/env python	COMMENT
LOW	tools/github_actions/third-party.sh	101		COMMENT
LOW	nltk/internals.py	21		COMMENT
LOW	nltk/internals.py	41	#	COMMENT
LOW	nltk/internals.py	61	{"-server", "-client", "-xbatch", "-xint", "-xcomp", "-xmixed"}	COMMENT
LOW	nltk/internals.py	381	# Rebuild from the verified entries only; never pass the raw string to -cp.	COMMENT
LOW	nltk/downloader.py	741		COMMENT
LOW	nltk/downloader.py	781	break	COMMENT
LOW	nltk/pathsec.py	1201	# apply: build_opener() would install a default ProxyHandler from	COMMENT
LOW	nltk/grammar.py	1	# Natural Language Toolkit: Context Free Grammars	COMMENT
LOW	nltk/text.py	261	#: :meth:`Text.findall`). The search applies an attacker-influenceable token	COMMENT
LOW	nltk/redos.py	61	#: Wall-clock seconds any single caller-supplied-pattern match may run before it	COMMENT
LOW	nltk/probability.py	1	# Natural Language Toolkit: Probability and Statistics	COMMENT
LOW	nltk/probability.py	1301	##//////////////////////////////////////////////////////	COMMENT
LOW	nltk/probability.py	1321	# where c is the original count, N(i) is the number of event types	COMMENT
LOW	nltk/probability.py	1341	# appropriate for high values of r. For low values of r, they use the	COMMENT
LOW	nltk/probability.py	1661	##//////////////////////////////////////////////////////	COMMENT
LOW	nltk/probability.py	1681	# titled "Improved backing-off for n-gram language modeling." In the same paper	COMMENT
LOW	nltk/collocations.py	381	)	COMMENT
LOW	nltk/data.py	761	#: i.e. payloads crafted purely to maximize expansion; legitimate text/markup	COMMENT
LOW	nltk/decorators.py	241	## "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT	COMMENT
LOW	nltk/misc/sort.py	1	# Natural Language Toolkit: List Sorting	COMMENT
LOW	nltk/tree/immutable.py	1	# Natural Language Toolkit: Text Trees	COMMENT
LOW	nltk/tree/parented.py	1	# Natural Language Toolkit: Text Trees	COMMENT
LOW	nltk/tree/tree.py	1	# Natural Language Toolkit: Text Trees	COMMENT
LOW	nltk/tree/parsing.py	1	# Natural Language Toolkit: Text Trees	COMMENT
LOW	nltk/tree/probabilistic.py	1	# Natural Language Toolkit: Text Trees	COMMENT
LOW	nltk/metrics/confusionmatrix.py	61	indices = {val: i for (i, val) in enumerate(values)}	COMMENT
LOW	nltk/metrics/distance.py	21		COMMENT
LOW	nltk/app/srparser_app.py	821	# Display the available productions.	COMMENT
LOW	nltk/app/srparser_app.py	841	# color='#000000', font=self._font)	COMMENT
LOW	nltk/app/chartparser_app.py	2561	# ChartComparer(*charts).mainloop()	COMMENT
LOW	nltk/app/wordnet_app.py	241	# 'Shutdown' that first shutsdown the server and closes the window or	COMMENT
LOW	nltk/classify/weka.py	201	"of weka may not be supported.\n"	COMMENT
LOW	nltk/classify/api.py	161	# strs or ints, but can be any immutable type. The set	COMMENT
LOW	nltk/classify/api.py	181	# the i\ th element of ``featuresets``.	COMMENT
LOW	nltk/classify/maxent.py	1361	for label in encoding.labels():	COMMENT
LOW	nltk/chunk/regexp.py	901		COMMENT
LOW	nltk/test/pytest.ini	1	[pytest]	COMMENT
LOW	nltk/test/unit/test_quadratic_dos.py	221	def test_low_vowel_run_is_linear_and_unbounded(self):	COMMENT
LOW	nltk/test/unit/test_distance.py	21	"left,right,substitution_cost,expecteds",	COMMENT
LOW	nltk/test/unit/test_distance.py	41	#	COMMENT
LOW	nltk/test/unit/test_distance.py	61	# with transpositions:	COMMENT
LOW	nltk/test/unit/test_distance.py	81	# with transpositions:	COMMENT
LOW	nltk/test/unit/test_pickle_allowlist_security.py	301	restored = punkt_pickle_load(io.BytesIO(pickle.dumps(obj, protocol=4)))	COMMENT
LOW	nltk/test/unit/test_tokenize.py	1001		COMMENT
LOW	nltk/test/unit/lm/test_models.py	101	def test_mle_bigram_entropy_perplexity_unseen(mle_bigram_model):	COMMENT
LOW	nltk/test/unit/lm/test_models.py	161	return model	COMMENT
LOW	nltk/test/unit/lm/test_models.py	261	("d", ["c"], 2.0 / 9),	COMMENT
LOW	nltk/test/unit/lm/test_models.py	321	"word, context, expected_score",	COMMENT
LOW	nltk/test/unit/lm/test_models.py	381	# = 1 / 14	COMMENT
LOW	nltk/test/unit/lm/test_models.py	441	# P(c\|b) = alpha('bc') + gamma('b') * P(c)	COMMENT
LOW	nltk/chat/zen.py	81	),	COMMENT
LOW	nltk/chat/eliza.py	1	# Natural Language Toolkit: Eliza	COMMENT
LOW	nltk/translate/phrase_based.py	1	# Natural Language Toolkit: Phrase Extraction Algorithm	COMMENT
LOW	nltk/translate/ibm1.py	1	# Natural Language Toolkit: IBM Model 1	COMMENT
LOW	nltk/sem/chat80.py	701	else:	COMMENT
LOW	nltk/sem/hole.py	221	plug_acc = plug_acc0.copy()	COMMENT
LOW	nltk/sem/boxer.py	241	temp_filename,	COMMENT
LOW	nltk/sem/boxer.py	481	def _handle_duplex(self):	COMMENT
55 more matches not shown…

Redundant / Tautological Comments53 hits · 79 pts

Severity	File	Line	Snippet	Context
LOW	nltk/jsontags.py	64	# Check if we have a tagged object.	COMMENT
LOW	nltk/internals.py	614	# Check if a method has been overridden	COMMENT
LOW	nltk/internals.py	833	# Check if the alternative is inside a 'file' directory	COMMENT
LOW⚡	nltk/internals.py	853	# Check if the environment variable contains a direct path to the bin	COMMENT
LOW⚡	nltk/internals.py	859	# Check if the possible bin names exist inside the environment variable directories	COMMENT
LOW⚡	nltk/internals.py	867	# Check if the alternative is inside a 'file' directory	COMMENT
LOW⚡	nltk/internals.py	870	# Check if the alternative is inside a 'bin' directory	COMMENT
LOW	nltk/downloader.py	1206	# Check if the file has the correct size.	COMMENT
LOW	nltk/downloader.py	1214	# Check if the file's checksum matches.	COMMENT
LOW	nltk/downloader.py	1275	# Check if the index is already up-to-date. If so, do nothing.	COMMENT
LOW	nltk/downloader.py	1404	# Check if we are on GAE where we cannot write into filesystem.	COMMENT
LOW	nltk/downloader.py	1408	# Check if we have sufficient permissions to install in a	COMMENT
LOW	nltk/downloader.py	2493	# Check if we've been told to kill ourselves:	COMMENT
LOW	nltk/util.py	738	queue.append(child) # Add child to queue	CODE
LOW	nltk/__init__.py	189	# Check if tkinter exists without importing it to avoid crashes after	COMMENT
LOW	nltk/collections.py	468	# Return the value	COMMENT
LOW	nltk/probability.py	2499	# Print the results in a formatted table.	COMMENT
LOW	nltk/featstruct.py	1788	# Print the result.	COMMENT
LOW	nltk/featstruct.py	2374	# Check if it's a special feature.	COMMENT
LOW	nltk/featstruct.py	2380	# Check if this feature has a value already.	COMMENT
LOW	nltk/data.py	1133	# Check if the resource name includes a zipfile name	COMMENT
LOW	nltk/data.py	2144	# Return the result	COMMENT
LOW	nltk/app/chunkparser_app.py	714	# Check if we've seen this grammar already. If so, then	COMMENT
LOW	nltk/app/chunkparser_app.py	748	# Check if we're done	COMMENT
LOW	nltk/app/chunkparser_app.py	1301	# Display the results	COMMENT
LOW	nltk/app/rdparser_app.py	644	# Check if we just completed a parse.	COMMENT
LOW	nltk/app/chartparser_app.py	1273	# Check if we can fit the edge in this level.	COMMENT
LOW	nltk/app/wordnet_app.py	158	# Set type to plain to prevent XSS by printing the path as HTML	COMMENT
LOW	nltk/classify/weka.py	143	# Check if something went wrong:	COMMENT
LOW	nltk/classify/weka.py	373	# Check if the tokens are labeled or unlabeled. If unlabeled,	COMMENT
LOW	nltk/classify/maxent.py	731	# Return the result	COMMENT
LOW	nltk/classify/megam.py	100	# Write the file, which contains one line per instance.	COMMENT
LOW	nltk/translate/phrase_based.py	71	# Check if alignment points are consistent.	COMMENT
LOW	nltk/corpus/reader/util.py	248	# Check if it's in the cache.	COMMENT
LOW	nltk/corpus/reader/util.py	260	# Check if it's in the cache.	COMMENT
LOW	nltk/corpus/reader/wordnet.py	1923	# Open the file for reading. Note that we can not re-use	COMMENT
LOW	nltk/parse/pchart.py	266	# Assign probabilities to the trees.	COMMENT
LOW	nltk/parse/chart.py	685	# Add it to the list of edges.	COMMENT
LOW	nltk/parse/nonprojectivedependencyparser.py	646	# Set roots to attempt	COMMENT
LOW	nltk/parse/earleychart.py	539	# Print results.	COMMENT
LOW	nltk/stem/porter.py	736	# Print the results.	COMMENT
LOW	nltk/stem/lancaster.py	176	# Check if a user wants to strip prefix	COMMENT
LOW	nltk/stem/lancaster.py	178	# Check if a user wants to use his/her own rule tuples.	COMMENT
LOW	nltk/inference/tableau.py	143	# Check if the branch is closed. Return 'True' if it is	COMMENT
LOW	nltk/inference/tableau.py	166	# Check if the branch is closed. Return 'True' if it is	COMMENT
LOW	nltk/inference/tableau.py	189	# Check if the branch is closed. Return 'True' if it is	COMMENT
LOW	nltk/inference/tableau.py	203	# Check if the branch is closed. Return 'True' if it is	COMMENT
LOW	nltk/tag/brill_trainer.py	531	# Check if the change causes any rule at this position to	STRING
LOW	nltk/tag/brill_trainer.py	540	# Check if the change causes our templates to propose any	STRING
LOW	nltk/tokenize/punkt.py	1739	# Check if any initials or ordinals tokens that are marked	COMMENT
LOW	nltk/ccg/chart.py	178	# Check if the two edges are permitted to combine.	COMMENT
LOW	nltk/ccg/chart.py	311	# Output the resulting parses	COMMENT
LOW	nltk/ccg/chart.py	443	# Print the resulting category on a new line.	COMMENT

AI Slop Vocabulary27 hits · 46 pts

Severity	File	Line	Snippet	Context
MEDIUM	nltk/util.py	259	# FIX: Use is_relative_to for robust boundary check	COMMENT
LOW	nltk/collections.py	267	# If the slice is small enough, just use a tuple.	COMMENT
LOW	nltk/text.py	124	# nothing in common -- just return an empty freqdist.	COMMENT
LOW	nltk/probability.py	1330	# Simple Good-Turing. As a smoothing curve they simply use a power curve:	COMMENT
MEDIUM	nltk/picklesec.py	382	# scipy.datasets network + cache fetchers, robust to a submodule move.	COMMENT
LOW	nltk/featstruct.py	319	# their children a second time, so just return true.	COMMENT
LOW	nltk/data.py	1494	# If we've cached the resource, then just return it.	COMMENT
LOW	nltk/data.py	2070	# If nothing's buffered, then just return our current filepos:	COMMENT
LOW	nltk/app/chunkparser_app.py	715	# just use the old evaluation values.	COMMENT
MEDIUM⚡	nltk/test/unit/test_quadratic_dos.py	155	# O(n). A ratio test is robust to the (high) linear constant: 4x input	COMMENT
MEDIUM	nltk/test/unit/test_quadratic_dos.py	236	# This is a wall-clock ratio test, so it must be robust to CI timing	COMMENT
MEDIUM	nltk/test/unit/test_quadratic_dos.py	240	# (2) the MEDIAN of a few runs, robust to both a fast and a slow outlier.	COMMENT
MEDIUM	nltk/test/unit/test_stopwords_extended.py	115	# paradigm completions (NLTK already has `should've`, `mustn't`, ...)	COMMENT
LOW	nltk/test/unit/test_downloader.py	89	# Cannot mock a zip here as we are trying to validate file checksums, so just create a simple one with the XML	COMMENT
LOW	nltk/translate/ribes_score.py	368	# To avoid this, we can just return the lowest possible score.	COMMENT
MEDIUM	nltk/sem/drt.py	151	"""This method serves as a hook for other logic parsers that	STRING
MEDIUM	nltk/sem/drt.py	156	"""This method serves as a hook for other logic parsers that	STRING
MEDIUM	nltk/sem/logic.py	456	"""This method serves as a hook for other logic parsers that	STRING
MEDIUM	nltk/sem/logic.py	494	"""This method serves as a hook for other logic parsers that	STRING
MEDIUM	nltk/sem/logic.py	529	"""This method serves as a hook for other logic parsers that	STRING
LOW	nltk/corpus/__init__.py	194	# [XX] This should probably just use TaggedCorpusReader:	COMMENT
LOW	nltk/corpus/reader/verbnet.py	167	# File identifier: just return the xml.	COMMENT
LOW	nltk/corpus/reader/framenet.py	1183	# as it's easier to just call frame().	COMMENT
LOW	nltk/corpus/reader/semcor.py	201	# solution: just use the lemma name as a string	COMMENT
LOW	nltk/parse/chart.py	1230	# just return (no new edges to add).	STRING
LOW	nltk/parse/featurechart.py	377	# just return (no new edges to add).	STRING
LOW	nltk/parse/featurechart.py	562	# already in the chart, then just return it as-is.	STRING

Modern Structural Boilerplate37 hits · 34 pts

Severity	File	Line	Snippet	Context
LOW	nltk/jsontags.py	76	__all__ = ["register_tag", "json_tags", "JSONTaggedEncoder", "JSONTaggedDecoder"]	CODE
LOW	nltk/pathsec.py	1622	__all__ = [	CODE
LOW	nltk/xmlsec.py	60	__all__ = ["HAVE_DEFUSEDXML", "EntitiesForbidden", "fromstring", "parse"]	CODE
LOW	nltk/treeprettyprinter.py	28	__all__ = ["TreePrettyPrinter"]	CODE
LOW	nltk/grammar.py	1744	__all__ = [	CODE
LOW	nltk/text.py	822	__all__ = [	CODE
LOW	nltk/redos.py	59	__all__ = ["DEFAULT_TIMEOUT", "TimedPattern", "compile"]	STRING
LOW	nltk/probability.py	2553	__all__ = [	CODE
LOW	nltk/treetransforms.py	126	__all__ = ["chomsky_normal_form", "un_chomsky_normal_form", "collapse_unary"]	STRING
LOW	nltk/collocations.py	417	__all__ = [	CODE
LOW	nltk/featstruct.py	2801	__all__ = [	STRING
LOW	nltk/data.py	2209	__all__ = [	CODE
LOW	nltk/decorators.py	16	__all__ = ["decorator", "new_wrapper", "getinfo"]	CODE
LOW	nltk/tree/immutable.py	119	__all__ = [	CODE
LOW	nltk/tree/transforms.py	344	__all__ = ["chomsky_normal_form", "un_chomsky_normal_form", "collapse_unary"]	CODE
LOW	nltk/tree/parented.py	587	__all__ = [	CODE
LOW	nltk/tree/tree.py	1037	__all__ = [	STRING
LOW	nltk/tree/parsing.py	63	__all__ = [	CODE
LOW	nltk/tree/__init__.py	37	__all__ = [	CODE
LOW	nltk/tree/probabilistic.py	74	__all__ = ["ProbabilisticTree"]	CODE
LOW	nltk/tree/prettyprinter.py	624	__all__ = ["TreePrettyPrinter"]	CODE
LOW	nltk/app/concordance_app.py	709	__all__ = ["app"]	CODE
LOW	nltk/app/chunkparser_app.py	1500	__all__ = ["app"]	CODE
LOW	nltk/app/rdparser_app.py	1052	__all__ = ["app"]	CODE
LOW	nltk/app/srparser_app.py	937	__all__ = ["app"]	CODE
LOW	nltk/app/wordfreq_app.py	36	__all__ = ["app"]	CODE
LOW	nltk/app/chartparser_app.py	2570	__all__ = ["app"]	CODE
LOW	nltk/app/collocations_app.py	438	__all__ = ["app"]	CODE
LOW	nltk/app/nemo_app.py	163	__all__ = ["app"]	STRING
LOW	nltk/app/wordnet_app.py	1062	__all__ = ["app"]	STRING
LOW	nltk/classify/scikitlearn.py	43	__all__ = ["SklearnClassifier"]	CODE
LOW	nltk/test/unit/security_probes/__init__.py	26	__all__ = ["PROBES", "FIXED", "VULNERABLE", "STATIC", "BENIGN", "run"]	CODE
LOW	nltk/lm/__init__.py	225	__all__ = [	CODE
LOW	nltk/corpus/reader/__init__.py	118	__all__ = [	CODE
LOW⚡	nltk/parse/bllip.py	82	__all__ = ["BllipParser"]	CODE
LOW	nltk/parse/nonprojectivedependencyparser.py	26	logger = logging.getLogger(__name__)	CODE
LOW	nltk/huggingface/__init__.py	9	__all__ = ["download"]	CODE

Modern AI Meta-Vocabulary5 hits · 15 pts

Severity	File	Line	Snippet	Context
MEDIUM	nltk/translate/ribes_score.py	218	# (an O(L^2) scan) for every context window, giving up to O(L^4) time: a	COMMENT
MEDIUM	nltk/translate/ribes_score.py	277	# A context window longer than the reference can never occur in it	COMMENT
MEDIUM	nltk/translate/ribes_score.py	286	# Retrieve the right context window.	COMMENT
MEDIUM	nltk/translate/ribes_score.py	301	# Retrieve the left context window.	COMMENT
MEDIUM	nltk/tokenize/punkt.py	1647	# Debug scaffold: write through pathsec to a fresh private (0700) dir under	COMMENT

AI Structural Patterns11 hits · 10 pts

Severity	File	Line	Context
LOW	nltk/downloader.py	244	CODE
LOW	nltk/downloader.py	1053	CODE
LOW	nltk/tree/prettyprinter.py	331	CODE
LOW	nltk/metrics/agreement.py	229	CODE
LOW	nltk/metrics/agreement.py	381	CODE
LOW	nltk/twitter/twitterclient.py	320	CODE
LOW	nltk/corpus/reader/conll.py	67	CODE
LOW	nltk/tbl/demo.py	143	CODE
LOW	nltk/sentiment/util.py	327	CODE
LOW	nltk/tokenize/stanford_segmenter.py	94	CODE
LOW	nltk/tokenize/texttiling.py	67	CODE

Fake / Example Data5 hits · 4 pts

Severity	File	Line	Snippet	Context
LOW	nltk/app/nemo_app.py	18	Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna ali	CODE
LOW	nltk/app/nemo_app.py	18	Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna ali	CODE
LOW⚡	nltk/test/unit/test_quadratic_dos.py	176	"555-1234",	CODE
LOW	nltk/test/unit/test_tokenize.py	1324	== "Lorem ipsum dolor sit amet. consectetur adipiscing elit."	CODE
LOW	nltk/test/unit/test_tokenize.py	1324	== "Lorem ipsum dolor sit amet. consectetur adipiscing elit."	CODE

Slop Phrases1 hit · 3 pts

Severity	File	Line	Snippet	Context
MEDIUM	nltk/app/chunkparser_app.py	114	#: for a list of tags you can use for colorizing.	COMMENT

Dead Code1 hit · 2 pts

Severity	File	Line	Snippet	Context
MEDIUM	nltk/app/chartparser_app.py	628		CODE

Analysis Overview

What These Metrics Mean

Score History

Severity Breakdown

Directory Score Breakdown

Pattern Findings