Debian Science Linguistics packages

Apertium, open source jadro na strojový preklad pomocou plytkého prekladu, je v prvom rade zamerané na súvisiace dvojice jazykov.

Na lexikálne spracovanie používa konečné prevodníky, skryté Markovove modely na označovanie slovných druhov a konečné rozdeľovanie na prenos štruktúry.

Systém je vo veľkej miere založený na systémoch, ktoré už vyvinula skupina Transducens na Universitat d'Alacant, ako interNOSTRUM (Spanish-Catalan, http://www.internostrum.com/welcome.php) a Traductor Universia (Spanish- Portuguese, http://traductor.universia.net).

Jednoduchým poskytnutím lingvistických dát v správnom formáte bude možné použiť Apertium na zostavenie systémov na strojový preklad pre rozličné dvojice súvisiacich jazykov.

This package contails Perl scripts to evaluate Apertium-based machine translation output against reference: WER, PER, TER, BLEU.

Module for compiling lexical selection rules and processing them in the pipeline.

Artha je offline anglický synonymický slovník s výraznými vlastnosťami:

• vyhľadanie slova stlačením klávesovej skratky (vyberte text v ľubovoľnom okne, stlačte kláves a vyhľadáte slovo)
• hľadanie podľa regulárnych výrazov (rozšírenie vyhľadávania pomocou zástupných znakov ako *, ? atď.)
• pasívne oznámenia prac. prostredia (definícií slov na nerušenú prácu)
• pravopisné návrhy (keď nie je presný pravopis známy alebo je nejasný)

Po spustení monitoruje nastavenú klávesovú skratku. Ak je v niektorom okne vybraný text a stlačená klávesová skratka, zobrazí sa okno s vyhľadaným slovom. Ak používateľ uprednostňuje pasívne oznámenia, môže si ich zapnúť.

Ak sa hľadaný termín nenájde alebo je nejasný, je možné buď vo vyhľadávacom reťazci rozšíriť vyhľadávanie pomocou regulárnych výrazov (*, ? atď.) alebo použiť návrhy, ak je termín nesprávny.

Aby regulárne výrazy fungovali, musíte mať nainštalovaný balík wordnet-sense-index.

Constraint Grammar compiler and applicator for the 3rd edition of CG that is developed and maintained by VISL SDU and GrammarSoft ApS.

CG-3 can be used for disambiguation of morphology, syntax, semantics, etc; dependency markup, target language lemma choice for MT, QA systems, and much more. The core idea is that you choose what to do based on the whole available context, as opposed to n-grams.

See https://visl.sdu.dk/cg3.html for more documentation

Collatinus can be used to lemmatise latin texts, i.e. extract words and make a lexicon which indicates for each word its canonic form, and how the form actually found in the text was derived from it, for instance by declining it. Example : rosam gives : rosa-rosae -- acc. sing. Collatinus provides a nice graphic front-end to each operation.

Collatinus-nouus (stands for Collatinus, new generation) replaces every previous version of Collatinus.

This package provides a documentation in HTML format.

Dimbl obaľuje klasifikátor k najbližších susedov v TiMBL. Ponúka paralelnú klasifikáciu na počítačoch s viacerými CPU. Dimbl rozdelí pôvodnú tréningovú množinu, zostaví samostatné klasifikátory TiMBL pre každú podmnožinu a zlúči ich sady najbližších susedov podľa klasifikovanej inštancie.

Vlastnosti Dimbl:

• je pekným obalom TiMBL a zachováva voľby príkazového riadka
• vie pracovať s viacerými jadrami
• využíva špecifikáciu OpenMP na paralelné programovanie
• dosahuje superlineárne zlepšenie rýchlosti v porovnaní so štandardným TiMBL

Dimbl je produktom výskumnej skupiny ILK (Tilburg University, Holandsko).

Ak sa venujete výskumu spracovania prirodzených jazykov pomocou metódy pamäťovej výuky, Dimbl sa vám pravdepodobne hodí.

fastText is a library for efficient learning of word representations and sentence classification, which refers subword information to enrich word vectors.

Memory-Based Learning (MBL) is a machine-learning method applicable to a wide range of tasks in Natural Language Processing (NLP).

Frog is a modular system integrating a morphosyntactic tagger, lemmatizer, morphological analyzer, and dependency parser for natural languages. It is based upon it's predecessor TADPOLE (TAgger, Dependency Parser, and mOrphoLogical analyzEr). Using Memory-Based Learning techniques, frog tokenizes, tags, lemmatizes, and morphologically segments word tokens in incoming UTF-8 text files, and assigns a dependency graph to each sentence. Frog is particularly targeted at the increasing need for fast, automatic NLP systems applicable to very large (multi-million to billion word) document collections that are becoming available due to the progressive digitization of both new and old textual data. Up to now, frog has only been tested and used using corpora of Dutch natural language (see the frogdata package for samples).

Frog is a product of the Centre of Language and Speech Technology at Radboud University Nijmegen, it subsumes previous work by the ILK Research Group (Tilburg University, The Netherlands) and the CLiPS Research Centre (University of Antwerp, Belgium). It is currently maintained at the KNAW Humanities Cluster.

If you do scientific research in NLP, Frog will likely be of use to you.

Data package providing Giellatekno language resources for North Saami.

The Helsinki Finite-State Transducer software is intended for the implementation of morphological analysers and other tools which are based on weighted and unweighted finite-state transducer technology.

Minimal HFST optimized lookup format based spell checker library and a demonstrational implementation of command line based spell checker.

The IRST Language Modeling Toolkit can be used to learn a language model from data. The generated n-gram models should be usable on any system supporting ARPA language model format.

This package provides the command line tools.

Detects over 80 languages in UTF-8 text, based largely on groups of four letters. Also tables for 160+ language version.

This is the development package.

Sleator, D. a Temperley, D. v „Parsing English with a Link Grammar“ (1991) definujú nový systém formálnych gramatík zvaný „spojová gramatika“. Postupnosť slov je v jazyku spojovej gramatiky ak existuje možnosť ako nakresliť „spojenia“ medzi slovami tak, aby boli uspokojené lokálne požiadavky každého slova, aby sa spojenia nepretínali a slová tvorili súvislý graf. Autori do takéhoto systému zakódovali gramatiku angličtiny a napísali tento program na syntaktickú analýzu angličtiny pomocou tejto gramatiky.

link-grammar je možné použiť na lingvistickú syntaktickú analýzu za účelom získavania informácií alebo extrakcie z dokumentov v prirodzenom jazyku. Tiež je možné použiť ho na kontrolu gramatiky.

Tento balík obsahuje používateľský spustiteľný súbor.

The lttoolbox contains the augmented letter transducer tools for natural language processing used by Apertium, a platform for building rule-based and hybrid machine translation systems. The software is also useful for making morphological analysers and generators for natural language processing applications.

MBT is a memory-based tagger-generator and tagger in one. The tagger-generator part can generate a sequence tagger on the basis of a training set of tagged sequences; the tagger part can tag new sequences. MBT can, for instance, be used to generate part-of-speech taggers or chunkers for natural language processing. Features:

• Tagger generation: tagged text in, tagger out,
• Optional feedback loop: feed previous tag decision back to input of next decision,
• Easily customizable feature representation; can incorporate user-provided features,
• Automatic generation of separate sub-taggers for known words and unknown words,
• Can make use of full algorithmic parameters of TiMBL.

MBT is a product of the Centre of Language and Speech Technology (Radboud University Nijmegen, The Netherlands), the ILK Research Group (Tilburg University, The Netherlands) and the CLiPS Research Centre (University of Antwerp, Belgium).

If you do scientific research in natural language processing, MBT will likely be of use to you.

MbtServer extends Mbt with a server layer, running as a TCP server. Mbt is a memory-based tagger-generator and tagger for natural language processing. MbtServer provides the possibility to access a trained tagger from multiple sessions. It also allows one to run and access different taggers in parallel.

MbtServer is a product of the Centre for Language and Speech Technology (Radboud University, Nijmegen, The Netherlands), the ILK Research Group (Tilburg University, The Netherlands) and the CLiPS Research Centre (University of Antwerp, Belgium).

If you do scientific research in natural language processing, MbtServer will likely be of use to you.

The Apache OpenNLP library is a machine learning based toolkit for the processing of natural language text. It supports the most common NLP tasks, such as tokenization, sentence segmentation, part-of-speech tagging, named entity extraction, chunking, parsing, and coreference resolution. These tasks are usually required to build more advanced text processing services. OpenNLP also included maximum entropy and perceptron based machine learning.

This package contains the command line wrapper.

PyNLPl, pronounced as 'pineapple', is a Python library for Natural Language Processing. It contains various modules useful for common, and less common, NLP tasks. PyNLPl can be used for basic tasks such as the extraction of n-grams and frequency lists, and to build simple language models. It also contains complex data types and algorithms. Moreover, it includes parsers for file formats common in NLP (e.g. FoLiA/Giza/Moses/ARPA/Timbl/CQL) and clients to interface with various NLP specific servers. PyNLPl most notably features a very extensive library for working with FoLiA XML (Format for Linguistic Annotation).

This is the Python 3 version.

Thinc is the machine learning library powering spaCy https://spacy.io. It features a battle-tested linear model designed for large sparse learning problems, and a flexible neural network model under development for spaCy v2.0 https://spacy.io/usage/v2.

Thinc is a practical toolkit for implementing models that follow the "Embed, encode, attend, predict" architecture. It's designed to be easy to install, efficient for CPU usage and optimised for NLP and deep learning with text – in particular, hierarchically structured input and variable-length sequences.

An R implementation of the LexRank algorithm implementing stochastic graph-based method for computing relative importance of textual units for Natural Language Processing. The technique on the problem of Text Summarization (TS) is tested. Extractive TS relies on the concept of sentence salience to identify the most important sentences in a document or set of documents. Salience is typically defined in terms of the presence of particular important words or in terms of similarity to a centroid pseudo-sentence.

An R interface to the C libstemmer library that implements Porter's word stemming algorithm for collapsing words to a common root to aid comparison of vocabulary. Currently supported languages are Danish, Dutch, English, Finnish, French, German, Hungarian, Italian, Norwegian, Portuguese, Romanian, Russian, Spanish, Swedish and Turkish.

SentencePiece is an unsupervised text tokenizer/detokenizer mainly designed for Neural Network-based text generation systems where the vocabulary size is predetermined prior to the neural model training.

Memory-Based Learning (MBL) is a machine-learning method applicable to a wide range of tasks in Natural Language Processing (NLP).

The Tilburg Memory Based Learner, TiMBL, is a tool for NLP research, and for many other domains where classification tasks are learned from examples. It is an efficient implementation of k-nearest neighbor classifier.

TiMBL's features are:

• Fast, decision-tree-based implementation of k-nearest neighbor classification;

• Implementations of IB1 and IB2, IGTree, TRIBL, and TRIBL2 algorithms;

• Similarity metrics: Overlap, MVDM, Jeffrey Divergence, Dot product, Cosine;

• Feature weighting metrics: information gain, gain ratio, chi squared, shared variance;

• Distance weighting metrics: inverse, inverse linear, exponential decay;

• Extensive verbosity options to inspect nearest neighbor sets;
• Server functionality and extensive API;
• Fast leave-one-out testing and internal cross-validation;
• and Handles user-defined example weighting.

TiMBL is a product of the Centre of Language and Speech Technology (Radboud University, Nijmegen, The Netherlands), the ILK Research Group (Tilburg University, The Netherlands) and the CLiPS Research Centre (University of Antwerp, Belgium).

If you do scientific research in NLP, timbl will likely be of use to you.

timblserver is a TiMBL wrapper; it adds server functionality to TiMBL. It allows TiMBL to run multiple experiments as a TCP server, optionally via HTTP.

The Tilburg Memory Based Learner, TiMBL, is a tool for Natural Language Processing research, and for many other domains where classification tasks are learned from examples.

TimblServer is a product of the ILK Research Group (Tilburg University, The Netherlands) and the CLiPS Research Centre (University of Antwerp, Belgium).

If you do scientific research in NLP, TimblServer will likely be of use to you.

Ucto can tokenize UTF-8 encoded text files (i.e. separate words from punctuation, split sentences, generate n-grams), and offers several other basic preprocessing steps that make your text suited for further processing such as indexing, part-of-speech tagging, or machine translation.

This package provides the command-line tool itself.

Ucto was written by Maarten van Gompel and Ko van der Sloot. Work on Ucto was funded by NWO, the Netherlands Organisation for Scientific Research, under the Implicit Linguistics project, the CLARIN-NL program, and the CLARIAH project.

Ucto is a product of the Centre of Language and Speech Technology (Radboud University Nijmegen), and previously the ILK Research Group (Tilburg University, The Netherlands).

If you are interested in machine parsing of UTF-8 encoded text files, e.g. to do scientific research in natural language processing, ucto will likely be of use to you.

Ucto can tokenize UTF-8 encoded text files (i.e. separate words from punctuation, split sentences, generate n-grams), and offers several other basic preprocessing steps that make your text suited for further processing such as indexing, part-of-speech tagging, or machine translation.

This package provides necessary language-specific datafiles for running Ucto.

Ucto was written by Maarten van Gompel and Ko van der Sloot. Work on Ucto was funded by NWO, the Netherlands Organisation for Scientific Research, under the Implicit Linguistics project, the CLARIN-NL program, and the CLARIAH project.

Ucto is a product of the Centre of Language and Speech Technology (Radboud University Nijmegen), and previously the ILK Research Group (Tilburg University, The Netherlands).

WordNet(C) je online systém lexikálnych odkazov, ktorého dizajn sa inšpiruje aktuálnymi psycholinguistickými teóriami ľudskej lexikálnej pamäti. Anglické podstatné mená, slovesá, prídavné mená a príslovky sú organizované do množín synoným, z ktorých každá reprezentuje ich lexikálny koncept. Množiny synoným sú prepojené pomocou rôznych vzťahov.

WordNet vyvinuli v Cognitive Science Laboratory na Princeton University pod vedením hlavného riešiteľa, ktorým bol profesor George A. Miller.

WordNet sa považuje za najdôležitejší dostupný zdroj pre v oblasti výskumníkov výpočtovej lingvistiky, analýzy diskurzu a mnohých súvisiacich oblastiach.

Spustiteľný súbor, manuálové stránky programu WordNet a tiež všeobecné manuálové stránky.

This is a transitional dummy package. It can safely be removed.

Tento balík obsahuje Apertium APY, čo je jednoduché API Apertium napísané v jazyku Python 3 určené ako okamžitá náhrada za ScaleMT.

Balík dát, ktorý poskytuje zdroje Apertium v jazyku aragónčina.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi aragónčina a katalánčina.

Data package providing Apertium language resources for Belarusian.

Data package providing Apertium language resources for translating between the Belarusian and Russian languages.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi bretónčina a francúzština.

This is a transitional dummy package. It can safely be removed.

Data package providing Apertium language resources for Catalan.

Data package providing Apertium language resources for translating between the Catalan and Sardinian languages.

Data package providing Apertium language resources for Crimean Tatar

Data package providing Apertium language resources for translating between the Crimean Tatar and Turkish languages.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi waleština a angličtina.

Data package providing Apertium language resources for Danish.

Data package providing Apertium language resources for translating from the Danish to the Norwegian Nynorsk/Norwegian Bokmål variants and from Danish to Norwegian Nynorsk.

This is a transitional dummy package. It can safely be removed.

This is a transitional dummy package. It can safely be removed.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi angličtina a galícijčina.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi esperanto a katalánčina.

This is a linguistic package for the Apertium shallow-transfer machine translation system. The package can be used to translate between Esperanto and English.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi esperanto a španielčina.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi esperanto a francúzština.

This is a transitional dummy package. It can safely be removed.

Toto je prechodový fiktívny balík. Je možné ho bezpečne odstrániť.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi španielčina a galícijčina.

Toto je prechodový fiktívny balík. Je možné ho bezpečne odstrániť.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi španielčina a portugalčina.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi španielčina a rumunčina.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi baskičtina a angličtina.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi baskičtina a španielčina.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi francúzština a španielčina.

Data package providing Apertium language resources for French.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi francúzština a katalánčina.

Data package providing Apertium language resources for Serbo-Croatian.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi srbochorvátčina a angličtina.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi srbochorvátčina a macedónčina.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi srbochorvátčina a slovinčina.

Data package providing Apertium language resources for Hindi.

This is a transitional dummy package. It can safely be removed.

This is a transitional dummy package. It can safely be removed.

Data package providing Apertium language resources for Icelandic.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi islandčina a angličtina.

Data package providing Apertium language resources for Italian.

Balík dát, ktorý poskytuje zdroje Apertium v jazyku kazaština.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi kazaština a tatárčina.

This is a transitional dummy package. It can safely be removed.

This is a transitional dummy package. It can safely be removed.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi maltčina a arabčina.

Data package providing Apertium language resources for Norwegian Nynorsk.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi nórsky nynorsk a nórsky bokmål.

Data package providing Apertium language resources for Norwegian Bokmål.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi okcitánčina a katalánčina.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi okcitánčina a španielčina.

Data package providing Apertium language resources for Occitan.

Data package providing Apertium language resources for Polish.

Data package providing Apertium language resources for translating between the Polish and Silesian languages.

This is a transitional dummy package. It can safely be removed.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi portugalčina a galícijčina.

Data package providing Apertium language resources for Russian

Apertium module for reordering separable/discontiguous multiwords.

Balík dát, ktorý poskytuje zdroje Apertium na preklad medzi jazykmi severná saamčina a nórsky bokmål.

Data package providing Apertium language resources for Spanish.