Debian Science Linguistics packages

Apertium ist ein quelloffenes, zunächst auf Paare verwandter Sprachen gerichtetes, Programm für maschinelle Übersetzung mit »flacher Übertragung«.

Es verwendet endliche Übertrager (finite-state transducers) für die lexikalische Verarbeitung, Hidden-Markov-Modelle für die Auszeichnung von Teilen der Sprache und Unterteilung mit endlichen Automaten (finite-state based chunking) für die Übertragung der Struktur.

Das System stützt sich weitgehend auf bereits von der »Transducens group« an der »Universitat d'Alacant« entwickelte Systeme wie interNOSTRUM (Spanisch-Katalanisch, http://www.internostrum.com/welcome.php) und Traductor Universia (Spanisch-Portugiesisch, http://traductor.universia.net).

Es wird möglich sein, mit Apertium maschinelle Übersetzungssysteme für eine Reihe von verwandten Sprachpaaren zu erstellen. Sie müssen nur die linguistischen Daten im richtigen Format zur Verfügung stellen.

This package contails Perl scripts to evaluate Apertium-based machine translation output against reference: WER, PER, TER, BLEU.

Module for compiling lexical selection rules and processing them in the pipeline.

Artha ist ein Offline-Englisch-Thesaurus mit verschiedenen Eigenschaften:

• Nachschlagen mit einem globalem Tastaturkürzel (wählen Sie Text in einem beliebigen Fenster aus und drücken Sie das voreingestellte Tastaturkürzel, um nachzuschlagen)
• Suchen basierend auf regulären Ausdrücken (erweitern Sie die Suche durch Platzhalter wie *, ?, usw.)
• Passive Arbeitsflächen-Mitteilungen (von Wortdefinitionen für ungestörten Arbeitsfluss)
• Vorschläge zur Schreibweise (wenn die genaue Schreibweise unklar oder unbekannt ist)

Einmal gestartet, wartet Artha auf eine vorher definierte Tastenkombination. Wurde Text in einem beliebigen Fenster ausgewählt und das Tastaturkürzel gedrückt, erscheint ein Nachrichtenfenster mit dem nachgeschlagenen Ausdruck. Möchte der Benutzer lieber passive Benachrichtigungen, kann dies in den Benachrichtigungsoptionen aktiviert werden.

Wenn der gesuchte Ausdruck unklar/unbekannt ist, kann die Suche entweder mit regulären Ausdrücken (*, ?, usw.) im Suchausdruck erweitert oder es können Vorschläge zur Schreibweise betrachtet werden, wenn der Ausdruck falsch ist.

Damit die Suche mit regulären Ausdrücken funktioniert, muss das Paket wordnet-sense-index installiert sein.

Constraint Grammar compiler and applicator for the 3rd edition of CG that is developed and maintained by VISL SDU and GrammarSoft ApS.

CG-3 can be used for disambiguation of morphology, syntax, semantics, etc; dependency markup, target language lemma choice for MT, QA systems, and much more. The core idea is that you choose what to do based on the whole available context, as opposed to n-grams.

See https://visl.sdu.dk/cg3.html for more documentation

Collatinus kann zur Lemmatisierung von lateinischen Texten verwendet werden. Das heißt, Sie können Wörter extrahieren und ein Lexikon erstellen. Das Lexikon enthält für jedes Wort seine kanonische Form und erklärt, wie die im Text gefundene Form daraus abgeleitet wurde, z.B. durch Deklinieren. Beispiel: rosam ergibt: rosa-rosae -- akk. sing. Collatinus stellt für alle Operationen eine nette grafische Oberfläche bereit.

Collatinus-nouus (steht für Collatinus, die neue Generation) ersetzt alle älteren Versionen von Collatinus.

Dieses Paket enthält die Dokumentation im HTML-Format.

Dimbl ist eine KNN-Schnittstelle für TiMBL, die parallele Klassifikation auf Mehrprozessorsystemen erlaubt. Dimbl unterteilt den ursprünglichen Satz von Trainingsdaten, erzeugt separate TimBL Klassen für jede Untergruppe des Trainingsdatensatzes und fügt ihre zugeordneten Klassen k-nächster Nachbarn separat für jede Gruppe zusammen.

Die Hauptmerkmale von Dimbl sind:

• Es fügt sich nahtlos mit TiMBL zusammen und übernimmt alle Befehlszeilenoptionen.
• Es kann sowohl mit Vielkern- als auch mit Zweikern- und Vierkernprozessoren umgehen.
• Die OpenMP Spezifikation für parallele Datenverarbeitung wird zugrundegelegt.
• Dimbl kann gegenüber normalem TiMBL einen nichtlinearen Geschwindigkeitszuwachs ermöglichen.

Dimbl wurde von der ILK Forschungsgruppe der Universität Tilburg in den Niederlanden bereitgestellt.

Dimbl wendet sich an Forscher aus dem Bereich der maschinellen Verarbeitung natürlicher Sprache auf Basis überwachten Lernens.

fastText is a library for efficient learning of word representations and sentence classification, which refers subword information to enrich word vectors.

Memory-Based Learning (MBL) is a machine-learning method applicable to a wide range of tasks in Natural Language Processing (NLP).

Frog is a modular system integrating a morphosyntactic tagger, lemmatizer, morphological analyzer, and dependency parser for natural languages. It is based upon it's predecessor TADPOLE (TAgger, Dependency Parser, and mOrphoLogical analyzEr). Using Memory-Based Learning techniques, frog tokenizes, tags, lemmatizes, and morphologically segments word tokens in incoming UTF-8 text files, and assigns a dependency graph to each sentence. Frog is particularly targeted at the increasing need for fast, automatic NLP systems applicable to very large (multi-million to billion word) document collections that are becoming available due to the progressive digitization of both new and old textual data. Up to now, frog has only been tested and used using corpora of Dutch natural language (see the frogdata package for samples).

Frog is a product of the Centre of Language and Speech Technology at Radboud University Nijmegen, it subsumes previous work by the ILK Research Group (Tilburg University, The Netherlands) and the CLiPS Research Centre (University of Antwerp, Belgium). It is currently maintained at the KNAW Humanities Cluster.

If you do scientific research in NLP, Frog will likely be of use to you.

Data package providing Giellatekno language resources for North Saami.

The Helsinki Finite-State Transducer software is intended for the implementation of morphological analysers and other tools which are based on weighted and unweighted finite-state transducer technology.

Minimale, für das HFST-Nachschlageformat optimierte Rechtschreibprüfungsbibliothek und eine Demonstrationsimplementierung einer befehlszeilenbasierten Rechtschreibprüfung.

Das IRST Sprachmodulierungs-Toolkit dient dazu, ein Sprachmodell aus Daten einzulernen. Die erzeugten n-gram Modelle sollten auf jedem System nutzbar ein, das ARPA-Sprachmodelle unterstützt.

Dieses Paket enthält die Kommandozeilen-Tools.

Detects over 80 languages in UTF-8 text, based largely on groups of four letters. Also tables for 160+ language version.

This is the development package.

Die Autoren von »Parsing English with a Link Grammar«, D. Sleator und D. Temperley, definierten ein neues formales Grammatiksystem, genannt »link grammar«. Eine Folge von Wörtern gehört zu der Sprache einer »link grammar«, wenn eine Möglichkeit existiert, »Verbindungen« zwischen den Wörtern so herzustellen, dass die lokalen Anforderungen jedes Wortes erfüllt sind, die Verbindungen sich nicht überschneiden und die Worte einen zusammenhängenden Graphen bilden. Die Autoren kodierten die englische Grammatik mit diesem System und schrieben dieses Programm, um die englische Sprache zu analysieren.

Link-grammar kann zur linguistischen Analyse, zur Informationsbeschaffung oder zur Extraktion aus natürlichsprachigen Dokumenten genutzt werden. Es kann auch zur grammatikalischen Prüfung genutzt werden.

Dieses Paket enthält den vom Benutzer ausführbaren Binärcode.

Die Lttoolbox enthält die von Apertium genutzten erweiterten Zeichenübertragungswerkzeuge für die Verarbeitung natürlicher Sprache. Apertium ist eine Plattform für die Erzeugung regelbasierter und hybrider maschineller Übersetzungssysteme. Die Software ist auch nützlich für die Erstellung morphologischer Analysatoren und Generatoren für Anwendungen zur Verarbeitung natürlicher Sprache.

MBT is a memory-based tagger-generator and tagger in one. The tagger-generator part can generate a sequence tagger on the basis of a training set of tagged sequences; the tagger part can tag new sequences. MBT can, for instance, be used to generate part-of-speech taggers or chunkers for natural language processing. Features:

• Tagger generation: tagged text in, tagger out,
• Optional feedback loop: feed previous tag decision back to input of next decision,
• Easily customizable feature representation; can incorporate user-provided features,
• Automatic generation of separate sub-taggers for known words and unknown words,
• Can make use of full algorithmic parameters of TiMBL.

MBT is a product of the Centre of Language and Speech Technology (Radboud University Nijmegen, The Netherlands), the ILK Research Group (Tilburg University, The Netherlands) and the CLiPS Research Centre (University of Antwerp, Belgium).

If you do scientific research in natural language processing, MBT will likely be of use to you.

MbtServer extends Mbt with a server layer, running as a TCP server. Mbt is a memory-based tagger-generator and tagger for natural language processing. MbtServer provides the possibility to access a trained tagger from multiple sessions. It also allows one to run and access different taggers in parallel.

MbtServer is a product of the Centre for Language and Speech Technology (Radboud University, Nijmegen, The Netherlands), the ILK Research Group (Tilburg University, The Netherlands) and the CLiPS Research Centre (University of Antwerp, Belgium).

If you do scientific research in natural language processing, MbtServer will likely be of use to you.

The Apache OpenNLP library is a machine learning based toolkit for the processing of natural language text. It supports the most common NLP tasks, such as tokenization, sentence segmentation, part-of-speech tagging, named entity extraction, chunking, parsing, and coreference resolution. These tasks are usually required to build more advanced text processing services. OpenNLP also included maximum entropy and perceptron based machine learning.

This package contains the command line wrapper.

PyNLPl, pronounced as 'pineapple', is a Python library for Natural Language Processing. It contains various modules useful for common, and less common, NLP tasks. PyNLPl can be used for basic tasks such as the extraction of n-grams and frequency lists, and to build simple language models. It also contains complex data types and algorithms. Moreover, it includes parsers for file formats common in NLP (e.g. FoLiA/Giza/Moses/ARPA/Timbl/CQL) and clients to interface with various NLP specific servers. PyNLPl most notably features a very extensive library for working with FoLiA XML (Format for Linguistic Annotation).

This is the Python 3 version.

Thinc is the machine learning library powering spaCy https://spacy.io. It features a battle-tested linear model designed for large sparse learning problems, and a flexible neural network model under development for spaCy v2.0 https://spacy.io/usage/v2.

Thinc is a practical toolkit for implementing models that follow the "Embed, encode, attend, predict" architecture. It's designed to be easy to install, efficient for CPU usage and optimised for NLP and deep learning with text – in particular, hierarchically structured input and variable-length sequences.

An R implementation of the LexRank algorithm implementing stochastic graph-based method for computing relative importance of textual units for Natural Language Processing. The technique on the problem of Text Summarization (TS) is tested. Extractive TS relies on the concept of sentence salience to identify the most important sentences in a document or set of documents. Salience is typically defined in terms of the presence of particular important words or in terms of similarity to a centroid pseudo-sentence.

An R interface to the C libstemmer library that implements Porter's word stemming algorithm for collapsing words to a common root to aid comparison of vocabulary. Currently supported languages are Danish, Dutch, English, Finnish, French, German, Hungarian, Italian, Norwegian, Portuguese, Romanian, Russian, Spanish, Swedish and Turkish.

SentencePiece is an unsupervised text tokenizer/detokenizer mainly designed for Neural Network-based text generation systems where the vocabulary size is predetermined prior to the neural model training.

Memory-Based Learning (MBL) is a machine-learning method applicable to a wide range of tasks in Natural Language Processing (NLP).

The Tilburg Memory Based Learner, TiMBL, is a tool for NLP research, and for many other domains where classification tasks are learned from examples. It is an efficient implementation of k-nearest neighbor classifier.

TiMBL's features are:

• Fast, decision-tree-based implementation of k-nearest neighbor classification;

• Implementations of IB1 and IB2, IGTree, TRIBL, and TRIBL2 algorithms;

• Similarity metrics: Overlap, MVDM, Jeffrey Divergence, Dot product, Cosine;

• Feature weighting metrics: information gain, gain ratio, chi squared, shared variance;

• Distance weighting metrics: inverse, inverse linear, exponential decay;

• Extensive verbosity options to inspect nearest neighbor sets;
• Server functionality and extensive API;
• Fast leave-one-out testing and internal cross-validation;
• and Handles user-defined example weighting.

TiMBL is a product of the Centre of Language and Speech Technology (Radboud University, Nijmegen, The Netherlands), the ILK Research Group (Tilburg University, The Netherlands) and the CLiPS Research Centre (University of Antwerp, Belgium).

If you do scientific research in NLP, timbl will likely be of use to you.

timblserver is a TiMBL wrapper; it adds server functionality to TiMBL. It allows TiMBL to run multiple experiments as a TCP server, optionally via HTTP.

The Tilburg Memory Based Learner, TiMBL, is a tool for Natural Language Processing research, and for many other domains where classification tasks are learned from examples.

TimblServer is a product of the ILK Research Group (Tilburg University, The Netherlands) and the CLiPS Research Centre (University of Antwerp, Belgium).

If you do scientific research in NLP, TimblServer will likely be of use to you.

Ucto can tokenize UTF-8 encoded text files (i.e. separate words from punctuation, split sentences, generate n-grams), and offers several other basic preprocessing steps that make your text suited for further processing such as indexing, part-of-speech tagging, or machine translation.

This package provides the command-line tool itself.

Ucto was written by Maarten van Gompel and Ko van der Sloot. Work on Ucto was funded by NWO, the Netherlands Organisation for Scientific Research, under the Implicit Linguistics project, the CLARIN-NL program, and the CLARIAH project.

Ucto is a product of the Centre of Language and Speech Technology (Radboud University Nijmegen), and previously the ILK Research Group (Tilburg University, The Netherlands).

If you are interested in machine parsing of UTF-8 encoded text files, e.g. to do scientific research in natural language processing, ucto will likely be of use to you.

Ucto can tokenize UTF-8 encoded text files (i.e. separate words from punctuation, split sentences, generate n-grams), and offers several other basic preprocessing steps that make your text suited for further processing such as indexing, part-of-speech tagging, or machine translation.

This package provides necessary language-specific datafiles for running Ucto.

Ucto was written by Maarten van Gompel and Ko van der Sloot. Work on Ucto was funded by NWO, the Netherlands Organisation for Scientific Research, under the Implicit Linguistics project, the CLARIN-NL program, and the CLARIAH project.

Ucto is a product of the Centre of Language and Speech Technology (Radboud University Nijmegen), and previously the ILK Research Group (Tilburg University, The Netherlands).

WordNet(C) ist ein lexikalisches Online-Referenzsystem. Sein Entwurf wurde von aktuellen psycholinguistischen Theorien über das menschliche lexikalische Gedächtnis beeinflusst. Englische Substantive, Verben, Adjektive und Adverbien sind in Synonym-Sätzen organisiert, die jeweils ein zugrunde liegendes lexikalisches Konzept repräsentieren. Verschiedene Beziehungen verbinden die Synonym-Sätze.

WordNet wurde vom Cognitive Science Laboratory der Princeton University unter Leitung von Professor George A. Miller (Untersuchungsleiter) entwickelt.

WordNet wird als die wichtigste Ressource für Forscher in den Gebieten Computational Linguistics, Textanalyse und vielen verwandten Bereichen angesehen.

Das Paket enthält neben dem Binärprogramm und dem Handbuch von WordNet auch allgemeine Handbücher.

This is a transitional dummy package. It can safely be removed.

This package contains Apertium APY which is simple Apertium API written in Python 3 meant as a drop-in replacement for ScaleMT.

Data package providing Apertium language resources for Aragonese

Data package providing Apertium language resources for translating between the Aragonese and Catalan languages.

Data package providing Apertium language resources for Belarusian.

Data package providing Apertium language resources for translating between the Belarusian and Russian languages.

This is a linguistic package for the Apertium shallow-transfer machine translation system. The package can be used to translate between Breton and French.

This is a transitional dummy package. It can safely be removed.

Data package providing Apertium language resources for Catalan.

Data package providing Apertium language resources for translating between the Catalan and Sardinian languages.

Data package providing Apertium language resources for Crimean Tatar

Data package providing Apertium language resources for translating between the Crimean Tatar and Turkish languages.

Data package providing Apertium language resources for translating between the Welsh and English languages.

Data package providing Apertium language resources for Danish.

Data package providing Apertium language resources for translating from the Danish to the Norwegian Nynorsk/Norwegian Bokmål variants and from Danish to Norwegian Nynorsk.

This is a transitional dummy package. It can safely be removed.

This is a transitional dummy package. It can safely be removed.

Data package providing Apertium language resources for translating between the English and Galician languages.

Data package providing Apertium language resources for translating between the Esperanto and Catalan languages.

This is a linguistic package for the Apertium shallow-transfer machine translation system. The package can be used to translate between Esperanto and English.

Data package providing Apertium language resources for translating between the Esperanto and Spanish languages.

Data package providing Apertium language resources for translating between the Esperanto and French languages.

This is a transitional dummy package. It can safely be removed.

This is a transitional dummy package. It can safely be removed.

Data package providing Apertium language resources for translating between the Spanish and Galician languages.

This is a transitional dummy package. It can safely be removed.

Data package providing Apertium language resources for translating between the Spanish and Portuguese languages.

Data package providing Apertium language resources for translating between the Spanish and Romanian languages.

Data package providing Apertium language resources for translating between the Basque and English languages.

Data package providing Apertium language resources for translating between the Basque and Spanish languages.

Data package providing Apertium language resources for translating between the French and Spanish languages.

Data package providing Apertium language resources for French.

Data package providing Apertium language resources for translating between the French and Catalan languages.

Data package providing Apertium language resources for Serbo-Croatian.

Data package providing Apertium language resources for translating between the Serbo-Croatian and English languages.

Data package providing Apertium language resources for translating between the Serbo-Croatian and Macedonian languages.

Data package providing Apertium language resources for translating between the Serbo-Croatian and Slovenian languages.

Data package providing Apertium language resources for Hindi.

This is a transitional dummy package. It can safely be removed.

This is a transitional dummy package. It can safely be removed.

Data package providing Apertium language resources for Icelandic.

Data package providing Apertium language resources for translating between the Icelandic and English languages.

Data package providing Apertium language resources for Italian.

Data package providing Apertium language resources for Kazakh

Data package providing Apertium language resources for translating between the Kazakh and Tatar languages.

This is a transitional dummy package. It can safely be removed.

This is a transitional dummy package. It can safely be removed.

Data package providing Apertium language resources for translating between the Maltese and Arabic languages.

Data package providing Apertium language resources for Norwegian Nynorsk.

Data package providing Apertium language resources for translating between the Norwegian Nynorsk and Norwegian Bokmål languages.

Data package providing Apertium language resources for Norwegian Bokmål.

Data package providing Apertium language resources for translating between the Occitan and Catalan languages.

Data package providing Apertium language resources for translating between the Occitan and Spanish languages.

Data package providing Apertium language resources for Occitan.

Data package providing Apertium language resources for Polish.

Data package providing Apertium language resources for translating between the Polish and Silesian languages.

This is a transitional dummy package. It can safely be removed.

Data package providing Apertium language resources for translating between the Portuguese and Galician languages.

Data package providing Apertium language resources for Russian

Apertium module for reordering separable/discontiguous multiwords.

Data package providing Apertium language resources for translating between the Northern Sami and Norwegian Bokmål languages.

Data package providing Apertium language resources for Spanish.