NLP Studio is a web application that allows users to prepare their own text mining experiments for natural language analysis without having to use a programming language. There is a development environment in which the user can work in the form of visual programming. Components for preprocessing, training or visualization are ready. The software is intended both for commercial and educational purposes and makes it easier for those interested to become familiar with the subject of natural language processing. The application is available at https://mta.pef.mendelu.cz/lab/ in both language versions (Czech and English).
Application Backend
The server part of the system is responsible for solving user tasks. From this point of view, the backend architecture can be divided into two levels.
The first level represents the communication interface with the frontend and the database. Basically, it is a regular web server that communicates using a REST API. At this level, the application receives and responds to requests, authenticates users, but also queues jobs.
The second level consists of a more complex, analytical part. This is used to solve various NLP tasks and is oriented towards processing a large amount of data, which is usually time- and computationally demanding. Therefore, we implemented the part of the application that handles these tasks as a separate software. Analytical selects tasks from the queue and solves them in the background based on the specified parameters. It stores intermediate states and the final result in the database. Therefore, the first level is practically not involved in solving the tasks itself. If it did, the API would respond to requests with high latency. In addition, such an arrangement does not allow for efficient task management or background running.
Communication interface
The Rest API is implemented in the language of the modern FastAPI framework. One of the main reasons for FastAPI is that it allows asynchronous processing of HTTP requests. Normally, Python executes code sequentially. For example, in the case of the HTTP protocol, this means that the server accepts the request only after the previous request has been completely processed. However, asynchronous methods can be temporarily suspended (e.g. while waiting for data from the database), thereby not blocking other, so-called competing requests waiting in the execution queue. Thus, the asynchronous paradigm significantly reduces the time required to process a request and is suitable for a concurrent task environment such as a web server. The tasks are processed by the analytical part of the application at the second level, as we mentioned at the beginning of this chapter.
Communication is initiated by the client by sending a POST request to the /task API endpoint. The endpoint accepts requests for all job types. Specific specifications and parameters are sent in the request body in JSON format. The request body can contain any number of parameters depending on the type of task.
Instead of directly executing the request, i.e. running the appropriate method, the API creates a new object for the task and inserts a new record in the Mongo database containing the necessary metadata. The API returns the ID of the created job to the user, but the result is processed asynchronously in the background.
The task running itself is provided by Celery, it is an implementation of a queue of tasks that are executed asynchronously in the background. However, Celery requires a message transport to send and receive messages between the client and the so-called worker (the thread that executes the code itself). For these purposes, we used the Redis service, in which the API stores a record of the task. This is important because the API and analytics layer work independently. Redis thus acts as a communication interface between them.
Usage of the result
The NLPStudio application was integrated by CCFR company into the Analytical Platform tool as one of the modules that allows users to analyze data in various formats and from various sources. Based on these analyses, users can then decide on the choice of specific securities and their inclusion in their trading portfolio. NLPStudio was included in this platform under the name DPS Lab, and supporting materials showing the possibilities of using this tool are available on the mentioned pages.
The application is also used for teaching topics related to natural language processing at PEF MENDELU. This is the subject of Text Mining in AJ and the Circle of text mining applications.
Publication output
A detailed description of the application can be found in the article presented at the ECOS 2023 conference: PŘICHYSTAL, J. VALOVIČ, R. JAKÚBEK, T. NLP Studio, Working paper [online] https://akela.mendelu.cz/~jprich/prichystal-NLPstudio.pdf