Implementation of an Automated Information Query System for NoSQL Databases with Artificial Intelligence

 

Implementation of an Automated Information Query System for NoSQL Databases with Artificial Intelligence

Author: Francisco Prats Quílez

Introduction

In today's data management context, NoSQL databases such as MongoDB play a crucial role due to their flexibility and scalability. However, querying and extracting relevant information from these databases can be challenging due to the diversity and variability of data structures. This case study presents the development and implementation of an automated system for searching and querying information in NoSQL databases using artificial intelligence, specifically large language models (LLM).

Objective

The objective of this project is to design a system that automates the detection of data structures in MongoDB databases and allows for natural language queries. This system should generate and execute Python code to extract and visualize the requested information, significantly improving the efficiency and accessibility of data management.

Development

A web platform developed in Vue.js will be used to manage the entire process and facilitate user interaction with the system.

1. Loading the Connection String:
• The first step involves loading the MongoDB database connection string, which is obtained through MongoDB Atlas. This string is essential for establishing the connection to the database.
2. Detection and Description of Collections:
• The system loads a document describing the existing collections or, alternatively, automatically detects the collections present in the database.
• Once the collections are identified, the system analyzes and detects the different data structures within each collection.
3. Automatic Generation of Descriptions:
• Using an LLM, the system generates a detailed description of the detected data structures. This description is stored to facilitate future queries and analyses.
4. Natural Language Query:
• The user can perform natural language queries about the information contained in the database.
• These queries, along with the database description, are sent to an LLM to generate the necessary Python code to obtain the result. The prompt specifies the available libraries, such as Pandas, NumPy, and Matplotlib, previously installed in the backend environment.
5. Execution and Visualization of Results:
• The generated Python code is executed, producing graphs or data that respond to the query.
• This process includes the creation of a temporary Python file that is deleted after execution to keep the environment clean and ready for future queries.
6. Deletion of the Temporary File:
• After executing the code and obtaining the results, the generated Python file is automatically deleted, ensuring that the system is prepared for new queries without accumulating unnecessary files.

Conclusions

The development of this automated system demonstrates a significant improvement in the efficiency of querying and managing data in NoSQL databases. The use of LLMs allows users to interact with the database more intuitively, reducing the need for advanced technical knowledge. Automating the detection of data structures and the generation of code optimizes the process of extracting and analyzing information.

Future Development

For future iterations of the project, several improvements and extensions are considered:

• Optimization of Prompts: Refine the prompts used to generate Python code, improving the accuracy and efficiency of the generated code.
• Expansion of Libraries: Integrate more libraries and data analysis tools to offer a wider range of functionalities.
• Continuous Learning: Incorporate continuous learning capabilities so that the system improves over time based on queries and results obtained.
• Query History: Possibility of having a query history.

This system represents a significant advance in data management in NoSQL databases, providing a powerful and accessible tool for querying and analyzing information.