Automated Generation of Test Procedures with AI

 Automated Generation of Test Procedures with AI

Author: Francisco Prats Quilez

 

Introduction

This project presents an innovative solution for the automated generation of test procedures, leveraging the capabilities of large language models (LLMs). By integrating an intuitive user interface in Vue.js with a backend that manages prompt generation and interaction with a local LLM, the system offers an efficient way to create high-quality documentation from a variety of sources.

Objective

The primary objective of the project is to streamline and improve the accuracy of test procedure development, reducing the workload of validation and verification engineers. By automating a large portion of the process, the goal is to ensure the consistency and comprehensiveness of documentation, while minimizing the risk of manual errors.

Development

• User Interface: A Vue.js-based interface was developed to allow users to easily upload initial project documents. The interface provides an intuitive experience and guides the user in selecting the type of test procedure to generate.

• Prompt Generation: The system's backend is responsible for generating customized prompts for the LLM, depending on the selected test procedure type. These prompts are designed to guide the model towards generating relevant and structured content.
• Local LLM Processing: The LLM, running in a local environment to ensure the security of confidential information, receives the prompt and input documents. The model processes this information and generates a draft of the test procedure.
• Document Generation: The draft generated by the LLM is transformed into a .docx file, ready for review by a validation and verification engineer.

Conclusions

The preliminary results of the project demonstrate the potential of AI to automate the generation of technical documentation. The system has shown a remarkable ability to produce coherent and well-structured test procedures, significantly reducing the time spent on this task.

Future Development

• Prompt Improvement:
• Prompt Engineering: Experiment with different prompt engineering techniques to achieve more accurate and customized results.
• Expanding the Knowledge Base:
• Continuous Learning: Develop a continuous learning system that allows the model to adapt to new document types and requirements.
• Model Evaluation:
• Model Comparison: Evaluate the performance of different LLM models (e.g., GPT-4, Cloud) to identify the most suitable model for this task.
• Human-Machine Interaction:
• Collaborative Editing: Implement collaborative editing features to allow multiple users to work simultaneously on the generated document.

Additional Considerations

• Scalability: Design the system to be scalable and adaptable to larger projects.
• Integration with Existing Tools: Explore integrating the system with other tools used in the software development process, such as requirements management systems and defect tracking tools.

This case study provides an overview of the project and establishes a clear roadmap for