AI-Powered Test Automation: GenAI Without Security Risks!

By Eduard Dubilyer , CTO of Skipper Soft

🔍 Introduction

AI-powered test automation is transforming how we design and execute tests. Large Language Models (LLMs) can generate test ideas, automation scripts, and intelligent test coverage suggestions without manual scripting. However, when third parties access your environment, there’s always a risk of exposure or misuse. But what if you could run these models locally, eliminating the need for cloud-based APIs and enhancing data security?

In this article, we’ll explore how to use Ollama to run LLMs locally and leverage Page Assist to generate test ideas and automation test scripts for any web application.

🔬 Why to try Running AI Models Locally for Test Automation:

As a Quality Architect, we always on the lookout for new innovations in test automation. While cloud-based AI tools like ChatGPT and Copilot are helpful, I was curious to explore:

1️⃣ Can AI models generate meaningful test scripts for real applications?

2️⃣ Is it possible to run LLMs locally without relying on cloud-based APIs?

3️⃣ Would this approach be mature enough for enterprise-level automation?

With these questions in mind, I decided to set up Ollama—an open-source framework for running LLMs locally—and integrate it with Page Assist, a browser extension that interacts with the AI model.

📊 Results & Observations: Is Local AI Ready for Enterprise Testing?

After experimenting with various test scenarios, here’s what I found:

✅ What Worked Well?

✔ Generated automation test ideas based on UI elements

✔ Produced Selenium & Playwright scripts with locators & assertions in language of my choice.

✔ Handled negative & boundary case generation better than traditional test case writing

✔ Worked offline—eliminating API costs and data privacy concerns

⚠️ Where It Struggled?

❌ Generated scripts weren’t always production-ready—some locators were incorrect

❌ Lacked built-in test execution capabilities—needed integration into existing frameworks

❌ High resource consumption—required at least 16GB RAM for smooth performance

🏆 Is It Ready for Enterprise Test Automation?

🔹 For now, it’s an exciting tool but not a complete replacement.

🔹 AI-generated test cases still need human validation & refinement before execution.

🔹 Running LLMs locally is resource-intensive, making cloud alternatives more scalable.

🔹 However, for quick test case ideation, exploratory testing, and automation script generation, it’s incredibly useful.

🎯 Step-by-Step Guide to Running AI Agents Locally

✅ Step 1: Install Ollama for Local AI Execution

To run LLMs locally, download Ollama, a tool for setting up AI models offline:

1️⃣ Visit Ollama’s website(https://ollama.com/) and download the software for your OS (Windows, macOS, or Linux).

2️⃣ Install and verify that Ollama runs correctly.

✅ Step 2: Choose an AI Model for Test Generation

Ollama supports various AI models for different use cases. For test automation, choose a model that excels in code generation:

1️⃣ Open a terminal or command prompt.

2️⃣ Search for available AI models.

3️⃣ Select DeepSeek-R1, optimized for reasoning and code generation.

✅ Step 3: Set Up the AI Model Locally

Now that you’ve selected your AI model, set it up with a simple command:

>> ollama run deepseek-r1

✅ Step 4: Install PageAssist Browser Extension

To integrate AI with your web browser and enable test automation, install PageAssist, a Chrome/Firefox extension that connects locally running AI models.

🔹 Install PageAssist from the Chrome Web Store or Firefox Add-ons

🔹 This extension allows you to generate test ideas and automation scripts directly from the browser

✅ Step 5: Open a Website for Test Automation

Once the AI model is running locally and PageAssist is installed:

🔹 Navigate to the website for which you want to generate test scenarios

🔹 Ensure the PageAssist extension is enabled

✅ Step 6: Enable PageAssist Side Panel for Better Visibility

For an enhanced experience:

1️⃣ Click on the PageAssist extension in the browser

2️⃣ Open the side panel to view AI-generated responses in real-time

3️⃣ Select the locally running AI model (DeepSeek-R1):

✅ Step 7: Configure RAG Settings in PageAssist

To optimize test generation, enable Retrieval-Augmented Generation (RAG):

1️⃣ Click on Settings in PageAssist

2️⃣ Select AI model as an embedding model

3️⃣ This ensures the AI considers context from the web page before generating test scripts

✅ Step 8: Provide Effective Prompts for Automation Test Ideas & Scripts.

1️⃣ Enter a well-structured prompt in the message field to generate test ideas or automation scripts.

Example 1: Prompt – Generate Selenium Java tests for Login Page. Example 2: Prompt – Analyse the webpage https://www.skipper-soft.com/ and generate UI test cases to validate all UI components, including the logo, navigation menu, links, buttons, images, and interactive elements. Ensure tests verify visibility, functionality, responsiveness, and broken links. Provide Playwright JS test scripts for execution.

2️⃣ The AI model will take some time to analyse and think like a human tester before generating meaningful test cases. You can expand the response to understand its reasoning process.

3️⃣ Once processed, the AI agent will provide detailed test cases and automation scripts.

Output:

💡 Final Verdict:

AI-assisted test automation is evolving rapidly, offering faster test creation, intelligent suggestions, and enhanced coverage. Running AI models locally not only boosts efficiency but also ensures data privacy and security by keeping sensitive test data(credentials or secret information) within your infrastructure. While promising, enterprise adoption will require better integration, script validation, and optimized resource management. With human oversight and a secure local setup, AI can revolutionize testing—explore it now to stay ahead! 🚀