Optics Framework

Optics Framework is a powerful, extensible no code test automation framework designed for vision powered, data-driven testing and production app synthetic monitoring. It enables seamless integration with intrusive action & detection drivers such as Appium / WebDriver as well as non-intrusive action drivers such as BLE mouse / keyboard and detection drivers such as video capture card and external web cams.

This framework was designed primarily for the following use cases:

1. Production app monitoring where access to USB debugging / developer mode and device screenshots is prohibited
2. Resilient self-healing test automation that rely on more than one element identifier and multiple fallbacks to ensure maximum recovery
3. Enable non-coders to build test automation scripts

---

🚀 Features

• Vision powered detections: UI object detections are powered by computer vision and not just on XPath elements.
• No code automation: No knowledge of programming languages or access to IDE needed to build automations scripts
• Supports non-intrusive action drivers: Non-intrusive action drivers such as BLE mouse and keyboard are supported
• Data-Driven Testing (DDT): Execute test cases dynamically with multiple datasets, enabling parameterized testing and iterative execution.
• Extensible & Scalable: Easily add new keywords and modules without any hassle.
• AI Integration: Choose which AI models to use for object recognition and OCR.
• Self-healing capability: Configure multiple drivers, screen capture methods, and detection techniques with priority-based execution. If a primary method fails, the system automatically switches to the next available method in the defined hierarchy

---

📦 Installation

Install via pip

pip install optics-framework

---

🚀 Quick Start

1 Install Optics Framework

Note: Ensure Appium server is running and a virtual Android device is enabled before proceeding.

mkdir ~/test-code
cd ~/test-code
python3 -m venv venv
source venv/bin/activate
pip install optics-framework

⚠️ Important: Conda environments are not supported for easyocr and optics-framework together, due to conflicting requirements for numpy (version 1.x vs 2.x). Please use a standard Python virtual environment instead.

2 Create a New Test Project

optics setup --install Appium EasyOCR
optics init --name my_test_project --path . --template contact

📌 Dry Run Test Cases

optics dry_run my_test_project

📌 Execute Test Cases

optics execute my_test_project

---

🛠️ Usage

Execute Tests

optics execute <project_name>

Initialize a New Project

optics init --name <project_name> --path <directory> --template <contact/youtube> --force

List Available Keywords

optics list

Display Help

optics --help

Check Version

optics version

---

🏗️ Developer Guide

Project Structure

Optics_Framework/
├── LICENSE
├── README.md
├── dev_requirements.txt
├── samples/            # Sample test cases and configurations
|   ├── contact/
|   ├── youtube/
├── pyproject.toml
├── tox.ini
├── docs/               # Documentation using Sphinx
├── optics_framework/   # Main package
│   ├── api/            # Core API modules
│   ├── common/         # Factories, interfaces, and utilities
│   ├── engines/        # Engine implementations (drivers, vision models, screenshot tools)
│   ├── helper/         # Configuration management
├── tests/              # Unit tests and test assets
│   ├── assets/         # Sample images for testing
│   ├── units/          # Unit tests organized by module
│   ├── functional/     # Functional tests organized by module

Available Keywords

The following keywords are available and organized by category. These keywords can be used directly in your test cases or extended further for custom workflows.

🔹 Core Keywords

• Clear Element Text (element, event_name=None)
  Clears any existing text from the given input element.
• Detect and Press (element, timeout, event_name=None)
  Detects if the element exists, then performs a press action on it.
• Enter Number (element, number, event_name=None)
  Enters a numeric value into the specified input field.
• Enter Text (element, text, event_name=None)
  Inputs the given text into the specified element.
• Get Text (element)
  Retrieves the text content from the specified element.
• Press by Coordinates (x, y, repeat=1, event_name=None)
  Performs a tap at the specified absolute screen coordinates.
• Press by Percentage (percent_x, percent_y, repeat=1, event_name=None)
  Taps on a location based on percentage of screen width and height.
• Press Element (element, repeat=1, offset_x=0, offset_y=0, event_name=None)
  Taps on a given element with optional offset and repeat parameters.
• Press Element with Index (element, index=0, event_name=None)
  Presses the element found at the specified index from multiple matches.
• Press Keycode (keycode, event_name)
  Simulates pressing a hardware key using a keycode.
• Scroll (direction, event_name=None)
  Scrolls the screen in the specified direction.
• Scroll from Element (element, direction, scroll_length, event_name)
  Scrolls starting from a specific element in the given direction.
• Scroll Until Element Appears (element, direction, timeout, event_name=None)
  Continuously scrolls until the target element becomes visible or the timeout is reached.
• Select Dropdown Option (element, option, event_name=None)
  Selects an option from a dropdown field by visible text.
• Sleep (duration)
  Pauses execution for a specified number of seconds.
• Swipe (x, y, direction='right', swipe_length=50, event_name=None)
  Swipes from a coordinate point in the given direction and length.
• Scroll from Element (element, direction, scroll_length, event_name)
  Scrolls starting from the position of a given element.
• Swipe Until Element Appears (element, direction, timeout, event_name=None)
  Swipes repeatedly until the element is detected or timeout is reached.

🔹 AppManagement

• Close And Terminate App(package_name, event_name)
  Closes and fully terminates the specified application using its package name.
• Force Terminate App(event_name)
  Forcefully terminates the currently running application.
• Get App Version
  Returns the version of the currently running application.
• Initialise Setup
  Prepares the environment for performing application management operations.
• Launch App (event_name=None)
  Launches the default application configured in the session.
• Start Appium Session (event_name=None)
  Starts a new Appium session for the current application.
• Start Other App (package_name, event_name)
  Launches a different application using the provided package name.

🔹 FlowControl

• Condition
  Evaluates multiple conditions and executes corresponding modules if the condition is true.
• Evaluate (param1, param2)
  Evaluates a mathematical or logical expression and stores the result in a variable.
• Read Data (input_element, file_path, index=None)
  Reads data from a CSV file, API URL, or list and assigns it to a variable.
• Run Loop (target, *args)
  Runs a loop either by count or by iterating over variable-value pairs.

🔹 Verifier

• Assert Equality (output, expression)
  Compares two values and checks if they are equal.
• Assert Images Vision (frame, images, element_status, rule)
  Searches for the specified image templates within the frame using vision-based template matching.
• Assert Presence (elements, timeout=30, rule='any', event_name=None)
  Verifies the presence of given elements using Appium or vision-based fallback logic.
• Assert Texts Vision (frame, texts, element_status, rule)
  Searches for text in the given frame using OCR and updates element status.
• Is Element (element, element_state, timeout, event_name)
  Checks if a given element exists.
• Validate Element (element, timeout=10, rule='all', event_name=None)
  Validates if the given element is present on the screen using defined rule and timeout.
• Validate Screen (elements, timeout=30, rule='any', event_name=None)
  Validates the presence of a set of elements on a screen using the defined rule.
• Vision Search (elements, timeout, rule)
  Performs vision-based search to detect text or image elements in the screen.

Setup Development Environment

git clone git@github.com:mozarkai/optics-framework.git
cd Optics_Framework
pipx install poetry
poetry install --with dev

Running Tests

poetry install --with tests
poetry run pytest

Build Documentation

poetry install --with docs
poetry run mkdocs serve

Packaging the Project

poetry build

---

📜 Contributing

We welcome contributions! Please follow these steps:

1. Fork the repository.
2. Create a new feature branch.
3. Commit your changes.
4. Open a pull request.

Ensure your code follows PEP8 standards and is formatted with Black.

---

🎯 Roadmap

Here are the key initiatives planned for the upcoming quarter:

1. MCP Servicer: Introduce a dedicated service to handle MCP (Model Context Protocol), improving scalability and modularity across the framework.
2. Omniparser Integration: Seamlessly integrate Omniparser to enable robust and flexible element extraction and location.
3. Playwright Integration: Add support for Playwright to enhance browser automation capabilities, enabling cross-browser testing with modern and powerful tooling.
4. Audio Support: Extend the framework to support audio inputs and outputs, enabling testing and verification of voice-based or sound-related interactions.

---

📄 License

This project is licensed under the Apache 2.0 License. See the LICENSE file for details.

---

📞 Support

For support, please open an issue on GitHub or contact us at [@malto101], [@davidamo9] or [lalit@mozark.ai] .

Happy Testing! 🚀