Overview

This paper describes our rigorous approach for securing our users’ tests and data. Testim assumes all data is highly sensitive and always strives to use the best security practices and encryption technologies to protect it.

This paper covers network, storage, and cloud policies and describes security considerations used in our products.

As of October 2019, Testim has been certified in compliance with the AICPA’s Service Organization Controls (SOC 2 Type II). For more information about this report or any security concerns, contact your Testim representative.

Cloud general architecture

Testim backend is hosted on AWS and Azure cloud providers. Testim servers are running on top of Amazon AWS. The servers run inside an AWS Virtual Private Cloud (VPC) with an independent security group for each resource and separate subnet. Mongo, Redis, and MySQL databases aren’t exposed publicly. Communication between Testim servers and Mongo servers is done over SSL, and data disks are encrypted (data encryption at rest).

Raw data (images, test locator results, etc.) are stored in Azure with a randomly generated GUID name. The Testim scheduler mechanism runs on a Kubernetes cluster.

Communication between Testim servers and Azure Storage and Kubernetes is encrypted and uses auth.

Test recording/playback

The Testim Record/Playback functionality is powered by our Google Chrome extension. The extension is controlled from our dashboard by an authenticated user. The extension is designed to meet Google’s content security policy (CSP), restricting the actions and resources that the extension requires to those directly associated with Testim Record/Playback.

All recorded data is saved on our secured data service and is transmitted using a secure connection. All tests and data generated from Record/Playback are only available to authenticated users through our dashboard.

Activation Policy

Our Google Chrome extension’s default state is set to inactive. It never collects or tracks any of the activity performed by the user in the Chrome browser. Once the user initiates a new recording, either with the Testim editor or the Testim bug capture feature, the extension will start collecting data representing the user’s actions (click, text insertion, navigations, etc.).

The recording is always limited to the selected tab and its descendants (other tabs opened from the selected tab or those opened, recursively, from them) and never affects other browser tabs or windows.

Playback Isolation

Test playback is performed using the data generated from the recording. Testim will never initiate any action or communication other than one recorded by the user. All test data is limited to the domain used in the recording (base URL), and those will never be used on different domains if not strictly selected by the user when running a test.

Storage policy

Test Data

Test data and other artifacts are never stored on the browser machine. They exist in RAM on the tested browser and are sent to the client via a secure connection. When interim data is stored as part of the test service, the storage is in a secure cloud-based location, and communication is over transport layer security (TLS).

Test history is stored in a secure cloud database. The access mechanism is encrypted and is therefore only accessible to authorized users via their Testim account.

Logging

All Testim products generate usage logs used for analytical purposes and monitoring (i.e., error reporting). These usage logs do not contain personal data about the user or any browsing data generated during tests.

Screenshots and Browser Console Logs

The Testim Automate & Capture feature collects screenshots and browser console log data. Both are created or captured during the test sessions and subsequently displayed on the user’s dashboard.

Video

Testim bug capture creates a scenario record. Starting a video capture is always triggered by user action, and the user is prompted to agree to record their desktop upon beginning to record. Testim will never auto-capture a video of a user’s browser or desktop.

While recording, Testim continuously displays a recording notification to make the user aware that their desktop is being recorded and serves as a reminder to turn it off once completed. The generated video is sent to our cloud storage over a secure connection and is visible only to the user and any other authorized project members through the dashboard.

Testim cloud policy

Testim Grid – Public Cloud

Testim’s Selenium-based grid in a public cloud configuration is built on top of Ubuntu LTS, an operating system well-known for being fast and secure and deployed in the AWS cloud service.

Ubuntu LTS is designed to be enterprise-focused, well-tested, and provides a mandatory access control (MAC) system. We chose Ubuntu LTS specifically because it receives regular security patches and upgrades, so we can be confident that it will remain secure over time.

Every test run in the Testim grid is performed on a new and clean browser session with no previous data, state, or storage. Every browser session is terminated upon test completion.

All data generated locally during the test run, in the browser session, is not saved or sent anywhere in the Testim cloud or externally. The data exists only for the lifetime of the browser session.

Testim Grid – Private Cloud

The Testim Selenium grid in a private cloud configuration is a dedicated Testim grid for a single customer. The grid is built on top of Ubuntu LTS and deployed as an AWS service similar to the Testim Grid – Public Cloud.

In addition to the security features mentioned in the Testim Grid – Public Cloud section above, in a private cloud Testim grid, the AWS VPC where the Testim grid is deployed connects to the customer’s data center using a dedicated VPN connection through AWS VPN Gateway or VPC peering connection, as shown in the diagram below.

The Testim grid can be accessed only through the VPN connection. The AWS VPC NACL is configured to communicate (send/receive traffic) with the customer data center only. This makes Testim Grid – Private Cloud isolated and secure from the internet.

Testim saves test results in the Testim SaaS Production environment. All communication to the servers is over TLS.

Testim Cloud Services

All Testim cloud services are accessed over a secure connection (HTTPS) and authenticated to verify the user’s identity accessing the service.

Network policy

To ensure that users’ test runs are always secure, we utilize HTTPS by default. Every time a user communicates with Testim, they are redirected through a secure connection using HTTPS. It uses a transport layer security (TLS), formerly known as secure sockets layer (SSL), and ensures that the communication between the user’s browser and the Testim servers is protected.

In cases where rich user interaction or real-time experience is required, a bidirectional and persistent connection with Testim’s service is needed. For that purpose, we use WebSockets to establish the connection.

WebSockets allow extensive interaction between the client’s browser and the servers and devices. To protect the privacy of transferred data during the testing session, we use WSS exclusively. WSS uses SSL over port 443 for transport and therefore only transmits encrypted data.

Encryption policy

Testim uses the following encryption practices:

• Data-in-transit, meaning the network between the client and server is encrypted (TLS).
• Passwords are encrypted using bcrypt password hashing.
• Data-at-rest, meaning disks are encrypted.

Cloud providers

Testim uses trusted and secured cloud providers to host our customers’ information and enable parts of its service. This includes:

1. Secured databases

2. Secured file storage

3. Secured SAAS services

Our cloud providers utilize firewalls to protect their network and encrypt all communication to Testim services using TLS. Our providers offer an additional layer of encryption using dedicated, hardware-based cryptographic key storage.

Our cloud providers meet the following compliance certificates:

• SOC 1/SSAE 16/ISAE 3402 (formerly SAS 70)
• SOC2
• SOC3
• FISMA, DIACAP, and FedRAMP
• DOD CSM Levels 1-5
• PCI DSS Level 1
• ISO 9001 / ISO 27001
• ITAR
• FIPS 140-2
• MTCS Level 3

Our Cloud Providers:

1. Amazon AWS – Database & cloud computing

2. Microsoft Azure – Cloud storage

3. Cloudinary – Image processing service

4. Coralogix – Real-time log service

5. Applitools Eyes – Image comparison service

6. Mongo Atlas – Cloud service security

Penetration testing

Testim performs annual penetration tests:

• Application vulnerability assessment (black box and grey box)
• External penetration assessment

Testim passed these assessments with no significant security vulnerabilities (last updated: August 2022).