Skip to content

Commit

Permalink
Feature/docker support (Codeception#3067)
Browse files Browse the repository at this point in the history
* added basic docker support

* updated selenium container hostnames

* added docker docs

* added GitLab CI config

* updated GitLab build jobs

* updated GitLab reports

* updated GitLab testing to run suites in parallel

* removed fixed port

* updated paths and test-stack configuration

* removed GitLab configuration

* updated testing examples and host-volume paths

* reverted docker network host-names

* ignore docker-compose config (.env)

* updated Docker comments & commands

* updated docs

* updated PATH variable

* build from PHP 7.0

* added Dockerfile maintainer

* updated working dir config to use empty dir (host-volume) by default

* moved local testing documentation to tests/

* fixed typo

* fixed typo
  • Loading branch information
schmunk42 authored and DavertMik committed May 25, 2016
1 parent ce47c7c commit e4514ba
Show file tree
Hide file tree
Showing 5 changed files with 178 additions and 9 deletions.
4 changes: 2 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
.idea
.roboci
Dockerfile
composer.phar
vendor
package/codecept.phar
Expand All @@ -19,4 +18,5 @@ tests/data/claypit/tests/_output/*
tests/data/claypit/c3tmp

.DS_Store
robo.phar
robo.phar
.env
51 changes: 51 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
FROM php:7.0-cli

MAINTAINER Tobias Munk [email protected]

# Install required system packages
RUN apt-get update && \
apt-get -y install \
git \
zlib1g-dev \
libssl-dev \
--no-install-recommends && \
apt-get clean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

# Install php extensions
RUN docker-php-ext-install \
bcmath \
zip

# Install pecl extensions
RUN pecl install mongodb xdebug && \
docker-php-ext-enable mongodb && \
docker-php-ext-enable xdebug

# Configure php
RUN echo "date.timezone = UTC" >> /usr/local/etc/php/php.ini

# Install composer
ENV COMPOSER_ALLOW_SUPERUSER=1
RUN curl -sS https://getcomposer.org/installer | php -- \
--filename=composer \
--install-dir=/usr/local/bin
RUN composer global require --optimize-autoloader \
"hirak/prestissimo"

# Prepare application
WORKDIR /repo

# Install vendor
COPY ./composer.json /repo/composer.json
RUN composer install --prefer-dist --optimize-autoloader

# Add source-code
COPY . /repo

ENV PATH /repo:${PATH}
ENTRYPOINT ["codecept"]

# Prepare host-volume working directory
RUN mkdir /project
WORKDIR /project
41 changes: 34 additions & 7 deletions docs/12-ParallelExecution.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,33 @@ Docker works really well for isolating testing environments.
By the time of writing this chapter, we didn't have an awesome tool like it. This chapter demonstrates how to manage parallel execution manually. As you will see we spend too much effort trying to isolate tests which Docker does for free. Today we **recommend using Docker** for parallel testing.
</div>

## What to do

## Docker

> :construction: Section is under construction
### Requirements

- `docker` or [Docker Toolbox](https://www.docker.com/products/docker-toolbox)


### Using Codeception Docker image

Run Docker image

docker run codeception/codeception

Running tests from a project, by mounting the current path as a host-volume into the container.
The default working directory in the container is `/project`.

docker run -v ${PWD}:/project codeception/codeception run

For local testing of the Codeception repository with Docker and `docker-copmose`, please refer to the [testing documentation](../tests/README.md).


## Robo

### What to do

Parallel Test Execution consists of 3 steps:

Expand All @@ -34,7 +60,7 @@ To conclude, we need:
* [Robo](http://robo.li), a task runner.
* [robo-paracept](https://github.com/Codeception/robo-paracept) - Codeception tasks for parallel execution.

## Preparing Robo
### Preparing Robo

`Robo` is recommended to be installed globally. You can either do [a global install with Composer](https://getcomposer.org/doc/03-cli.md#global) or download `robo.phar` and put it somewhere in PATH.

Expand Down Expand Up @@ -103,7 +129,7 @@ parallel
parallel:split-tests
```

## Sample Project
### Sample Project

Let's say we have long running acceptance tests and we want to split them into 5 processes. To make tests not be conflicting with each other they should use different hosts and databases. Thus, before proceeding we need to configure 5 different hosts in Apache/Nginx (or just run application on different ports in PHP Built-in web server). Based on host our application should use corresponding databases.

Expand All @@ -113,7 +139,7 @@ You can also think about running your tests on remote hosts using SSH. `Robo` ha

In current example we assume that we have prepared 5 databases and 5 independent hosts for our application.

### Step 1: Split Tests
#### Step 1: Split Tests

Codeception can organize tests into [groups](http://codeception.com/docs/07-AdvancedUsage#Groups). Starting from 2.0 it can load information about a group from a files. Sample text file with a list of file names can be treated as a dynamically configured group. Take a look into sample group file:

Expand Down Expand Up @@ -173,7 +199,7 @@ Let's try to execute tests from the second group:
$ php codecept.phar run functional -g p2
```

### Step 2: Running Tests
#### Step 2: Running Tests

As it was mentioned, Robo has `ParallelExec` task to spawn background processes. But you should not think of it as the only option. For instance, you can execute tests remotely via SSH, or spawn processes with Gearman, RabbitMQ, etc. But in our example we will use 5 background processes:

Expand Down Expand Up @@ -275,7 +301,7 @@ Now, we can execute tests with
$ robo parallel:run
```

### Step 3: Merge Results
#### Step 3: Merge Results

We should not rely on console output when running our tests. In case of `parallelExec` task, some text can be missed. We recommend to save results as JUnit XML, which can be merged and plugged into Continuous Integration server.

Expand All @@ -295,7 +321,7 @@ We should not rely on console output when running our tests. In case of `paralle

`result.xml` file will be generated. It can be processed and analyzed.

### All Together
#### All Together

To create one command to rule them all we can define new public method `parallelAll` and execute all commands. We will save the result of `parallelRun` and use it for our final exit code:

Expand All @@ -311,6 +337,7 @@ To create one command to rule them all we can define new public method `parallel
?>
```


## Conclusion

Codeception does not provide tools for parallel test execution. This is a complex task and solutions may vary depending on a project. We use [Robo](http://robo.li) task runner as an external tool to perform all required steps. To prepare our tests to be executed in parallel we use Codeception features of dynamic groups and environments. To do even more we can create Extensions and Group classes to perform dynamic configuration depending on a test process.
78 changes: 78 additions & 0 deletions tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,3 +155,81 @@ There are various test suites in `tests/data/claypit`. Maybe you will need to cr
## Coverage

Acceptance tests that use demo application and `c3` collector, to check that a code coverage can be collected remotely. This tests are rarely updated, they should just work )

---

## Dockerized testing

### Local testing and development with `docker-compose`

Using `docker-compose` for test configurations

cd tests

Build the `codeception/codeception` image

docker-compose build

Start

docker-compose up -d

By default the image has `codecept` as its entrypoint, to run the tests simply supply the `run` command

docker-compose run --rm codecept help

Run suite

docker-compose run --rm codecept run cli

Run folder

docker-compose run --rm codecept run unit Codeception/Command

Run single test

docker-compose run --rm codecept run cli ExtensionsCest

Development bash

docker-compose run --rm --entrypoint bash codecept

Cleanup

docker-compose run --rm codecept clean

In parallel

docker-compose --project-name test-cli run -d --rm codecept run --html report-cli.html cli & \
docker-compose --project-name test-unit-command run -d --rm codecept run --html report-unit-command.html unit Codeception/Command & \
docker-compose --project-name test-unit-constraints run -d --rm codecept run --html report-unit-constraints.html unit Codeception/Constraints


### Adding services

Add Redis to `docker-compose.yml`

services:
[...]
redis:
image: redis:3

Update `host`

protected static $config = [
'database' => 15,
'host' => 'redis'
];

Run Redis tests

docker-compose run --rm codecept run unit Codeception/Module/RedisTest

Further Examples

firefox:
image: selenium/standalone-firefox-debug:2.52.0
chrome:
image: selenium/standalone-chrome-debug:2.52.0
mongo:
image: mongo
13 changes: 13 additions & 0 deletions tests/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
version: '2'
services:
codecept:
image: testing/codeception/codeception
build: ..
working_dir: /repo
volumes:
- ../tests:/repo/tests
- ../codeception.yml:/repo/codeception.yml
- ../src:/repo/src
- ../shim.php:/repo/shim.php
#redis:
# image: redis:3

0 comments on commit e4514ba

Please sign in to comment.