PhpStorm & PHPUnit & Docker
Keboola Connection (KBC) is composed of many components, most of which run in Docker. At the heart of this there is another component — the Docker Runner, which takes care of transforming KBC jobs into docker commands.
I use PhpStorm a lot and I quite love it. It also has decent support for running things in Docker. What it does not have is support for Docker compose. In a recent release, support for PHP Docker interpreter was also added. This sort of replaces Docker Compose, but not entirely. I was more interested in using a standard
docker-compose.yml so that the tests would run in the same environment both on Travis and my local machine. Also I wanted to use Xdebug with the tests.
There is a great hack for running PHPUnit in with Docker Compose by pointing PhpStorm to a fake script which actually runs PHP inside a running container. The problem with this approach is that Xdebug does not work correctly. After many failed attempts I went with running PHP inside a container connecting through it with SSH. This is similar to running tests in a Vagrant machine.
So I started with the following
version: '2' services: elastic: container_name: test_runner_elastic image: elasticsearch:2.4 ports: - "9200:9200" database: container_name: test_runner_database image: mysql:5.7 ports: - "3306:3306" volumes: - ./db/data/:/var/lib/mysql/ - ./db/init/:/docker-entrypoint-initdb.d/ runner: container_name: test_runner_php build: context: . dockerfile: ./DockerfileTests volumes: - ../:/code/ ports: - "800:80" - "220:22" links: - database - elastic
DockerfileTests includes installed SSH server and enabled remote access. Then I can configure the PHP remote interpreter:
Obviously this works only after I start all the containers with
docker-compose up. Then I can configure a PHPUnit interpreter:
Notice that in the
docker-compose.yml file I have mapped the current directory to the
/code/ directory inside the container. This is very important so that PhpStorm can access the
vendor libraries. PhpStorm connects to the container via SSH and everything works like a charm.
More or less anyway.
The trouble is that inside the container I was testing our Docker Runner, which as the name suggests, requires access to Docker. So one option is running Docker in Docker (DIND). I read and for a moment ignored the warning about not using DIND. So I added another container with DIND and connected to it. This worked, but the performance was incredibly appalling (pulling a simple image took tens of minutes). So I had to switch to using Docker installed on the host machine.
Connecting to docker host machine is best done using a mapped volume so adding
volumes: — /var/run/docker.sock:/var/run/docker.sock in docker compose solves this. Unless of course you’re using Windows, in which case, you have to connect through the API. Fortunately this can be done too, you need to set
DOCKER_HOST must be set to external address of the host machine. I.e it cannot be set to
localhost because inside the container this would not be the host. But this is not enough because docker does not listen to external IPs, so we need to make a loop back, e.g.:
netsh interface portproxy add v4tov4 listenport=2376 listenaddress=192.168.0.1 connectport=2375 connectaddress=127.0.0.1
If all works well, the code inside the container should now talk with the host Docker. Then I ran into an issue that the tests were terminating after some time. Unfortunately they were terminated with no error message and with exit code 0. By a lucky guess I discovered that the culprit is the setting of
COMPOSE_HTTP_TIMEOUT. I set it to about 10000 seconds which is enough to run our entire test suite.
It is also possible to debug command-line scripts with the same setup. The first step is to configure remote PHP debugging:
The important part is mapping which converts local PhpStorm directory to
/code/ in docker container. Then the magic Start Listening for PHP Debug connections must be enabled. Then I have to connect into the running container (either via
docker exec or via
ssh) and run another magic command:
DockerServer is the name of the Remote debug configuration. And now, I can actually run (still inside the container) the PHP command to debug a remote script:
/usr/local/bin/php -dxdebug.remote_enable=1 -dxdebug.remote_mode=req -dxdebug.remote_port=9000 -dxdebug.remote_connect_back=0 -dxdebug.remote_autostart=1 -dxdebug.idekey=PHPSTORM -dxdebug.remote_host=192.168.0.103 /code/vendor/keboola/syrup/app/console syrup:run-job 123456
The most important part is setting
idekey=PHPSTORM , I also like to set
remote_host to the IP of the PhpStorm machine to make the connection reliable. PhpStorm will now stop at the specified breakpoints — it will stop the command line script executed inside a container created through docker-compose.
It’s all a little bit complicated, but once it works, its awesome.
So what are the important parts:
docker-compose.ymlwith SSH enabled.
- Make sure that the code directory is mapped into the container.
- Always run
composer installinside the container.
- To run the tests, first run
- From inside the container connect to the docker host either via docker socket or
- If the tests crash randomly, set or raise
- Setting up command line debugger is also possible.
- Really do not use Docker in Docker.
Of course, this only solves the part of running the tests, not running the entire application. But since we do TDD, this is the most important for our development process.