Docker support

[giabao]

features request:

• Add Dockerfile to build ambry
• Add guide for deploying servers, frontends, (maybe) load balancer in docker environment (swarm / rancher / kubenetes / ...)

[maciejbak85]

Ive tried this way:

FROM quantumobject/docker-alpine

USER root

RUN
apk update &&
apk upgrade &&
apk add git &&
apk add bash &&
apk add openjdk8 &&
rm -rf /var/cache/apk/*

ENV JAVA_HOME /usr/lib/jvm/java-1.8-openjdk

RUN git clone https://github.com/linkedin/ambry.git

WORKDIR ambry/target

RUN cd .. && ./gradlew allJar

RUN mkdir logs

RUN java -Dlog4j.configuration=file:../config/log4j.properties -jar ambry.jar --serverPropsFilePath ../config/server.properties --hardwareLayoutFilePath ../config/HardwareLayout.json --partitionLayoutFilePath ../config/PartitionLayout.json > logs/server.log && /sbin/my_init &

EXPOSE 1174

ENTRYPOINT java -Dlog4j.configuration=file:../config/log4j.properties -cp "*" com.github.ambry.frontend.AmbryFrontendMain --serverPropsFilePath ../config/frontend.properties --hardwareLayoutFilePath ../config/HardwareLayout.json --partitionLayoutFilePath ../config/PartitionLayout.json > logs/frontend.log && /sbin/my_init

but it does not want to work.. healthCheck shows GOOD but internal server error on upload..
Did some one managed how to do it ?

[vgkholla]

@maciejbak85 About the internal server error, anything in the logs (frontend or server)? Internal server errors usually get logged

[maciejbak85]

@vgkholla nothing

[vgkholla]

well thats odd - all internal server errors on the frontend get logged. You can also check the public access logs to make sure the request was received and recorded. Without logs, its hard to diagnose what happened. You can try changing the log levels to DEBUG or TRACE in log4j.properties and see if that helps

I'm not very familiar with docker but I see that you have a statement explicitly exposing port 1174 (the frontend's port) - do you have expose the port of the server too (6667)?

[maciejbak85]

This is from frontend.log

ERROR Connection localhost/127.0.0.1 disconnected (com.github.ambry.network.Selector)
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at com.github.ambry.network.Transmission.finishConnect(Transmission.java:54)
at com.github.ambry.network.Selector.poll(Selector.java:337)
at com.github.ambry.network.NetworkClient.sendAndPoll(NetworkClient.java:107)
at com.github.ambry.router.NonBlockingRouter$OperationController.run(NonBlockingRouter.java:576)
at java.lang.Thread.run(Thread.java:748)
[2017-08-25 17:01:59,929] ERROR Internal error handling request / with method POST. (com.github.ambry.rest.NettyResponseChannel)
com.github.ambry.rest.RestServiceException: com.github.ambry.router.RouterException: No writable partitions available.
at com.github.ambry.frontend.AmbryBlobStorageService.submitResponse(AmbryBlobStorageService.java:322)
at com.github.ambry.frontend.AmbryBlobStorageService$PostCallback.onCompletion(AmbryBlobStorageService.java:893)
at com.github.ambry.frontend.AmbryBlobStorageService$PostCallback.onCompletion(AmbryBlobStorageService.java:818)
at com.github.ambry.router.NonBlockingRouter.completeOperation(NonBlockingRouter.java:392)
at com.github.ambry.router.PutManager.onComplete(PutManager.java:255)
at com.github.ambry.router.PutManager.handleResponse(PutManager.java:200)
at com.github.ambry.router.NonBlockingRouter$OperationController.onResponse(NonBlockingRouter.java:544)
at com.github.ambry.router.NonBlockingRouter$OperationController.run(NonBlockingRouter.java:577)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.github.ambry.router.RouterException: No writable partitions available.
at com.github.ambry.router.PutOperation$PutChunk.getPartitionForPut(PutOperation.java:967)
at com.github.ambry.router.PutOperation$PutChunk.prepareForSending(PutOperation.java:789)
at com.github.ambry.router.PutOperation$PutChunk.checkAndMaybeComplete(PutOperation.java:856)
at com.github.ambry.router.PutOperation$PutChunk.handleResponse(PutOperation.java:1049)
at com.github.ambry.router.PutOperation.handleResponse(PutOperation.java:263)
at com.github.ambry.router.PutManager.handleResponse(PutManager.java:193)
... 3 more

[maciejbak85]

In docker-compose.yml I have:

ambry:
    build: resources/ambry
    ports:
        - "1174:1174"
        - "6667:6667" 

I have added also to Dockerfile:

EXPOSE 6667
EXPOSE 1174
EXPOSE 1175

but does not helped.

[vgkholla]

I'm not familiar with docker but the from the error message I can glean that the frontend is unable to connect to the server. It might be some port binding issues. Please try connecting to the server port via telnet with the exact parameters used by the frontend (in the clustermap). For the default cluster map this should be localhost and 6667. So please see if telnet localhost 6667 is successful.

[maciejbak85]

but frontend is on same docker container as server, should not be a problem

[maciejbak85]

@vgkholla :

telnet localhost 6667
Trying ::1...
Connected to localhost.
Escape character is '^]'.
Connection closed by foreign host.

[maciejbak85]

Ok Ive managed it: https://github.com/maciejbak85/ambry-docker