Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions docker/tidb-bench/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
FROM golang:1.12.1-alpine3.9 AS goycsb

RUN apk add git build-base \
&& git clone https://github.com/pingcap/go-ycsb $GOPATH/src/github.com/pingcap/go-ycsb \
&& cd $GOPATH/src/github.com/pingcap/go-ycsb \
&& make

FROM alpine:3.9
MAINTAINER PingCAP <cloud@pingcap.com>

COPY --from=pingcap/tidb-sysbench /usr/local/bin/sysbench /usr/local/bin/sysbench
COPY --from=pingcap/tidb-sysbench /usr/local/share/sysbench /usr/local/share/sysbench

COPY --from=dmonakhov/alpine-fio /usr/local/bin/fio /usr/local/bin/fio
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just copy this binary to make it work? Is the fio a static binary or dynamically linked binary? From its Dockerfile I don't see any special instruction for static build.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't tested this yet.


COPY --from=goycsb /go/src/github.com/pingcap/go-ycsb/bin/go-ycsb /go-ycsb/bin/go-ycsb
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also does this work in alpine? And maybe you can copy from pingcap/go-ycsb directly instead of build the go-ycsb yourself.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't tested this, but it should work. Copying from pingcap/go-ycsb creates additional fragility unless we define a contract for go-ycsb (linux distro, binary location, static build)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The go-ycsb repo already has a Dockerfile. It also uses alpine as builder and final image.

COPY --from=goycsb /go/src/github.com/pingcap/go-ycsb/workloads /go-ycsb/workloads
COPY --from=goycsb /go/src/github.com/pingcap/go-ycsb/tool /go-ycsb/tool

ADD README.md /README.md

CMD ["bash"]
54 changes: 54 additions & 0 deletions docker/tidb-bench/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
This image includes various utilities used to benchmark TiDB.

# Performance tests

* sysbench
* go-ycsb


# Disk tests

*Note:* All the commands need to be run under the disk mount point. And the test.data file need to be deleted before test.

## fio
* Read
```
fio -ioengine=libaio -bs=32k -direct=1 -thread -rw=read -size=10G -filename=test.data -name="PingCAP max throughput" -iodepth=4 -runtime=60
```

* Write
```
fio -ioengine=libaio -bs=32k -direct=1 -thread -rw=write -size=10G -filename=test.data -name="PingCAP max throughput" -iodepth=4 -runtime=60
```

* Random read
```
fio -ioengine=libaio -bs=32k -direct=1 -thread -rw=randread -size=10G -filename=test.data -name="PingCAP max throughput" -iodepth=4 -runtime=60
```

* Random write
```
fio -ioengine=libaio -bs=32k -direct=1 -thread -rw=randwrite -size=10G -filename=test.data -name="PingCAP max throughput" -iodepth=4 -runtime=60
```

* Random read & write
```
fio -ioengine=libaio -bs=32k -direct=1 -thread -rw=randrw -percentage_random=100,0 -size=10G -filename=test.data -name="PingCAP max throughput" -iodepth=4 -runtime=60
```

## dd

* Cache write
```
dd bs=4k count=400000 if=/dev/zero of=test.data
```

* Direct write
```
dd bs=4k count=400000 if=/dev/zero of=test.data oflag=direct
```

* Direct read
```
dd bs=4k count=400000 if=test.data of=/dev/null& oflag=direct
```
6 changes: 6 additions & 0 deletions docker/tidb-bench/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#!/usr/bin/env bash
set -euo pipefail
pushd ../tidb-sysbench
./build.sh
popd
docker build -t pingcap/tidb-bench .
31 changes: 31 additions & 0 deletions docker/tidb-sysbench/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
FROM alpine:3.9
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to make this a separate image?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, because we patch sysbench. If we can get our patch upstream it won't really be needed.

MAINTAINER PingCAP <cloud@pingcap.com>

# Install sysbench
# gcc and mariadb-dev install shared libraries used at runtime
RUN apk add gcc mariadb-dev

RUN apk add --virtual .build-deps git build-base automake autoconf libtool --update \
&& git clone https://github.com/akopytov/sysbench.git \
&& cd sysbench \
&& git checkout 1.0.17 \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make the version as an environment variable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay

&& ./autogen.sh \
&& ./configure --disable-shared \
&& make \
&& make install \
&& cd .. \
&& rm -r sysbench \
&& apk del .build-deps

# TiDB patch for sysbench data loading (prepare)
RUN apk add --virtual .build-deps wget \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why run apk add again? A wget to download the oltp_common.lua is enough.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is easy to apk add again if you need it later.

&& wget https://raw.githubusercontent.com/pingcap/tidb-bench/master/sysbench-patch/oltp_common.lua \
&& apk del .build-deps \
&& mv oltp_common.lua /usr/local/share/sysbench/oltp_common.lua \
&& chmod +x /usr/local/share/sysbench/oltp_common.lua

RUN apk add mysql-client
RUN apk add bash
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why add bash in this docker image? The tidb bench script is not included in this image.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I agree that these last two can be left out and put in tidb-bench so that this image is just the patched sysbench. It just depends if you want to attach to the container for debugging.


WORKDIR "/usr/local/share/sysbench"
CMD ["sysbench"]
3 changes: 3 additions & 0 deletions docker/tidb-sysbench/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/usr/bin/env bash
set -euo pipefail
docker build -t pingcap/tidb-sysbench .