Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

simplified CI setup #163

Merged
merged 3 commits into from
Sep 4, 2019
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .ci/install.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/bin/bash

# failure is a natural part of life
set -e

if [[ "$TASK" == "rpkg" ]]; then
R_PACKAGE_DIR=$(pwd)/r-pkg
R CMD INSTALL \
--clean \
${R_PACKAGE_DIR}
fi

if [[ "$TASK" == "pypkg" ]]; then
PY_PACKAGE_DIR=$(pwd)/py-pkg
pip install ${R_PACKAGE_DIR}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be PY_PACKAGE_DIR

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh good catch

fi
16 changes: 16 additions & 0 deletions .ci/report_to_covr.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/bin/bash

# failure is a natural part of life
set -e

if [[ "$TASK" == "rpkg" ]]; then
Rscript -e " \
install.packages('covr'); \
Sys.setenv(NOT_CRAN = 'true'); \
covr::codecov('r-pkg/') \
"
fi

if [[ "$TASK" == "pypkg" ]]; then
echo "This is a Python build. No post-build actions configured."
fi
152 changes: 152 additions & 0 deletions .ci/seed_es_on_travis.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
#!/bin/bash

# failure is a natural part of life
set -e

# set up some parameters
ES_HOST=http://127.0.0.1:9200
SLEEP_TIL_STARTUP_SECONDS=20

# where can you get ES binaries?
ES1_ARCHIVE=https://download.elasticsearch.org/elasticsearch/elasticsearch
ES2_ARCHIVE=https://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/distribution/deb/elasticsearch
ES5PLUS_ARCHIVE=https://artifacts.elastic.co/downloads/elasticsearch

# where is the test data? what file has the mapping for the test "shakespeare" index?
TEST_DATA_DIR=$(pwd)/test_data
LEGACY_MAPPING_FILE="${TEST_DATA_DIR}/legacy_shakespeare_mapping.json"
ES5_MAPPING_FILE="${TEST_DATA_DIR}/es5_shakespeare_mapping.json"
ES6_MAPPING_FILE="${TEST_DATA_DIR}/es6_shakespeare_mapping.json"


case "$ES_VERSION" in
"") ;;

"1.0.0")
export ES_VERSION=1.0.0;
export MAPPING_FILE=${LEGACY_MAPPING_FILE};
export ES_BINARY_URL="${ES1_ARCHIVE}/elasticsearch-$ES_VERSION.deb"
;;

"1.4.4")
export ES_VERSION=1.4.4;
export MAPPING_FILE=${LEGACY_MAPPING_FILE};
export ES_BINARY_URL="${ES1_ARCHIVE}/elasticsearch-$ES_VERSION.deb"
;;

"1.7.2")
export ES_VERSION=1.7.2;
export MAPPING_FILE=${LEGACY_MAPPING_FILE};
export ES_BINARY_URL="${ES1_ARCHIVE}/elasticsearch-$ES_VERSION.deb"
;;

"2.0.2")
export ES_VERSION=2.0.2;
export MAPPING_FILE=${LEGACY_MAPPING_FILE};
export ES_BINARY_URL="${ES2_ARCHIVE}/$ES_VERSION/elasticsearch-$ES_VERSION.deb"
;;

"2.1.2")
export ES_VERSION=2.1.2;
export MAPPING_FILE=${LEGACY_MAPPING_FILE};
export ES_BINARY_URL="${ES2_ARCHIVE}/$ES_VERSION/elasticsearch-$ES_VERSION.deb"
;;

"2.2.2")
export ES_VERSION=2.2.2;
export MAPPING_FILE=${LEGACY_MAPPING_FILE};
export ES_BINARY_URL="${ES2_ARCHIVE}/$ES_VERSION/elasticsearch-$ES_VERSION.deb"
;;

"2.3.5")
export ES_VERSION=2.3.5;
export MAPPING_FILE=${LEGACY_MAPPING_FILE};
export ES_BINARY_URL="${ES2_ARCHIVE}/$ES_VERSION/elasticsearch-$ES_VERSION.deb"
;;

"5.0.2")
export ES_VERSION=5.0.2;
export MAPPING_FILE=${ES5_MAPPING_FILE};
export ES_BINARY_URL="${ES5PLUS_ARCHIVE}/elasticsearch-$ES_VERSION.deb"
;;

"5.3.3")
export ES_VERSION=5.3.3;
export MAPPING_FILE=${ES5_MAPPING_FILE};
export ES_BINARY_URL="${ES5PLUS_ARCHIVE}/elasticsearch-$ES_VERSION.deb"
;;

"5.4.3")
export ES_VERSION=5.4.3;
export MAPPING_FILE=${ES5_MAPPING_FILE};
export ES_BINARY_URL="${ES5PLUS_ARCHIVE}/elasticsearch-$ES_VERSION.deb"
;;

"5.6.9")
export ES_VERSION=5.6.9;
export MAPPING_FILE=${ES5_MAPPING_FILE};
export ES_BINARY_URL="${ES5PLUS_ARCHIVE}/elasticsearch-$ES_VERSION.deb"
;;

"6.0.1")
export ES_VERSION=6.0.1;
export MAPPING_FILE=${ES6_MAPPING_FILE};
export ES_BINARY_URL="${ES5PLUS_ARCHIVE}/elasticsearch-$ES_VERSION.deb"
;;

"6.1.4")
export ES_VERSION=6.1.4;
export MAPPING_FILE=${ES5_MAPPING_FILE};
export ES_BINARY_URL="${ES5PLUS_ARCHIVE}/elasticsearch-$ES_VERSION.deb"
;;

"6.2.4")
export ES_VERSION=6.2.4;
export MAPPING_FILE=${ES5_MAPPING_FILE};
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah here's the error (probably?), I used the wrong mapping file.

Python tests didn't fail because there isn't a get_fields() in the Python package yet. Created #164 to document it.

export ES_BINARY_URL="${ES5PLUS_ARCHIVE}/elasticsearch-$ES_VERSION.deb"
;;
esac

# pull the binary
curl -O ${ES_BINARY_URL}

# start the service and wait a bit
sudo dpkg \
-i \
--force-confnew \
elasticsearch-$ES_VERSION.deb
sudo service elasticsearch start
sleep ${SLEEP_TIL_STARTUP_SECONDS}
sudo service elasticsearch status

# seed ES with data
mv ${MAPPING_FILE} shakespeare_mapping.json
echo $(ls)

curl -X PUT \
"${ES_HOST}/shakespeare" \
--silent \
-H 'Content-Type:application/json' \
-d @shakespeare_mapping.json

curl -X PUT \
"${ES_HOST}/empty_index" \
--silent \
-H 'Content-Type:application/json' \
-d @shakespeare_mapping.json

mv test_data/sample.json sample.json

curl -X POST \
"${ES_HOST}/shakespeare/_bulk" \
--silent \
-H 'Content-Type:application/json' \
--data-binary @sample.json

# test that the expected data made it
curl -X POST \
"${ES_HOST}/_refresh"

curl -X GET \
"${ES_HOST}/shakespeare/_search?size=1" \
-H 'Content-Type:application/json'
19 changes: 19 additions & 0 deletions .ci/test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#!/bin/bash

# failure is a natural part of life
set -e

if [[ "$TASK" == "rpkg" ]]; then
R_PACKAGE_DIR=$(pwd)/r-pkg
R CMD build ${R_PACKAGE_DIR}
R CMD check \
--as-cran \
*.tar.gz
fi

if [[ "$TASK" == "pypkg" ]]; then
PY_PACKAGE_DIR=$(pwd)/py-pkg
pytest \
--verbose \
${PY_PACKAGE_DI}
fi
Loading