Repository navigation
Description
I am working to use the bitsensor Elastalert for Docker setup. Unfortunately I have to build on a Windows WSL Linux. My corporate RHEL servers don't have a high enough version of Docker to build there, and I can't upgrade them.
I am able to build an image using Dockerfile from pull request #174. This is working great to build. I docker save the image and move it to my RHEL server. Loading the image is fine, but running it I am getting the following error.
Any guidance would be appreciated. I am not too familiar with Python to understand the versioning and packaging.
server:/elk/docker> dc_reup elastalert
No stopped containers
Creating elastalert ... done
Creating elastalert ...
Attaching to elastalert
elastalert |
elastalert | > @bitsensor/elastalert@3.0.0-beta.0 start /opt/elastalert-server
elastalert | > sh ./scripts/start.sh "sh" "./scripts/start.sh"
elastalert |
: not found | ./scripts/start.sh: line 2:
"lastalert | bunyan: error: unknown output mode: "short
elastalert | events.js:292
elastalert | throw er; // Unhandled 'error' event
elastalert | ^
elastalert |
elastalert | Error: write EPIPE
elastalert | at afterWriteDispatched (internal/stream_base_commons.js:154:25)
elastalert | at writeGeneric (internal/stream_base_commons.js:145:3)
elastalert | at Socket._writeGeneric (net.js:782:11)
elastalert | at Socket._write (net.js:794:8)
elastalert | at writeOrBuffer (_stream_writable.js:352:12)
elastalert | at Socket.Writable.write (_stream_writable.js:303:10)
elastalert | at Logger._emit (/opt/elastalert-server/node_modules/bunyan/lib/bunyan.js:923:22)
elastalert | at Logger.info (/opt/elastalert-server/node_modules/bunyan/lib/bunyan.js:1045:24)
elastalert | at Logger.info (/opt/elastalert-server/src/common/logger/index.js:9:5)
elastalert | at /opt/elastalert-server/src/common/config/server_config.js:115:22
elastalert | Emitted 'error' event on Socket instance at:
elastalert | at emitErrorNT (internal/streams/destroy.js: 100:8)
elastalert | at emitErrorCloseNT (internal/streams/destroy.js:68:3)
elastalert | at processTicksAndRejections (internal/process/task_queues.js:84:21) {
elastalert | errno: -32,
elastalert | code: 'EPIPE',
elastalert | syscall: 'write'
elastalert | }
elastalert | npm ERR! code ELIFECYCLE
elastalert | npm ERR! errno 1
elastalert | npm ERR! @bitsensor/elastalert@3.0.0-beta.0 start:sh ./scripts/start.sh "sh" "./scripts/start.sh"
elastalert | npm ERR! Exit status 1
elastalert | npm ERR!
elastalert | npm ERR! Failed at the @bitsensor/elastalert@3.0.0-beta.0 start script.
elastalert | npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
elastalert exited with code 1
My Dockerfile is below.
FROM alpine:3.11 as py-ea
ARG ELASTALERT_VERSION=v0.2.4
ENV ELASTALERT_VERSION=${ELASTALERT_VERSION}
# URL from which to download Elastalert.
ARG ELASTALERT_URL=https://github.com/Yelp/elastalert/archive/$ELASTALERT_VERSION.zip
ENV ELASTALERT_URL=${ELASTALERT_URL}
# Elastalert home directory full path.
ENV ELASTALERT_HOME /opt/elastalert
WORKDIR /opt
RUN apk add --update --no-cache ca-certificates openssl-dev openssl python3-dev python3 py3-pip py3-yaml libffi-dev gcc musl-dev wget && \
# Download and unpack Elastalert.
wget -O elastalert.zip "${ELASTALERT_URL}" && \
unzip elastalert.zip && \
rm elastalert.zip && \
mv e* "${ELASTALERT_HOME}"
WORKDIR "${ELASTALERT_HOME}"
# Install Elastalert.
RUN python3 setup.py install
FROM node:14-alpine
LABEL maintainer="BitSensor <dev@bitsensor.io>"
# Set timezone for this container
ENV TZ Etc/UTC
RUN apk add --update --no-cache curl tzdata python3 make libmagic && \
ln -s /usr/bin/python3 /usr/bin/python
COPY --from=py-ea /usr/lib/python3.8/site-packages /usr/lib/python3.8/site-packages
COPY --from=py-ea /opt/elastalert /opt/elastalert
COPY --from=py-ea /usr/bin/elastalert* /usr/bin/
WORKDIR /opt/elastalert-server
COPY . /opt/elastalert-server
RUN npm install --production --quiet
COPY config/elastalert.yaml /opt/elastalert/config.yaml
COPY config/elastalert-test.yaml /opt/elastalert/config-test.yaml
COPY config/config.json config/config.json
COPY rule_templates/ /opt/elastalert/rule_templates
COPY elastalert_modules/ /opt/elastalert/elastalert_modules
# Add default rules directory
# Set permission as unpriviledged user (1000:1000), compatible with Kubernetes
RUN mkdir -p /opt/elastalert/rules/ /opt/elastalert/server_data/tests/ \
&& chown -R node:node /opt
USER node
EXPOSE 3030
ENTRYPOINT ["npm", "start"]
Activity
nsano-rururu commentedon Jun 23, 2020
・Does docker save mean "docker save" command?
・What are Elasticsearch and Kibana version
・Are you aware that it worked properly in the Windows WSL Linux environment?
・What is the folder structure used by Elasticsearch, Kibana and ElastAlert docker on Windows WSL Linux and RHEL server? .. Is there any difference?
Answer example)
docker-compose.yml
kibana/config/kibana.yml
es/config/elasticsearch.yml
Dockerfiles/Dockerfile-elastalert
elastalert/bin/elastic_search_status.sh
elastalert/bin/elastalert-start.sh
elastalert/config/config.json
elastalert/config/elastalert-test.yml
elastalert/config/elastalert.yml
MinnMoto commentedon Jun 23, 2020
@nsano-rururu You have given me much to consider here. Let me see what I can do.
My Elastalert install in WSL gives an eniterly different error: 500. There might be more there for me to pour through though.
Thank you. I'll look at the files you recommend and try that.
nsano-rururu commentedon Jun 24, 2020
ElastAlert manages ElastAlert execution history and delta information with Elasticsearch index.
The ElastAlert index created in Elasticsearch is as follows.
elastalert_status_error
https://elastalert.readthedocs.io/en/latest/elastalert_status.html#elastalert-error
elastalert_status_status
https://elastalert.readthedocs.io/en/latest/elastalert_status.html#elastalert-status
elastalert_status
https://elastalert.readthedocs.io/en/latest/elastalert_status.html#elastalert
elastalert_status_silence
https://elastalert.readthedocs.io/en/latest/elastalert_status.html#silence
elastalert_status_past
nsano-rururu commentedon Jun 24, 2020
The main ElastAlert docker images are as follows.
bitsensor/elastalert:2.0.1 and bitsensor/elastalert:latest use a version of ElastAlert that doesn't work with Elasticsearch 7.x.
Please be careful.
Babel 6 → 7
bug fix
MinnMoto commentedon Jun 25, 2020
@nsano-rururu Thank you. I am trying to build a more current version of Elastalert (7.x compatible) to use in a new Docker image, based on bitsensor's build. Still running in to issues though. I appreciate your feedback.
nsano-rururu commentedon Jun 25, 2020
If the issue is still occurring, please tell me the command you ran and the error message
mpokale commentedon Jun 26, 2020
Hi
I created a docker image following instructions above.
This is what my Dockerfile looks like
Elastalert is not able to create an index . These are the errors I see when
mpokale commentedon Jun 26, 2020
Not sure what I am missing. Any help would be appreciated.
nsano-rururu commentedon Jun 26, 2020
@mpokale
There was an unnecessary description in elastalert/bin/elastalert-start.sh due to an edit error when posting.
I'm sorry. Please delete the last "es/config/elasticsearch.yml"
The following is the modified file.
elastalert/bin/elastalert-start.sh
MinnMoto commentedon Jun 26, 2020
nsano-rururu commentedon Jun 27, 2020
No more information from me. Do your best afterwards
MinnMoto commentedon Jun 27, 2020
mpokale commentedon Jun 29, 2020
@nsano-rururu Thank you!