Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optional progress bar on task page, updated with task-progress events #1071

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

EugZol
Copy link

@EugZol EugZol commented Feb 7, 2021

Progress bar which appears automatically on task page when task-progress event is received.

2021-02-07_22-33-44 (2)

It was discussed in #453 and #143 previously, but as far as I understand wasn't implemented. That PR provides unobtrusive built-in solution.

Example task code:

@shared_task(bind=True)
def test_task(self):
    from time import sleep

    for i in range(1, 102):
        sleep(1)
        self.send_event('task-progress', current=i, total=101)

@EugZol EugZol mentioned this pull request Feb 7, 2021
@ftaralle
Copy link

ftaralle commented Apr 27, 2021

Hi there, This feature would be realy cool !

I installed this branch of flower : pip3 install git+https://github.com/EugZol/flower.git
I use redis as broker and backend
I have a task that uses the send_event('task-progress', pattern.
Worker is started with the -E option.
But I see no progressbar in flower.

Here is my task code

@app.task(name='train', bind=True)
@wraps(raw_train) # used to inject doc & param information
def train(self, *args, **kwargs):
    # define a callback to update a progress state in celery
    def callback(progress):
        print('PROGRESS: %d%%' % int(100*progress))
        # send task-progress event for flower
        self.send_event('task-progress', current=int(100*progress), total=100)

    # execute the true train method
    return raw_train(*args, **kwargs, callback=callback)
# ###############################

starting flower (i dont know if broker_api is usefull in any way)
flower -A celery_train_worker --port=6006 --broker_api=redis://localhost:6379/0

starting the worker
celery -A celery_train_worker worker --loglevel=info -E -c1

@adampl
Copy link

adampl commented Dec 15, 2022

why isn't it merged already?

@JRGit4UE
Copy link

I would like to see a progress bar in flower as well, but unfortunately, the PR has never been merged. So I tried to build the fork EugZol/flower, which seems to be outdated as well ERROR: for flower_flower_1 Cannot start service flower: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "flower": executable not found in $PATH: unknown on my Ubuntu 20.04, Docker 23.0.1, docker-compose 1.25.0

@geekalaa
Copy link

geekalaa commented Sep 17, 2023

It's working but the UI and the version is old:
celery==5.3.1
redis==5.0.0

DockerFile :

FROM python:3.8.18-alpine3.18

# Get latest root certificates
RUN apk add --no-cache ca-certificates && update-ca-certificates
COPY ./flower-master /flower-master
# Install the required packages
RUN pip install --no-cache-dir redis 
RUN pip install --no-cache-dir -e /flower-master

# PYTHONUNBUFFERED: Force stdin, stdout and stderr to be totally unbuffered. (equivalent to `python -u`)
# PYTHONHASHSEED: Enable hash randomization (equivalent to `python -R`)
# PYTHONDONTWRITEBYTECODE: Do not write byte files to disk, since we maintain it as readonly. (equivalent to `python -B`)
ENV PYTHONUNBUFFERED=1 PYTHONHASHSEED=random PYTHONDONTWRITEBYTECODE=1

# Default port
EXPOSE 8888

ENV FLOWER_DATA_DIR /data
ENV PYTHONPATH ${FLOWER_DATA_DIR}

WORKDIR $FLOWER_DATA_DIR

# Add a user with an explicit UID/GID and create necessary directories
RUN set -eux; \
    addgroup -g 1000 flower; \
    adduser -u 1000 -G flower flower -D; \
    mkdir -p "$FLOWER_DATA_DIR"; \
    chown flower:flower "$FLOWER_DATA_DIR"
USER flower

VOLUME $FLOWER_DATA_DIR

TASK :

@app.task(bind=True,default_retry_delay=30, max_retries=1,time_limit=9000)
def test_progress(self):
    from time import sleep
    for progress in range(1, 101):
        sleep(1)
        self.send_event('task-progress', current=progress, total=100)

Docker-compose:

flower:
    build: 
      context: ./flower
    container_name: flower
    command: python -m flower flower 
    environment:
      - 'CELERY_BROKER_URL=redis://redis:6379'
      - FLOWER_PORT=8888
      - TZ=Africa/Tunis
    ports:
      - 8989:8888 
    volumes:
      - ./flower/flower_data:/data
    depends_on:
      - redis
    restart: always

@invoker-bot
Copy link

It's cool!But is there any progress with this feature?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants