Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flux executor: Remove mini from submit command #5229

Merged
merged 7 commits into from
Sep 10, 2024

Conversation

ewels
Copy link
Member

@ewels ewels commented Aug 15, 2024

Porting a set of changes from an email thread for discussion.

Context is that a user was trying to use Nextflow with Flux. The executor is written for flux mini and didn't work for him.

After some back and forth and investigation, he found the following set of changes got the executor to work on their system. This led to the creation of the following docs page for the system users: https://doku.lrz.de/nextflow-on-hpc-systems-test-operation-788693597.html

This was all only in an email thread, so moving here so that it doesn't get lost. I'm curious whether the flux executor support could be tweaked to work with both flux core and flux mini? (maybe with some configuration?). The changes seem pretty minor.

'cc @pditommaso and I think the original implementation was by @vsoch

@nextflow-io nextflow-io deleted a comment from netlify bot Aug 15, 2024
@pditommaso
Copy link
Member

Who is going to maintain this?

@vsoch
Copy link
Contributor

vsoch commented Aug 15, 2024

This looks OK - the only change with flux is we deprecated mini, so it's just flux submit.

@ewels
Copy link
Member Author

ewels commented Aug 15, 2024

Thanks @vsoch! Are you happy / do you understand the logic behind the other changes to the flags? (I haven't read up on them yet so don't).

@ewels
Copy link
Member Author

ewels commented Aug 15, 2024

Also do you mean that flux / flux mini are not two different systems? So we can just make this change and it's valid for all people using flux?

We might need some docs updates here as well.

@vsoch
Copy link
Contributor

vsoch commented Aug 16, 2024

And this title is not correct:

Draft proposal for supporting Flux (different to flux mini)

They are exactly the same.

@ewels ewels changed the title Draft proposal for supporting Flux (different to flux mini) Draft: proposal for supporting Flux Aug 16, 2024
@ewels ewels changed the title Draft: proposal for supporting Flux Flux executor updates Aug 16, 2024
@pditommaso
Copy link
Member

@ewels not sure what to do with this PR

@vsoch
Copy link
Contributor

vsoch commented Sep 4, 2024

@ewels could you please address my questions/comments so we can move the PR forward?

@ewels
Copy link
Member Author

ewels commented Sep 4, 2024

@vsoch because I just pasted the code diff from an email, I don't know the answers sorry. I'll follow up with him again and try to find out.

Signed-off-by: Phil Ewels <[email protected]>
@ewels ewels changed the title Flux executor updates Flux executor: Remove mini from submit command Sep 9, 2024
@ewels
Copy link
Member Author

ewels commented Sep 9, 2024

ok after feedback from the original author, this PR gets much simpler - now just a single line PR. I've also removed the mini from the docs.

Is this good to merge now @vsoch? Thanks for the review, and apologies for the slightly odd nature of this PR.

@vsoch
Copy link
Contributor

vsoch commented Sep 9, 2024

This now LGTM! 👍

@ewels ewels marked this pull request as ready for review September 9, 2024 13:51
@ewels ewels requested a review from a team as a code owner September 9, 2024 13:51
Signed-off-by: Phil Ewels <[email protected]>
@pditommaso
Copy link
Member

Tests green, there are some unresolved comments

Co-authored-by: Christopher Hakkaart <[email protected]>
Signed-off-by: Phil Ewels <[email protected]>
@ewels
Copy link
Member Author

ewels commented Sep 9, 2024

Comments all resolved 👌🏻

@pditommaso pditommaso merged commit a0f6902 into nextflow-io:master Sep 10, 2024
22 checks passed
@ewels ewels deleted the flux-support branch September 10, 2024 12:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants