Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kill all pids when killing pash via signal #454

Closed
wants to merge 21 commits into from
Closed

Conversation

dkarnikis
Copy link
Collaborator

This should fix #452

Signed-off-by: Dimitris Karnikis [email protected]

dkarnikis and others added 19 commits October 3, 2021 10:55
Signed-off-by: DIMITRIS KARNIKIS <[email protected]>
- fix broken link for evaluation scripts
- make markdown formatting consistent

Signed-off-by: Vladislav Doster <[email protected]>
Signed-off-by: DIMITRIS KARNIKIS <[email protected]>
Fix for python pkgs in pash setup
Signed-off-by: dkarnikis <dkarnikis>
Signed-off-by: dkarnikis <dkarnikis>
Signed-off-by: dkarnikis <dkarnikis>
Signed-off-by: DIMITRIS KARNIKIS <[email protected]>
Signed-off-by: DIMITRIS KARNIKIS <[email protected]>
Signed-off-by: DIMITRIS KARNIKIS <[email protected]>
Signed-off-by: DIMITRIS KARNIKIS <[email protected]>
@dkarnikis
Copy link
Collaborator Author

The only issue here is that I do not know any way to silence the produced output by the killed processes:

temp-analytics.sh:^CTraceback (most recent call last):                                              
  File "/home/dkarnikis/pash/compiler/pash.py", line 258, in <module>                               
    main()                                                                                          
  File "/home/dkarnikis/pash/compiler/pash.py", line 35, in main                                    
    return_code = preprocess_and_execute_asts(ast_objects, args, input_script_arguments, shell_name)
  File "/home/dkarnikis/pash/compiler/pash.py", line 73, in preprocess_and_execute_asts             
    return_code = execute_script(fname, args.command, input_script_arguments, shell_name)           
  File "/home/dkarnikis/pash/compiler/pash.py", line 254, in execute_script                         
    exec_obj = subprocess.run(subprocess_args, env=new_env, close_fds=False)                        
  File "/usr/lib/python3.8/subprocess.py", line 495, in run                                         
    stdout, stderr = process.communicate(input, timeout=timeout)                                    
  File "/usr/lib/python3.8/subprocess.py", line 1020, in communicate                                
    self.wait()
  File "/usr/lib/python3.8/subprocess.py", line 1083, in wait
    return self._wait(timeout=timeout)
  File "/usr/lib/python3.8/subprocess.py", line 1806, in _wait
    (pid, sts) = self._try_wait(0) 
  File "/usr/lib/python3.8/subprocess.py", line 1764, in _try_wait
    (pid, sts) = os.waitpid(self.pid, wait_flags)
KeyboardInterrupt
Killed
1.477
Terminated
25.665

gzip: stdin: unexpected end of file
cat: /tmp/pash_d3DWqNu/4a72fcee018249e68d6af4f1b1d0fc45/#fifo248: No such file or directory
cat: /tmp/pash_d3DWqNu/4a72fcee018249e68d6af4f1b1d0fc45/#fifo249: No such file or directory
cat: /tmp/pash_d3DWqNu/4a72fcee018249e68d6af4f1b1d0fc45/#fifo250: No such file or directory
cat: /tmp/pash_d3DWqNu/4a72fcee018249e68d6af4f1b1d0fc45/#fifo251: No such file or directory
cat: /tmp/pash_d3DWqNu/4a72fcee018249e68d6af4f1b1d0fc45/#fifo252: No such file or directory
cat: /tmp/pash_d3DWqNu/4a72fcee018249e68d6af4f1b1d0fc45/#fifo253: No such file or directory
cat: /tmp/pash_d3DWqNu/4a72fcee018249e68d6af4f1b1d0fc45/#fifo254: No such file or directory
cat: /tmp/pash_d3DWqNu/4a72fcee018249e68d6af4f1b1d0fc45/#fifo255: No such file or directory
cat: /tmp/pash_d3DWqNu/4a72fcee018249e68d6af4f1b1d0fc45/#fifo256: No such file or directory
cat: /tmp/pash_d3DWqNu/4a72fcee018249e68d6af4f1b1d0fc45/#fifo257: No such file or directory
cat: /tmp/pash_d3DWqNu/4a72fcee018249e68d6af4f1b1d0fc45/#fifo258: No such file or directory
cat: /tmp/pash_d3DWqNu/4a72fcee018249e68d6af4f1b1d0fc45/#fifo259: No such file or directory
cat: /tmp/pash_d3DWqNu/4a72fcee018249e68d6af4f1b1d0fc45/#fifo260: No such file or directory
cat: /tmp/pash_d3DWqNu/4a72fcee018249e68d6af4f1b1d0fc45/#fifo261: No such file or directory

All the processes are killed however :)

pa.sh Outdated
## In case we kill pash via signal, kill all the pending processes
function kill_all() {
# common prefix for all the pash processes
pkill -f -9 tmp/pash
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There could be processes that run commands like grep and sort right? I don't think this kills those. Also, could this kill the processes of another pash instance?

I think that in order to kill all processes we need to do something inside this: https://github.com/binpash/pash/blob/main/runtime/wait_for_output_and_sigpipe_rest.sh in addition to whatever we catch here.

Also, whenever killing processes, we also need to wait for them.

Also, this should not go to main but to future since it is an experimental change.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It actually kills everything!
In the current state of the script yeah, it will kill other pash instances. However, don't we assume that only a single instance of pash is running on machine?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It actually kills everything!

I don't see why tmp/pash would match a grep that is left running. Maybe I am missing something though

In the current state of the script yeah, it will kill other pash instances. However, don't we assume that only a single instance of pash is running on machine?

I don't think this is a valid assumption. We did a lot of modifications to allow multiple instances of pash to be run by the same user.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I am not mistaken, all the pash commands operate on /tmp/pash_XXXXX/blah, that's why the pkill -f -9 tmp/pash works for us.

Is there even a point to optimize the script to only kill the children of the current pash instance?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I am not mistaken, all the pash commands operate on /tmp/pash_XXXXX/blah, that's why the pkill -f -9 tmp/pash works for us.

A grep in the dataflow might read and write to two completely arbitrary files, therefore I am not sure if /tmp/pash would appear anywhere in its invocation.

Is there even a point to optimize the script to only kill the children of the current pash instance?

This is not an optimizations but rather correctness! If two instances of pash are running we don't want to kill both because one is killed.

@dkarnikis dkarnikis changed the base branch from main to future February 14, 2022 08:55
@dkarnikis
Copy link
Collaborator Author

I quite confident that the issue with the failing tests on the CI are already fixed on the fixes PR

@dkarnikis dkarnikis changed the base branch from future to main February 14, 2022 09:07
@dkarnikis dkarnikis changed the base branch from main to future February 14, 2022 09:08
@dkarnikis
Copy link
Collaborator Author

Closing this PR. Moved to #458

@dkarnikis dkarnikis closed this Feb 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

PaSh cleanup should remove all nodes
3 participants